Kubernetes二进制安装实战：从原理到生产部署

1. 为什么选择二进制安装Kubernetes在容器编排领域Kubernetes简称K8s的安装方式主要分为两大类使用kubeadm等工具快速部署和手动二进制安装。二进制安装虽然步骤繁琐但却是理解K8s架构最彻底的方式。我经历过从kubeadm到二进制安装的完整迁移过程深刻体会到二进制方式带来的三大优势架构透明性每个组件都以独立进程运行日志分离便于故障排查定制灵活性可自由指定各个组件的版本、启动参数和部署位置生产适用性国内多数企业的生产环境仍采用二进制部署方式重要提示二进制安装适合需要深度掌控K8s的团队新手建议先用kubeadm熟悉基本操作后再尝试。2. 环境准备与组件规划2.1 硬件资源配置建议根据我参与的多个生产级部署案例建议采用如下配置以4节点集群为例角色CPU内存磁盘网络数量Master节点4核8G100G千兆双网卡2Worker节点8核16G200G万兆生产网卡2负载均衡2核4G50G千兆双网卡22.2 软件版本选型策略版本兼容性是二进制安装的最大挑战之一。经过多次踩坑验证推荐以下稳定组合# 组件版本清单经过生产验证 K8S_VERSIONv1.23.6 # 长期支持版本 ETCD_VERSIONv3.5.3 # 匹配K8s版本要求 CNI_PLUGINcalico-v3.22.1 # 网络插件 CONTAINERD1.6.4 # 容器运行时避坑经验避免使用各组件的最新版本奇数版本如1.25通常含有实验性功能生产环境应选择偶数版本。3. 核心组件安装实战3.1 证书体系构建二进制安装最大的技术难点在于证书管理。这是我总结的证书生成模板# 生成CA根证书有效期10年 openssl genrsa -out ca.key 2048 openssl req -x509 -new -nodes -key ca.key \ -subj /CNkubernetes-ca -days 3650 -out ca.crt # 生成apiserver证书注意SAN配置 cat apiserver-csr.json EOF { CN: kube-apiserver, hosts: [ 127.0.0.1, 192.168.101.100, # Master节点IP 10.96.0.1, # Service Cluster IP kubernetes, kubernetes.default ], key: { algo: rsa, size: 2048 } } EOF证书生成后需要严格验证# 验证证书链完整性 openssl verify -CAfile ca.crt apiserver.crt # 检查证书有效期 openssl x509 -in apiserver.crt -noout -dates3.2 etcd集群部署etcd作为K8s的大脑其部署质量直接影响集群稳定性。这是我优化过的启动脚本# etcd启动参数关键配置 ETCD_OPTS --nameetcd01 \ --data-dir/var/lib/etcd \ --listen-peer-urlshttps://192.168.101.100:2380 \ --listen-client-urlshttps://192.168.101.100:2379,https://127.0.0.1:2379 \ --advertise-client-urlshttps://192.168.101.100:2379 \ --initial-advertise-peer-urlshttps://192.168.101.100:2380 \ --initial-clusteretcd01https://192.168.101.100:2380,etcd02https://192.168.101.101:2380 \ --client-cert-authtrue \ --trusted-ca-file/etc/kubernetes/pki/ca.crt \ --cert-file/etc/kubernetes/pki/etcd/server.crt \ --key-file/etc/kubernetes/pki/etcd/server.key 健康检查方法# 检查集群成员 ETCDCTL_API3 etcdctl \ --endpointshttps://192.168.101.100:2379 \ --cacert/etc/kubernetes/pki/ca.crt \ --cert/etc/kubernetes/pki/etcd/healthcheck-client.crt \ --key/etc/kubernetes/pki/etcd/healthcheck-client.key \ member list4. 控制平面组件配置4.1 kube-apiserver关键参数apiserver是集群的网关这些参数经过生产环境验证# /usr/lib/systemd/system/kube-apiserver.service [Service] ExecStart/usr/local/bin/kube-apiserver \ --advertise-address192.168.101.100 \ --allow-privilegedtrue \ --authorization-modeNode,RBAC \ --client-ca-file/etc/kubernetes/pki/ca.crt \ --enable-admission-pluginsNodeRestriction \ --enable-bootstrap-token-authtrue \ --etcd-cafile/etc/kubernetes/pki/ca.crt \ --etcd-certfile/etc/kubernetes/pki/apiserver-etcd-client.crt \ --etcd-keyfile/etc/kubernetes/pki/apiserver-etcd-client.key \ --etcd-servershttps://192.168.101.100:2379,https://192.168.101.101:2379 \ --kubelet-client-certificate/etc/kubernetes/pki/apiserver-kubelet-client.crt \ --kubelet-client-key/etc/kubernetes/pki/apiserver-kubelet-client.key \ --service-account-key-file/etc/kubernetes/pki/sa.pub \ --service-account-signing-key-file/etc/kubernetes/pki/sa.key \ --service-cluster-ip-range10.96.0.0/12 \ --tls-cert-file/etc/kubernetes/pki/apiserver.crt \ --tls-private-key-file/etc/kubernetes/pki/apiserver.key \ --request-timeout300s \ # 调大超时防止长操作失败 --max-mutating-requests-inflight500 \ # 提升写操作并发 --max-requests-inflight1500 # 提升读操作并发4.2 高可用负载均衡配置生产环境必须配置负载均衡这是用Nginx实现LB的配置示例stream { upstream kube-apiserver { server 192.168.101.100:6443 max_fails3 fail_timeout30s; server 192.168.101.101:6443 max_fails3 fail_timeout30s; } server { listen 6443; proxy_pass kube-apiserver; proxy_connect_timeout 2s; proxy_timeout 900s; # 保持长连接 } }健康检查脚本#!/bin/bash APISERVERS(192.168.101.100 192.168.101.101) for ip in ${APISERVERS[]}; do if ! curl -sk --connect-timeout 2 https://${ip}:6443/healthz; then systemctl restart kube-apiserver fi done5. 工作节点部署5.1 containerd容器运行时优化containerd的配置直接影响容器性能关键配置如下# /etc/containerd/config.toml [plugins.io.containerd.grpc.v1.cri] sandbox_image registry.aliyuncs.com/google_containers/pause:3.6 [plugins.io.containerd.grpc.v1.cri.containerd] snapshotter overlayfs [plugins.io.containerd.grpc.v1.cri.registry] [plugins.io.containerd.grpc.v1.cri.registry.mirrors] [plugins.io.containerd.grpc.v1.cri.registry.mirrors.docker.io] endpoint [https://registry-1.docker.io] [plugins.io.containerd.grpc.v1.cri.containerd.runtimes.runc] runtime_type io.containerd.runc.v2 [plugins.io.containerd.grpc.v1.cri.containerd.runtimes.runc.options] SystemdCgroup true # 使用systemd cgroup驱动5.2 kubelet关键参数调优kubelet是节点核心组件这些参数能显著提升稳定性# /var/lib/kubelet/config.yaml apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration evictionHard: memory.available: 500Mi nodefs.available: 10% imagefs.available: 15% maxPods: 150 # 根据节点规格调整 kubeAPIQPS: 50 kubeAPIBurst: 100 serializeImagePulls: false # 并行拉取镜像 rotateCertificates: true # 自动轮换证书6. 网络插件与集群验证6.1 Calico网络部署技巧Calico是生产环境首选网络方案部署时注意# 修改calico.yaml关键参数 sed -i s|192.168.0.0/16|10.244.0.0/16|g calico.yaml # 修改Pod网段 sed -i /FELIX_IPTABLESBACKEND/s/^# //g calico.yaml # 启用iptables-nft # 启用BGP路由提升网络性能 cat calico.yaml EOF - name: CALICO_IPV4POOL_IPIP value: Never - name: IP_AUTODETECTION_METHOD value: interfaceeth.* EOF6.2 集群健康检查大全部署完成后必须执行的检查项# 检查节点状态 kubectl get nodes -o wide # 检查核心组件健康状态 kubectl get --raw/readyz?verbose # 检查证书过期时间 openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -dates # 压力测试创建100个nginx pod kubectl run test --imagenginx --replicas100 --port807. 生产环境运维要点7.1 证书自动轮换方案证书过期是生产环境常见故障推荐使用cert-manager实现自动续期# 安装cert-manager kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml # 创建CA Issuer cat EOF | kubectl apply -f - apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: ca-issuer namespace: kube-system spec: ca: secretName: ca-key-pair EOF7.2 监控与日志方案必须部署的监控三件套指标监控Prometheus-Operator Grafanahelm install prometheus-stack prometheus-community/kube-prometheus-stack \ --namespace monitoring \ --set alertmanager.enabledfalse日志收集Loki Promtailhelm install loki grafana/loki-stack \ --namespace logging \ --set promtail.enabledtrue事件审计启用K8s审计日志# /etc/kubernetes/audit-policy.yaml apiVersion: audit.k8s.io/v1 kind: Policy rules: - level: Metadata resources: - group: resources: [secrets, configmaps]经过多次生产环境部署验证二进制安装的K8s集群在稳定性方面明显优于工具化安装方案。特别是在需要深度定制的场景下二进制方式能让你对集群的每个细节都了如指掌。记得定期检查证书有效期这是运维过程中最容易忽视的风险点。