技术博客

ArgoCD 多环境部署实战:Helm + Kustomize 管理 Dev/Staging/Prod 差异

详解如何用 ArgoCD 配合 Helm values 文件或 Kustomize overlays 管理 dev/staging/production 三套环境的配置差异,包括镜像标签策略、环境变量注入、资源配额差异、ApplicationSet 批量管理,避免重复 YAML。

ArgoCDGitOpsHelmKustomize多环境Kubernetes

多环境管理是 GitOps 落地最常遇到的挑战:dev/staging/prod 三套环境需要相同的应用,但副本数、资源限制、镜像标签、外部依赖地址各不相同。如果为每个环境复制一份完整 YAML,维护三份重复代码噩梦级别。本文讲解用 Helm + ArgoCD 或 Kustomize + ArgoCD 优雅解决这个问题。

两种方案对比

方案 优点 适合场景
Helm values 文件 模板功能强、条件逻辑丰富 第三方 Chart、复杂配置
Kustomize overlays 无模板语法、纯 YAML、学习成本低 自研应用、不喜欢模板

两种方案都可以与 ArgoCD 无缝集成,本文各讲一套完整流程。


方案一:Helm + ArgoCD 多环境

仓库结构

k8s-configs/
├── charts/
│   └── myapp/           ← 自研 Helm Chart(或引用官方 Chart)
│       ├── Chart.yaml
│       ├── templates/
│       │   ├── deployment.yaml
│       │   ├── service.yaml
│       │   └── hpa.yaml
│       └── values.yaml  ← 默认值(通用基础配置)
├── envs/
│   ├── dev/
│   │   └── values.yaml  ← dev 环境覆盖
│   ├── staging/
│   │   └── values.yaml  ← staging 环境覆盖
│   └── prod/
│       └── values.yaml  ← prod 环境覆盖
└── argocd/
    ├── app-dev.yaml     ← ArgoCD Application(dev)
    ├── app-staging.yaml ← ArgoCD Application(staging)
    └── app-prod.yaml    ← ArgoCD Application(prod)

Chart 模板示例

# charts/myapp/values.yaml(默认值)
image:
  repository: registry.cn-hangzhou.aliyuncs.com/mycompany/myapp
  tag: "latest"
  pullPolicy: IfNotPresent

replicaCount: 1

resources:
  requests:
    cpu: "100m"
    memory: "128Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

env:
  APP_ENV: "development"
  LOG_LEVEL: "debug"
  DB_HOST: "postgres.dev.svc.cluster.local"
  REDIS_HOST: "redis.dev.svc.cluster.local"

hpa:
  enabled: false
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilization: 70

ingress:
  enabled: true
  host: "myapp.dev.example.com"
# charts/myapp/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
  namespace: {{ .Release.Namespace }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app: {{ .Release.Name }}
        version: {{ .Values.image.tag | quote }}
    spec:
      containers:
      - name: myapp
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        env:
        {{- range $key, $val := .Values.env }}
        - name: {{ $key }}
          value: {{ $val | quote }}
        {{- end }}
        resources:
          {{- toYaml .Values.resources | nindent 10 }}
        ports:
        - containerPort: 8080

环境差异化 values 文件

# envs/dev/values.yaml(开发环境)
image:
  tag: "dev-latest"   # CI 自动更新这里

replicaCount: 1

resources:
  requests:
    cpu: "50m"
    memory: "64Mi"
  limits:
    cpu: "200m"
    memory: "256Mi"

env:
  APP_ENV: "development"
  LOG_LEVEL: "debug"
  DB_HOST: "postgres.dev.svc.cluster.local"

ingress:
  host: "myapp.dev.example.com"
# envs/staging/values.yaml(预发布环境)
image:
  tag: "v1.2.3-rc1"   # 固定 RC 版本

replicaCount: 2

resources:
  requests:
    cpu: "200m"
    memory: "256Mi"
  limits:
    cpu: "1000m"
    memory: "1Gi"

env:
  APP_ENV: "staging"
  LOG_LEVEL: "info"
  DB_HOST: "postgres.staging.svc.cluster.local"

ingress:
  host: "myapp.staging.example.com"
# envs/prod/values.yaml(生产环境)
image:
  tag: "v1.2.3"   # 正式版本,手动发布

replicaCount: 5

resources:
  requests:
    cpu: "500m"
    memory: "512Mi"
  limits:
    cpu: "2000m"
    memory: "2Gi"

env:
  APP_ENV: "production"
  LOG_LEVEL: "warn"
  DB_HOST: "postgres.prod.svc.cluster.local"

hpa:
  enabled: true
  minReplicas: 5
  maxReplicas: 20
  targetCPUUtilization: 60

ingress:
  host: "myapp.example.com"

ArgoCD Application(Helm 多 values 文件)

# argocd/app-prod.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-prod
  namespace: argocd
  annotations:
    # 通知 Slack 部署结果(需要安装 argocd-notifications)
    notifications.argoproj.io/subscribe.on-sync-succeeded.slack: prod-deploys
spec:
  project: production   # 使用生产项目(有额外 RBAC 限制)
  source:
    repoURL: https://github.com/your-org/k8s-configs
    targetRevision: HEAD
    path: charts/myapp
    helm:
      # 叠加多个 values 文件:先加载 chart 默认值,再叠加环境值
      valueFiles:
      - values.yaml              # chart 内置默认值
      - ../../envs/prod/values.yaml  # 生产环境覆盖(相对路径)
  destination:
    server: https://k8s-prod.internal:6443   # 生产集群
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
    retry:
      limit: 3
      backoff:
        duration: 30s
        factor: 2
        maxDuration: 5m

方案二:Kustomize + ArgoCD 多环境

仓库结构

k8s-configs/
├── base/                    ← 通用基础配置
│   ├── deployment.yaml
│   ├── service.yaml
│   └── kustomization.yaml
└── overlays/
    ├── dev/                 ← dev 环境差异
    │   ├── kustomization.yaml
    │   └── patches/
    │       ├── replicas.yaml
    │       └── resources.yaml
    ├── staging/
    │   ├── kustomization.yaml
    │   └── patches/
    │       └── replicas.yaml
    └── prod/
        ├── kustomization.yaml
        └── patches/
            ├── replicas.yaml
            ├── resources.yaml
            └── hpa.yaml

Base 配置

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: registry.cn-hangzhou.aliyuncs.com/mycompany/myapp:latest
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
        env:
        - name: APP_ENV
          value: "development"
        - name: LOG_LEVEL
          value: "debug"
# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml

Overlay 配置(生产环境)

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# 继承 base
resources:
- ../../base

# 命名空间
namespace: production

# 为所有资源添加 prod 标签
commonLabels:
  environment: production

# 镜像版本替换(CI 流水线更新这里)
images:
- name: registry.cn-hangzhou.aliyuncs.com/mycompany/myapp
  newTag: "v1.2.3"   # 每次发布更新此处

# 应用 patches
patches:
- path: patches/replicas.yaml
- path: patches/resources.yaml
- path: patches/env.yaml

# 追加 HPA(仅生产有)
resources:
- patches/hpa.yaml
# overlays/prod/patches/replicas.yaml
# 修改副本数为 5
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 5
# overlays/prod/patches/resources.yaml
# 覆盖资源限制
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
      - name: myapp
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"
          limits:
            cpu: "2000m"
            memory: "2Gi"
# overlays/prod/patches/env.yaml
# 覆盖环境变量
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
      - name: myapp
        env:
        - name: APP_ENV
          value: "production"
        - name: LOG_LEVEL
          value: "warn"
        - name: DB_HOST
          value: "postgres.prod.svc.cluster.local"

ArgoCD Application(Kustomize)

# argocd/app-prod-kustomize.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/k8s-configs
    targetRevision: HEAD
    path: overlays/prod   # 直接指向 overlay 目录,ArgoCD 自动识别 Kustomize
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

本地预览渲染结果:

# 在提交 PR 前预览最终 YAML
kubectl kustomize overlays/prod

# 对比 dev 和 prod 的差异
diff <(kubectl kustomize overlays/dev) <(kubectl kustomize overlays/prod)

ApplicationSet:批量管理多环境

手动创建 3 个 Application 对象还算简单,但如果有 20 个微服务 × 3 个环境 = 60 个 Application,手动管理噩梦级别。ApplicationSet 可以用模板批量生成:

# applicationset-multi-env.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapp-all-envs
  namespace: argocd
spec:
  # 生成器:为每个 (app, env) 组合生成一个 Application
  generators:
  - matrix:
      generators:
      - list:
          elements:
          - app: myapp
            chart: charts/myapp
          - app: api-gateway
            chart: charts/api-gateway
          - app: user-service
            chart: charts/user-service
      - list:
          elements:
          - env: dev
            cluster: https://k8s-dev.internal:6443
            namespace: development
            valuesFile: envs/dev/values.yaml
          - env: staging
            cluster: https://k8s-staging.internal:6443
            namespace: staging
            valuesFile: envs/staging/values.yaml
          - env: prod
            cluster: https://k8s-prod.internal:6443
            namespace: production
            valuesFile: envs/prod/values.yaml

  # Application 模板
  template:
    metadata:
      name: "{{app}}-{{env}}"
      namespace: argocd
    spec:
      project: default
      source:
        repoURL: https://github.com/your-org/k8s-configs
        targetRevision: HEAD
        path: "{{chart}}"
        helm:
          valueFiles:
          - values.yaml
          - "../../{{valuesFile}}"
      destination:
        server: "{{cluster}}"
        namespace: "{{namespace}}"
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
        - CreateNamespace=true
kubectl apply -f applicationset-multi-env.yaml

# 自动生成 9 个 Application(3 app × 3 env)
kubectl get application -n argocd
# NAME                    SYNC STATUS  HEALTH
# myapp-dev               Synced       Healthy
# myapp-staging           Synced       Healthy
# myapp-prod              Synced       Healthy
# api-gateway-dev         Synced       Healthy
# api-gateway-staging     Synced       Healthy
# api-gateway-prod        Synced       Healthy
# user-service-dev        Synced       Healthy
# user-service-staging    Synced       Healthy
# user-service-prod       Synced       Healthy

CI/CD 集成:自动更新镜像标签

GitOps 流水线的关键节点是 CI 构建完镜像后,自动更新 Git 中的镜像标签:

# CI 流水线脚本示例(GitHub Actions/Jenkins)

# 构建并推送镜像
docker build -t registry.cn-hangzhou.aliyuncs.com/mycompany/myapp:${GIT_SHA} .
docker push registry.cn-hangzhou.aliyuncs.com/mycompany/myapp:${GIT_SHA}

# 更新 Git 中的镜像标签(触发 ArgoCD 自动同步)
git clone https://github.com/your-org/k8s-configs
cd k8s-configs

# Kustomize 方式:更新 overlay
cd overlays/dev
kustomize edit set image \
  registry.cn-hangzhou.aliyuncs.com/mycompany/myapp=registry.cn-hangzhou.aliyuncs.com/mycompany/myapp:${GIT_SHA}

# 或直接 sed 修改 Helm values
# sed -i "s/tag: .*/tag: \"${GIT_SHA}\"/" envs/dev/values.yaml

git config user.email "ci@company.com"
git config user.name "CI Bot"
git add -A
git commit -m "chore: update myapp dev image to ${GIT_SHA}"
git push

# ArgoCD 检测到 Git 变更后自动同步(约 3 分钟,或配置 Webhook 立即触发)

生产环境同步策略建议

# 生产环境建议:不开 automated,手动审批同步
spec:
  syncPolicy:
    # automated: 注释掉,需要手动触发
    syncOptions:
    - CreateNamespace=true
    retry:
      limit: 3
      backoff:
        duration: 30s
        maxDuration: 10m
# 查看 prod 是否有待同步的变更
argocd app diff myapp-prod

# 确认无误后手动同步
argocd app sync myapp-prod --prune

# 同步后观察健康状态
argocd app wait myapp-prod --health --timeout 300

小结

Helm values 文件叠加和 Kustomize overlays 都能优雅解决多环境配置差异问题,核心原则一致:基础配置只写一份,差异只在各环境的覆盖文件中。ApplicationSet 进一步消除了多 Application 的手动管理负担。生产实践中,dev 环境适合全自动同步(快速迭代),prod 环境适合手动审批(稳定优先),staging 环境居中。下一篇将讲解 ArgoCD App of Apps 模式实现多集群统一管理。