Skip to content

UNT162-15: promoção para produção atrás de aprovação (digests do staging, rollback automático, docs/deploy-gke.md) - #22

Merged
devin-ai-integration[bot] merged 3 commits into
masterfrom
devin/unt162-15-prod-approval-gate
Nov 5, 2026
Merged

devin-ai-integration[bot] merged 3 commits into
masterfrom
devin/unt162-15-prod-approval-gate

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Resumo

Passo s4.3: depois de um deploy verde em staging, a produção é promovida atrás do environment production (required reviewers), com os mesmos digests que passaram em staging (sem rebuild), values-prod.yaml, smoke + e2e contra prod e helm rollback automático em qualquer falha.

ci.yml: publish ─▶ deploy-staging (result=deployed, artefato staging-release) ─▶ deploy-prod
deploy-prod.yml:
  release  (sem environment) : confere PROD_* vars → skip com notice se ausentes; mostra tag + 4 digests no summary
  promote  (environment: production → espera aprovação)
     WIF staging (token) + WIF prod (token)        # sem chave de service account
     ci/deploy/promote-images.sh                    # skopeo copy --all --preserve-digests staging@digest → prod:<sha>, confere o digest
     ci/deploy/helm-deploy.sh -f values-prod.yaml   # o mesmo script do staging (#20): rollout, smoke, e2e, rollback
  • Skip: sem PROD_GCP_WORKLOAD_IDENTITY_PROVIDER / PROD_GCP_CI_SERVICE_ACCOUNT / PROD_GCP_ARTIFACT_REGISTRY / PROD_GKE_CLUSTER o job release emite ::notice e o promote nem entra na fila de aprovação (hoje não há projeto GCP). Com essas setadas, falta de PROD_CLOUDSQL_* / PROD_URL / GCP_* de staging é erro.
  • Re-promover uma release anterior: workflow_dispatch com staging-run-id (só runs de master) baixa o artefato staging-release daquele run (90 dias) e passa pela mesma aprovação.
  • Checkout do próprio SHA da release no promote: chart/values iguais aos que staging testou.
  • Terraform: novo allowed_environments no módulo WIF → ci_allowed_environments = ["production"] no env prod. O pool prod só troca tokens de jobs master do environment production, então não existe token de prod sem aprovação. Nada foi aplicado.
  • deploy-staging.yml: novo output result e upload do staging-release (release.json: tag + digests) quando o deploy passa.
  • promote-images.sh não reescreve tag que já existe em prod (tags imutáveis): se já serve o digest, pula (re-promoção/rollback); se serve outro, falha.
  • promote-images.sh roda o skopeo 1.16 de uma imagem pinada por digest: o skopeo 1.13 do Ubuntu/runner não tem --preserve-digests e reescreve o índice com attestations (falhou assim no kind).
  • docs/deploy-gke.md (novo, linkado no README): pipeline, setup do environment production (UI + gh api), variáveis, release, rollback automático, rollback manual (helm rollback + como achar os digests anteriores com helm get values --revision) e workflow local no kind. trading-engine segue com 1 réplica (d-engine-replicas).

Ainda pendente (fora do alcance sem acesso): criar o environment production com reviewers (precisa de [REDACTED SECRET] no repo, comando em docs/deploy-gke.md) e o dry-run de promoção num cluster prod real; não existe projeto GCP.

Como testar

mvn -B -DskipTests package jib:dockerBuild && docker build -t esarch/trader-ui:latest trader-app-ui
bash ci/kind/verify-prod-promotion.sh
deploy/helm/scripts/validate.sh && infra/terraform/scripts/validate.sh
actionlint && shellcheck -x ci/kind/verify-prod-promotion.sh ci/deploy/*.sh

Evidência (local, kind v0.29 / k8s 1.33)

bash ci/kind/verify-prod-promotion.sh: dois registries locais fazem o papel do Artifact Registry de staging e de prod, e o cluster só consegue puxar do de "prod":

== 1. promote release 1 (staging digests, no rebuild) and deploy with values-prod.yaml
trading-engine sha256:d5bfc9a2…   trader-app sha256:4e87f5a5…   discovery-server sha256:efc4344d…   trader-ui sha256:a7ae9011…
| Postman smoke (newman): 10/10 requests, 23/23 assertions | pass |
SMOKE PASS
prod registry serves the staging digests under 8d22a2db8614cde80bbea7fa46ba9560cc4451b8
trading-engine: spec + pods on localhost:5002/trading-engine@sha256:d5bfc9a29a26ad9c634acfc175c87d66ceefd4b67ceb314a71076feefe7a5650
trader-app: spec + pods on localhost:5002/trader-app@sha256:4e87f5a5de0669303f38c2234a48339a49b9009d775a8c6c4ec4db049e581d59
discovery-server: spec + pods on localhost:5002/discovery-server@sha256:efc4344d3f49d463439e94d6230af2d06546ecf36e6107f6f410f6b3eb9d415b
trader-ui: spec + pods on localhost:5002/trader-ui@sha256:a7ae90110f871a31b20ec5b93d209ef943098557a759d6f3f8b2a90cbb59553e
trader-app   Deployment/trader-app   cpu: <unknown>/70%   3   10   3     (HPA de values-prod)
release: 1 deployed Install complete

== 2. promote a release whose trader-app crash-loops: expect rollout failure + automatic rollback to release 1
DEPLOY FAILED (rollout of trader-app): rolled back axon-trader to revision 1 (ok)
exit status: 1 (expected non-zero)
release: 3 deployed Rollback to 1
trader-app: spec + pods on localhost:5002/trader-app@sha256:4e87f5a5…   (de volta à release 1)

== 3. promote release 2, then the manual rollback of docs/deploy-gke.md back to release 1
SMOKE PASS                                        (release 2: trader-ui sha256:f8a07fb6…)
previous revision 3 ran: {"trading-engine":"sha256:d5bfc9a2…","trader-app":"sha256:4e87f5a5…","discovery-server":"sha256:efc4344d…","trader-ui":"sha256:a7ae9011…"}
Rollback was a success! Happy Helming!
SMOKE PASS

REVISION  STATUS      DESCRIPTION
1         superseded  Install complete
2         superseded  Upgrade complete
3         superseded  Rollback to 1
4         superseded  Upgrade complete
5         deployed    Rollback to 3
NAME               READY   REPLICAS   IMAGE
discovery-server   1       1          localhost:5002/discovery-server@sha256:efc4344d…
trader-app         3       3          localhost:5002/trader-app@sha256:4e87f5a5…
trader-ui          3       3          localhost:5002/trader-ui@sha256:a7ae9011…
trading-engine     1       1          localhost:5002/trading-engine@sha256:d5bfc9a2…
PASS: staging digests promoted without rebuild and deployed with values-prod.yaml (smoke + e2e green); crash-looping release rolled back automatically; manual helm rollback to the previous digests green

Checagens estáticas:

$ deploy/helm/scripts/validate.sh            → helm lint --strict + kubeconform OK (default, staging, prod, kind, kind-gateway)
$ infra/terraform/scripts/validate.sh        → fmt/validate OK, todos os terraform test pass (inclui o novo assert
                                                "... && assertion.ref in ['refs/heads/master'] && assertion.environment in ['production']"), tflint: no issues
$ actionlint                                 → sem saída (OK)
$ shellcheck -x ci/kind/verify-prod-promotion.sh ci/deploy/*.sh → OK
$ zizmor --offline ci.yml deploy-staging.yml deploy-prod.yml → nenhum achado novo (o deploy-prod recebe só SLACK_WEBHOOK_URL, não `secrets: inherit`)
$ promote-images.sh com digest inexistente → "manifest unknown", exit 1
$ promote-images.sh (tags imutáveis em prod): 1ª promoção de A → copia, exit 0; re-promover A (rollback) → "trader-ui sha256:a7ae… (already promoted)", exit 0;
  mesma tag com outro digest → "FAIL: …:shaA already holds sha256:a7ae…, staging passed sha256:d5bf…", exit 1

Nenhuma credencial commitada; nada aplicado em GCP nem em TAS.

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/811de16662ea4ae1b8632c50739ae1fb
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/811de16662ea4ae1b8632c50739ae1fb?variant=devin
Requested by: @matthewguerra-cog


Devin Review

Status Commit
⚪ Not started —

Run Devin Review

Enable auto-review to review your pull requests automatically.

Devin Review (Staging)


Devin Review

…gests do staging, sem rebuild), rollback automático e docs/deploy-gke.md (s4.3)

Co-Authored-By: Matthew Guerra <matthew.guerra@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration[bot]

This comment was marked as resolved.

… imutáveis), falha se servem outro

Co-Authored-By: Matthew Guerra <matthew.guerra@cognition.ai>
devin-ai-integration[bot]

This comment was marked as resolved.

…es exatos do manifesto

Co-Authored-By: Matthew Guerra <matthew.guerra@cognition.ai>
@devin-ai-integration
devin-ai-integration Bot merged commit bac0396 into master Nov 5, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant