Deployment Pipeline

This document is the source of truth for the GitLab deployment flow.

Secret source

The deployment pipeline supports two modes:

• preferred: SOPS-encrypted env files committed in the repository
• fallback: environment-scoped GitLab CI variables rendered into .env

Priority order:

1. infra/env/{environment}.env.enc
2. scoped GitLab CI variables

Environments

• staging
  • deployed automatically on every push to main
  • used to validate the current main state before production rollout
• production
  • deployed manually from the main pipeline
  • intended as an explicit promotion step after staging validation

GitLab jobs

• deploy:staging
  • environment: staging
  • resource group: staging
  • trigger: automatic on main
• deploy:prod
  • environment: production
  • resource group: production
  • trigger: manual on main

Both jobs reuse the same deployment flow:

1. sync application sources to the target server
2. decrypt infra/env/{environment}.env.enc or generate .env from CI variables
3. validate env with scripts/check-env.sh
4. run infra/deploy.sh
5. run infra/migrate-improved.sh

SOPS setup

Required in CI when using encrypted env files:

• preferred: SOPS_AGE_KEY_B64
  • base64-encoded private age key content on one line
  • recommended for GitLab variables
• fallback: SOPS_AGE_KEY
  • raw multiline private age key content

Generate SOPS_AGE_KEY_B64 with:

base64 < ~/.config/sops/age/aaperture.txt | tr -d '\n'

Repository files:

• .sops.yaml
• infra/env/staging.env.enc
• infra/env/production.env.enc

Helper files:

• infra/env/README.md
• scripts/bootstrap-sops-config.sh
• scripts/render-deploy-env.sh
• scripts/export-gitlab-env.mjs
• Makefile (make sops-decrypt-*, make sops-encrypt-*, make sops-edit-*)

SSH deploy key in GitLab

Recommended CI variable:

• SSH_PRIVATE_KEY_B64
  • base64-encoded SSH private key on a single line
  • avoids multiline formatting issues in GitLab variables

Generate the recommended base64 value with:

base64 < ~/.ssh/<your-deploy-key> | tr -d '\n'

Variable strategy

Use GitLab environment-scoped variables with the same variable names for both environments.

Examples of variables that should be scoped separately for staging and production:

• SERVER_HOST
• SERVER_USER
• SERVER_APP_DIR
• COMPOSE_PROJECT_NAME
• SSH_PRIVATE_KEY_B64
• DOMAIN
• NAME
• FRONTEND_URL
• FRONTEND_BASE_URL
• API_BASE_URL
• PUBLIC_ROOT_DOMAIN
• ALLOWED_ORIGINS
• GOOGLE_CLIENT_ID
• GOOGLE_CLIENT_SECRET
• GOOGLE_CALLBACK_URL
• POSTGRES_DB
• POSTGRES_USER
• POSTGRES_PASSWORD
• POSTGRES_HOST_PORT
• API_HOST_PORT
• WEB_HOST_PORT
• PROMETHEUS_HOST_PORT
• GRAFANA_HOST_PORT
• R2_*
• STRIPE_*
• SENTRY_*
• VAPID_*

Recommendation:

• keep shared secrets global only when they are truly identical across environments
• scope host/domain/database credentials per environment
• prefer SERVER_APP_DIR over legacy aliases in new configuration
• keep distinct host ports when staging and production run on the same server

Expected URLs

Staging should use its own dedicated domains, for example:

• marketing: https://staging.aaperture.com
• app: https://app.staging.aaperture.com
• tenant: https://{slug}.staging.aaperture.com

Production keeps:

• marketing: https://aaperture.com
• app: https://app.aaperture.com
• tenant: https://{slug}.aaperture.com

The exact staging domains are up to infrastructure, but they must be reflected consistently in:

• DOMAIN
• FRONTEND_URL
• FRONTEND_BASE_URL
• API_BASE_URL
• PUBLIC_ROOT_DOMAIN
• OAuth redirect URIs
• DNS and certificates

Runtime isolation on the same server

If staging and production run on the same host, they must not share:

• SERVER_APP_DIR
• COMPOSE_PROJECT_NAME
• Postgres data directory
• published host ports

Recommended staging values:

• SERVER_APP_DIR=/var/www/aaperture-staging
• COMPOSE_PROJECT_NAME=aaperture-staging
• POSTGRES_HOST_PORT=15432
• API_HOST_PORT=18080
• WEB_HOST_PORT=18081
• PROMETHEUS_HOST_PORT=19090
• GRAFANA_HOST_PORT=13001

Recommended production values:

• SERVER_APP_DIR=/var/www/aaperture
• COMPOSE_PROJECT_NAME=aaperture
• POSTGRES_HOST_PORT=5432
• API_HOST_PORT=8080
• WEB_HOST_PORT=8081
• PROMETHEUS_HOST_PORT=9090
• GRAFANA_HOST_PORT=3001

Rollout expectation

Recommended workflow:

1. push to main
2. let deploy:staging finish automatically
3. validate staging manually
4. trigger deploy:prod

Current status:

• staging deployment with SOPS: validated
• production deployment with SOPS: validated

Notes

• NODE_ENV remains production in both staging and production deployments
• APP_ENV / ENVIRONMENT carry the runtime label (staging, production, etc.)
• backups, backup emails, and any runtime environment label must read APP_ENV / ENVIRONMENT, never NODE_ENV
• migration script receives ENVIRONMENT=staging or ENVIRONMENT=production
• staging and production use different resource_group values, so they do not block each other unnecessarily

Ongoing maintenance

After SOPS becomes the source of truth:

• keep infra/env/staging.env.enc and infra/env/production.env.enc committed
• do not keep *.plain.env or *.from-gitlab.env in the repository
• use GitLab scoped variables only for:
  • deployment access
  • bootstrap/migration fallback
  • SOPS_AGE_KEY_B64

Practical minimum GitLab set once the migration is complete:

• SOPS_AGE_KEY_B64
• SERVER_HOST
• SERVER_USER
• SERVER_APP_DIR
• SSH_PRIVATE_KEY_B64

Everything else should preferably come from:

• infra/env/staging.env.enc
• infra/env/production.env.enc

When adding a new runtime variable:

1. update .env.example
2. update scripts/check-env.sh if required
3. update and re-encrypt both environment files

Recommended workflow:

make sops-decrypt-staging
make sops-encrypt-staging

make sops-decrypt-production
make sops-encrypt-production

Immediate priorities

The deployment stack is now stable enough to move away from deployment setup work.

Current priority order:

1. keep staging as the default validation gate for every significant change
2. remove remaining runtime drift between staging, production, and local development
3. improve pipeline/build efficiency only after measuring the real bottlenecks
4. use the new platform baseline to validate public tenant, booking, portal, and email quality

Staging smoke test checklist

Keep this checklist as the standard regression pass for staging:

• marketing host: https://staging.aaperture.com
• app host: https://app.staging.aaperture.com
• tenant host: https://{slug}.staging.aaperture.com
• public booking flow
• client portal login flow
• public lead form flow
• Google OAuth callback on staging
• R2 uploads and public asset delivery
• Sentry event ingestion
• backup naming and backup notification email

Database/runtime checks:

• staging writes to /var/www/aaperture-staging
• COMPOSE_PROJECT_NAME=aaperture-staging
• POSTGRES_DB=aaperture_staging
• POSTGRES_HOST_PORT=15432
• migrations run against staging DB, not production

Production promotion checklist

Only trigger deploy:prod after:

• deploy:staging passed on the same commit
• staging smoke test is green
• encrypted env files were updated and committed if needed
• GitLab bootstrap variables still match production
• DNS / cert / Nginx remain valid for:
  • aaperture.com
  • app.aaperture.com
  • *.aaperture.com

This checklist has already been validated once with the SOPS-based setup. Keep using it as the release gate for future production promotions.

Improvement plan

Short-term improvement topics for the deployment platform:

• add a documented smoke test runbook for staging and production
• measure pipeline duration by stage and introduce further optimizations only after measurement
• consider Docker layer caching if build:web and build:api remain slow
• keep GitLab limited to bootstrap secrets only
• periodically rotate:
  • deploy SSH key
  • SOPS age key
  • high-impact production secrets copied from legacy config

Application-adjacent priorities unlocked by the new staging setup:

• full staging validation of tenant/app/public multi-host behavior
• booking perceived performance and slot loading improvements
• transactional email visual QA across real providers