GitOps on AWS EKS with App Mesh

The problem

Java Spring Boot microservices are great to write and interesting to run. But the platform I built around them — spanning 15+ customer systems — had three concrete failures before the work started. Releases were hand-triggered shell scripts run by whoever was on call, so nobody could answer "what is actually running in the cluster right now?". Services talked to each other over plain HTTP with no retries, no mTLS, and no way to see traffic between them. And scaling was a guess: replicas were bumped by hand during incidents and forgotten after.

The resolution was three separate but reinforcing layers: GitOps delivery (ArgoCD as the source of truth), a service mesh (AWS App Mesh for secure service-to-service traffic), and autoscaling on business metrics instead of CPU.

GitOps delivery

ArgoCD became the source of truth for everything running in the cluster:

• Repositories hold the desired state — manifests, Helm charts, and kustomization layers.
• ArgoCD continuously reconciles the cluster toward the repository. Drift is an anomaly, not a norm.
• GitLab CI/CD builds and pushes images to Amazon ECR, then updates the Git state. Deploys are just merges.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: checkout
  namespace: argocd
spec:
  source:
    repoURL: https://gitlab.com/team/platform.git
    path: apps/checkout
    targetRevision: main
  destination:
    server: https://kubernetes.default.svc
    namespace: checkout
  syncPolicy:
    automated: {}

GitLab CI was the other half of the loop — it builds and pushes the image, then updates the desired state in Git. Deploys are just merges:

build-and-push:
  stage: deploy
  image: docker:24
  services: [docker:24-dind]
  script:
    - docker build -t $CI_REGISTRY_IMAGE/checkout:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE/checkout:$CI_COMMIT_SHA
    - kustomize edit set image checkout=$CI_REGISTRY_IMAGE/checkout:$CI_COMMIT_SHA
    - git add kustomization.yaml && git commit -m "release checkout $CI_COMMIT_SHA"
    - git push origin main
  only:
    - main

Service mesh

The Java Spring Boot services communicated over gRPC, and AWS App Mesh handled service-to-service traffic with mTLS, retries, and observability without touching application code. Each service declared its listeners and backends as a VirtualNode — a piece of config, not an application change:

apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualNode
metadata:
  name: checkout
  namespace: checkout
spec:
  listeners:
    - portMapping: { port: 8080, protocol: grpc }
  serviceDiscovery:
    dns: { hostname: checkout.checkout.svc.cluster.local }
  backends:
    - virtualService:
        virtualServiceName: orders.checkout.svc.cluster.local

That decoupling meant the mesh could evolve independently of the services it connected.

Autoscaling

Horizontal Pod Autoscaling ran on custom metrics — not CPU, but the business signals that actually mattered (queue depth, request latency, error rate):

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout
  minReplicas: 3
  maxReplicas: 24
  metrics:
    - type: External
      external:
        metric:
          name: sqs_approximate_number_of_messages
        target:
          type: AverageValue
          averageValue: "50"

The platform scaled with demand and costed proportionally — replicas went up when the queue backed up and came back down when it drained.

Cost governance

The same work touched multi-cloud cost governance across Azure and AWS: resource tagging standards, budget alerts, and cost allocation reporting. If you cannot attribute spend, you cannot control it.

Lessons learned

• GitOps converts "deploys" into "code review." The same discipline you use for application code now applies to the platform.
• A service mesh is a contract, not a feature. Introduce it early when the service count is small; retrofitting is far harder.
• Scale on the metric that hurts. CPU autoscaling keeps you alive; business-metric autoscaling keeps you profitable.

The best platform outcome is invisible: developers merge code, traffic routes safely, and the bill stays predictable.