AWS EKS

Production deployment on Amazon EKS.

For a full map of all secrets, service accounts, and namespaces, see Credentials Reference.

Prerequisites#

AWS CLI configured
kubectl 1.24+
Helm 3.0+
eksctl (optional, for cluster creation)
EKS cluster 1.24+

Required Components#

1. VPC and Networking#

Requirements:

VPC with public and private subnets
NAT gateway for private subnet internet access
Security groups allowing pod-to-pod communication

See: AWS VPC Best Practices

2. IAM Roles and Permissions#

EKS Pod Identity (recommended):

Crossplane AWS provider role (crossplane-provider-aws-role):

{
  "Effect": "Allow",
  "Action": [
    "sqs:CreateQueue",
    "sqs:DeleteQueue",
    "sqs:GetQueueAttributes",
    "sqs:SetQueueAttributes",
    "sqs:TagQueue",
    "sqs:GetQueueUrl"
  ],
  "Resource": "arn:aws:sqs:*:*:asya-*"
}

Actor role (asya-actor-role) - shared IAM role for all actor sidecars. Provides access to SQS queues and S3 bucket for persisting messages. This role is assigned via IRSA (or EKS Pod Identity) to a shared asya-actors ServiceAccount in each actor namespace — no static AWS credentials are stored in the cluster:

Note: For local development with LocalStack, IRSA is unavailable. Use a static aws-creds Secret with AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY instead. See Quickstart for the dev setup.

{
  "Effect": "Allow",
  "Action": [
    "sqs:ReceiveMessage",
    "sqs:SendMessage",
    "sqs:DeleteMessage",
    "sqs:ChangeMessageVisibility",
    "sqs:GetQueueAttributes"
  ],
  "Resource": "arn:aws:sqs:*:*:asya-*"
},
{
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:PutObject",
    "s3:DeleteObject",
    "s3:ListBucket"
  ],
  "Resource": [
    "arn:aws:s3:::asya-results-bucket",
    "arn:aws:s3:::asya-results-bucket/*"
  ]
}

KEDA role (keda-operator-role):

{
  "Effect": "Allow",
  "Action": [
    "sqs:GetQueueAttributes",
    "sqs:GetQueueUrl",
    "sqs:ListQueues"
  ],
  "Resource": "arn:aws:sqs:*:*:asya-*"
}

3. EKS Addons#

# Install Pod Identity Agent
eksctl create addon --cluster my-cluster \
  --name eks-pod-identity-agent

# Install VPC CNI
eksctl create addon --cluster my-cluster \
  --name vpc-cni --version v1.16.2

4. KEDA Operator#

# Create namespace
kubectl create namespace keda

# Add Helm repo
helm repo add kedacore https://kedacore.github.io/charts
helm repo update

# Install KEDA
helm install keda kedacore/keda \
  --namespace keda \
  --version 2.15.1

Configure Pod Identity for KEDA:

aws eks create-pod-identity-association \
  --cluster-name my-cluster \
  --namespace keda \
  --service-account keda-operator \
  --role-arn arn:aws:iam::ACCOUNT:role/keda-operator-role

5. S3 Bucket for Results#

Create a sample S3 bucket for persisting result messages:

aws s3 mb s3://asya-results-bucket --region us-east-1

Optional Components#

GPU Node Group#

For AI/ML workloads:

eksctl create nodegroup \
  --cluster my-cluster \
  --name gpu-nodes \
  --node-type g4dn.xlarge \
  --nodes-min 0 \
  --nodes-max 10 \
  --node-ami-family AmazonLinux2 \
  --node-taints nvidia.com/gpu=true:NoSchedule

Install NVIDIA Device Plugin:

kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.17.0/deployments/static/nvidia-device-plugin.yml

Cluster Autoscaler#

For automatic node provisioning:

helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm install cluster-autoscaler autoscaler/cluster-autoscaler \
  --namespace kube-system \
  --set autoDiscovery.clusterName=my-cluster \
  --set awsRegion=us-east-1

Metrics Server#

For resource metrics:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

CloudWatch Container Insights#

For centralized logging:

eksctl create iamserviceaccount \
  --cluster my-cluster \
  --namespace amazon-cloudwatch \
  --name cloudwatch-agent \
  --attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
  --approve

# Install CloudWatch agent
kubectl apply -f https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluentd-quickstart.yaml

Asya🎭 Deployment#

1. Install Crossplane#

helm repo add crossplane-stable https://charts.crossplane.io/stable
helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system --create-namespace

2. Configure Crossplane Values#

# crossplane-values.yaml
providers:
  aws:
    enabled: true   # opt-in: disabled by default

irsa:
  enabled: true     # opt-in: disabled by default
  roleArnPattern: "arn:aws:iam::ACCOUNT_ID:role/asya-actors-{namespace}"

awsRegion: us-east-1
awsProviderConfig:
  name: default
  credentialsSource: Secret
  secretRef:
    namespace: crossplane-system
    name: aws-creds
    key: credentials

3. Install Asya Crossplane Chart#

helm repo add asya https://asya.sh/charts
helm repo update asya

helm install asya-crossplane asya/asya-crossplane --version $ASYA_VERSION \
  -n crossplane-system \
  -f crossplane-values.yaml

4. Install Gateway (Optional)#

# gateway-values.yaml
config:
  sqsRegion: us-east-1
  s3Bucket: asya-results-bucket
  postgresHost: postgres.default.svc.cluster.local

serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/asya-gateway-role

routes:
  tools:
  - name: example
    description: Example tool
    parameters:
      text:
        type: string
        required: true
    route: [example-actor]

helm install asya-gateway asya/asya-gateway --version $ASYA_VERSION \
  -n default \
  -f gateway-values.yaml

6. Install Crew Actors#

Crew actors are pre-defined system actors for handling common scenarios. For example, actors x-sink and x-sump are the common flow finalizers and can persist messages to S3-compatible storage.

Suppose, we want to save all messages to the bucket s3://asya-results-bucket. Note that the bucket name should be globally unique.

# crew-values.yaml
x-sink:
  enabled: true
  env:
    ASYA_PERSISTENCE_MOUNT: /state/checkpoints

x-sump:
  enabled: true
  env:
    ASYA_PERSISTENCE_MOUNT: /state/checkpoints

helm install asya-crew asya/asya-crew --version $ASYA_VERSION \
  -n default \
  -f crew-values.yaml

Note: IRSA annotation can be set per-actor in AsyncActor spec if needed.

7. Deploy Your Actors#

apiVersion: asya.sh/v1alpha1
kind: AsyncActor
metadata:
  name: my-actor
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/asya-actor-role
spec:
  scaling:
    minReplicaCount: 0
    maxReplicaCount: 50
  image: my-actor:v1
  handler: handler.process

kubectl apply -f my-actor.yaml

Verification#

# Check Crossplane
kubectl get pods -n crossplane-system

# Check KEDA
kubectl get pods -n keda

# Check actor
kubectl get asyncactor my-actor
kubectl get pods -l asya.sh/actor=my-actor

# Check queue created
aws sqs list-queues | grep asya-my-actor
kubectl get sqsqueue

Cost Optimization#

Use Spot Instances for GPU nodes
Enable cluster autoscaler scale-to-zero
Use KEDA scale-to-zero (minReplicaCount: 0)
Set appropriate queueLength for scaling efficiency
Monitor SQS costs (first 1M requests free)

See: AWS EKS Best Practices

Asya Docs

AWS EKS#