EKS

Guide for understanding and configuring an EKS stack

The EKS stack creates and manages an Amazon Elastic Kubernetes Service (EKS) cluster in your AWS account. Webscale deploys Helm charts to the EKS cluster through the Control Plane. Your own tools, such as kubectl or Helm, reach it from the networks you allow. See Create an EKS stack .

An EKS stack consists of:

  • An EKS cluster with a private API endpoint inside its VPC and a public one that only the Webscale Control Plane reaches by default
  • A fixed-size node group of Arm nodes
  • The vpc-cni, kube-proxy, coredns, and metrics-server add-ons
  • External Secrets Operator with a ClusterSecretStore for AWS Secrets Manager (enabled by default)
  • Karpenter node autoscaling (enabled by default)
  • Supporting infrastructure (subnets and their route tables, a NAT gateway, IAM roles, and the security group rules, plus a VPC in the new network mode)

Network topology

The stack can be deployed in two modes:

  • Existing network: Uses the subnets you list in node_subnets and api_subnets (the stack creates no network). The subnets must already have a route to the internet, or the nodes cannot pull images. Their VPC must have DNS support and DNS hostnames enabled, or the stack update fails. The stack opens the NodePort range to the primary CIDR block of that VPC. Private subnets in the VPC of your dedicated data plane need nothing more, because the proxies already reach the nodes over that VPC’s own routing. Connect the two networks yourself when the subnets are in another VPC, and add the network of the proxies to service_ingress_cidrs.
  • New network: Creates a VPC from network_cidr with a public and a private subnet in each of two availability zones. The EKS cluster takes all four subnets for its API. The nodes take the private subnets. They reach the internet through one NAT gateway, whose address the nat_ip output reports for allowlisting. The proxies have no route to those nodes. Connect the two networks yourself and add the network of the proxies to service_ingress_cidrs, or every deployment fails its reachability test. Set public_nodes to put the nodes in the public subnets instead. Each node then gets its own public address, the stack creates no NAT gateway, and nat_ip is null. service_ingress_cidrs needs 0.0.0.0/0, because the proxies reach the nodes from their public addresses. Ports 30000-32767 are then reachable from any address.

The mode cannot change for the life of the EKS cluster, which cannot move between VPCs. A different mode needs a new stack.

Data plane

A data plane is a set of Webscale proxies that receive the traffic for your applications, and a dedicated data plane serves your account alone. Its proxies send traffic directly to the EKS cluster’s nodes on the node port of the release’s Service. Because the port can change with every deployment, the stack opens the whole NodePort range (30000-32767) to the network of the proxies. It takes that network from the primary CIDR block of the VPC the nodes are in. service_ingress_cidrs opens the range to any other network you name, which is how proxies outside that VPC reach the nodes.

Variables

Variable nameTypeDescription
regionstring(Required) (Permanent) The AWS region to create the EKS cluster in.
permissions_boundary_arnstringOptional ARN of a customer-managed permissions boundary policy.
provisioner_role_namestringOptional custom role name to assume in customer account. Defaults to AccountProvisioner.
kubernetes_versionstringKubernetes version of the EKS cluster, its nodes, and its add-ons. Defaults to "1.36". A change upgrades all three. A version past standard support keeps running under EKS extended support, which AWS bills at a higher price. See Amazon EKS pricing .
network_cidrstring(Permanent) Network block of the created VPC. Defaults to 10.0.0.0/16. The stack splits it into a public and a private subnet in each of two availability zones. Provide your own block when the default overlaps a network the EKS cluster must reach. Ignored when node_subnets is set. The EKS cluster cannot move between VPCs, and its NAT gateway cannot move between subnets.
node_subnetslist(string)(Permanent) Existing subnet IDs to adopt for the nodes. They must span two or more availability zones and share one VPC with api_subnets. When set, the stack creates no network, and api_subnets must be set with it. See Network topology .
api_subnetslist(string)(Permanent) Existing subnet IDs to adopt for the EKS cluster’s API. They must span two or more availability zones and share one VPC with node_subnets. The EKS cluster places network interfaces in these subnets to reach the nodes, and its private endpoint answers on those interface addresses. Setting one of the two without the other fails the update. See Network topology .
public_nodesboolWhether the nodes launch in the created public subnets instead of the private ones. Defaults to false. Ignored when node_subnets is set, where the adopted subnets decide the addressing. When true, each node carries its own public address and egresses from it, the stack creates no NAT gateway, and nat_ip is null. service_ingress_cidrs needs 0.0.0.0/0, because the proxies reach the nodes from those addresses. Do not change the value on a live EKS cluster. The change forces a node group migration to the other subnets and removes or adds the NAT gateway. The old nodes can lose their internet access during the migration.
base_instance_typestringArm instance type of the base nodes, such as t4g.large (default) or m7g.large. A change builds a new node group before the stack removes the old one.
base_node_countnumberNode count of the base node group. Defaults to 2. The group never scales. Karpenter adds temporary nodes on top when enabled.
base_node_labelsmap(string)Kubernetes labels of the base nodes. Empty by default. The base nodes already carry webscale.com/nodegroup, set to the stack name. Temporary nodes never carry these labels. A key under kubernetes.io or k8s.io, such as node-role.kubernetes.io/infra, fails the stack update. A change updates the nodes in place. See Node autoscaling .
disk_sizenumberRoot disk size in GiB of every node. Defaults to 80. A change builds a new node group before the stack removes the old one.
kubernetes_adminslist(string)Extra IAM principal ARNs the stack admits to the EKS cluster through access entries. The AccountProvisioner role, the node role, and the AWS provider’s user already have entries. Listing one of them fails the apply. Entries grant permission only. A principal outside the EKS cluster’s VPC also needs its network in api_ingress_cidrs. Empty by default. See Cluster access .
kubernetes_admins_policy_arnstringEKS cluster access policy ARN granted to every principal in kubernetes_admins. Defaults to AmazonEKSClusterAdminPolicy (full cluster admin); pass one of the other AWS-managed EKS cluster access policies (AmazonEKSAdminPolicy, AmazonEKSEditPolicy, AmazonEKSViewPolicy, or AmazonEKSAdminViewPolicy) to scope them down instead. No other policy ARNs are accepted.
api_ingress_cidrslist(string)Networks your API clients reach the public endpoint from, such as your office network or your CI provider’s ranges. Defaults to an empty list, so only the Webscale Control Plane reaches it. The public endpoint accepts 40 networks, a limit this list shares with the Control Plane. A change updates the live EKS cluster in place. See Cluster access .
service_ingress_cidrslist(string)Networks allowed to reach the NodePort range 30000-32767 on the nodes. Defaults to an empty list. The stack already opens the range to the primary CIDR block of the VPC the node subnets are in, and listing that block again fails the stack update. Anything on a listed network reaches every Service the EKS cluster exposes on a node port. See Data plane .
enable_external_secretsboolWhether to install External Secrets Operator and its aws-secretsmanager store. Defaults to true. When false, a chart that declares an ExternalSecret fails to install. Setting it to false on a live EKS cluster deletes every ExternalSecret and the Secrets it created. See Secrets .
enable_karpenterboolWhether to install Karpenter node autoscaling. Defaults to true. When false, the EKS cluster stays at base_node_count nodes. See Node autoscaling .
karpenter_cpu_limitnumberTotal vCPUs of the temporary nodes Karpenter may add. Defaults to 512. Karpenter stops adding nodes at the cap.
karpenter_node_lifetimestringLifetime of each temporary node, a duration built from s, m, and h. Defaults to 720h. Karpenter replaces a node at this age, and Never disables the rotation. The base node group never expires.
permissions_boundary_arnstringOptional ARN of a customer-managed permissions boundary policy. See AWS Setup . Not necessary for Webscale managed AWS accounts.
provisioner_role_namestringOptional role name to assume for Stack provisioning in the customer account. Defaults to AccountProvisioner. See AWS Setup . Not necessary for Webscale managed AWS accounts.

Example:

{
  "type": "eks",
  "environment": "/v2/environments/3ps666yxcanq",
  "service": "/v2/services/u91dkrui39cj",
  "name": "my-eks-cluster",
  "variables": {
    "region": "us-east-1",
    "node_subnets": ["subnet-0123456789abcdef0", "subnet-0fedcba9876543210"],
    "api_subnets": ["subnet-0aaaa1111bbbb2222", "subnet-0cccc3333dddd4444"],
    "kubernetes_admins": ["arn:aws:iam::123456789012:role/PlatformAdmin"],
    "api_ingress_cidrs": ["203.0.113.0/24"]
  }
}

Outputs

Output nameDescription
kubernetes_cluster_nameThe name of the EKS cluster. Enter it in Cluster Name when you enable Use for Kubernetes on the AWS provider.
nat_ipPublic address of the NAT gateway, the fixed address the nodes use to reach the internet. Null when node_subnets is set or public_nodes is true.

Cluster access

The stack grants cluster admin through an EKS access entry to the AWS provider’s IAM user. Use kubernetes_admins for your own roles and users. Their entries carry the policy in kubernetes_admins_policy_arn (cluster admin by default). The provider’s user and the stack’s own IAM roles already have entries, and listing one of them fails the stack update. A provider whose access key belongs to the account root fails the stack update, because EKS cannot grant root an access entry. The stack reads the provider’s access key only when it runs, so a key moved to another IAM user fails every deployment until the next stack update.

An access entry grants permissions, not network access. Only the Webscale Control Plane reaches the public API endpoint until you list your own networks in api_ingress_cidrs. A client inside the EKS cluster’s VPC reaches the private endpoint instead. api_ingress_cidrs does not apply to it, and the cluster security group decides its access.

To deploy through Webscale, enable Use for Kubernetes on the AWS provider and enter the kubernetes_cluster_name output in Cluster Name. Then create a Kubernetes cluster with a Helm blueprint. See Working with Providers for the provider settings.

Secrets

When enable_external_secrets is true, the stack installs External Secrets Operator in the external-secrets namespace and a ClusterSecretStore named aws-secretsmanager for AWS Secrets Manager. The operator reads each value directly from Secrets Manager, so no value passes through Webscale. It can read only secrets whose names start with webscale/. Your chart declares an ExternalSecret that names the secret, and the operator writes a Kubernetes Secret in the namespace of the deployment.

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: example
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: aws-secretsmanager
  target:
    name: secret-to-be-created
  dataFrom:
    - extract:
        key: webscale/acme-api/prod

Reference the Secret in the container spec of your Deployment with envFrom:

envFrom:
  - secretRef:
      name: secret-to-be-created

Containers read environment variables at start. A rotated value reaches the application on the next deployment.

Node autoscaling

When enable_karpenter is true, Karpenter adds temporary nodes when pods do not fit on the existing nodes and removes them as demand falls. It launches on-demand Arm instances from the c, m, and r families, generation 3 or newer, size large or larger. They share the disk size, node role, and security group of the base nodes. karpenter_cpu_limit caps the total, and karpenter_node_lifetime sets when Karpenter replaces a node.

Karpenter evicts pods when it removes a node and honors a PodDisruptionBudget for up to 20 minutes, then deletes the remaining pods. Add a budget that selects each Deployment’s pods.

The base nodes carry the label webscale.com/nodegroup, set to the stack name. Use it in a nodeSelector to keep a Deployment’s pods on the base nodes, such as a monitoring tool’s cluster agent. Temporary nodes carry karpenter.sh/nodepool instead, set to default. Do not select on eks.amazonaws.com/nodegroup, which carries the node group’s name. A change of base_instance_type, disk_size, or public_nodes builds a new node group with a new name, and that selector would then match nothing.

Deleting the stack

Delete the Kubernetes clusters that deploy to the EKS cluster before you delete the stack, and wait for Karpenter to remove its temporary nodes. Karpenter launches and terminates those instances itself, so the stack cannot delete them. If temporary nodes are still running, the delete waits for Karpenter to drain them, which can take up to 20 minutes. The stack releases the NAT address when you delete it, and a replacement stack gets a different one. The stack removes only the network pieces it created.

Have questions not answered here? Contact Support to get more help.

Last modified on September 17, 2026