> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/operate/v1/dataos-operator/provision-data-plane/aws.md).

# AWS

Deploy the DataOS Data Plane on an existing Amazon EKS cluster within your AWS account. Your SRE team is responsible for provisioning and operating the underlying AWS infrastructure in accordance with Modern’s published requirements. DataOS components are then installed on the cluster using a Data Plane manifest applied through `dataos-ctl`.

The Data Plane establishes an outbound connection to the DataOS Instance hosted in Modern Cloud.

## Prerequisites

<details>

<summary>1. Instance FQDN</summary>

Use a working DataOS Instance FQDN. For example: `pacific-staging.instance.dataos.cloud`

</details>

<details>

<summary>2. DataOS CLI</summary>

Install `dataos-ctl` on the bastion host, from the **Setup CLI** section on the Instance home page. Use the DataOS CLI version named in the release inputs below.

If the binary is not on the bastion host's `PATH`, call it by its full install path, for example `/home/jumpvmadmin/.dataos/v2/bin/dataos-ctl`. This guide uses the short form.

</details>

<details>

<summary>3. DataOS permissions</summary>

Your own DataOS user needs all three of these roles on the Instance, so that you can create the application user and apply the Data Plane manifest:

* `roles:id:user`
* `roles:id:operator`
* `roles:id:super-user`

Check them in **Bifrost** at `https://<instance-fqdn>/platform/bifrost/users`.

</details>

<details>

<summary>4. Database endpoints and credentials</summary>

The Data Plane connects to an external PostgreSQL database provisioned during infrastructure setup. Have these values ready; you supply them directly in the Data Plane manifest.

* `PG_HOST`, `PG_PORT`: PostgreSQL host and port.
* `SYSTEM_HEALTHD_DB_NAME`, `SYSTEM_HEALTHD_DB_USERNAME`, `SYSTEM_HEALTHD_DB_PASSWORD`: Database name and credentials for System HealthD.
* `VESTA_DB_NAME`, `VESTA_DB_USERNAME`, `VESTA_DB_PASSWORD`: Database name and credentials for Vesta.

</details>

<details>

<summary>5. AWS credentials and kubeconfig access</summary>

**Credentials**

* **AWS\_ACCESS\_KEY\_ID:** Access key ID for the AWS identity used during Data Plane setup.
* **AWS\_SECRET\_ACCESS\_KEY:** Secret access key associated with the AWS access key ID.

**Kubeconfig access**

You need a valid kubeconfig for the provisioned EKS cluster, and `kubectl` must reach the cluster with it. You base64-encode this kubeconfig during deployment.

</details>

<details>

<summary>6. Release inputs (from Modern)</summary>

These values are supplied by Modern for the release you are installing; Confirm all of them before you apply.

| Input                | Manifest field       | Example                      |
| -------------------- | -------------------- | ---------------------------- |
| Install artifact     | `artifactIdentifier` | `<install-artifact>`         |
| DataOS Manager image | `imageTag`           | `<dataos-manager-image-tag>` |
| DataOS CLI version   | :                    | `<cli-version>`              |

The examples are placeholders: take the exact values from the release notes for the release you are installing. Install the named CLI version on the bastion host, because an older CLI may not support the manifest fields the release expects.

In production, use the snapshot form of `artifactIdentifier` (`artifact:aft_abc123abc123`), which pins one exact snapshot. The release form (`release:dataos-v2-install-manifests:prd_abc123abc123`) always installs the latest artifact behind that release, so a re-apply can pick up a newer build.

</details>

## Deploy Data Plane

Work through the steps in order, and run every command from the bastion host. Each step prepares one input the final `dataos-ctl domain apply` needs: a deployment application user and API key, a base64 kubeconfig for the EKS cluster, the variables the manifest resolves, and the Data Plane manifest itself.

### 1. Authenticate from the bastion host

Provisioning is always initiated from the **bastion host** (for example, `jumpvmadmin`). In production the EKS API server is restricted to authorized CIDR ranges, so both `kubectl` and `dataos-ctl` must run from inside your network. Confirm the access path with your infrastructure team.

The bastion host keeps no logged-in session, so you never run `dataos-ctl init` or `dataos-ctl login`. Every command carries these four values instead; replace the placeholders with your own each time:

| Variable           | Purpose                           |
| ------------------ | --------------------------------- |
| `USER_ID`          | User the command authenticates as |
| `DATAOS_FQDN`      | Target DataOS Instance domain     |
| `APIKEY`           | API key belonging to `USER_ID`    |
| `DATAOS_TENANT_ID` | Always `system`                   |

Which user they name changes as you go: your **personal user** creates the application user's API key in Step 2, and the **application user** runs every `domain` command after that. So start by collecting your own two values.

{% hint style="warning" %}
Never provision a Data Plane with your personal API key. It ties the Data Plane to an individual account.
{% endhint %}

**1.1 Get your user ID.** It is shown on your profile in the DataOS UI, and by `dataos-ctl user get` on a machine that has a logged-in CLI session:

```bash
dataos-ctl user get
```

```
INFO[0000] 😃 user get...
INFO[0001] 😃 user get...complete

    NAME    │        ID        │  TYPE  │       EMAIL       │             TAGS
────────────┼──────────────────┼────────┼───────────────────┼──────────────────────────────
 iamgroot   │ iamgroottmdcio   │ person │ iamgroot@tmdc.io  │ roles:id:user,
            │                  │        │                   │ roles:id:operator,
            │                  │        │                   │ users:id:iamgroottmdcio
```

The value under **ID** is your `USER_ID`.

**1.2 Create your personal API key.** Issue it from **Profile → Tokens** in the DataOS UI, or from a machine with a logged-in CLI session:

```bash
dataos-ctl user apikey create --id <your-user-id> --name <key-name> -d 24h
```

This key authenticates the one command in Step 2 that creates the application user's key. It is not used to provision anything.

### 2. Create the deployment application user and API key

Provision a dedicated **application user** for this Data Plane and issue it a short-lived **API key**.

**2.1 Provision the application user.** In **Bifrost** on your DataOS Instance (`https://<instance-fqdn>/platform/bifrost`), create an application user named `<dataplane-name>-user` and assign these roles:

* `roles:id:user`
* `roles:id:operator`
* `roles:id:super-user`

**2.2 Create the API key.** This is the one command you run as your **personal user**: `USER_ID` and `APIKEY` are your own. Issue a short-lived key (for example, a 24-hour lifetime) for the application user.

```bash
USER_ID=<your-user-id> \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<your-personal-apikey-token> \
  DATAOS_TENANT_ID=system \
  dataos-ctl user apikey create -i <dataplane-name>-user -d 24h
```

The command prints the new API key in the terminal. Copy it: you supply it as `APIKEY` in the lint, apply, and monitor commands that follow.

### 3. Base64-encode kubeconfig

Confirm `kubectl` reaches the cluster, then base64-encode the kubeconfig into `KUBECONFIG_BASE64`:

```bash
kubectl get nodes   # confirm access first
```

The encoding command differs by operating system, because `base64` takes different flags on each and Windows has no `base64` at all. Run the one that matches the machine you are deploying from.

{% tabs %}
{% tab title="Linux" %}

```bash
export KUBECONFIG_BASE64=$(kubectl config view --minify --flatten | base64 -w 0)
```

{% endtab %}

{% tab title="macOS" %}

```bash
export KUBECONFIG_BASE64=$(kubectl config view --minify --flatten | base64 | tr -d '\n')
```

{% endtab %}

{% tab title="Windows" %}

```powershell
$env:KUBECONFIG_BASE64 = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes((kubectl config view --minify --flatten | Out-String)))
```

{% endtab %}
{% endtabs %}

This value gives the DataOS Instance access to the EKS API. Treat it as a secret.

### 4. Set the variables the manifest resolves

`dataos-ctl domain apply` resolves `${...}` references in the manifest from the shell you run it in. Set these values there before you apply:

```bash
export KUBECONFIG_BASE64=<base64-encoded-kubeconfig>   # from the previous step
export IMAGE_TAG=<dataos-manager-image-tag>            # provided by Modern
export ARTIFACT_IDENTIFIER=<install-artifact>          # provided by Modern
export REGION=<aws-region>                             # region of the EKS cluster
export AWS_ACCESS_KEY_ID=<base64-encoded-key-id>
export AWS_SECRET_ACCESS_KEY=<base64-encoded-secret>
```

`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` must be **base64-encoded** before you set them. Encode the raw keys first:

```bash
export AWS_ACCESS_KEY_ID=$(echo -n "<raw-access-key-id>" | base64)
export AWS_SECRET_ACCESS_KEY=$(echo -n "<raw-secret-access-key>" | base64)
```

| Variable                                     | Purpose                                                   | Example          |
| -------------------------------------------- | --------------------------------------------------------- | ---------------- |
| `DATAPLANE_ID`                               | Stable Data Plane identifier                              | `acme-aws-dp-01` |
| `INSTANCE_TENANT_ID`                         | DataOS Instance this Data Plane attaches to               | `pacific-ocean`  |
| `KUBECONFIG_BASE64`                          | Base64-encoded kubeconfig from the previous step          | :                |
| `IMAGE_TAG`                                  | DataOS Manager image tag to pin (from Modern)             | :                |
| `ARTIFACT_IDENTIFIER`                        | Install artifact to apply (from Modern)                   | :                |
| `REGION`                                     | AWS region where the EKS cluster runs                     | `ap-south-1`     |
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | Base64-encoded AWS credentials for ongoing AWS operations | :                |

`DATAPLANE_ID` and `INSTANCE_TENANT_ID` are supplied inline on the lint and apply commands in Step 6, so you do not need to set them here.

### 5. Prepare the Data Plane manifest

Copy the sample manifest below. You only need to fill in a handful of values; everything else is fixed and wires the Data Plane to the DataOS Instance over the tunnel.

**What you fill in**

Identity and naming:

* `name` / `DATAPLANE_ID`: the dataplane identifier. Use a name that encodes the Instance and cloud, for example `<instance-name>-aws-dp-01`.
* `description`: a short, Instance- and Tenant-specific description for traceability.
* `workpiece.name`: must end with the `.dataplane` suffix. This becomes the Data Plane domain.

Cluster and cloud access:

* `region`: resolved from `${REGION}` (the AWS region where your EKS cluster runs).
* `kubeConfigBase64`: the base64 kubeconfig from Step 3.
* `directInputs` and `kubeAuthConfigs`: your base64-encoded AWS credentials from Step 4.

Version and install:

* `dataOsManagerConfigs.imageTag`: resolved from `${IMAGE_TAG}` (the `dataos-manager` version from Modern).
* `installFromPrime.artifactIdentifier`: resolved from `${ARTIFACT_IDENTIFIER}` (the install artifact from Modern).

PostgreSQL databases:

* `pgHost`, `pgPort`: the PostgreSQL endpoint. `pgPort` is a quoted string, for example `"5432"`.
* `systemHealthDb*` and `vestaDb*` values: database names and credentials for System HealthD and Vesta.

Leave every other field exactly as shown.

```yaml
name: ${DATAPLANE_ID}
version: v1alpha
entity: domain
type: dataplane
description: the ${DATAPLANE_ID} dataplane for ${INSTANCE_TENANT_ID} instance
v1alpha:
  dataplane:
    name: ${DATAPLANE_ID}
    tenantDomainNames:
      - system-entities
    networkType: tunnel
    workpiece:
      name: ${DATAPLANE_ID}-${INSTANCE_TENANT_ID}.dataplane
      zone: dataos.cloud
      kernelRequest:
        template: dataplane-kernel-system-existing-compute-tunnel-v1
        cloud: aws
        inputs:
          region: ${REGION}
          dataOsManagerConfigs:
            imageTag: ${IMAGE_TAG}
            logLevel: info
            replicas: 1
            nodeSelector:
              dataos.io/purpose: core-kernel
          kubeConfigBase64: ${KUBECONFIG_BASE64}
        configs:
          - cloudflare:tmdc-prime
        directInputs:
          aws_access_key_id: ${AWS_ACCESS_KEY_ID}
          aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
      kubeAuthConfigs:
        - key: AWS_ACCESS_KEY_ID
          value: ${AWS_ACCESS_KEY_ID}
          base64: false
          envVar: true
        - key: AWS_SECRET_ACCESS_KEY
          value: ${AWS_SECRET_ACCESS_KEY}
          base64: false
          envVar: true
    dataPlaneKernelSystemInstall:
      installFromPrime:
        artifactIdentifier: ${ARTIFACT_IDENTIFIER}
        installFileRootDir: /install/dataplane-kernel-system
        installFile: release-prd-install.yaml
        applicationsFile: install.applications.yaml
        valuesFile: install.values.yaml
        onDemandCloudKernel: true
      envPurpose: prd
      pgHost: <PG_HOST>
      pgPort: "5432"
      systemHealthDbName: <SYSTEM_HEALTHD_DB_NAME>
      systemHealthDbUsername: <SYSTEM_HEALTHD_DB_USERNAME>
      systemHealthDbPassword: <SYSTEM_HEALTHD_DB_PASSWORD>
      vestaDbName: <VESTA_DB_NAME>
      vestaDbUsername: <VESTA_DB_USERNAME>
      vestaDbPassword: <VESTA_DB_PASSWORD>
      azureEndpointSuffix: core.windows.net
      awsEndpointSuffix: amazonaws.com
      coreKernelNodeSelector:
        dataos.io/purpose: core-kernel
      features:
        clusterAutoScaler:
          enable: false
        cloudnativePg:
          enable: false
```

{% hint style="warning" %}
The kubeconfig, AWS credentials, and database credentials inside the manifest are sensitive. Store the manifest encrypted and limit who can decrypt it.
{% endhint %}

### 6. Lint and apply the Data Plane manifest

Authenticate as the **application user** with the API key from Step 2, then **lint** the manifest before applying. `KUBECONFIG_BASE64`, `IMAGE_TAG`, and `REGION` are resolved from the variables you set in Step 4.

**Lint** (validate the manifest with `-l`):

```bash
USER_ID=<dataplane-name>-user \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<application-user-apikey-token> \
  DATAOS_TENANT_ID=system \
  INSTANCE_TENANT_ID=<instance-name> \
  DATAPLANE_ID=<dataplane-name> \
  dataos-ctl domain apply -f aws-dataplane.yaml -l
```

Lint validates the manifest and resolves the variables without provisioning anything. If it succeeds but nothing is created, you left the `-l` flag on.

**Apply** (once lint passes, rerun the same command without `-l`):

```bash
USER_ID=<dataplane-name>-user \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<application-user-apikey-token> \
  DATAOS_TENANT_ID=system \
  INSTANCE_TENANT_ID=<instance-name> \
  DATAPLANE_ID=<dataplane-name> \
  dataos-ctl domain apply -f aws-dataplane.yaml
```

The DataOS Instance validates the request and starts installing the Data Plane onto your cluster.

<details>

<summary>What happens after you apply</summary>

When you run the apply, the DataOS Instance reads the `kubeConfigBase64` you supplied and uses it to reach the EKS API for the first time. It installs `dataos-manager` onto the cluster, which pulls the install manifests defined in `installFromPrime` and lays down the DataOS components.

A tunnel client inside the cluster opens an outbound, encrypted connection from a worker node out through the NAT Gateway to Modern Cloud. No inbound ports on your cluster are exposed. Once the tunnel is established, the DataOS Instance registers the Data Plane and manages it over that connection.

The kubeconfig is used only for this DataOS Instance-to-cluster management path. It is not part of your data traffic. Every hop is TLS-encrypted end to end.

</details>

### 7. Monitor provisioning

Provisioning may take some time to complete. Check the Data Plane status until it reports active.

```bash
USER_ID=<dataplane-name>-user \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<application-user-apikey-token> \
  DATAOS_TENANT_ID=system \
  dataos-ctl domain get -t dataplane -n <dataplane-name> -v v1alpha -d
```

To watch the domain list refresh instead of rerunning the command, use `domain ls -r`:

```bash
USER_ID=<dataplane-name>-user \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<application-user-apikey-token> \
  DATAOS_TENANT_ID=system \
  dataos-ctl domain ls -r
```

{% hint style="info" %}
Spinning up the infrastructure and installing the platform typically completes within 30 minutes: the Instance reaches the cluster, installs `dataos-manager`, and waits for the tunnel to come up. If the status has not changed well past that, follow the `dataos-manager` logs as shown under [Check status](#check-status).
{% endhint %}

## Manage Data Plane

### Check status

```bash
USER_ID=<dataplane-name>-user \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<application-user-apikey-token> \
  DATAOS_TENANT_ID=system \
  dataos-ctl domain get
#add -a to include Data Planes created by other users
```

{% hint style="info" %}
If the command above shows <mark style="color:$success;">`workpiece-forge-kernel: success`</mark>, the infrastructure setup is complete.
{% endhint %}

Then access the Data Plane cluster with `kubectl` and inspect the `dataos-manager` logs.

```bash
kubectl logs -n <instance_name>-0-dsm dataos-manager-0 -f
```

### Describe applied configuration

```bash
USER_ID=<dataplane-name>-user \
  DATAOS_FQDN=<instance-fqdn> \
  APIKEY=<application-user-apikey-token> \
  DATAOS_TENANT_ID=system \
  dataos-ctl domain get -t dataplane -n <instance-name>-aws-dp-01 -v v1alpha -d
```

## Dataplane configuration

A field-by-field reference for the Data Plane manifest.

#### Metadata

Top-level fields that identify the Data Plane resource.

| Attribute     | Description                                                       |
| ------------- | ----------------------------------------------------------------- |
| `name`        | Keep it stable and readable: it becomes the dataplane identifier. |
| `version`     | API version for the resource spec.                                |
| `entity`      | Always `domain`.                                                  |
| `type`        | Always `dataplane`: declares what you are provisioning.           |
| `description` | Make it environment- and Tenant-specific for traceability.        |

```yaml
name: ${DATAPLANE_ID}
version: v1alpha
entity: domain
type: dataplane
description: the ${DATAPLANE_ID} dataplane for ${INSTANCE_TENANT_ID} instance
```

{% hint style="info" %}
Use a naming convention that clearly encodes the Instance and cloud. Example: `<instance-name>-<cloud>-dp-01`.
{% endhint %}

#### Dataplane

Core Data Plane identity and DataOS Instance connectivity.

| Attribute           | Description                                                                                               |
| ------------------- | --------------------------------------------------------------------------------------------------------- |
| `dataplane.name`    | The dataplane identifier.                                                                                 |
| `tenantDomainNames` | Default domain mappings (platform-specific).                                                              |
| `networkType`       | Decides how the Data Plane reaches the DataOS Instance. `tunnel` uses outbound-only Cloudflare tunneling. |

```yaml
v1alpha:
  dataplane:
    name: ${DATAPLANE_ID}
    tenantDomainNames:
      - system-entities
    networkType: tunnel
```

#### Workpiece

Represents the execution unit for the provisioning pipeline.

| Attribute        | Description                                                                                                                        |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `workpiece.name` | Must end with the `.dataplane` suffix. This produces the Data Plane domain, for example `awsdataplane0123.dataplane.dataos.cloud`. |
| `workpiece.zone` | Always set to `dataos.cloud`.                                                                                                      |

```yaml
workpiece:
  name: ${DATAPLANE_ID}-${INSTANCE_TENANT_ID}.dataplane
  zone: dataos.cloud
```

#### Kernel Request

Selects the kernel provisioning blueprint and target cloud.

| Attribute  | Description                                                                                                                                                                  |
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `template` | Defines which kernel provisioning blueprint is used. `dataplane-kernel-system-existing-compute-tunnel-v1` installs the platform onto an existing EKS cluster in tunnel mode. |
| `cloud`    | Selects cloud-specific behavior. Use `aws`.                                                                                                                                  |

```yaml
kernelRequest:
  template: dataplane-kernel-system-existing-compute-tunnel-v1
  cloud: aws
```

#### Inputs (region, DataOS Manager, kubeconfig)

| Attribute                           | Description                                                                             |
| ----------------------------------- | --------------------------------------------------------------------------------------- |
| `region`                            | AWS region of the EKS cluster, resolved from `${REGION}`, for example `ap-south-1`.     |
| `dataOsManagerConfigs.imageTag`     | `dataos-manager` image version, resolved from `${IMAGE_TAG}`. Provided by Modern.       |
| `dataOsManagerConfigs.logLevel`     | Prefer `info` for production, `debug` temporarily during incidents.                     |
| `dataOsManagerConfigs.replicas`     | Keep as `1`; a single replica is sufficient.                                            |
| `dataOsManagerConfigs.nodeSelector` | Schedules `dataos-manager` onto the node pool labeled `dataos.io/purpose: core-kernel`. |
| `kubeConfigBase64`                  | Base64-encoded kubeconfig that gives the DataOS Instance access to the EKS cluster.     |

```yaml
inputs:
  region: ${REGION}
  dataOsManagerConfigs:
    imageTag: ${IMAGE_TAG}
    logLevel: info
    replicas: 1
    nodeSelector:
      dataos.io/purpose: core-kernel
  kubeConfigBase64: ${KUBECONFIG_BASE64}
```

#### Config Bundle

| Attribute | Description                                                                                                                                                                      |
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `configs` | Config references applied to the kernel. `cloudflare:tmdc-prime` supplies the Cloudflare tunnel configuration for the Data Plane domain. This value is fixed: leave it as shown. |

```yaml
configs:
  - cloudflare:tmdc-prime
```

#### Auth (AWS credentials)

AWS credentials used by the platform for ongoing AWS API operations. Supply the credentials in two places: `directInputs` passes them to the kernel provisioning inputs, and `kubeAuthConfigs` injects them into the cluster as environment variables. Both reference the same base64-encoded values.

| Attribute | Description                                                                                            |
| --------- | ------------------------------------------------------------------------------------------------------ |
| `key`     | Environment variable name.                                                                             |
| `value`   | Base64-encoded credential value.                                                                       |
| `base64`  | Whether the supplied value is already base64-encoded. Keep `false` for this guide. See the note below. |
| `envVar`  | Set `true`: inject the value into the cluster as an environment variable.                              |

```yaml
kernelRequest:
  directInputs:
    aws_access_key_id: ${AWS_ACCESS_KEY_ID}
    aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
kubeAuthConfigs:
  - key: AWS_ACCESS_KEY_ID
    value: ${AWS_ACCESS_KEY_ID}
    base64: false
    envVar: true
  - key: AWS_SECRET_ACCESS_KEY
    value: ${AWS_SECRET_ACCESS_KEY}
    base64: false
    envVar: true
```

{% hint style="info" %}
**Choosing `base64: true` or `false`**

The `base64` flag tells the platform whether the value you supply still needs encoding before it is stored as a Kubernetes Secret in the cluster.

This guide encodes the keys in Step 4, so keep `base64: false`. Setting `true` on an already-encoded value double-encodes it, and the credentials reach the cluster unusable.
{% endhint %}

| Set     | Supply the credential as            | The platform then                        |
| ------- | ----------------------------------- | ---------------------------------------- |
| `true`  | the **raw** key (plain text)        | base64-encodes it for you before storing |
| `false` | a value that is **already encoded** | stores it as-is, without re-encoding     |

#### Installation (Prime-based)

Defines which installation manifests are applied for Data Plane components, pulled from Prime rather than a Git repository.

| Attribute                                       | Description                                                                                                                                                                                                                             |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `artifactIdentifier`                            | The install artifact to apply, resolved from `${ARTIFACT_IDENTIFIER}`. Provided by Modern for the target release. Prefer a snapshot identifier (`artifact:aft_...`) over a release identifier, so a re-apply cannot pull a newer build. |
| `installFileRootDir`                            | Root directory within the artifact for install files (`/install/dataplane-kernel-system`).                                                                                                                                              |
| `installFile`, `applicationsFile`, `valuesFile` | Install manifests and Helm-style values within the artifact.                                                                                                                                                                            |
| `onDemandCloudKernel`                           | When infrastructure provisioning generates a new output, `dataos-manager` creates a new Cloud Kernel JSON. **This value must remain `true`.**                                                                                           |
| `envPurpose`                                    | Environment purpose for the install (`prd`).                                                                                                                                                                                            |

```yaml
installFromPrime:
  artifactIdentifier: ${ARTIFACT_IDENTIFIER}
  installFileRootDir: /install/dataplane-kernel-system
  installFile: release-prd-install.yaml
  applicationsFile: install.applications.yaml
  valuesFile: install.values.yaml
  onDemandCloudKernel: true
envPurpose: prd
```

#### PostgreSQL databases

Two databases provisioned on a PostgreSQL server: one for System HealthD and one for Vesta. Supply the connection details and credentials provisioned during infrastructure setup.

| Attribute                                                                | Description                                       |
| ------------------------------------------------------------------------ | ------------------------------------------------- |
| `pgHost`, `pgPort`                                                       | PostgreSQL host and port.                         |
| `systemHealthDbName`, `systemHealthDbUsername`, `systemHealthDbPassword` | Database name and credentials for System HealthD. |
| `vestaDbName`, `vestaDbUsername`, `vestaDbPassword`                      | Database name and credentials for Vesta.          |

```yaml
pgHost: <PG_HOST>
pgPort: "5432"
systemHealthDbName: <SYSTEM_HEALTHD_DB_NAME>
systemHealthDbUsername: <SYSTEM_HEALTHD_DB_USERNAME>
systemHealthDbPassword: <SYSTEM_HEALTHD_DB_PASSWORD>
vestaDbName: <VESTA_DB_NAME>
vestaDbUsername: <VESTA_DB_USERNAME>
vestaDbPassword: <VESTA_DB_PASSWORD>
```

#### Cloud Endpoints

Used for cloud integrations and endpoint resolution.

```yaml
azureEndpointSuffix: core.windows.net
awsEndpointSuffix: amazonaws.com
```

#### Node Selectors

Ensures platform components schedule onto the intended node pool.

| Attribute                | Description                                   |
| ------------------------ | --------------------------------------------- |
| `coreKernelNodeSelector` | Label selector for the core kernel node pool. |

```yaml
coreKernelNodeSelector:
  dataos.io/purpose: core-kernel
```

{% hint style="info" %}
**Do not change this:** the DataOS installer applies the same node label to node pools.
{% endhint %}

#### Features

Optional platform feature toggles.

| Attribute                  | Description                                                                          |
| -------------------------- | ------------------------------------------------------------------------------------ |
| `clusterAutoScaler.enable` | Enables the cluster autoscaler. Keep `false` unless your deployment requires it.     |
| `cloudnativePg.enable`     | Enables the CloudNativePG operator. Keep `false` unless your deployment requires it. |

```yaml
features:
  clusterAutoScaler:
    enable: false
  cloudnativePg:
    enable: false
```

## Troubleshooting

<details>

<summary><code>user apikey create</code> fails</summary>

The personal user's credentials or the target Instance are wrong. Verify `USER_ID`, your personal API key, and `DATAOS_FQDN`.

</details>

<details>

<summary><code>domain apply</code> returns forbidden</summary>

The application user is missing a role, or you are authenticating as yourself. Confirm the three roles on the application user, and that `USER_ID` and `APIKEY` belong to it.

</details>

<details>

<summary>Lint succeeds but nothing is provisioned</summary>

The `-l` flag is still on the command. Rerun the apply without it.

</details>

<details>

<summary>Manifest decryption fails</summary>

The identity decrypting the manifest lacks access to the key. Check its AWS credentials and KMS key permissions.

</details>

<details>

<summary>Forge task fails</summary>

The target region or availability zones are wrong. Validate `REGION` and that the availability zones in use exist in that region.

</details>

<details>

<summary>Apply fails with an auth error</summary>

Invalid or expired kubeconfig, or insufficient RBAC. Regenerate the kubeconfig with the required RBAC, re-encode it, and re-apply. If the failure is on the CLI authentication itself, confirm the application user's API key has not expired.

</details>

<details>

<summary>Pods stuck <code>Pending</code></summary>

The `dataos.io/purpose: core-kernel` label is missing, or the node pool lacks capacity. Label the nodes and scale the node pool.

</details>

<details>

<summary>Image pull failures</summary>

Egress to the registry is blocked, or image pull secrets are missing for a private mirror. Allow TCP 443 to the registry and configure image pull secrets.

</details>

<details>

<summary>Tunnel does not establish</summary>

Outbound egress to the Cloudflare tunnel edge is blocked, or DNS resolution is failing. The tunnel connects to `region1.v2.argotunnel.com` and `region2.v2.argotunnel.com` on port `7844`: UDP (QUIC) primary, TCP `7844` as fallback. If both are blocked, the tunnel cannot connect. If your firewall uses TLS inspection or SNI-based filtering, allow the SNI hostname pattern `*.cftunnel.com`. Confirm NAT Gateway egress and DNS resolution from a worker node; the tunnel client retries automatically once egress is restored.

</details>

<details>

<summary>Reconciliation stalls (cluster unreachable)</summary>

The EKS API is unreachable, the IAM access entry is missing, or `aws-auth` is mis-bound. Verify the kubeconfig's IAM identity is mapped to a Kubernetes RBAC scope on the cluster.

</details>

<details>

<summary>Install step fails</summary>

The Prime `artifactIdentifier` is wrong or unavailable for this release. Confirm the artifact identifier with Modern and that the Data Plane can reach Prime.

</details>

## Best practices

* Dedicate the `core-kernel` node pool to platform components. Do not co-schedule user Data Product workloads on it.
* Monitor AWS quotas and EKS node-group capacity ahead of production Data Product workload growth.
* Capture provisioning logs from `dataos-ctl` and AWS CloudWatch during the apply for assertion and troubleshooting.
* Store the manifest encrypted in version control; decrypt it only at apply time.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/operate/v1/dataos-operator/provision-data-plane/aws.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.