> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/engine-guide/athena.md).

# Athena

The full technical reference for building and running a Data Product on Amazon Athena, covering connection, IAM permissions, materialization, operations, performance, and troubleshooting.

Amazon Athena is a serverless, Presto/Trino-family query engine over data registered in the AWS Glue catalog. Vulcan runs against Athena to manage transformations with version control and gated deployments. This page is the full manual: it covers how you connect, what IAM permissions you need, how each model kind materializes, how to set operational guardrails, and how to recover from the failures you are most likely to hit.

## When to use

Choose Athena when your data already lands in S3 as Parquet or Iceberg and is registered in Glue, and you want serverless, pay-per-scan SQL rather than a cluster to manage. Athena suits teams standardized on the AWS data stack who want to avoid running warehouse infrastructure, at the cost of per-query scan-based billing instead of cluster hours.

{% hint style="info" %}
Athena has no SQL `GRANT` statements. Access is enforced through AWS IAM (and optionally Lake Formation) on Glue databases/tables and the underlying S3 prefixes, not through database-side grants.
{% endhint %}

### Supported Model Kinds

`FULL`, `VIEW`, `INCREMENTAL_BY_TIME_RANGE`, `INCREMENTAL_BY_UNIQUE_KEY`, and `INCREMENTAL_BY_PARTITION`. Validate incrementals at your data volume. Python models run on the Athena adapter's pandas path where supported; validate against your specific Vulcan image before using them in production.

### Engine Characteristics

| Property               | Value                                                                                    |
| ---------------------- | ---------------------------------------------------------------------------------------- |
| Engine adapter type    | `athena`                                                                                 |
| Model dialect          | `athena`                                                                                 |
| Support level          | GA (SQL + Glue catalog path; validate Python models per image)                           |
| Execution vehicle      | Athena **workgroup**; queries via PyAthena / the AWS API                                 |
| Catalog                | Glue: default `awsdatacatalog`; set `catalog_name` in the connection                     |
| Database (schema)      | Glue database: `schema_name` / `ATHENA_SCHEMA_NAME`                                      |
| Warehouse storage      | **`s3_warehouse_location`** (required for managed tables)                                |
| Query results          | **`s3_staging_dir`** (Athena query output location)                                      |
| Identifier casing rule | Match semantic model identifiers to physical column casing (often lowercase snake\_case) |
| Auth (local / CI)      | IAM access key + secret, or a role chain per your org standard                           |
| Auth (DataOS)          | Instance secret → env projection on the Vulcan resource                                  |

Once you choose Athena as your Data Product engine, use this guide to explore **Authorize, Connect, Build, Serve, Operate, and Troubleshoot**: covering access, connectivity, model development, data serving, performance, and issue resolution.

Depending on where you are starting from, you may enter this path at a different point.

## Authorize

Access setup is required before gateway configuration.

Provision AWS IAM and DataOS access before configuring a gateway or deploying a Data Product. See [Permissions and grants](https://v2.dataos.info/references/engine-guide/athena/permissions-and-grants.md) for the IAM role breakdown, minimum capabilities, required DataOS permissions, and the pre-connect checklist.

## Connect

This is the first hands-on step after access provisioning.

For connection options, gateway configuration, local development, and production deployment, see [Connect to Athena](https://v2.dataos.info/references/engine-guide/athena/connect-to-athena.md).

## Build Models

Choose a model strategy after you validate the connection.

For model design, Athena-native behavior, support boundaries, and required AWS resources, see [Materialization behavior](https://v2.dataos.info/references/engine-guide/athena/materialization-behavior.md).

## Serve

### Endpoints

Use endpoints to make deployed Data Product data available to consumers.

Every deployed Data Product on Athena gets REST, GraphQL, and MySQL-wire endpoints, all pushing down to the same Athena gateway. Keep `api.resources.limits.memory` at 3 GiB for large result sets.

### MCP Tools

| MCP tool   | Athena behavior                                    |
| ---------- | -------------------------------------------------- |
| `about`    | Static; no engine call                             |
| `lineage`  | Parsed graph; no live Athena query                 |
| `quality`  | Last check/audit results; typically no live call   |
| `data`     | Live Athena query; consumes scanned-byte capacity  |
| `run`      | Triggers workflow; same cost as a scheduled run    |
| `activity` | Reads workflow history from DataOS; no engine call |

Treat agent-driven `data` tool queries like production dashboard queries: each one scans bytes and counts against your workgroup's cost and concurrency limits.

### Metadata Scanning and Catalog

DataOS scans AWS Glue (and optionally Athena query history, depending on scanner configuration) for discovery and governance.

| Object                | Catalog            | Lineage        |
| --------------------- | ------------------ | -------------- |
| Glue databases        | ✓                  | N/A            |
| Tables, views         | ✓                  | ✓ as nodes     |
| Columns               | ✓                  | ✓ column-level |
| S3 external locations | ✓ via table params | ✓ when parsed  |

Use a read-only IAM principal distinct from the Vulcan service role:

| Area | Actions (illustrative)                                                     |
| ---- | -------------------------------------------------------------------------- |
| Glue | `glue:GetDatabase`, `glue:GetDatabases`, `glue:GetTable`, `glue:GetTables` |
| S3   | `s3:GetObject`, `s3:ListBucket` on published prefixes only                 |

Scanner cadence is instance-configurable, often every 6 to 12 hours. Glue changes appear after the next scan.

## Operate Safely

Set workload limits before scaling consumers or production schedules.

For compute sizing, concurrency, scheduling, cost guardrails, and performance guidance, see [Limits and performance](https://v2.dataos.info/references/engine-guide/athena/limits-and-performance.md).

### Before Production

* Confirm least-privilege IAM for the service, scanner, and consumer principals.
* Validate the gateway.
* Set workgroup concurrency and API replica limits.
* Test recovery procedures.

## Troubleshooting

Use these checks when configuration, deployment, or runtime behavior needs investigation.

For failure modes, health checks, and recovery procedures, see [Troubleshooting](https://v2.dataos.info/references/engine-guide/athena/troubleshooting.md).

## Start Faster with a Recipe

Skip tuning from a blank slate. [Deployment recipes](https://v2.dataos.info/references/engine-guide/athena/recipes/deployment-recipes.md) has a daily-incremental Kimball shape, a split build-vs-query workgroup pattern, first-time backfill guidance, and the local full-stack commands.

## Full Implementation Example

Use this complete Athena Data Product as a production-shaped starting point.

{% file src="/files/t781ywyHL0lHnZICtbwj" %}

## Related

* [Overview](https://v2.dataos.info/references/engine-guide/engine-guide.md) for the cross-engine rules.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/references/engine-guide/athena.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
