> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/engine-guide/redshift.md).

# Redshift

The full technical reference for building and running a Data Product on Amazon Redshift, covering connection, grants, materialization, operations, performance, and troubleshooting.

Amazon Redshift is a cloud data warehouse with a Postgres-family SQL dialect, running on RA3 provisioned clusters or Redshift Serverless. Vulcan runs against Redshift to manage transformations with version control and gated deployments. This page is the full manual: it covers how you connect, what permissions you need, how each model kind materializes, how to set operational guardrails, and how to recover from the failures you are most likely to hit.

## When to use

Choose Redshift when your data already lands in a Redshift cluster or Serverless workgroup and you want governed transformations to run close to it on AWS. Redshift suits teams standardized on the AWS data stack, workloads that benefit from Spectrum/S3 federation, and projects that need SQL `GRANT`-based access control rather than a separate role hierarchy.

{% hint style="info" %}
Redshift's adapter is Postgres-family but not identical. Set `dialect: redshift`, not `postgres`. Identifier casing, `MERGE`/replace-table behavior, and view comments all differ from vanilla PostgreSQL.
{% endhint %}

### Supported model kinds

`FULL`, `VIEW`, `INCREMENTAL_BY_TIME_RANGE`, `INCREMENTAL_BY_UNIQUE_KEY`, and `INCREMENTAL_BY_PARTITION` — validate at volume before relying on them for large backfills. Python models run on the Redshift adapter where supported; validate against your specific Vulcan image before using them in production.

### Engine characteristics

| Property                   | Value                                                                                     |
| -------------------------- | ----------------------------------------------------------------------------------------- |
| Engine adapter type        | `redshift`                                                                                |
| Model dialect              | `redshift`                                                                                |
| Support level              | GA (SQL cluster path; validate Python models per image)                                   |
| Execution vehicle          | Redshift **cluster** or **Serverless** (SQL on port **5439** default)                     |
| Database                   | Cluster database, for example `dev` or `analytics`                                        |
| Schemas                    | Vulcan creates schemas and tables, such as `bronze`, `silver`, and `gold`                 |
| Identifier casing rule     | Match semantic identifiers to physical column casing (often **lowercase snake\_case**)    |
| SQL grants                 | Adapter **`SUPPORTS_GRANTS`**. Use SQL `GRANT` for consumer roles where appropriate       |
| Recommended authentication | DB user + password (SSL required) locally; **Redshift depot** + instance secret in DataOS |

Once you choose Redshift as your Data Product engine, use this guide to explore **Authorize, Connect, Build, Serve, Operate, and Troubleshoot**. It covers access, connectivity, model development, data serving, performance, and issue resolution.

Depending on where you are starting from, you may enter this path at a different point.

## Authorize

Access setup is required before gateway configuration.

Provision Redshift and DataOS access before configuring a gateway or deploying a Data Product. See [Permissions and Grants](https://v2.dataos.info/references/engine-guide/redshift/permissions-and-grants.md) for role design, grant templates, required DataOS permissions, and the pre-connect checklist.

## Connect

This is the first hands-on step after access provisioning.

For connection options, gateway configuration, local development, and production deployment, see [Connect to Redshift](https://v2.dataos.info/references/engine-guide/redshift/connect-to-redshift.md).

## Build models

Choose a model strategy after you validate the connection.

For model-kind mappings, engine-native capabilities, lifecycle SQL, semantic-model and metric guidance, see [Materialization Behavior](https://v2.dataos.info/references/engine-guide/redshift/materialization-behavior.md).

## Serve

### Endpoints

Use endpoints to make deployed Data Product data available to consumers.

Every deployed Data Product on Redshift gets REST, GraphQL, and MySQL-wire endpoints, all pushing down to the same Redshift gateway. Large result sets are buffered before streaming; size `api.resources.limits.memory` for expected payloads, with 3 GiB a practical starting point for high-row responses.

### MCP tools

| MCP tool   | Redshift behavior                                  |
| ---------- | -------------------------------------------------- |
| `about`    | Static; no engine call                             |
| `lineage`  | Parsed graph; no live Redshift query               |
| `quality`  | Last check/audit results; typically no live call   |
| `data`     | Live Redshift query, subject to WLM queueing       |
| `run`      | Triggers workflow; same cost as a scheduled run    |
| `activity` | Reads workflow history from DataOS; no engine call |

Treat agent-driven `data` tool queries like production dashboard queries: they compete for the same WLM queue slots as everything else on the cluster.

### Metadata scanning and catalog

DataOS scans Redshift for discovery and governance. The scan catalogs available assets and maps their lineage.

| Object             | Catalog      | Lineage        |
| ------------------ | ------------ | -------------- |
| Databases, schemas | ✓            | Not applicable |
| Tables, views      | ✓            | ✓ as nodes     |
| Columns            | ✓ with types | ✓ column-level |

DataOS scans `information_schema` (and the depot's `scan` purpose) for structural metadata. Use a read-only scanner role separate from the Vulcan service role:

```sql
GRANT USAGE ON SCHEMA bronze, silver, gold TO scanner_role;
GRANT SELECT ON ALL TABLES IN SCHEMA bronze, silver, gold TO scanner_role;
```

Scanner cadence is instance-configurable, often every 6 to 12 hours. If lineage is missing after `vulcan apply`, verify `inputs.yaml`, then wait for the next scan.

## Operate safely

Set workload limits before scaling consumers or production schedules.

For compute sizing, concurrency, WLM, scheduling, cost guardrails, and performance guidance, see [Limits and Performance](https://v2.dataos.info/references/engine-guide/redshift/limits-and-performance.md).

### Before production

* Confirm least-privilege grants.
* Validate the gateway.
* Set WLM concurrency and API replica limits.
* Test recovery procedures.

## Troubleshooting

Use these checks when configuration, deployment, or runtime behavior needs investigation.

For failure modes, health checks, and recovery procedures, see [Troubleshooting](https://v2.dataos.info/references/engine-guide/redshift/troubleshooting.md).

## Start faster with a recipe

Skip tuning from a blank slate. [Deployment Recipes](https://v2.dataos.info/references/engine-guide/redshift/recipes/deployment-recipes.md) has a daily-incremental Kimball shape, a split build-vs-query WLM pattern, first-time backfill guidance, and local full-stack commands.

## Full implementation example

Use this complete Redshift Data Product as a production-shaped starting point.

{% file src="/files/kdGlYhLAfTohtNFuA5ah" %}

## Related

* [Overview](https://v2.dataos.info/references/engine-guide/engine-guide.md) for the cross-engine rules.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/references/engine-guide/redshift.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
