> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/v1/resources/lakehouse.md).

# Lakehouse

Lakehouse is a DataOS Resource that combines Apache Iceberg with cloud object storage. It gives you warehouse-style management on open storage formats.

When you create a Lakehouse, DataOS provisions the storage integration, a REST Catalog backed by PostgreSQL, and the query and maintenance services (Spark Cluster and Sherpa) that run alongside it.

### Key features of Lakehouse

* **Decoupled storage and compute:** Scale storage and compute independently.
* **ACID support:** Keep writes and reads consistent during concurrent operations.
* **Open table and file formats:** Use Apache Iceberg for metadata and Apache Parquet for data files.
* **Unified workload support:** Run analytics, transformation, and maintenance from one storage layer.
* **Managed orchestration:** Bundle REST Catalog, Spark Cluster, and Sherpa as one resource.

Learn how the Lakehouse components work together in the [Lakehouse architecture](/references/v1/resources/lakehouse/architecture.md).

### Supported storage system

Lakehouse supports Amazon S3, Azure ABFSS, and Google Cloud Storage. Each backend needs provider-specific Secrets and manifest settings.

See [supported storage system configurations](/references/v1/resources/lakehouse/supported-sources.md) to choose your backend and configure it.

### Supported data formats

* Lakehouse uses Apache Iceberg as the table format
* Lakehouse stores data in Apache Parquet files
* The Spark Cluster can also handle standalone Parquet use cases outside Lakehouse

### Create a Lakehouse Depot

A Depot of type `lakehouse` connects to an existing Lakehouse instance, exposing its REST Catalog and object storage through the DataOS catalog layer. Create the Depot after the Lakehouse is running.

Follow [Create a Lakehouse Depot](/references/v1/resources/lakehouse/create-a-lakehouse-depot.md) for the prerequisites, configuration flow, and deployment steps.

### Lakehouse access

Lakehouse access is managed through purposes(for example):

* `scan` for scanner use cases
* `read` for read-only access
* `rw` for read-write access

Access enforcement is handled by DataOS's policy engine, which evaluates policies and use-case grants before allowing catalog, namespace, or table operations.

### Use a Lakehouse

Once the Lakehouse Depot is created successfully, there are two ways to actually run queries against it, and they serve different purposes: Minerva is for ad hoc/interactive SQL, Vulcan is for building governed, reusable data Products on top of the same data.

#### Via Minerva

Minerva exposes the Depot as a Trino-compatible SQL catalog for interactive querying.

Prerequisites:

* The Depot exists and its source credentials are valid.
* A Minerva domain/Cluster is available in the instance.
* A compute resource is available for the Minerva workload.
* The querying user has access to the Lakehouse Depot (a `Can Use` grant)

#### Via Vulcan

Reference the Lakehouse Depot from a Vulcan project when the goal is a governed, reusable Data Product. Include it directly in a model's `spec.depots[]` to build Data Products on top of it.

Vulcan models run against data made available through that Depot connection. This is the right path for building semantic models for governed analytics, defining reusable metrics/dimensions for BI tools, or powering reporting that needs to stay consistent across teams.

when a Vulcan model targets a Lakehouse, the engine underneath is commonly Spark itself

### Manage a Lakehouse in DataOS

* Manage operations and datasets with the [Lakehouse command reference](/references/v1/resources/lakehouse/command-reference.md)
* Review manifest fields in [Lakehouse manifest reference](/references/v1/resources/lakehouse/configurations.md)

### Troubleshooting

<table data-header-hidden><thead><tr><th width="138.690185546875"></th><th width="168.578369140625"></th><th width="139.6295166015625"></th><th></th></tr></thead><tbody><tr><td><strong>Category</strong></td><td><strong>Symptom</strong></td><td><strong>Likely Cause</strong></td><td><strong>Recommended Action</strong></td></tr><tr><td><strong>Provisioning</strong></td><td>Unable to create the Lakehouse resource.</td><td>Missing or incorrect Lakehouse domain.</td><td>Run <code>dataos-ctl domain get -t lakehouse -a</code> to verify the domain. If it's missing, contact your <strong>Operator</strong> or <strong>Tenant Admin</strong> for the correct domain before reapplying.</td></tr><tr><td><strong>Metastore</strong></td><td>Catalog initialization fails or metadata is not persisted.</td><td>Invalid metastore credentials or PostgreSQL is unreachable.</td><td>Verify the metastore Secret (<code>dataos-ctl resource get -t secret</code>) and ensure the PostgreSQL host, port, and database are reachable. Checking logs will help here. Use <code>-d</code> to check the metastore runtime status before assuming it's a credential issue.</td></tr><tr><td><strong>Storage Access</strong></td><td>Read/write operations fail (for example, <strong>403</strong> or <strong>Permission Denied</strong>).</td><td>Storage credentials are invalid or lack bucket/container permissions.</td><td>Verify that the storage Secret has the required IAM permissions for the target bucket/container. Use <code>-b</code> to inspect pod-level errors during write operations.</td></tr><tr><td><strong>Storage Configuration</strong></td><td>Storage access fails even though credentials are valid.</td><td>Incorrect region, endpoint, or storage scheme.</td><td>Verify the configured region, endpoint, and storage scheme (<code>storage.s3.region</code>, <code>abfss.endpointSuffix</code>, etc.) against your storage account settings.</td></tr><tr><td><strong>Network</strong></td><td>Depot queries fail while the Lakehouse appears healthy.</td><td>Connectivity or DNS issues between the Depot and the Lakehouse gateway/rest-catalog.</td><td>Verify that the <code>metastoreUrl</code> resolves correctly from the Depot environment and that network connectivity to the gateway is available.</td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/references/v1/resources/lakehouse.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.