> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/resources/lakehouse.md).

# Lakehouse

What a Lakehouse is, how it's built, and how to create and manage one.

Lakehouse is a DataOS Resource that combines Apache Iceberg with cloud object storage. It gives you warehouse-style management on open storage formats.

When you create a Lakehouse, DataOS provisions the storage integration, a REST Catalog backed by PostgreSQL, and the query and maintenance services (Spark Cluster and Sherpa) that run alongside it.

### Key features of Lakehouse

* **Decoupled storage and compute:** Scale storage and compute independently.
* **ACID support:** Keep writes and reads consistent during concurrent operations.
* **Open table and file formats:** Use Apache Iceberg for metadata and Apache Parquet for data files.
* **Unified workload support:** Run analytics, transformation, and maintenance from one storage layer.
* **Managed orchestration:** Bundle REST Catalog, Spark Cluster, and Sherpa as one resource.

Learn how the Lakehouse components work together in the [Lakehouse architecture](/references/resources/lakehouse/architecture.md).

### Supported storage system

Lakehouse supports Amazon S3, Azure ABFSS, and Google Cloud Storage. Each backend needs provider-specific Secrets and manifest settings.

See [Storage-specific configuration](/references/resources/lakehouse/supported-sources.md) to choose your backend and configure it.

### Supported data formats

* Lakehouse uses Apache Iceberg as the table format
* Lakehouse stores data in Apache Parquet files
* The Spark Cluster can also handle standalone Parquet use cases outside Lakehouse

### Create a Lakehouse

A Lakehouse Resource provisions the Iceberg REST Catalog, PostgreSQL-backed metastore, object storage integration, Spark Cluster, and Sherpa.

You need a Lakehouse domain, an active `lakehouse-mruntime` stack, object storage credentials, and a dedicated PostgreSQL database. Choose the storage backend first, then create the required Secrets and apply the Lakehouse manifest.

Follow [Create a Lakehouse](/references/resources/lakehouse/create-a-lakehouse.md) for prerequisites, manifests, deployment, and verification.

### Permissions to create a Lakehouse

Creating a Lakehouse requires the **Tenant Admin**, **Data Admin**, or **Data Developer** Tenant role. You'll also need a `Can Use` grant on the Compute, Secret, and Depot the Lakehouse depends on — ask their owner, or anyone holding `Can Manage Access` on them, if you don't already hold it.

Once created, you become the Lakehouse's owner and automatically get `Can Edit` and `Can Manage Access` on it.

See [Lakehouse: permissions](https://v2.dataos.info/operate/access-control/roles-and-permissions/resource-level-permissions/lakehouse-access-control) for the full access model, including the common access pattern and what happens after the Lakehouse exists.

### Create a Lakehouse Depot

A Depot of type `lakehouse` connects to an existing Lakehouse instance, exposing its REST Catalog and object storage through the DataOS catalog layer. Create the Depot after the Lakehouse is running.

Follow [Create a Lakehouse Depot](/references/resources/lakehouse/create-a-lakehouse-depot.md) for the prerequisites, configuration flow, and deployment steps.

### Lakehouse access

Lakehouse access is managed through purposes(for example):

* `scan` for scanner use cases
* `read` for read-only access
* `rw` for read-write access

Access enforcement is handled by DataOS's policy engine, which evaluates policies and use-case grants before allowing catalog, namespace, or table operations.

### Use a Lakehouse

Once the Lakehouse Depot is created successfully, you query it the same way you'd query any other Depot in DataOS, through a compute engine that reads from it, not through the Lakehouse resource directly.\
There are two ways to actually run queries against it, and they serve different purposes: Minerva is for ad hoc/interactive SQL, Vulcan is for building governed, reusable data Products on top of the same data.

#### Via Minerva

Minerva exposes the Depot as a Trino-compatible SQL catalog for interactive querying.

Prerequisites:

* The Depot exists and its source credentials are valid.
* A Minerva domain/Cluster is available in the instance.
* A compute resource is available for the Minerva workload.
* The querying user has access to the Lakehouse Depot (a `Can Use` grant)

#### Via Vulcan

Reference the Lakehouse Depot from a Vulcan project when the goal is a governed, reusable Data Product. Include it directly in a model's `spec.depots[]` to build Data Products on top of it.

Vulcan models run against data made available through that Depot connection. This is the right path for building semantic models for governed analytics, defining reusable metrics/dimensions for BI tools, or powering reporting that needs to stay consistent across teams.

when a Vulcan model targets a Lakehouse, the engine underneath is commonly Spark itself.

### Manage a Lakehouse in DataOS

After deployment, use the Lakehouse CLI commands to monitor the Lakehouse resource, inspect datasets, and perform operational tasks.

Use the [Lakehouse command reference](/references/resources/lakehouse/command-reference.md) to inspect status, manage operations, and work with datasets.

{% hint style="info" %}
Review the [Lakehouse manifest reference](/references/resources/lakehouse/configurations.md) before changing configuration, especially storage, metastore, or compute settings.
{% endhint %}

### Lakehouse data access

Data in a Lakehouse is organized hierarchically. At the top level, the Lakehouse contains namespaces, each of which groups related Iceberg tables and views. Use namespaces to separate domains, teams, or environments and apply access policies at the appropriate scope.

See [Manage Lakehouse data access](https://v2.dataos.info/operate/data-admin/access-control) to learn how to control access to namespaces, tables, and views.

### Troubleshooting

<table data-header-hidden><thead><tr><th width="138.690185546875"></th><th width="168.578369140625"></th><th width="139.6295166015625"></th><th></th></tr></thead><tbody><tr><td><strong>Category</strong></td><td><strong>Symptom</strong></td><td><strong>Likely Cause</strong></td><td><strong>Recommended Action</strong></td></tr><tr><td><strong>Provisioning</strong></td><td>Unable to create the Lakehouse resource.</td><td>Missing or incorrect Lakehouse domain.</td><td>Run <code>dataos-ctl domain get -t lakehouse -a</code> to verify the domain. If it's missing, contact your <strong>Operator</strong> or <strong>Tenant Admin</strong> for the correct domain before reapplying.</td></tr><tr><td><strong>Metastore</strong></td><td>Catalog initialization fails or metadata is not persisted.</td><td>Invalid metastore credentials or PostgreSQL is unreachable.</td><td>Verify the metastore Secret (<code>dataos-ctl resource get -t secret</code>) and ensure the PostgreSQL host, port, and database are reachable. Checking logs will help here. Use <code>-d</code> to check the metastore runtime status before assuming it's a credential issue.</td></tr><tr><td><strong>Storage Access</strong></td><td>Read/write operations fail (for example, <strong>403</strong> or <strong>Permission Denied</strong>).</td><td>Storage credentials are invalid or lack bucket/container permissions.</td><td>Verify that the storage Secret has the required IAM permissions for the target bucket/container. Use <code>-b</code> to inspect pod-level errors during write operations.</td></tr><tr><td><strong>Storage Configuration</strong></td><td>Storage access fails even though credentials are valid.</td><td>Incorrect region, endpoint, or storage scheme.</td><td>Verify the configured region, endpoint, and storage scheme (<code>storage.s3.region</code>, <code>storage.s3.endpoint</code>, <code>storage.gcs.endpoint</code>, <code>abfss.endpointSuffix</code>, etc.) against your storage account settings.</td></tr><tr><td><strong>Network</strong></td><td>Depot queries fail while the Lakehouse appears healthy.</td><td>Connectivity or DNS issues between the Depot and the Lakehouse gateway/rest-catalog.</td><td>Verify that the <code>metastoreUrl</code> resolves correctly from the Depot environment and that network connectivity to the gateway is available.</td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/references/resources/lakehouse.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.