> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/concepts/resources/depot.md).

# Depot

## Overview <a href="#depot" id="depot"></a>

Depot in DataOS is a Resource used to connect different data sources to DataOS by abstracting the complexities associated with the underlying source system (including protocols, credentials, and connection schemas). It enables users to establish connections and retrieve data from various data sources, such as file systems (e.g., AWS S3, Google GCS, Azure Blob Storage), data lake systems, database systems (e.g., Redshift, SnowflakeDB, BigQuery, Postgres), and event systems (e.g., Kafka), without moving the data.

{% hint style="success" icon="circle-info" %}
**Depot in the Data Product lifecycle**

A Depot is the DataOS Resource that connects to a data source and exposes it for use, without copying the data. It hides differences between storage and database systems behind one access pattern so a Data Product can read from any registered source the same way.

Key roles of a Depot in the lifecycle include:

* **Data integration and access:** A Depot connects to file systems (AWS S3, Google GCS), databases (PostgreSQL, Snowflake), and event streams (Kafka) through Uniform Data Links (UDLs). Data Products read through the UDL without duplicating data.
* **Standardization and abstraction:** By registering data locations within DataOS, a Depot standardizes access mechanisms, eliminating inconsistencies in connection protocols, schemas, and credentials across different systems. This simplifies Data Product development and ensures interoperability.
* **Data transformation and querying:** Depots support ingestion, transformation, and query acceleration through DataOS Stacks (Flare, Nilus, and others), so data lands ready for analytical and operational use.
* **Foundation for Data Products:** Depots are essential for creating Data Products, as they enable teams to query and transform source data.

By using Depots, organizations can build scalable, secure, and efficient data ecosystems within DataOS, ensuring Data Products have reliable access to well-governed and high-quality data.
{% endhint %}

The Depot serves as the registration of data locations to be made accessible to DataOS. Through the Depot Service, each source system is assigned a unique address, referred to as a **Uniform Data Link (UDL)**. The UDL grants convenient access and manipulation of data within the source system, eliminating the need for repetitive credential entry. The UDL follows this format:

<p align="center"><code>dataos://[depot]/[source-path]</code></p>

The UDL gives a Data Product a single address for a dataset across reads, writes, and transformations.

Once this mapping is established, Depot Service automatically generates the Uniform Data Link (UDL) that can be used throughout DataOS to access the data. As a reminder, the UDL has the format: `dataos://[depot]/[source-path]`.

The path after the Depot name follows the native hierarchy of the connected source system.\
For example, a PostgreSQL Depot can be consumed as `dataos://postgresdepot/public/orders` where `public` is the schema and `orders` is the table. Similarly, object storage or other sources follow their own native path structure.

{% hint style="warning" icon="hand-back-point-right" %}
Depot Service is the DataOS Service that manages the Depot Resource. It introspects Depots and their underlying storage engines. After a Depot is created, you can read dataset details from it: constraints, partitions, indexing, and other metadata.
{% endhint %}

{% hint style="warning" icon="hand-back-point-right" %}
A Depot provides access to data without moving or duplicating it. The data stays in the source system. When ingestion, querying, syndication, or copying is needed, use a DataOS Stack such as Nilus.
{% endhint %}

## Prerequisites

To create a Depot, you need a tenant-specific role (**Tenant Admin**, **Data Admin**, or **Data Developer**).

To use a Depot, you need resource-specific permission granted by the Depot owner.

### Structure of a Depot manifest file <a href="#structure-of-a-depot-manifest" id="structure-of-a-depot-manifest"></a>

To know more about the attributes of Depot manifest Configuration, refer to [Manifest Configurations](/concepts/resources/depot/configurations.md).

### How to create a Depot? <a href="#how-to-create-a-depot" id="how-to-create-a-depot"></a>

This section involves steps for creating a Depot for different data sources supported by DataOS.

* [Amazon Redshift](/concepts/resources/depot/supported-sources/redshift.md)
* [Azure Blob File System Secure (ABFSS)](/concepts/resources/depot/supported-sources/abfss.md)
* [Google BigQuery](/concepts/resources/depot/supported-sources/bigquery.md)
* [Google Cloud Storage (GCS)](/concepts/resources/depot/supported-sources/gcs.md)
* [Java Database Connectivity (JDBC)](/concepts/resources/depot/supported-sources/jdbc.md)
* [Kafka](/concepts/resources/depot/supported-sources/kafka.md)
* [MongoDB](/concepts/resources/depot/supported-sources/mongo.md)
* [Microsoft SQL Server (MSSQL) or Azure SQL](/concepts/resources/depot/supported-sources/mssql.md)
* [MySQL](/concepts/resources/depot/supported-sources/mysql.md)
* [OpenSearch](/concepts/resources/depot/supported-sources/opensearch.md)
* [Oracle](/concepts/resources/depot/supported-sources/oracle.md)
* [PostgreSQL](/concepts/resources/depot/supported-sources/postgresql.md)
* [Simple Storage Service (Amazon S3)](/concepts/resources/depot/supported-sources/s3.md)
* [Snowflake](/concepts/resources/depot/supported-sources/snowflake.md)

### How to consume Depots? <a href="#how-to-use-depots" id="how-to-use-depots"></a>

Once a Depot is created, you can use its Uniform Data Links (UDLs) to access data without physically moving it and perform further operations. The UDLs play a crucial role in various scenarios within DataOS.

See the [consume](/concepts/resources/depot/consume.md) section for common Depot consumption patterns.

### Best Practices <a href="#best-practices" id="best-practices"></a>

This section involves dos and don'ts for managing a Depot.

* Give your Depot a meaningful name that represents its purpose.

  **Example:** This makes it easier to manage and identify Depots.

  * ✅ `s3depot` instead of ❌ `s3-1`, as by `s3depot` it is easily interpretable that this Depot is establishing a connection with the Amazon S3 source.
  * ✅ `dbmysql` instead of ❌ `db1`, by `dbmysql` it is clear that this Depot is establishing the connection with the MySQL database, making it clear.
* Before deleting a Depot, ensure no other downstream Resources are consuming it.

### Data integration - Supported connectors in DataOS <a href="#data-integration-supported-connectors-in-dataos" id="data-integration-supported-connectors-in-dataos"></a>

The catalogue of data sources accessible by one or more components within DataOS is available on [Supported Connectors](/concepts/resources/depot/supported-connectors.md).

### FAQs <a href="#faqs" id="faqs"></a>

**1. How do I know which data sources are supported in DataOS?**

You can refer to [Supported Connectors](/concepts/resources/depot/supported-connectors.md) to view all supported data sources.

**2. What are Uniform Data Links (UDLs), and how do they work?**

UDLs are standardized links that let you access and reference data across the environment without moving it. They provide consistent data access regardless of where the data resides within DataOS.

**3. How can I use the data in a Depot without moving it?**

DataOS allows compute-on-read and virtual access, so you can query and analyze data in-place without physically moving it.

**4. Can users see everyone’s Depots or only their own?**

Users can only see the Depots they have been granted access to by the Depot owner.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/concepts/resources/depot.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.