> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/build/get-started/readme.md). # Introduction Building a Data Product in DataOS follows one pattern: **Extract, Load, Productize (ELP)**. You pull raw data from a source, land it where your engine can reach it, and turn it into an owned, contracted product that people and AI agents can trust.

Note: Supported sources extend beyond those shown in the image.

That's the same journey traced end to end, source to consumer. Data starts in an engine such as Snowflake, Databricks, Postgres, Spark, Trino, SQL Server, or Microsoft Fabric. **Connect Data** registers a Secret, Depot, or environment variables so DataOS can reach it. From there the path forks and rejoins: **Nilus** brings raw data in through stream, CDC, or batch ingestion when it isn't already reachable, while **Vulcan** builds the product itself, models, semantic model, metrics, tests, assertions, and data quality, on whatever is reachable. **Govern** attaches the policy: an `after_authorize` plugin resolves the caller's group, and semantic-model policies mask or filter accordingly. **Publish** deploys the result as a governed, discoverable product, served out to BI tools, APIs, a DB client, SDKs, and MCP. ELP extends ETL with productization. A transformed table can be correct yet remain difficult for others to discover, understand, and use safely. Productizing closes that gap. This section walks that pattern end to end. ## The three stages "Build" has three stages. They map directly onto ELP, with one stage of reconnaissance before it.
StageWhat you doELP
UnderstandInspect the sources you might use: schema, lineage, quality, freshness.Before the pattern
MoveBring raw data in when it isn't already where your engine can reach it.Extract + Load
ProductizeTurn the data into a tested, governed, served product.Productize
```mermaid flowchart LR U[Understand
inspect the sources] --> M[Move
Extract + Load raw data] M --> P[Productize
build the product] U -.->|data already reachable| P style U fill:#EDE9E5,stroke:#35505B,color:#242422 style M fill:#54DED1,stroke:#009293,color:#202F36 style P fill:#35505B,stroke:#202F36,color:#EDE9E5 ``` "Move" is conditional. If the data already lives where your engine can read it, you skip straight from Understand to Productize. You move data only when there is a gap to close. ## What each stage produces **Understand** answers one question: is this the right data? You inspect metadata, check lineage and freshness, and run sample queries. You leave with evidence, not assumptions. If a source has no metadata in DataOS yet, you scan it first. **Move** brings raw data into reach. Connect Data first: a Secret, Depot, or environment variables give DataOS a path to the source. Then Nilus, DataOS's data movement framework, picks up from there with a mode (batch, change data capture, or streaming), points it at a source and a destination, and lands the data unchanged. Move never transforms; it only relocates. **Productize** is the core of Build. On the Vulcan resource, you author the transform, the semantic model, the metrics, and the tests, assertions, and data quality checks that make the product enforce its own contract. Govern attaches the access policy, an `after_authorize` plugin plus semantic-model policies that mask or filter by caller group, before Publish deploys everything as one object, a governed, discoverable Data Product. The product carries its own meaning: what a measure is, how fresh it must be, and who can see which rows. That is what makes it safe to serve out to BI tools, APIs, a DB client, SDKs, and MCP, and safe for an AI agent to consume, not just a person. ## The worked example Every page in this section uses one running example: **`orders-analytics`**, an e-commerce Data Product built on PostgreSQL. It tracks order revenue, customer segmentation, product performance, and fulfillment conversion, organized in a bronze, silver, and gold structure where each layer reads from the one above it. Following the example end-to-end shows what each step produces, from the first source scan to the deployed product. ## Where to start Pick the entry point that matches where you're starting from. * **Get ready to build?** Work through [Prerequisites](/build/get-started/prerequisites.md), and set up the requirements. * **New to Build?** Run the [Quickstart](/build/get-started/quickstart.md). It builds a small product using a local engine with guided file generation, so you can see the full pattern before setting up your environment. * **Building with AI?** Follow [Build journey with AI](/build/get-started/build-journey-with-ai.md) and let an agent design and generate alongside you. Productizing is the same discipline, whichever engine you choose and whichever stage you enter from. The engine is the choice; the pattern does not change. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://v2.dataos.info/build/get-started/readme.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.