Introduction
What building a Data Product involves, the three stages, and where to start.
Building a Data Product in DataOS follows one pattern: Extract, Load, Productize (ELP). You pull raw data from a source, land it where your engine can reach it, and turn it into an owned, contracted product that people and AI agents can trust.

That's the same journey traced end to end, source to consumer. Data starts in an engine such as Snowflake, Databricks, Postgres, Spark, Trino, SQL Server, or Microsoft Fabric. Connect Data registers a Secret, Depot, or environment variables so DataOS can reach it. From there the path forks and rejoins: Nilus brings raw data in through stream, CDC, or batch ingestion when it isn't already reachable, while Vulcan builds the product itself, models, semantic model, metrics, tests, assertions, and data quality, on whatever is reachable. Govern attaches the policy: an after_authorize plugin resolves the caller's group, and semantic-model policies mask or filter accordingly. Publish deploys the result as a governed, discoverable product, served out to BI tools, APIs, a DB client, SDKs, and MCP.
ELP extends ETL with productization. A transformed table can be correct yet remain difficult for others to discover, understand, and use safely. Productizing closes that gap. This section walks that pattern end to end.
The three stages
"Build" has three stages. They map directly onto ELP, with one stage of reconnaissance before it.
Understand
Inspect the sources you might use: schema, lineage, quality, freshness.
Before the pattern
Move
Bring raw data in when it isn't already where your engine can reach it.
Extract + Load
Productize
Turn the data into a tested, governed, served product.
Productize
"Move" is conditional. If the data already lives where your engine can read it, you skip straight from Understand to Productize. You move data only when there is a gap to close.
What each stage produces
Understand answers one question: is this the right data? You inspect metadata, check lineage and freshness, and run sample queries. You leave with evidence, not assumptions. If a source has no metadata in DataOS yet, you scan it first.
Move brings raw data into reach. Connect Data first: a Secret, Depot, or environment variables give DataOS a path to the source. Then Nilus, DataOS's data movement framework, picks up from there with a mode (batch, change data capture, or streaming), points it at a source and a destination, and lands the data unchanged. Move never transforms; it only relocates.
Productize is the core of Build. On the Vulcan resource, you author the transform, the semantic model, the metrics, and the tests, assertions, and data quality checks that make the product enforce its own contract. Govern attaches the access policy, an after_authorize plugin plus semantic-model policies that mask or filter by caller group, before Publish deploys everything as one object, a governed, discoverable Data Product. The product carries its own meaning: what a measure is, how fresh it must be, and who can see which rows. That is what makes it safe to serve out to BI tools, APIs, a DB client, SDKs, and MCP, and safe for an AI agent to consume, not just a person.
The worked example
Every page in this section uses one running example: orders-analytics, an e-commerce Data Product built on PostgreSQL. It tracks order revenue, customer segmentation, product performance, and fulfillment conversion, organized in a bronze, silver, and gold structure where each layer reads from the one above it.
Following the example end-to-end shows what each step produces, from the first source scan to the deployed product.
Where to start
Pick the entry point that matches where you're starting from.
Get ready to build? Work through Prerequisites, and set up the requirements.
New to Build? Run the Quickstart. It builds a small product using a local engine with guided file generation, so you can see the full pattern before setting up your environment.
Building with AI? Follow Build journey with AI and let an agent design and generate alongside you.
Productizing is the same discipline, whichever engine you choose and whichever stage you enter from. The engine is the choice; the pattern does not change.
Last updated
Was this helpful?