> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/dataos-resources/vulcan/ldk.md).

# LDK

The Local Development Kit (LDK) is how you build and run a Vulcan data product on your own machine before you deploy it. It bundles the Vulcan command-line interface (CLI), the project scaffold, and the local services you need to develop, test, plan, and serve a data product end to end.

This page covers the common path: install the CLI, scaffold a project, and move a data product through its lifecycle. For the production deployment steps, see the [Deployment checklist](/references/dataos-resources/vulcan/deployment-checklist.md).

## Prerequisites

You need two things before you install the CLI:

1. **Python 3.10.** Vulcan requires Python 3.10 for a local wheel installation.

   ```bash
   python3.10 --version
   python3.10 -m pip install --upgrade pip
   ```
2. **Docker.** The LDK uses Docker to run local services for development, including a local Postgres engine, the state store, the transpiler, and the API services. Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) and make sure it is running.

   ```bash
   docker --version
   docker compose version
   ```

{% hint style="info" %}
**Spark also needs Java.** For the Spark engine, install Vulcan locally in Python 3.10 like the other engines, and make a Java 17 SDK available, because the Spark driver runs in your local Python process.
{% endhint %}

## Install the CLI

You install Vulcan from a pre-built Python wheel, with the extra for the engine you want to use. Create and activate a virtual environment first.

1. Create and activate the environment:

   ```bash
   mkdir my-vulcan-project && cd my-vulcan-project
   python3.10 -m venv .venv
   source .venv/bin/activate
   ```

   After activation, your prompt includes `(.venv)`.
2. Install the wheel with your engine's extra. Quote the path so your shell does not interpret the square brackets:

   ```bash
   pip install "./vulcan-<version>-py3-none-any.whl[postgres]"
   ```

   Replace `postgres` with `snowflake`, `databricks`, `trino`, or another supported engine.
3. Verify the install:

   ```bash
   vulcan --version
   ```

{% hint style="info" %}
Alternatively, run the CLI from an engine-specific Docker image with an alias, which avoids a local Python install:

```bash
alias vulcan="docker run -it --network=vulcan --rm -v .:/workspace tmdcio/vulcan-postgres:<version> vulcan"
```

{% endhint %}

For the exact connection fields and authentication for your engine, see the [Engine guide](/references/engine-guide/engine-guide.md).

## Scaffold a project

Initialize the project structure:

```bash
vulcan init
```

The initializer creates the starter layout for models, seeds, tests, quality checks, macros, and semantic definitions:

```
my-vulcan-project/
├── config.yaml            # Project configuration
├── usage.yaml             # Business-facing usage guidance
├── audits/                # Audit queries (named SQL rules, blocking)
├── dq/                    # Data quality (DQ) checks (kind: dq, non-blocking)
├── macros/                # Reusable SQL patterns
├── models/                # SQL and Python transformation models
│   ├── semantics/         # Semantic models (kind: semantic)
│   └── metrics/           # Per-metric files (kind: metric)
├── seeds/                 # Static CSV data
└── tests/                 # Unit tests for model logic
```

Configure `config.yaml` with your engine connection and `model_defaults.dialect`, then verify your setup:

```bash
vulcan info
```

`vulcan info` checks connection status, project structure, and configuration. See [Configuration](/references/dataos-resources/vulcan/configuration.md) for the full `config.yaml` reference.

## Walk a data product through its lifecycle

With the project scaffolded, you move a data product through these steps. Each builds on the last.

1. **Write models.** Author SQL or Python transformations in `models/`. Most teams start with SQL and add Python when SQL gets painful, such as calling external APIs or running machine learning models. See [Model and kinds](/references/dataos-resources/vulcan/core-concepts/model-and-kinds.md).
2. **Lint.** `vulcan info` validates syntax, ambiguous columns, and invalid SQL patterns before execution.
3. **Test.** Write unit tests in `tests/`, then run them locally without touching the warehouse:

   ```bash
   vulcan test
   ```

   See [Test](/references/dataos-resources/vulcan/core-concepts/unit-test.md).
4. **Add quality.** Attach audits to models as assertions to block bad rows, and add DQ checks in `dq/` to monitor quality over time. See [Assertions](/references/dataos-resources/vulcan/core-concepts/assertions.md) and [Data quality](/references/dataos-resources/vulcan/core-concepts/data-quality.md).
5. **Define semantics.** Add semantic models in `models/semantics/` and metrics in `models/metrics/`. See [Semantic model and components](/references/dataos-resources/vulcan/core-concepts/semantic-model-and-components.md).
6. **Plan and apply.** Preview the change, classify its impact, and apply it:

   ```bash
   vulcan plan
   ```

   See [Plan](/references/dataos-resources/vulcan/core-concepts/plan.md).
7. **Run on schedule.** Keep the applied shape filled with fresh data:

   ```bash
   vulcan run
   ```

   See [Observability](/references/dataos-resources/vulcan/observability.md).
8. **Serve and monitor.** Query through the REST, GraphQL, or SQL APIs, and inspect logs and rendered SQL while you iterate.

## Useful commands

| Command                      | Purpose                                                     |
| ---------------------------- | ----------------------------------------------------------- |
| `vulcan init`                | Scaffold a new project.                                     |
| `vulcan info`                | Verify connection, structure, and configuration; lint code. |
| `vulcan test`                | Run unit tests against fixtures, no warehouse cost.         |
| `vulcan plan`                | Preview, classify, and apply project changes.               |
| `vulcan run`                 | Process new and missing data intervals on schedule.         |
| `vulcan render <model>`      | See the generated SQL for a model.                          |
| `vulcan fetchdf "<sql>"`     | Execute a read-only query and return rows.                  |
| `vulcan transpile`           | Convert a semantic query to engine SQL.                     |
| `vulcan create_test <model>` | Generate a starter test from warehouse data.                |

## Iterating

When you change a model, semantic definition, check, or metadata, run `vulcan plan` to review and apply the change. When the shape is already applied and you only need fresh data, run `vulcan run`. The rule of thumb: changed the shape, use `plan`; refreshing data, use `run`.

## Related docs

* [Configuration](/references/dataos-resources/vulcan/configuration.md): the full `config.yaml`, gateways, model defaults, and options.
* [Core concepts](/references/dataos-resources/vulcan/core-concepts.md): models, plan and state, semantics, quality, macros, and signals.
* [Deployment checklist](/references/dataos-resources/vulcan/deployment-checklist.md): promote the data product into a DataOS environment.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/references/dataos-resources/vulcan/ldk.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.