> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/resources/nilus/quick-start-guides.md).

# Quick start

Decision table for picking a pipeline mode, minimal batch/CDC config snippets, and a pre-production checklist for a first Nilus pipeline.

Use this page as the shortest path from "I need a pipeline" to "I know which page to open next."

## First successful pipeline

Start with a narrow batch pipeline before moving to Change Data Capture (CDC), stream, or metadata use cases. The recommended first path is a PostgreSQL table into a DataOS Lakehouse destination with one source table, one destination table, and a merge strategy only when a stable `primary_key` exists.

## Choose a pipeline mode

| Goal                                           | Start here                                                                                                        | Use when                                                                                                                        |
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| Periodic table, file, SaaS, or warehouse loads | [Batch Sample Configs](https://v2.dataos.info/references/resources/nilus/batch/sample-configs.md)                 | The source can be read on a schedule or on demand.                                                                              |
| Low-latency database change streams            | [CDC Sample Configs](https://v2.dataos.info/references/resources/nilus/cdc/sample-configs.md)                     | The source database exposes a supported change feed and retention is configured.                                                |
| Event systems                                  | [Stream Sample Configs](https://v2.dataos.info/references/resources/nilus/stream/sample-configs.md)               | Events arrive through Kafka or NATS and should keep flowing after startup.                                                      |
| Catalog introspection without data movement    | [Metadata Sample Configs](https://v2.dataos.info/references/resources/nilus/metadata-pipelines/sample-configs.md) | The source is a data warehouse or Lakehouse and you want to publish schema, lineage, or usage into the DataOS metadata catalog. |

## Fastest safe path

Confirm that your source, destination, and required pipeline mode are supported in [Supported connectors](https://v2.dataos.info/references/supported-connectors.md).

1. Pick the source and destination from [Sources](https://v2.dataos.info/references/resources/nilus/sources.md) and [Destinations](https://v2.dataos.info/references/resources/nilus/destinations.md).
2. Copy the closest sample config for the pipeline mode.
3. Configure credentials with [Secrets and Projections](https://v2.dataos.info/references/resources/nilus/concepts/secrets-and-projections.md) or use a `dataos://` depot address.
4. Keep the first run narrow: one table, one resource, or a bounded time range.
5. Confirm rows, schema, and write behavior before widening the scope.
6. Add observability and tuning only after the baseline run is correct.

## Minimal batch shape

```yaml
version: v1alpha
name: customers-batch
type: nilus
spec:
  type: batch
  compute: universe-compute
  source:
    address: dataos://postgressource?purpose=ro
    options:
      source_table: public.customers
      incremental_key: updated_at
      primary_key: id
  sink:
    address: dataos://warehouse?purpose=rw
    options:
      dest_table: raw.customers
      incremental_strategy: merge
```

## Minimal CDC shape

```yaml
version: v1alpha
name: customers-cdc
type: nilus
spec:
  type: cdc
  compute: universe-compute
  source:
    address: dataos://postgrescdc?purpose=ro
    options:
      source_table: public.customers
      strategy: flatten
    cdc:
      topic.prefix: customers
      schema.include.list: public
      table.include.list: public.customers
  sink:
    address: dataos://lakehouse?purpose=rw
    options:
      dest_table: raw.customers_changes
      incremental_strategy: append
```

## Pre-production checklist

| Check                                                            | Why it matters                                                                      |
| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| `source_table` points to the intended object                     | Prevents loading the wrong table, stream, sheet, or API resource.                   |
| `dest_table` is fully qualified where the destination expects it | Avoids writing into a default schema by accident.                                   |
| `incremental_strategy` matches the data model                    | `replace`, `append`, and `merge` produce different history semantics.               |
| `primary_key` is set for merge workloads                         | Merge-style writes need stable row identity.                                        |
| CDC retention is longer than expected downtime                   | A CDC connector that falls behind the retained change feed may need a new snapshot. |
| First run has logs and metrics                                   | Makes later troubleshooting far easier.                                             |

## Related docs

* [Batch pipeline config](https://v2.dataos.info/references/resources/nilus/batch/pipeline-config.md)
* [CDC pipeline config](https://v2.dataos.info/references/resources/nilus/cdc/pipeline-config.md)
* [Troubleshooting](https://v2.dataos.info/references/resources/nilus/troubleshooting.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/references/resources/nilus/quick-start-guides.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
