> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/concepts/resources/nilus/quick-start-guides.md).

# Quick Start

Use this page as the shortest path from "I need a pipeline" to "I know which page to open next."

## First successful pipeline

Start with a narrow batch pipeline before moving to Change Data Capture (CDC), stream, or metadata use cases. The recommended first path is a PostgreSQL table into a DataOS Lakehouse destination with one source table, one destination table, and a merge strategy only when a stable `primary_key` exists.

## Choose a pipeline mode

| Goal                                           | Start here                                                                  | Use when                                                                         |
| ---------------------------------------------- | --------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| Periodic table, file, SaaS, or warehouse loads | [Batch Sample Configs](/concepts/resources/nilus/batch/sample-configs.md)   | The source can be read on a schedule or on demand.                               |
| Low-latency database change streams            | [CDC Sample Configs](/concepts/resources/nilus/cdc/sample-configs.md)       | The source database exposes a supported change feed and retention is configured. |
| Event systems                                  | [Stream Sample Configs](/concepts/resources/nilus/stream/sample-configs.md) | Events arrive through Kafka or NATS and should keep flowing after startup.       |

## Fastest safe path

1. Pick the source and destination from [Sources](/concepts/resources/nilus/sources.md) and [Destinations](/concepts/resources/nilus/destinations.md).
2. Copy the closest sample config for the pipeline mode.
3. Configure credentials with [Secrets and Projections](/concepts/resources/nilus/concepts/secrets-and-projections.md) or use a `dataos://` depot address.
4. Keep the first run narrow: one table, one resource, or a bounded time range.
5. Confirm rows, schema, and write behavior before widening the scope.
6. Add observability and tuning only after the baseline run is correct.

## Minimal batch shape

```yaml
version: v1alpha
name: customers-batch
type: nilus
spec:
  type: batch
  compute: universe-compute
  source:
    address: dataos://postgres-source?purpose=ro
    options:
      source_table: public.customers
      incremental_key: updated_at
      primary_key: id
  sink:
    address: dataos://warehouse?purpose=rw
    options:
      dest_table: raw.customers
      incremental_strategy: merge
```

## Minimal CDC shape

```yaml
version: v1alpha
name: customers-cdc
type: nilus
spec:
  type: cdc
  compute: universe-compute
  source:
    address: dataos://postgres-cdc?purpose=ro
    options:
      source_table: public.customers
      strategy: flatten
    cdc:
      topic.prefix: customers
      schema.include.list: public
      table.include.list: public.customers
  sink:
    address: dataos://lakehouse?purpose=rw
    options:
      dest_table: raw.customers_changes
      incremental_strategy: append
```

## Pre-production checklist

| Check                                                            | Why it matters                                                                      |
| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| `source_table` points to the intended object                     | Prevents loading the wrong table, stream, sheet, or API resource.                   |
| `dest_table` is fully qualified where the destination expects it | Avoids writing into a default schema by accident.                                   |
| `incremental_strategy` matches the data model                    | `replace`, `append`, and `merge` produce different history semantics.               |
| `primary_key` is set for merge workloads                         | Merge-style writes need stable row identity.                                        |
| CDC retention is longer than expected downtime                   | A CDC connector that falls behind the retained change feed may need a new snapshot. |
| First run has logs and metrics                                   | Makes later troubleshooting far easier.                                             |

## Related docs

* [Understanding Batch Pipeline Config](/concepts/resources/nilus/batch/pipeline-config.md)
* [Understanding CDC Pipeline Config](/concepts/resources/nilus/cdc/service-config.md)
* [Troubleshooting](/concepts/resources/nilus/troubleshooting.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/concepts/resources/nilus/quick-start-guides.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.