> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/concepts/resources/nilus/cdc.md).

# Change Data Capture

Change Data Capture reads inserts, updates, and deletes from a source system and applies them to another system with low delay. Nilus uses CDC when a source exposes a durable change-log surface and the destination needs fresher state than a scheduled batch copy can provide.

CDC planning still matters because each source has its own log, permission, retention, schema-change, and recovery behavior. The connector pages document those source-specific requirements; this page explains the common Nilus model.

## CDC in Nilus

Nilus models CDC as `type: nilus` with `spec.type: cdc`. The authored manifest keeps connector-specific change-capture settings under `spec.source.cdc`, while Nilus handles the runtime service, state persistence, and sink loading path. In practice, Nilus CDC combines:

* source-specific log reading
* optional initial snapshotting for bootstrap
* event shaping through `source.options.strategy`
* durable offset and schema history storage

For the full list of `source.options`, `source.cdc`, and `sink.options` keys (tables, examples, and validation notes), see [Understanding CDC Pipeline Config](/concepts/resources/nilus/cdc/service-config.md). That page is the single reference for CDC manifest authoring; this page explains the conceptual model only.

## Core Capabilities

Nilus CDC mode provides the following capabilities:

### Source-Aware Change Capture

* Nilus reads the source system through connector-specific CDC settings such as include lists, topic prefixes, slot or server identifiers, and heartbeat controls.
* These settings stay in `spec.source.cdc` so the pipeline manifest reflects the source contract clearly.
* Source prerequisites still differ by connector, which is why the connector pages remain essential for production onboarding.

### Snapshot Plus Ongoing Capture

* CDC bootstrapping often needs both an initial snapshot and ongoing log consumption.
* Nilus supports that handoff inside the same CDC model instead of forcing users to wire two separate pipelines for baseline load and steady-state change capture.
* Include lists should stay narrow during onboarding so the first snapshot and ongoing state are easier to validate.

### Stable Event and State Handling

* Nilus can emit row-oriented records through strategies such as `flatten`, or keep change-oriented structure where the connector contract requires it.
* Offsets, schema history, and pipeline state are persisted through the Nilus observability layer so CDC runs can be restarted and inspected consistently.
* Keeping `topic.prefix` and other identity keys stable is important because those values affect how state is tracked over time.

## Flow

1. The source connector reads change events from the database log or equivalent change surface.
2. Nilus applies source-specific CDC configuration from `spec.source.cdc`.
3. The runtime emits row-shaped or change-shaped output depending on the configured strategy.
4. Nilus writes the output to the sink and stores offsets, schema history, and run metadata for recovery and observability.

## Constraints

* Every CDC source has its own permissions, replication, and retention requirements.
* Long-running lag can cause problems if the source’s change-log retention window is shorter than the pipeline recovery window.
* Schema evolution is a first-class operational concern and should be tested with the actual source and destination pair.
* CDC does not remove the need for destination-specific write planning. Sink behavior still depends on the destination contract.

## Related Docs

* [Understanding CDC Pipeline Config](/concepts/resources/nilus/cdc/service-config.md)
* [CDC Sample Configs](/concepts/resources/nilus/cdc/sample-configs.md)
* [Sources](/concepts/resources/nilus/sources.md)
* [Destinations](/concepts/resources/nilus/destinations.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/concepts/resources/nilus/cdc.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.