> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/build/move/overview.md).

# Move Data

Bring raw data into reach when it is not already where your engine can read it.

Move is the Extract and Load of the build pattern. You pull raw records from a source and land them, unchanged, where your engine can read them. Move never transforms; it only relocates. Producing meaning from the data is [Productize](https://v2.dataos.info/build/productize/overview.md), the step after.

You only do this when there is a gap. If [Understand](https://v2.dataos.info/build/understand/overview.md) showed the data already lives where your engine can reach it, skip Move and go straight to Productize. Reach for Move when the data you need is in a system your engine cannot query directly.

Data movement runs on **Nilus**, the DataOS data movement framework. You declare a `nilus` resource that names a source, a destination, and a type, then apply it with the CLI.

## Choose a data movement type

The type depends on how fresh the data must be and what the source supports.

| Type                                                                       | Best for                                                                                           | Freshness        |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------- |
| [Batch](https://v2.dataos.info/build/move/mode/batch.md)                   | Large volumes; scheduled full or incremental loads from databases, warehouses, and SaaS platforms  | Minutes to hours |
| [Change data capture (CDC)](https://v2.dataos.info/build/move/mode/cdc.md) | Operational databases where every insert, update, and delete must be captured without full reloads | Near real-time   |
| [Streaming](https://v2.dataos.info/build/move/mode/stream.md)              | Event streams and message queues processed continuously as records arrive                          | Near real-time   |

If you are unsure, start with **Batch**. Use **CDC** when you need row-level change capture from a transactional database. Use **Streaming** when the source produces continuous events through systems such as Kafka or NATS.

## The shape of a move

Every data movement type follows the same three steps, so once you have run one you can run any of them:

```mermaid
flowchart LR
    S[(Source)] -->|Extract| N[Nilus pipeline<br/>source + sink + type]
    N -->|Load, unchanged| D[(Destination<br/>your engine can read)]

    style S fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A
    style N fill:#E6F1FB,stroke:#185FA5,color:#0C447C
    style D fill:#E1F5EE,stroke:#0F6E56,color:#085041
```

1. **Pick a source and destination.** Each is addressed through a Depot, or through a direct connector URI with secrets projected via `spec.use.projection` for connectors without a Depot variant (such as Databricks, IBM Db2, NATS, and Microsoft Fabric Lakehouse). See [Supported sources and destinations](https://v2.dataos.info/build/move/supported-sources-and-destinations.md).
2. **Write the manifest.** A `nilus` resource with the right `spec.type` (`batch`, `cdc`, or `stream`).
3. **Apply and confirm.** `dataos-ctl resource apply`, then check that STATUS is `active` and RUNTIME shows the pipeline running.

## Before you begin

Across all types of data movement, you need:

* A Depot for the source and a Depot for the destination (or an approved connection with secrets projected via `spec.use.projection`).
* A compute profile for the pipeline.
* Two independent sets of permissions:
  * **DataOS access:** the Tenant roles to create a `nilus` resource, plus `Can Use` on the Compute, Depot, and Secret it depends on.
  * **Source and destination privileges:** SQL `GRANT`s or IAM roles for the pipeline's credential on the source and destination systems.

See [Nilus](https://v2.dataos.info/references/resources/nilus) in References.

When the data has landed, [explore the moved data](https://v2.dataos.info/build/move/explore-moved-data.md) to confirm it is correct, then move on to Productize.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/build/move/overview.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
