> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/build/v1/move/mode/batch.md).

# Batch

Batch movement extracts data from a source, normalizes it, and loads it into your engine in chunks, either on a schedule or as a one-time run. It is the default type and the widest in source support.

## When to use batch

* Periodic updates (hourly, daily, weekly) are acceptable.
* You need a full reload, or an incremental extraction driven by a timestamp or cursor.
* The source is a SaaS API, an analytics warehouse, object storage, or a file system.

## Before you begin

Ensure you have:

* A Depot for the source, or an approved connection with secrets projected via `use.projection`.
* A Depot or connection for the destination.
* A compute profile, read access to the source, and write access to the destination.

## 1. Choose your source

Batch supports the widest range of sources of any move type. See [supported sources and destinations](/build/v1/move/supported-sources-and-destinations.md) for the summary, and [supported batch sources](https://v2.dataos.info/references/resources/nilus/batch/batch-sources) in References for the full, current list plus per-connector prerequisites and options.

## 2. Write the manifest

Create a `nilus` resource with `spec.type: batch`:

```yaml
name: ${{pipeline-name}}
version: v1alpha
type: nilus
tags:
  - nilus-batch
description: ${{description}}
spec:
  type: batch
  compute: ${{compute-profile}}
  logLevel: INFO
  schedule:                       # optional; omit for a one-time run
    crons:
      - "0 2 * * *"              # daily at 02:00 UTC
    timezone: UTC
    concurrencyPolicy: Forbid
  source:
    address: dataos://${{source-depot}}?purpose=ro
    options:
      source_table: "${{schema.table}}"
      incremental_key: updated_at  # optional; for delta loads
  sink:
    address: dataos://${{sink-depot}}?purpose=rw
    options:
      dest_table: ${{schema.table}}
      incremental_strategy: replace
```

Pick the sink strategy that matches your refresh:

| Strategy  | Use it when                                                        |
| --------- | ------------------------------------------------------------------ |
| `append`  | Add new rows without touching existing ones                        |
| `replace` | Drop and reload the destination on every run                       |
| `merge`   | Upsert by `primary_key` (requires `primary_key` in source options) |

For the full attribute reference, see [batch configuration](https://v2.dataos.info/references/resources/nilus/batch/pipeline-config) in References.

## 3. Apply and confirm

```bash
dataos-ctl resource apply -f ${{path-to-manifest.yaml}}
dataos-ctl resource get -t nilus -n ${{pipeline-name}}
```

A status of `active` means the pipeline is running. When the data has landed, [explore the moved data](/build/v1/move/explore-moved-data.md) to confirm it.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/build/v1/move/mode/batch.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.