> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/consume/v1/recipes/retrieve-large-result-sets-with-pagination.md).

# Retrieve Large Result Sets with Pagination

Use this recipe when your Semantic REST query returns more than **50,000 records**.

Semantic REST queries are asynchronous. Retrieve large result sets by submitting the query with `limit`, `offset`, and a deterministic `order`, then poll the statement until execution completes before fetching the result.

```
POST (limit + offset + order)
        │
        ▼
   Poll statement
        │
        ▼
     SUCCESS
        │
        ▼
  Retrieve result
        │
        ▼
Increase offset and repeat
```

## Submit a paginated query

{% hint style="info" %}
Add `limit`, `offset`, and `order` to the **POST request body**.

* `limit` defines the maximum number of records in each batch.
* `offset` defines where the next batch begins.
* `order` must remain the same across all batches to avoid duplicate or missing records.
  {% endhint %}

The following request retrieves the first 50,000 records:

{% code title="Paginated Semantic REST query" %}

```bash
curl --request POST \
  'https://<DATAOS_FQDN>/vulcan/tenants/<DATAOS_TENANT_ID>/data-products/<DATAOS_TENANT_ID>-<resource-name>/api/v1/query/semantic/rest' \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer ${DATAOS_TOKEN}" \
  --data '{
    "query": {
      "measures": [
        "customer_profile.count"
      ],
      "dimensions": [
        "customer_profile.region_id"
      ],
      "order": {
        "customer_profile.region_id": "asc"
      },
      "limit": 50000,
      "offset": 0,
      "timezone": "UTC"
    }
  }'
```

{% endcode %}

To retrieve the next batch, keep the same `order` and increase only the `offset`.

```
Batch 1 → limit=50000, offset=0
Batch 2 → limit=50000, offset=50000
Batch 3 → limit=50000, offset=100000
```

## Poll the statement

Each POST request returns a statement ID.

```json
{
  "id": "01KYP6V3G1JTNSRARCN0HAF43B",
  "status": "QUEUED",
  "strategy": "EXECUTE"
}
```

Poll the statement until the status becomes `SUCCESS`.

{% code title="Poll statement status" %}

```bash
curl --request GET \
  'https://<DATAOS_FQDN>/vulcan/tenants/<DATAOS_TENANT_ID>/data-products/<DATAOS_TENANT_ID>-<resource-name>/api/v1/query/statement/<statement-id>' \
  --header "Authorization: Bearer ${DATAOS_TOKEN}"
```

{% endcode %}

Continue polling while the status is:

* `ACCEPTED`
* `QUEUED`
* `IN_PROGRESS`

If the status is `FAILED`, inspect the `error` field.

## Retrieve the result

### JSON

{% code title="Fetch result as JSON" %}

```bash
curl --request GET \
  'https://<DATAOS_FQDN>/vulcan/tenants/<DATAOS_TENANT_ID>/data-products/<DATAOS_TENANT_ID>-<resource-name>/api/v1/query/statement/<statement-id>/result?format=json' \
  --header "Authorization: Bearer ${DATAOS_TOKEN}"
```

{% endcode %}

### Parquet

{% code title="Download result as Parquet" %}

```bash
curl --location \
  'https://<DATAOS_FQDN>/vulcan/tenants/<DATAOS_TENANT_ID>/data-products/<DATAOS_TENANT_ID>-<resource-name>/api/v1/query/statement/<statement-id>/result?format=parquet' \
  --header "Authorization: Bearer ${DATAOS_TOKEN}" \
  --output result.parquet
```

{% endcode %}

Parquet downloads the complete result as a single file. `limit`, `offset`, and `columns` are ignored for the Parquet format.

## Complete workflow

```
POST (limit=50000, offset=0, order=...)      ──► Poll statement ──► SUCCESS ──► Retrieve result
POST (limit=50000, offset=50000, order=...)  ──► Poll statement ──► SUCCESS ──► Retrieve result
POST (limit=50000, offset=100000, order=...) ──► Poll statement ──► SUCCESS ──► Retrieve result

Continue until the returned batch contains fewer records than the requested `limit`.
```

## Best practices

* Keep the same `order` for every request.
* Increase only the `offset` between batches.
* Use `limit` based on your application's throughput requirements.
* Download results in **Parquet** format for large exports instead of retrieving many JSON batches.

## References

* [Query API reference](https://v2.dataos.info/references/interfaces/apis/data-product-apis/api-reference/query): full statement submission, polling, and result endpoints
* [Build applications with Semantic GraphQL](/consume/v1/recipes/query-semantic-model-with-graphql.md): a synchronous alternative for smaller, structured result sets


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://v2.dataos.info/consume/v1/recipes/retrieve-large-result-sets-with-pagination.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.