For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to page URLs.
Skip to content

Batch sources

Index of Nilus batch source connectors by category (database vs. SaaS API), linking to each connector's requirements, options, and behavior.

Batch sources read data on a scheduled or manually triggered cadence. Use them for full refreshes, incremental snapshots, and bounded query-based extraction.

SQL batch sources that accept source_table: "query:..." share one extract path. On a DataOS Lakehouse destination, table reads and query: write different Iceberg decimal types. See custom query (SQL sources) and schema evolution.

For batch manifest shape, see batch pipeline config. For worked examples, see batch sample configs.

Published batch sources

Source
Category
Notes

Database

Warehouse reads through the Redshift/PostgreSQL-compatible path.

Database

Supported through the MS SQL Server batch source against a compatible dedicated SQL pool.

Database

Batch reads through Databricks SQL warehouses.

Database

Batch reads from Delta table storage.

SaaS API

Batch API extraction from Google Analytics.

Database

Batch reads from BigQuery datasets and tables.

SaaS API

Batch extraction from spreadsheets.

SaaS API

Batch API extraction from HubSpot resources.

Database

Batch extraction from collections or aggregation output.

Database

Batch reads from SQL Server tables and views.

Database

Batch reads from MySQL tables and views.

Database

Batch reads from PostgreSQL tables and views.

SaaS API

Batch API extraction from Salesforce objects.

Database

Batch reads from Snowflake tables, views, or queries.

SaaS API

Batch API extraction from Stripe resources.

Last updated

Was this helpful?