> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/releasenotes/september-2026.md). # September 2026 More control over platform operations, broader engine support, and tighter governance over how data gets shared. August extended the surface; September turns inward, toward platform operations and governance. Dataplane credentials now rotate in place, service and worker resources get full lifecycle control, and a new business glossary arrives with a dedicated Data Steward role to govern it. Vulcan adds beta Athena and Redshift support, and Nilus makes the CDC JVM heap configurable, both runtimes also move to Python 3.13 as part of ongoing hardening. Skip to the parts of this release that match what you do. | You are | Start with | | ----------------------- | ------------------------------------------------------------------------------------------------------------------ | | Building data products | [Vulcan](#vulcan), [Nilus](#nilus), [Behavior changes](#behavior-changes), [Known limitations](#known-limitations) | | Consuming data products | [Data Product Hub](#data-product-hub), [Activation](#activation) | | Operating the platform | [Platform](#platform), [Action required](#action-required), [Component version](#component-version) | | Working with AI agents | [Platform](#platform) | ### New features #### Platform * DataOS operators can now rotate a Dataplane's cloud credentials ([AWS](https://v2.dataos.info/operate/operator/provision-data-plane/aws-ui#rotate-credentials) or [Azure](https://v2.dataos.info/operate/operator/provision-data-plane/azure-ui#rotate-credentials)) in place, without deleting and recreating it, cutting the downtime a credential update used to cause. * [Runtime lifecycle management](https://v2.dataos.info/references/interfaces/command-line-interface/resource#dataos-ctl-resource-runtime-commands) (`stop`, `resume`, `re-run`), previously available only for workflows**,** now extends to `service` and `worker` resources. Pause or restart workloads on demand via CLI, with no manifest changes. * A new [platform API](https://v2.dataos.info/references/interfaces/apis/core-apis/api-reference/user-tokens) lets you create, list, retrieve, and delete your own API tokens programmatically instead of using the API Tokens panel or CLI, making token provisioning easy to automate in CI and onboarding. * A new [platform API](https://v2.dataos.info/references/interfaces/apis/core-apis/api-reference/observability-metrics) lets you query workload metrics from Instance Prometheus with PromQL. Use it for custom dashboards, or let apps and AI agents read runtime metrics directly. * [App links](https://v2.dataos.info/references/interfaces/apps/deploy-a-data-application) can now be set as a simple path (like `/mdc/app`) instead of a full `http:// or https://` URL, so the same App definition works unchanged as it moves across environments. * App secrets can now reference their tenant explicitly (`tenant:secret-name`), making it clear and auditable which secret an App depends on. * Visibility for Apps and Links can now target exact tenants or a name pattern, not just "my tenant" or "everyone". Share precisely without overexposing. * Lakehouse Spark cluster CPU/memory config moves to `sparkCluster.resources` (was `sparkCluster.server`), separating resource sizing from the unrelated server-mode setting. * The [`unquarantine`](https://v2.dataos.info/references/interfaces/command-line-interface/resource#dataos-ctl-resource-unquarantine) command now accepts multiple resource ID formats (`NAME|VERSION|TYPE`, `TYPE|VERSION|NAME`, `TYPE:VERSION:NAME`) and validates that the resource exists before proceeding. * Configure [`highAvailabilityConfig` ](https://v2.dataos.info/references/resources/worker/core-concepts#high-availability)(`mode: preferred/default`, `level: hostname/zone/region`) in a service or worker manifest to spread replicas across nodes, zones, or regions via pod anti-affinity, keeping workloads resilient during infrastructure disruptions. Set multiple replicas before enabling placement rules. * Dataplane delete moves under Manage with type-to-confirm, matching the tenant-delete pattern. This reduces the risk of an accidental deletion from the listing. * A new role, [Data Steward](https://v2.dataos.info/operate/data-steward/overview), is introduced, which holds exclusive rights to update business glossary. #### Vulcan * Vulcan adds beta support for [Athena](https://v2.dataos.info/build/productize/connect-engine/athena) and [Redshift](https://v2.dataos.info/build/productize/connect-engine/redshift) engine stacks, so teams can validate Athena- and Redshift-backed data products through the standard Vulcan workflow. * Vulcan APIs now support tenant-scoped authorization, ensuring users and services can access a data product only when they have permission for it within the correct tenant. * [External models](https://v2.dataos.info/references/resources/vulcan/models/data-models/types/external-models#writing-yaml-by-hand) support a new `remote_engine` field to record where the underlying data actually lives (e.g., Redshift behind a Trino model), for more accurate lineage and planning. * Vulcan's runtime moves to Python 3.13 with added compliance-readiness wiring, reducing exposure from older packages and aligning with modern compliance expectations. * [Data Quality checks](https://v2.dataos.info/references/resources/vulcan/quality/data-quality) move to a new `expectations:` format as part of a dependency-stack security update. Legacy `rules:` definitions keep working, so you can migrate gradually. * [Deployment resource config](https://v2.dataos.info/references/resources/vulcan/deployment) now standardizes on plural keys (`resources`, `requests`, `limits`); older singular keys still work but now show deprecation warnings. * Stronger [query result cache](https://v2.dataos.info/references/resources/vulcan/concepts/query-result-caching#keeping-results-current) freshness checks reduce stale reads for repeated queries, UI polling, and API consumers. #### Nilus * The CDC JVM heap is now configurable via [`NILUS_JVM_HEAP_PERCENT`](https://v2.dataos.info/references/resources/nilus/pipeline-optimization/optimizing-cdc#nilus_jvm_heap_percent) (default 25%), so large CDC snapshots and high-rate streams can be sized without a Nilus rebuild. * The Nilus runtime moves to Python 3.13, with fewer High-severity findings than `2.0.28`. It is more current and hardened. * [`database_filter` behavior](https://v2.dataos.info/references/resources/nilus/metadata-pipelines/pipeline-config#how-database_filter-is-evaluated) is now documented and consistent across all seven [metadata connectors](https://v2.dataos.info/references/resources/nilus/metadata-pipelines/metadata-sources). For deep runs using a depot- or URI-bound database, `database_filter` cannot change that database. It only narrows discovery within the existing catalog. Use `schema_filter` or `table_filter` for further scoping. #### Activation * DB clients now get upfront query validation with clear, actionable errors for unsupported SQL. You know what to fix instead of hitting a generic failure. * [Policy-aware warnings](https://v2.dataos.info/references/interfaces/apis/data-product-apis/querying-data-products/semantic-sql#access-policy-warnings) now flag empty query results caused by an access policy. They are visible to DB clients that support `SHOW WARNINGS`. #### Data Product Hub * The Perspective owner can now delete that Perspective from the details menu. * A new instance-level business glossary adds auto-populated, auto-linked terms from Data Product Hub assets and links them to matching products, models, and columns. Administrators assign the Data Steward role; Data Stewards can edit term descriptions from the glossary. * After a query succeeds, Share copies a permalink that reopens the same Studio session with the query and results already loaded. Share or resume work without re-running anything. * Clone in Studio reopens a Perspective's query and results in its native mode. Today, it's also how you "edit" a Perspective, by cloning and deleting the original. * From an input or output on the Assets tab, open that model directly in Studio's Source SQL, prefilled with `select * from limit 10;`. ### Bug fixes * Dependent Connect Dataplane and Compute resources no longer stay orphaned after their parent Dataplane is deleted. They are cleaned up correctly in both the CLI and the UI. * Query execution on Minerva clusters now surfaces actionable storage errors for misconfigured Depot or Lakehouse endpoints, instead of opaque internal metadata/storage exceptions. * Newly provisioned resources for a recreated tenant now correctly appear under active resources, instead of being hidden or shown under the deleted list. * Bundle resources no longer stay stuck in `pending` once their child resources are actually running. * Vulcan: Spark models using `INCREMENTAL_BY_UNIQUE_KEY` no longer overwrite existing Iceberg data during incremental runs. * Vulcan: external model discovery for Snowflake now finds source models referenced only through semantic models, so dependency tracking and planning include them. * Vulcan generates Trino container group resources only for Trino data products, avoiding unnecessary deployment resources for other engine stacks. * Vulcan repository secrets are now optional for public repositories. A `secret_id` is only required when the repository needs authentication. * Grain dimensions stay hidden by default but can now be explicitly exposed with `public: true`, so selected grain columns remain queryable in Visual Builder and Semantic SQL. * Data Product Hub: Assets → Inputs now shows quality checks, including pass or fail, when results exist; Quality, Schedule, and Last Run are omitted when there's nothing to show. * Data Product Hub: Quality → Details now lists every model with data quality applied. It previously stopped at 100 rules when more were in scope. * Data Product Hub: Quality → Details coverage now counts distinct models and columns correctly, so a covered count no longer exceeds the total. * Data Product Hub: an ABFSS Lakehouse overview now shows the correct account, container, format, and path fields instead of S3 fields. * Data Product Hub: Studio now warns when you've edited a query that already ran, prompting a rerun for the latest results. ### Behavior changes * The CDC JVM heap defaults to 25% of the pod memory limit unless `NILUS_JVM_HEAP_PERCENT` is set. * The Vulcan as well as Nilus's runtime is now based on Python 3.13 (was 3.11). * Vulcan's singular resource-config keys (`resource`, `request`, `limit`) still work but now emit deprecation warnings; use the plural `resources`/`requests`/`limits` form going forward. * Experimental anomaly detection checks and change-over checks have been deprecated. Existing Vulcan projects that use these checks should review their usage, and reach out to Product Management team. * Lakehouse Spark cluster CPU/memory configuration moves to `sparkCluster.resources`; if both `resources` and the old `server` location are set, `resources` wins. * App links set as full URLs still work today, but support for them will be removed in a future release. Switch to a path. * The `unquarantine` command now requires a resource identifier: running it with an empty ID prompts for one instead of acting across all resources. * Data Product Hub detail pages now show `{Entity name} · {Module name} · DataOS` in the browser tab. Other pages retain `{Module} · DataOS`. * Creating a tenant named `system` now fails with the same style of message as a name that already exists. * Dataplane delete now lives under Manage on the details page, not on the listing row. ### Action required Here's exactly what to check and change if you're upgrading to the September release.
If your CDC pipelines need more than 25% of pod memory for the JVM heap, set NILUS_JVM_HEAP_PERCENT The embedded Debezium JVM now defaults to 25% of the pod memory limit. No YAML change is required if the default is enough; to raise it, set the `NILUS_JVM_HEAP_PERCENT` environment variable on the pipeline: ```yaml projection: projections: envVars: - key: NILUS_JVM_HEAP_PERCENT template: "30" ```
If you run Snowflake deep runs with database_filter, set it to your intended database `database_filter` doesn't override a database already bound on the depot or URI. Left unset, classification and profiling cover every database already in the catalog. Set `database_filter` to the intended database to scope it correctly.
If you maintain custom Nilus or Vulcan extensions, validate them against Python 3.13 Nilus and Vulcan have moved from Python 3.11 to Python 3.13. Re-validate any custom Nilus source or destination connector, plus any Vulcan plugin, library, or automation that runs inside either runtime. Check startup logs and run a representative pipeline or plan before production rollout.
If your Lakehouse Spark manifests set CPU or memory under sparkCluster.server, move them to sparkCluster.resources Resource sizing now belongs under `sparkCluster.resources`. The previous `sparkCluster.server` location remains supported temporarily. If both locations are set, `sparkCluster.resources` wins. Review your manifests before upgrading. Move CPU and memory settings to `sparkCluster.resources` to avoid unexpected sizing after the deprecated field is removed.
If App open links use full URLs, replace them with paths Full URLs still work today, but support will be removed in a future release. Set App open links as paths so one manifest works across environments. For example, replace `https://example.dataos.app/dashboard` with `/dashboard`.
If your Vulcan deployments use singular resource keys, migrate to plural keys `resource`, `request`, and `limit` remain supported, but now emit deprecation warnings. Update deployment configuration to use `resources`, `requests`, and `limits`. ```yaml deployment: workflow: resources: requests: cpu: 200m memory: 512Mi limits: cpu: 1000m memory: 1Gi ``` Validate the updated manifest in CI before rollout. This prevents a future release from blocking deployment configuration.
If you maintain custom Data Quality checks in Vulcan, plan a move to the expectations: format Legacy `rules:` definitions still load and are translated automatically, so no immediate action is needed. New and updated DQ checks should use the expectation-based format going forward: ```yaml kind: dq name: customer_users_dq depends_on: customer.users expectations: - expectation: expect_column_values_to_not_be_null column: email ```
### Known limitations * Microsoft Fabric Table View may not load Delta tables that combine `TIMESTAMP_NTZ` with a non-nullable column such as a primary key, due to an upstream Delta protocol/metadata mismatch; Nilus ingestion and merge still succeed and the data stays queryable via SQL. * `database_filter` cannot expand or retarget metadata for a depot- or URI-bound database. Use `schema_filter`/`table_filter` to narrow within it, or an unbound Snowflake/Databricks URI when account- or workspace-level discovery is required. * A Perspective can currently only be edited by cloning it and deleting the original. * A deleted Perspective may still briefly appear in the listing. This is a known issue, with a fix planned for an upcoming release. * Studio Share stays disabled until a query succeeds; the link stores the statement ID, not the query text. * Open in Studio from the Assets tab is Source SQL only, and only available from inputs and outputs. ### Component version | Component | Version | Notes | | --------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------- | | Vulcan | `0.228.1.30` | Beta Athena/Redshift engine stacks, tenant-scoped API authorization, `remote_engine` metadata, Python 3.13 runtime. | | Nilus | `2.0.29` | Configurable CDC JVM heap, Python 3.13 runtime, consistent `database_filter` behavior. | | Data Product Hub | `0.0.100` | Studio Share and Clone, Source SQL from Assets, instance-level glossary, Dataplane delete relocation. | | Data Product MCP | `0.4.13` | no changes same as Aug version | | Data Product Builder Skills | `2.1.0` | Adds Athena and Redshift support, with engine-specific build guidance. | September sharpens platform control. Vulcan adds Athena and Redshift support, while Data Product Hub gains stronger governance. Tell us about your experience working with it and what you’d like to see next. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://v2.dataos.info/releasenotes/september-2026.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.