> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/references/dataos-resources/vulcan/core-concepts/semantic-model-and-components.md). # Semantic model and components The semantic model sits between your physical models and the people who consume them. It maps technical objects (tables, columns, joins) to business concepts like "revenue," "active users," or "churn rate," so everyone queries data with the same vocabulary without understanding the underlying schema. The same definition runs in every BI tool, notebook, and API, so two dashboards never disagree because they wrote slightly different `SUM(...)` expressions. The semantic layer has two pieces that build on each other: semantic models first, then metrics on top. * **Semantic models** wrap a physical Vulcan model and expose dimensions, measures, segments, and joins. * **Metrics** combine a measure with a time column and dimensions to create a complete, time-series analytical definition. Vulcan validates both when you create a plan (`vulcan plan` triggers semantic validation), so a typo in a measure expression fails the plan instead of quietly returning zeros in a dashboard. ## Where the files live Co-locate semantic models with the physical models they wrap, and give each metric its own file: ``` project/ ├── models/ │ ├── customers.sql │ ├── orders.sql │ ├── semantics/ # Semantic models (kind: semantic) │ │ ├── customers.yml │ │ └── orders.yml │ └── metrics/ # Per-metric files (kind: metric) │ ├── arr_growth.yml │ └── churn_analysis.yml └── config.yaml ``` The filename does not matter; Vulcan merges all YAML files in `models/semantics/` and `models/metrics/`. Use one semantic model and one metric per file. {% hint style="info" %} If you still have a top-level `semantics/` directory next to `models/`, move those files under `models/semantics/` as part of the OSI GA migration. {% endhint %} {% hint style="success" %} **Case-sensitive engines.** Snowflake stores unquoted identifiers in uppercase by default. When targeting Snowflake, use uppercase column names in dimension lists, expressions, and filters. Lowercase examples here assume a case-insensitive engine such as DuckDB or Postgres. {% endhint %} ## Semantic models A semantic model wraps a single physical model with `kind: semantic` and `depends_on:`. `dimensions` is required and must be non-empty. ```yaml kind: semantic name: users # Business-friendly name used in queries depends_on: b2b_saas.users # Fully qualified Vulcan model this wraps description: Core user dimension (semantic layer) dimensions: [...] measures: [...] segments: [...] joins: [...] ``` | Field | Required | Description | | ----------------------------------------------------- | :------: | -------------------------------------------------------------------------------- | | `kind: semantic` | Yes | Declares the file as a semantic model. | | `name` | Yes | Business-friendly identifier used in `{name.column}` references everywhere else. | | `depends_on` | Yes | The fully qualified Vulcan model this wraps. | | `dimensions` | Yes | Non-empty list of dimensions. | | `measures`, `segments`, `joins`, `policies` | No | Aggregations, reusable filters, relationships, and access policies. | | `description`, `owner`, `tags`, `terms`, `ai_context` | No | Documentation and discovery metadata. | ### Naming rules Vulcan validates every identifier. Semantic model `name`, measure `name`, segment `name`, join `name`, granularity `name`, and metric `name` must match `^[a-z][a-z0-9_]{0,63}$` (lowercase, starts with a letter, underscores allowed, up to 64 characters). A dimension `name` (a column reference) may be mixed case (`^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`) so warehouse identifiers like `CUSTKEY` survive. | Identifier | Pattern | Notes | | ---------- | -------------------- | ------------------------------------------------------------------------------------------------------- | | `tags[*]` | `^[a-zA-Z0-9.:_-]+$` | Supports `key:value` patterns, e.g. `classification:PII`. | | `terms[*]` | `^[a-zA-Z0-9._-]+$` | Business-glossary references, typically dotted FQNs, e.g. `revenue.subscription` or `glossary.revenue`. | Unknown keys inside `ai_context`, `semantic_config`, `rolling_window`, and `granularities` fail validation. ### Dimensions `dimensions` is a required, non-empty list. Each item is a bare column name (shorthand) or a dict with metadata, semantic typing, granularities, masking, and formatting. ```yaml dimensions: - plan_type - status - name: signup_date description: When the user signed up tags: [temporal, acquisition] granularities: - name: day interval: 1 day - name: month interval: 1 month - name: user_id semantic_config: type: identifier # identifier | categorical - name: avg_order_value format: currency # free-form display hint, e.g. percent, currency ``` Attach `granularities` (each with `name` and `interval`, like `15 minutes` or `1 month`) on time-like dimensions. An `interval` is any positive quantity of `minute`, `hour`, `day`, `week`, `month`, or `year` (for example, `15 minutes`, `3 months`, `1 year`); granularities are only meaningful on `TIMESTAMP`/`DATETIME` columns. Each granularity entry also accepts `description` (string) and `ai_context` (object); granularity `name` values must be unique within the dimension. Set `semantic_config.type` to `identifier` (keys, IDs) or `categorical` (enums, status, grouping columns). Use `mask_expression` to control what restricted users see (see [Policies](#policies-masking-and-auth-context)). | Property | Required | Default | Description | | ----------------- | :------: | :-----: | ------------------------------------------------------------------------------------- | | `name` | Yes | — | Column name in the underlying model. Mixed case allowed (e.g. `CUSTKEY`). | | `description` | No | — | Human-readable explanation. | | `tags` | No | — | Categorization labels. | | `terms` | No | — | Business glossary references. | | `granularities` | No | — | List of time buckets. Only meaningful on `TIMESTAMP`/`DATETIME` columns. | | `format` | No | — | Free-form display hint (e.g. `percent`, `currency`). | | `semantic_config` | No | — | Semantic type: `identifier` or `categorical`. | | `mask_expression` | No | — | SQL expression returned when a policy masks this dimension. | | `ai_context` | No | — | Hints for AI/LLM consumers. | | `public` | No | `true` | Whether this dimension is exposed via the API. Set `false` to hide it from consumers. | ### Measures `measures` is a list of named aggregations. Reference columns with `{name.column}`, where `name` is the semantic model's `name`. ```yaml measures: - name: total_users type: count expression: "{users.user_id}" description: Total registered users - name: active_users type: count filters: - "{users.status} = 'active'" - name: avg_mrr_per_account type: avg expression: "{subscriptions.mrr}" filters: - "{subscriptions.status} = 'active'" ``` Measure types: `count`, `count_distinct`, `count_distinct_approx`, `sum`, `avg`, `min`, `max`, `number`, `string`, `time`, `boolean`. `expression` is required for every type except `count`, where omitting it (or using `"*"`) counts rows, and a `{name.column}` reference counts non-null values. `filters` are allowed on `count`, `count_distinct`, `count_distinct_approx`, `sum`, `avg`, `min`, and `max`, and never on `number`. | Property | Required | Default | Description | | ----------------- | :------: | :-----: | ---------------------------------------------------------------------------------------- | | `name` | Yes | — | Lowercase identifier. Must be unique among measures and segments in this semantic model. | | `type` | Yes | — | Aggregation type (see list above). | | `expression` | Cond. | — | Column reference or SQL expression. Required for every type except `count`. | | `filters` | No | — | SQL conditions restricting which rows are aggregated. Not allowed on `number`. | | `description` | No | — | Human-readable explanation. | | `tags` | No | — | Categorization labels. | | `terms` | No | — | Business glossary references. | | `rolling_window` | No | — | Sliding-window config (`trailing`, `leading`, `offset`). | | `semantic_config` | No | — | Semantic behavior (`simple`, `flow`, `stock`, `ratio`). | | `ai_context` | No | — | Hints for AI/LLM consumers. | | `public` | No | `true` | Whether this measure is exposed via the API. Set `false` to hide it from consumers. | {% hint style="warning" %} `count` is a reserved measure name. Vulcan adds an implicit `count` measure automatically, so use a name like `total_users` or `row_count`. {% endhint %} Set `semantic_config.type` on measures to declare behavior: `simple` (additive), `flow` (accumulates over time), `stock` (point-in-time value such as ARR or active users), or `ratio` (numerator over denominator). A `stock` measure adds `time_dimension`, `period_treatment: last`, and `period_grain: day` to pin the value to a point in time: ```yaml - name: total_arr type: sum expression: "{subscriptions.arr}" semantic_config: type: stock time_dimension: start_date period_treatment: last period_grain: day ``` A `ratio` measure references two other measures with `numerator:` and `denominator:`: ```yaml - name: churn_rate type: number semantic_config: type: ratio numerator: churn_count denominator: subscription_count ``` Attach a `rolling_window` (with `trailing`, `leading`, and `offset`) for sliding-window calculations. `trailing` and `leading` each accept `unbounded` or a signed duration (same grammar as granularity intervals, e.g. `7 days`, `-7 days`); `unbounded` removes the bound on that side. `offset` accepts `start` or `end` (default `end`) and controls whether the window is anchored to the start or end of the bucket. ```yaml - name: trailing_7d_revenue type: sum expression: "{subscriptions.mrr}" rolling_window: trailing: 7 days offset: end ``` ### Segments Segments are reusable filter conditions defined once and applied everywhere. `expression` must reference only columns of the current semantic model. Each segment entry also accepts `description`, `tags`, `terms`, `ai_context`, and `public`. ```yaml segments: - name: high_value_accounts expression: "{users.plan_type} IN ('pro', 'enterprise')" description: Paid plan users tags: [revenue, segment] ai_context: instructions: Use to restrict queries to paid-plan rows only. - name: recent_signups expression: "{users.signup_date} >= CURRENT_DATE - INTERVAL '7 days'" ``` Measure and segment names must be unique within a single semantic model. | Property | Required | Default | Description | | ------------- | :------: | :-----: | ----------------------------------------------------------------------------------- | | `name` | Yes | — | Lowercase identifier. Must be unique among measures and segments. | | `expression` | Yes | — | SQL boolean condition referencing only columns of the current semantic model. | | `description` | No | — | Human-readable explanation. | | `tags` | No | — | Categorization labels. | | `terms` | No | — | Business glossary references. | | `ai_context` | No | — | Hints for AI/LLM consumers. | | `public` | No | `true` | Whether this segment is exposed via the API. Set `false` to hide it from consumers. | {% hint style="info" %} Per-entry metadata (`description`, `tags`, `terms`, `ai_context`) is supported on segments defined in a semantic model but is not overridable per metric. To change segment metadata, edit the underlying semantic model. {% endhint %} ### Joins Joins define relationships so you can analyze across tables. The join `name` must match the `name` of another semantic model, and must not equal the current model's own name. ```yaml joins: - name: subscriptions type: one_to_many # one_to_one | one_to_many | many_to_one expression: "{users.user_id} = {subscriptions.user_id}" ``` `many_to_many` is not supported; model it through an intermediate bridge semantic model and chain two joins. Each join entry also accepts `ai_context` (object). Joins do not accept `description`, `tags`, `terms`, or `public`; extra keys fail validation. A join may also carry an optional `fqn` (the fully-qualified name of the join target); it is engine-set and rarely authored by hand. Once a join exists, a measure filter can reference columns from the joined model, and Vulcan resolves the join path automatically. ### AI context Most spec objects accept an optional `ai_context` block with `instructions`, `caveats`, `synonyms`, and `examples` to help AI and large language model (LLM) consumers interpret a measure, dimension, or model correctly (for example, "ARR and MRR are point-in-time values; do not sum them across dates"). Unknown keys fail validation. | Field | Type | Description | | -------------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `instructions` | String **or** list of strings | Free-form guidance for how to think about this object. Use a list when multiple instructions apply. | | `caveats` | List of strings | Warnings about aggregation behavior, time-grain assumptions, business interpretation, or ways the field can be misused. | | `synonyms` | List of strings | Alternate names consumers or LLMs might use when referring to this object. | | `examples` | List of objects | Example queries. Each example can include `description`, `format` (`sql`, `rest`, `graphql`), and `query`. | ## Metrics A metric is a time-series analytical definition built on top of semantic models. Use `kind: metric`, one metric per file under `models/metrics/`. Reference fields as `.`. ```yaml # models/metrics/monthly_revenue.yml kind: metric name: monthly_revenue measure: orders.total_revenue ts: orders.ORDER_DATE granularity: month dimensions: - customers.CUSTOMER_TIER - orders.REGION description: Monthly revenue by customer tier and region tags: [revenue, financial] ``` Every metric defines four required fields: | Field | Description | | ------------- | ---------------------------------------------------------------------------------------------- | | `name` | Unique identifier consumers use. | | `measure` | A measure on a semantic model, as `.`. | | `ts` | A time/date column, as `.`. Cannot equal `measure`. | | `granularity` | Default time bucket: `second`, `minute`, `hour`, `day`, `week`, `month`, `quarter`, or `year`. | Consumers can override the granularity at query time, so you define a metric once and query it daily, weekly, or monthly without separate definitions. Optional top-level metric fields: | Field | Type | Description | | ------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `description` | String | Human-readable explanation of the metric. | | `tags` | List of strings | Categorization labels for discovery. | | `owner` | String | Team or person responsible for this metric. | | `terms` | Array of strings | Glossary term references. Same pattern as semantic model `terms` (e.g. `glossary.revenue`). | | `ai_context` | Object | Hints for AI/LLM consumers. Accepts `instructions` (string or list of strings), `synonyms`, `examples`, and `caveats` — same structure as field-level `ai_context`, scoped to this metric. | `dimensions` items are a bare reference (the name auto-derives from the part after the `.`) or a named slice (`name` plus `ref`, with optional overriding metadata). If two bare references derive the same name from different refs, validation fails; switch to named slices. A named slice's `description`, `tags`, `terms`, and `ai_context` override the underlying semantic field's values, scoped only to this metric's view; other metrics referencing the same column still see what the semantic model defines. `segments` are qualified-reference strings only (`.`), with no per-entry metadata. Each segment's name auto-derives as `_` (lowercased) and must not collide with any dimension name on the metric. ```yaml kind: metric name: arr_growth measure: subscriptions.total_arr ts: subscriptions.start_date granularity: month dimensions: - subscriptions.plan_type - name: industry ref: users.industry segments: - subscriptions.active_subscriptions ``` A metric can pull its measure, ts, and dimensions from different semantic models, as long as those models are connected through joins. Vulcan validates that every reference resolves, that names are unique across dimensions and segments, and that cross-model references have valid join paths. **Forbidden legacy keys** — these cause validation failure. Do a global find-and-replace before running `vulcan plan`: | Forbidden key | Replace with | | --------------------------------- | ------------ | | `time` (on a dimension or metric) | `ts` | | `slices` (top-level) | `dimensions` | **Reserved names** — the following cannot be used as a dimension `name` (object form) or as an auto-derived segment name: * `measure` * `time` * `ts` A segment whose auto-derived name `_` collides with a reserved name also fails validation. ## Policies, masking, and auth context Semantic files can define row and column access policies close to the model they protect. Policies match against the user's resolved security context, usually populated after Heimdall authorization by a root-level `after_authorize` hook in `config.yaml`. ```yaml after_authorize: "plugins.auth_ext:resolve_user_groups" ``` The `after_authorize` config value follows the format `".:"`. For example, `"plugins.auth_ext:resolve_user_groups"` means: package = `plugins`, module = `auth_ext`, function = `resolve_user_groups`. The hook lives in a project-level `plugins/` package: ``` plugins/ ├── __init__.py └── auth_ext.py ``` {% hint style="warning" %} The `__init__.py` file can be empty, but it is required. Without it, the import fails. {% endhint %} `AuthExtensionContext` is the input object passed to the hook and `SecurityContext` is the output object it returns. The hook reads Heimdall role tags from `ctx.user_tags` (tags like `roles:id:operator`) and returns a `SecurityContext` with a primary `group` and a comma-separated `groups` string. Key constants used in the hook: * **`GROUP_DELIMITER`**: The `,` character used to join the resolved groups into the `groups` string. Users who customize the hook need to use this same delimiter. * **`POLICY_GROUP_PRIORITY`**: A tuple that controls which group becomes primary when a user has multiple roles. Example: `POLICY_GROUP_PRIORITY = ("operator", "developer")` — if a user has both `operator` and `developer`, `operator` is selected as the primary group because it appears first in the tuple. ```python from schema.auth import AuthExtensionContext, SecurityContext ROLE_ID_TAG_PREFIX = "roles:id:" GROUP_DELIMITER = "," POLICY_GROUP_PRIORITY = ("operator", "developer") async def resolve_user_groups(ctx: AuthExtensionContext) -> SecurityContext: groups = [ tag.replace(ROLE_ID_TAG_PREFIX, "", 1) for tag in ctx.user_tags if tag.startswith(ROLE_ID_TAG_PREFIX) ] group = next( (g for g in POLICY_GROUP_PRIORITY if g in groups), groups[0] if groups else "", ) return SecurityContext(group=group, groups=GROUP_DELIMITER.join(groups)) ``` Policies then use the resolved `group` to decide row and column access: ```yaml policies: - group: developer - group: operator mask: - email - customer_name filter: - member: customer_segment operator: notEquals values: - Churned ``` Here `developer` has full access; `operator` can query the model but cannot see raw `email` or `customer_name`, and cannot see rows where `customer_segment = Churned`. Each masked dimension should define a `mask_expression` (for example, `"CAST(NULL AS TEXT)"` or `"'***'"`) that controls what restricted users see. {% hint style="warning" %} For auth-backed policies or masking, make sure `config.yaml` includes the root-level `after_authorize` hook. Keep Heimdall connection settings under `heimdall` and put the extension hook at the root. {% endhint %} ## Querying the semantic layer Consumers query the semantic layer through REST, GraphQL, and a SQL (MySQL wire protocol) API, all backed by the same definitions. Vulcan transpiles a semantic query into engine SQL: business-friendly names in, warehouse-specific SQL out. Preview and validate that SQL locally with `vulcan transpile` before sending a query to the API. It accepts a semantic SQL query or a REST-style JSON payload: ```bash # Semantic SQL: MEASURE() wraps measures, FROM uses the semantic name vulcan transpile --format sql "SELECT users.plan_type, MEASURE(total_users) FROM users GROUP BY users.plan_type" # REST payload vulcan transpile --format json '{"query": {"measures": ["users.total_users"], "dimensions": ["users.plan_type"]}}' ``` For a semantic SQL query, wrap measures in `MEASURE()`, reference the semantic `name` (not the physical table) in `FROM`, and use `CROSS JOIN` to combine models (Vulcan infers the join condition). Segments only support `= true`. Transpiling catches errors before execution and shows exactly what SQL would run, which is useful for debugging unexpected results and for tuning join and filter placement. For the full asynchronous query path (submit, transpile, cache, execute, fetch) and the REST, GraphQL, and SQL endpoints, see [Observability](/references/dataos-resources/vulcan/observability.md) and the [semantic query lifecycle guide](/references/dataos-resources/vulcan/incremental-by-time/semantic-query-lifecycle.md). ## Importing existing semantic views If you already have a semantic view defined natively in Snowflake, you can import it instead of writing `kind: semantic` YAML by hand. Vulcan reads the view definition, translates it to the Vulcan semantic format, and writes one YAML file per table. See the [import Snowflake semantic views guide](/references/dataos-resources/vulcan/incremental-by-time/import-snowflake-semantic-views.md). ## Related docs * [Model and kinds](/references/dataos-resources/vulcan/core-concepts/model-and-kinds.md): the physical models semantic models wrap. * [Semantic query lifecycle guide](/references/dataos-resources/vulcan/incremental-by-time/semantic-query-lifecycle.md): how a query travels from request to result. * [Import Snowflake semantic views guide](/references/dataos-resources/vulcan/incremental-by-time/import-snowflake-semantic-views.md): generate semantic models from an existing view. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://v2.dataos.info/references/dataos-resources/vulcan/core-concepts/semantic-model-and-components.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.