> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/build/july2026/recipes/mask-sensitive-columns-by-role.md).

# Mask sensitive columns in a semantic model by role

Mask sensitive columns in a physical model per caller role using an access policy, column\_mask\_expressions, and an auth extension that maps DataOS roles to policy groups: no forked pipelines or models

A semantic model usually has more than one audience. An operator might need to query customer activity without ever seeing raw email addresses or names; a developer debugging the same pipeline needs the real values. The naive fix is two models, or two pipelines, one masked and one not, which means two things to keep in sync every time either changes.

Vulcan avoids the fork: a single semantic model can serve both audiences differently at query time. Row filters and column masks are authored once, against the **physical model** the semantic model wraps, as a policy under `policies/access/`. The semantic model inherits that policy automatically; it never authors access rules itself. The bridge between "who is calling" and "which policy group applies" is an `after_authorize` auth extension you write once per project.

{% hint style="warning" %}
Configure the auth extension (steps 1–2) before writing the policy. A policy rule matches against a resolved `group`. With no extension wired, there's nothing for it to match against.
{% endhint %}

## Before you begin

{% hint style="info" %}

* A physical model with the sensitive columns present as columns (e.g. `email`, `customer_name`), wrapped by a semantic model that exposes them as dimensions.
* Know your DataOS role taxonomy: the role tags (`roles:id:operator`, `roles:id:developer`, etc.) callers actually carry.
* Access to two identities (or bearer tokens) with different roles, to confirm the masking behaves differently per caller once deployed.
  {% endhint %}

## Steps

### 1. Write the auth extension hook

Create `plugins/__init__.py` (empty: it just marks the folder as a package) and `plugins/auth_ext.py`. The hook reads DataOS role tags off the request and resolves them into a `SecurityContext`.

```python
# plugins/auth_ext.py
from __future__ import annotations

from schema.auth import AuthExtensionContext, SecurityContext

ROLE_ID_TAG_PREFIX = "roles:id:"
GROUP_DELIMITER = ","
POLICY_GROUP_PRIORITY = ("operator", "developer")


async def resolve_user_groups(ctx: AuthExtensionContext) -> SecurityContext:
    """Derive policy groups from DataOS role tags."""

    groups = [
        tag.replace(ROLE_ID_TAG_PREFIX, "", 1)
        for tag in ctx.user_tags
        if tag.startswith(ROLE_ID_TAG_PREFIX)
    ]

    group = next(
        (policy_group for policy_group in POLICY_GROUP_PRIORITY if policy_group in groups),
        groups[0] if groups else "",
    )
    return SecurityContext(group=group, groups=GROUP_DELIMITER.join(groups))
```

`ctx.user_tags` carries the caller's DataOS role tags (e.g. `roles:id:operator`). `POLICY_GROUP_PRIORITY` decides which group wins when a caller holds more than one role. Reorder it to match how your roles should rank. The returned `group` is what policies match against; `groups` is the full comma-separated list, useful if a policy ever needs to check for more than one role at once.

### 2. Wire the hook in `config.yaml`

```yaml
after_authorize: "plugins.auth_ext:resolve_user_groups"
```

The value is `<package>.<module>:<function>`. Vulcan imports `plugins.auth_ext` and calls `resolve_user_groups` after DataOS authorizes each request.

### 3. Define the mask on the physical model

Masking is defined on the **physical model**, not the semantic model. Add a `column_mask_expressions` entry for every column you plan to mask, in the `MODEL(...)` block of the physical model the semantic model wraps:

```sql
-- silver.dim_customer_profile
MODEL (
  name silver.dim_customer_profile,
  column_mask_expressions (
    customer_name = CAST(NULL AS TEXT),
    email = CAST(NULL AS TEXT)
  )
);
```

Pick `NULL` when downstream consumers should treat the field as absent, or a fixed string like `'***'` when they need a visibly-redacted placeholder instead. Every column a policy rule later masks must have a matching entry here, or `vulcan plan` fails with `column_mask_expression is missing for proposed masking on column <name>`.

### 4. Add an access policy for the physical model

Create a policy file under `policies/access/`, one file per physical model that needs access control. Set `type: policy` and a single `depends_on` entry pointing at the physical model. Pointing `depends_on` at a semantic or metric model instead fails outright with `attach policy to a physical model only`.

```yaml
# policies/access/customer_profile_access.yml
type: policy
name: customer_profile_access
depends_on:
  - silver.dim_customer_profile

rules:
  - group: developer

  - group: operator
    mask:
      - email
      - customer_name
    filter:
      - member: customer_segment
        operator: notEquals
        values:
          - Churned
```

A rule with only `group` (here, `developer`) grants that group unrestricted access. `group` is required and must be lowercase snake\_case, matching `^[a-z][a-z0-9_]{0,63}$`. `mask` takes only bare column names; the actual masking SQL lives in `column_mask_expressions` on the physical model (step 3), not here. A caller resolving to `operator` can query the model, but `email` and `customer_name` come back masked, and rows where `customer_segment = 'Churned'` are filtered out entirely.

The semantic model itself doesn't declare any of this. It wraps the physical model with dimensions as usual, nothing more:

```yaml
# models/semantics/customer_profile.yml
kind: semantic
name: customer_profile
depends_on: silver.dim_customer_profile
description: Customer profile and lifetime purchase behavior.

dimensions:
  - name: customer_id
    behavior:
      type: identifier
  - name: customer_name
  - name: email
  - name: customer_segment
    behavior:
      type: categorical
  - total_orders
  - total_revenue

measures:
  - name: total_customers
    type: count_distinct
    expression: "{customer_profile.customer_id}"
    behavior:
      type: simple
```

{% hint style="warning" %}
Don't add a `policies:` block or a dimension-level `mask_expression` to the semantic model. Inline `policies:` on a semantic model is ignored with a warning, and a dimension's `mask_expression` is stripped at load time. Both live on the physical model and its policy file instead.
{% endhint %}

At export time, Vulcan resolves `customer_profile`'s physical anchor (`silver.dim_customer_profile`) and pulls that anchor's policy and `column_mask_expressions`, so the semantic layer reflects the physical model's access rules without redeclaring them.

Run `vulcan plan` to confirm the policy and physical model validate: Vulcan checks that every policy group, masked column, and filter reference resolves to something real.

{% hint style="info" %}
**Rollups only allow constant masks**

If the project sets `enable_rollup: true` and a dimension is used as a rollup group-by column, that column's `column_mask_expressions` entry must be a **constant** expression, such as `CAST(NULL AS TIMESTAMP)` or a string literal. A value-referencing mask like `CONCAT(LEFT(email, 2), '***')` can't be safely re-derived from already-aggregated rollup data, so Vulcan rejects it for rollup dimensions. See [Policies](https://v2.dataos.info/references/resources/vulcan/policies#rollups).
{% endhint %}

### 5. Confirm the same query returns different results per role

Once deployed, query the model with two identities that resolve to different groups. Same query, two bearer tokens:

```bash
# As an operator
curl --location 'https://<env-fqn>/<tenant>/vulcan/<data-product-name>/api/v1/query' \
  --header 'Authorization: Bearer <operator-token>' \
  --header 'Content-Type: application/json' \
  --data '{
    "query": {
      "dimensions": ["customer_profile.customer_id", "customer_profile.customer_name", "customer_profile.email"],
      "measures": ["customer_profile.total_customers"]
    }
  }'
```

```json
{
  "data": [
    { "customer_profile.customer_id": "42", "customer_profile.customer_name": null, "customer_profile.email": null, "customer_profile.total_customers": 1 }
  ]
}
```

```bash
# As a developer: identical query, different token
curl --location 'https://<env-fqn>/<tenant>/vulcan/<data-product-name>/api/v1/query' \
  --header 'Authorization: Bearer <developer-token>' \
  --header 'Content-Type: application/json' \
  --data '{
    "query": {
      "dimensions": ["customer_profile.customer_id", "customer_profile.customer_name", "customer_profile.email"],
      "measures": ["customer_profile.total_customers"]
    }
  }'
```

```json
{
  "data": [
    { "customer_profile.customer_id": "42", "customer_profile.customer_name": "Jordan Lee", "customer_profile.email": "jordan.lee@example.com", "customer_profile.total_customers": 1 }
  ]
}
```

Same semantic model, same query, no forked pipeline: the caller's resolved role is the only thing that changed the response.

{% hint style="warning" %}
Querying via `vulcan fetchdf` or a direct warehouse connection bypasses this entirely: policies are enforced at the API/query layer, where the auth context actually exists. Test masking against a real deployed endpoint, not a local `fetchdf` call.
{% endhint %}

## Common validation failures

* **Policy `depends_on` doesn't resolve to a physical model.** Fails with `attach policy to a physical model only`. Point `depends_on` at the physical model backing the semantic model, not at the semantic or metric model itself.
* **A masked column has no matching `column_mask_expressions` entry.** `vulcan plan` fails with `column_mask_expression is missing for proposed masking on column <name>`. Add the expression to the physical model's `MODEL(...)` block before deploying.
* **`after_authorize` isn't configured.** No group ever resolves, so policy rules beyond an unrestricted one have nothing to match against. Confirm step 2 is in place first.
* **`group` isn't lowercase snake\_case.** Every rule's `group` must match `^[a-z][a-z0-9_]{0,63}$`.

## Best practices

* Keep policy groups aligned 1:1 with your role taxonomy so a caller with multiple roles resolves to one predictable group via `POLICY_GROUP_PRIORITY`, not an ambiguous one.
* Give every masked column an explicit `column_mask_expressions` entry on the physical model. Restricted callers should get a defined placeholder, never an error.
* Test with two real identities against a deployed endpoint, not just by reading the YAML. Policy bugs usually only show up at query time.
* Keep `plugins/auth_ext.py` narrow: it derives groups from role tags, nothing more. Masking and filtering rules belong in the physical model's `column_mask_expressions` and the policy file under `policies/access/`, not in the auth hook.
* If a masked column is also a rollup group-by dimension, keep its `column_mask_expressions` value constant (see the rollup hint in step 4).

## Troubleshooting

| Symptom                                                                                            | Likely cause                                                                                            | Fix                                                                                                                                  |
| -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Every caller sees unmasked data, regardless of role                                                | `after_authorize` isn't configured in `config.yaml`, so no group ever resolves                          | Add `after_authorize: "plugins.auth_ext:resolve_user_groups"` and confirm `plugins/` imports cleanly                                 |
| `vulcan plan` fails with `attach policy to a physical model only`                                  | The policy's `depends_on` points at a semantic or metric model instead of a physical model              | Point `depends_on` at the physical model backing the semantic model                                                                  |
| `vulcan plan` fails with `column_mask_expression is missing for proposed masking on column <name>` | A policy rule's `mask` lists a column with no matching `column_mask_expressions` entry                  | Add the expression to the physical model's `MODEL(...)` block                                                                        |
| A user with two roles gets the wrong access level                                                  | `POLICY_GROUP_PRIORITY` resolves to the wrong role first                                                | Reorder `POLICY_GROUP_PRIORITY` in `auth_ext.py`                                                                                     |
| A filter policy doesn't hide the expected rows                                                     | The filter's `member`/`operator`/`values` don't match the real data (case, exact string)                | Re-check the filter block against actual column values                                                                               |
| Masking "works" in local testing but not in production                                             | Local test queried via `fetchdf`/direct connection, bypassing the API auth layer                        | Re-test against the deployed API endpoint with real tokens                                                                           |
| A rollup dimension's mask is rejected at plan time                                                 | `column_mask_expressions` for that column references a value (e.g. `CONCAT(...)`) instead of a constant | Use a constant expression, such as `CAST(NULL AS TIMESTAMP)` or a string literal, for any column used as a rollup group-by dimension |

## Related recipes

* [Route plan and run failures to Slack or Teams](https://v2.dataos.info/build/july2026/recipes/route-failures-to-slack-or-teams.md) - wire the same project to alert on a `vulcan plan` failure, including one caused by an invalid policy.

## References

* [Policies](https://v2.dataos.info/references/resources/vulcan/policies) - `policies/access/*.yml`, `column_mask_expressions`, rule schema, rollup constraints, and physical-model-only inheritance.
* [Plugins and auth](https://v2.dataos.info/references/resources/vulcan/plugins-and-auth) - the `plugins/` folder, `after_authorize`, `AuthExtensionContext`, and `SecurityContext`.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/build/july2026/recipes/mask-sensitive-columns-by-role.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
