> For the complete documentation index, see [llms.txt](https://v2.dataos.info/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://v2.dataos.info/build/july2026/recipes/wait-for-data-with-signals.md).

# Wait for data to be ready with signals

Make a scheduled Vulcan model wait for late-arriving source data instead of running on a fixed clock and computing an incomplete result.

Vulcan normally decides whether to run a model using two checks: has the `cron` schedule elapsed, and have upstream dependencies finished. Neither check confirms the data itself has actually arrived. If a daily model is scheduled for 12:15 and the source system's batch job runs late, the model still fires at 12:15 and processes an incomplete window.

A signal adds a third check: a Python function that asks the source system directly, "is today's data here yet?" before the model runs. Use this recipe when a model depends on an external system whose delivery time isn't guaranteed by the clock alone (for example, a daily sales rollup that must wait for that day's orders to land before it aggregates them).

{% hint style="info" %}
A signal only adds a readiness check. It does not replace `cron` or upstream dependencies; all three must pass before Vulcan evaluates an interval. If a model defines more than one signal, every signal must agree the interval is ready; Vulcan treats them as an AND, not an OR.
{% endhint %}

## Steps

### 1. Write the signal function

Create (or add to) `signals/__init__.py` at the project root. Define a function that checks whether the source data exists for the interval Vulcan wants to process.

```python
from vulcan import signal, DatetimeRanges, ExecutionContext


@signal()
def data_available_for_today(batch: DatetimeRanges, context: ExecutionContext) -> bool:
    overall_start = min(s for s, _ in batch)
    overall_end = max(e for _, e in batch)

    query = f"""
        SELECT COUNT(*) AS cnt
        FROM vulcan_demo.orders
        WHERE order_date::timestamp BETWEEN '{overall_start}' AND '{overall_end}'
    """
    df = context.engine_adapter.fetchdf(query)
    count = int(df.iloc[0]['cnt'])
    return count > 0
```

Vulcan can pass several intervals at once as `batch` (the model's `batch_size` controls how many). This function collapses them into one overall window and asks the source table: "does at least one row exist in this window?" `context.engine_adapter.fetchdf` runs the query against the live source, the same mechanism a Python model uses to pull data.

The function must return a single `True`/`False` for the whole batch, or a filtered list of the ready intervals, never a per-row result.

{% hint style="warning" %}
Start with the simple `True`/`False` version above. Only return a filtered list of intervals (see step 4) once a batch can contain some ready days and some that aren't.
{% endhint %}

### 2. Wire the signal into the model

Add a `signals (...)` block to the model's `MODEL(...)` definition, calling the function by name.

```sql
MODEL (
  name vulcan_demo.incremental_by_time_range,
  kind INCREMENTAL_BY_TIME_RANGE (
    time_column order_date
  ),
  start '2025-01-01',
  grains (
    order_date,
    product_id
  ),
  cron '15 12 * * *',
  signals (
    data_available_for_today()
  ),
  audits (
    not_null(columns := (order_date, product_id, product_name, category)),
    not_empty_string(column := (product_name)),
    accepted_range(column := total_sales_amount, min_v := 0)
  )
);

SELECT
  o.order_date,
  p.product_id,
  p.name AS product_name,
  p.category,
  COUNT(DISTINCT o.order_id) AS order_count,
  SUM(oi.quantity) AS total_quantity,
  SUM(oi.quantity * oi.unit_price) AS total_sales_amount
FROM vulcan_demo.orders AS o
JOIN vulcan_demo.order_items AS oi ON o.order_id = oi.order_id
JOIN vulcan_demo.products AS p ON oi.product_id = p.product_id
WHERE o.order_date BETWEEN @start_ds AND @end_ds
GROUP BY o.order_date, p.product_id, p.name, p.category
```

Vulcan now checks `data_available_for_today()` in addition to `cron` and upstream dependencies. On a normal day, the signal returns `True` right away and the model runs on schedule. On a day with delayed source data, the model waits and runs once the signal returns `True` on a later scheduler pass.

Run `vulcan plan` to confirm the signal name and arguments resolve correctly.

{% hint style="danger" %}
Never leave a `signals` block commented out in committed code. A commented-out signal silently reverts the model to clock-only scheduling, and the model can process incomplete data without any error.
{% endhint %}

### 3. Confirm the signal is actually gating runs

Compare `vulcan check_intervals` output with and without the signal against a deployed environment.

```bash
vulcan plan my_dev
vulcan check_intervals my_dev --select-model vulcan_demo.incremental_by_time_range
vulcan check_intervals my_dev --select-model vulcan_demo.incremental_by_time_range --no-signals
```

`check_intervals` only evaluates a deployed environment, so run `vulcan plan` first. Local-only signal changes aren't reflected until they're deployed. With signals on, an interval whose source data hasn't landed yet is excluded from the list. With `--no-signals`, that same interval shows up as ready, confirming the signal (not something else) is the reason it's being held back.

### 4. Handle partial readiness (optional)

If a batch can contain some ready intervals and some that aren't, return a filtered list instead of one boolean.

```python
from vulcan import signal, DatetimeRanges, ExecutionContext


@signal()
def data_available_for_today(batch: DatetimeRanges, context: ExecutionContext):
    ready = []
    for start, end in batch:
        query = f"""
            SELECT COUNT(*) AS cnt
            FROM vulcan_demo.orders
            WHERE order_date::timestamp BETWEEN '{start}' AND '{end}'
        """
        df = context.engine_adapter.fetchdf(query)
        if int(df.iloc[0]['cnt']) > 0:
            ready.append((start, end))
    return ready
```

If a model's `batch_size` groups several days together, this lets Vulcan process the days that already have data while it keeps waiting on the rest, instead of holding the whole batch back for one late day. Only return a subset of the exact `(start, end)` tuples you were given; never modify their bounds.

## Best practices

* Keep the readiness check close to the real question ("does the source have data for this window?") rather than a proxy like a fixed delay.
* Query the actual source table with `context.engine_adapter.fetchdf` (or `context.fetchdf` in a Python model), not a cached or derived copy.
* Return a filtered list of intervals instead of a single boolean once a model's `batch_size` spans more than a day, so one late day doesn't block everything else.
* Use `vulcan check_intervals --no-signals` whenever you need to confirm a signal, not something else, is the reason a model isn't running.

## Troubleshooting

| Symptom                                             | Likely cause                                                                                                    | Fix                                                                                |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| Model never runs, even though data has landed       | The signal's query checks the wrong table/column, or targets a different environment than the one with the data | Run the signal's query manually against the same environment the model targets     |
| Signal changes don't seem to take effect            | The signal was only changed locally, not deployed                                                               | Run `vulcan plan <env>` before `vulcan check_intervals <env>`                      |
| Model runs but processes incomplete data anyway     | The `signals` block was commented out or removed from the `MODEL` definition                                    | Confirm the `signals (...)` block is present and active                            |
| Type or assertion errors inside the signal function | The signal declares an argument that doesn't match what the model passes                                        | Match the signal function's parameters to the arguments in the `signals(...)` call |
| One slow day blocks an entire multi-day batch       | The signal returns a single `True`/`False` for the whole batch                                                  | Return a filtered list of ready intervals instead (step 4)                         |

## Related recipes

* [Cluster data with a Spark ML Python model](https://v2.dataos.info/build/july2026/recipes/spark-ml-python-model.md) - a complete Spark MLlib model with a scikit-learn fallback.

## References

* [Signals](https://v2.dataos.info/references/resources/vulcan/advanced-features/signals) - full signal reference: the `@signal` decorator, batch/context arguments, and testing with `check_intervals`. Vulcan ships no built-in signals; every signal is a custom function you write.
* [Run and scheduling](https://v2.dataos.info/references/resources/vulcan/concepts/run-and-plan/run-and-scheduling) - how `cron`, dependencies, and signals combine to decide when a model runs.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://v2.dataos.info/build/july2026/recipes/wait-for-data-with-signals.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
