Permissions and grants
Provision DataOS, lakehouse, compute, state-store, storage, and Git access for Spark deployments.
This is the authorize stage: the step before you touch a gateway config or write a model. Provision the platform resources, object-storage grants, and DataOS access before you deploy a Vulcan Data Product using Spark.
Platform dependencies
Provision the platform before you write any model code. Spark itself brings no compute, storage, or state store — the items below (compute pool, tenant stack, lakehouse, secrets, and DataOS access) all have to be in place first, or you won't even be able to connect, let alone deploy a model. Work through this list with your SRE team before starting.
Required platform resources
Request these from your SRE team, they cover the Spark runtime, Iceberg storage, state store, and source-code access:
Spark-capable compute pool
Verify with dataos-ctl resource -t compute get -a. Set spec.compute to this pool name.
Tenant-level vulcan-spark stack
Provides the runtime image, serving sidecars, dependency loading, and catalog templating.
DataOS lakehouse resource
Backed by S3 or ABFSS object storage.
Lakehouse depot
Attach as dataos://<depot>?purpose=rw in the Vulcan resource.
External Postgres for Vulcan state
The platform provisions and wires this in production. Do not add it to project config.yaml.
Object-storage credentials
Store these on the lakehouse or depot. Never hard-code them in project code.
Git-sync secret
Lets the Vulcan resource pull model code.
Roles and minimum storage grants
The three roles below separate platform administration, workload execution, and consumer access.
Admin role
Platform SRE provisions the compute pool, installs the stack, and creates the lakehouse and depot.
Vulcan service role
Configure as runAsUser. Runs Spark jobs and writes Iceberg tables through the lakehouse depot.
Consumer role
Provides read-only access through API endpoints.
The lakehouse depot owns storage credentials. Grant the required cloud access to its backing storage.
S3
s3:GetObject, s3:PutObject, s3:DeleteObject, and s3:ListBucket on the warehouse bucket.
ABFSS
Storage Blob Data Contributor on the container. Store Storage Account Key or SPN credentials on the depot.
DataOS permissions
After platform provisioning is complete, grant the following DataOS permissions:
Permission
What it Allows
Granted By
Tenant Access
Build and manage Data Products within your tenant.
Tenant Admin
roles:id:data-dev (or your tenant's equivalent role tag)
Create and apply Vulcan resources. Verify your role using dataos-ctl user get (check the TAGS column).
Tenant Admin
depot:rw:<lakehouse-depot-name>
Read and write access to the lakehouse depot.
Tenant Admin or Data Admin
depot:r:<lakehouse-depot-name>
Read-only access to the lakehouse depot (for consumers).
Tenant Admin or Data Admin
Can Use (Compute, Depots, Secrets)
Allows you to actually use Compute, Depots, and Secrets. Note: Resource access is deny-by-default. Even if you have a role tag or depot permission, you also need Can Use to use the resource.
Every permission can be granted to a tenant Role now -grant it once and the whole team, including future joiners, inherits it automatically.
git-sync Secret
Allows Vulcan to pull model code from Git using credentials stored in a Secret. Request a git-sync Secret, not raw Git credentials. If you create the Secret yourself, you still need Can Use permission on it.
Tenant Admin, Data Admin, or the Secret owner (with Can Use granted)
Run dataos-ctl get depot to confirm your read access before you start. If the lakehouse depot is missing, request depot:r from your operator.
Python version
Local development requires Python 3.10 to 3.13 (python --version must report 3.10.x to 3.13.x). The Spark runtime inside the Vulcan image manages its own Python version.
Next steps
With access and platform resources ready, configure your gateway in Connect to Spark.
Last updated
Was this helpful?