Persistent storage & databases
Scripts and flows in Orvanta are stateless by default: nothing survives from one job to the next unless you explicitly put it somewhere durable. This page is the map of where that “somewhere” can be, depending on how much structure your data has and how much of it there is.
Primary storage options
Section titled “Primary storage options”The recommended approach is to use data tables to store relational data, and ducklakes to store massive datasets, with external databases available as connections.
Data tables
Section titled “Data tables”Built-in relational database functionality requiring minimal setup for storing structured data directly within Orvanta.
Ducklake
Section titled “Ducklake”A data lakehouse solution enabling massive amounts of data stored in parquet files on S3 while querying it with natural SQL syntax.
External databases
Section titled “External databases”Connect existing databases through integrations or create custom connection types if your provider isn’t pre-integrated.
Additional storage methods
Section titled “Additional storage methods”Large unstructured data: S3, Cloudflare R2, MinIO, Azure Blob Storage, and Google Cloud Storage are highly scalable and durable object storage services with native Orvanta integration.
Structured SQL data: Postgres-based solutions like Supabase and Neon.tech are recommended for schema-based storage with defined entity relationships.
File persistence: Volumes provide persistent file storage that can be attached to scripts, with automatic syncing to object storage between runs.
NoSQL solutions: MongoDB Atlas, Redis, and Upstash support flexible, non-structured data management.
Internal storage
Section titled “Internal storage”Orvanta offers internal persistence between job executions, but this approach is explicitly marked as not recommended.