Within Orvanta (not recommended)

This page is part of the Persistent storage & databases section covering data storage and management options in Orvanta.

States and resources

Within Orvanta, you can use States and Resources to store transient state represented as small JSON.

States

States allow scripts to maintain data persistence between runs of the same script triggered by schedules or users.

States are considered as resources, but they are excluded from the Workspace tab for clarity. They are displayed on the Resources menu, under a dedicated tab.

A state is stored as a resource with type state meant to persist across distinct script executions.

Example usage in Python:

import requests
from orvanta import set_state, get_state

def main():
    # Get temperature from last execution
    last_temperature = get_state()
    # Fetch the temperature in Paris from wttr.in
    response = requests.get("http://wttr.in/Paris?format=%t")
    new_temperature = response.text.strip("°F")
    # Set current temperature to state
    set_state(new_temperature)
    # Compare temperatures
    if last_temperature < new_temperature:
        return "The temperature has increased."
    elif last_temperature > new_temperature:
        return "The temperature has decreased."
    else:
        return "The temperature has remained the same."

States enable Flows to watch for changes in event-watching scenarios. The pattern involves:

• Retrieving the last state or assuming first execution if undefined
• Retrieving the current state from the external system
• Calculating differences between current and last internal state
• Setting the new state to avoid reprocessing
• Returning differences for subsequent processing steps

TypeScript convenience functions:

• getState() retrieves an object at a path determined by getStatePath
• setState(value: any) sets the new state

Python convenience functions:

• get_state() retrieves an object at a path determined by get_state_path
• set_state(value: Any) sets the new state

Both require importing the orvanta client library.

Custom flow states

Custom flow states store data across steps in a flow. You can set and retrieve values given a key from any flow step, available globally within that flow. The state persists for the flow’s job lifetime.

It’s a powerful escape hatch when passing data as output/input is not feasible and using getResource/setResource has the issue of cluttering the workspace.

TypeScript example:

import * as orvanta from 'orvanta-client';

export async function main(x: string) {
    await orvanta.setFlowUserState('FOO', 42);
    return await orvanta.getFlowUserState('FOO');
}

Python example:

import orvanta

def main(x: str):
    orvanta.set_flow_user_state("foobar", 43)
    return orvanta.get_flow_user_state("foobar")

Resources

States are a specific type of resource where the type is state and the path is automatically calculated. For arbitrary paths or different types, use setResource and getResource functions. Resources can be shared across different scripts and flows.

• setResource(value: any, path?: string, initializeToTypeIfNotExist?: string) sets a resource at a given path with optional custom type
• getResource(path: string) retrieves a resource at a given path

States appear in the Resources section with a Resource Type of state.

Variables are similar to resources but have no types, can be tagged as secret (encrypted by workspace key), and only store strings. setVariable/getVariable may be preferable in some situations.

In conclusion, setState and setResource are convenient ways to persist JSON between multiple script executions.

Shared directory

For heavier ETL processes or sharing data between flow steps, Orvanta provides a Shared Directory feature allowing steps to share data via a designated ./shared folder.

Caution

• All steps execute on the same worker
• Data in Shared Directory is ephemeral to the flow execution
• Contents are not preserved across suspends and sleeps

To enable Shared Directory:

1. Open the Settings menu in the Orvanta interface
2. Go to the Shared Directory section
3. Toggle on the option for “Shared Directory on ’./shared’”

Once enabled, steps can read and write files to ./shared to pass data. This is useful for:

• Handling larger datasets impractical to pass as step inputs/outputs
• Temporary storage of intermediate processing results
• Sharing files between steps in ETL pipelines