Temple Compute Docs

Handling secrets

How Secret fields keep credentials out of exported YAML and packaged zips, and how to resolve them at run time.

A plugin target can hold a credential in a plain field, SSHTarget.password for instance. Left as a plain str, that value travels through every export path unmarked: to_yaml, horus package, and anything downstream that reads the dumped workflow dict would carry it verbatim.

horus_runtime.secrets marks such fields and redacts them wherever a workflow is exported.

Marking a field

from horus_runtime.secrets import Secret

class SSHTarget(BaseTarget):
    password: Secret | None = None

Secret is a SecretStr subclass, so it keeps pydantic's masked repr/str/ JSON dump and format: "password" / writeOnly in the generated JSON schema. A value built from a ${secret:<ref>} string is a reference: .ref is set, and .resolve() looks the value up. Any other value is a literal: .ref is None, and .resolve() returns it as-is.

What gets redacted, and where

  • BaseWorkflow.to_yaml walks the model with iter_secret_fields, then redacts the dumped dict before writing. Each secret field becomes ${secret:<ref>}, never the pydantic mask. A mask round-trips back in as a literal password on re-import, so this is why redaction targets a reference instead.
  • packaging.package_workflow does the same for the workflow YAML it writes into the zip, so horus package (see Packaging workflows) never ships a literal credential either.

iter_secret_refs(data) is the dict-side twin of iter_secret_fields: it walks an already-dumped workflow dict (no models) and yields (path, ref) for every ${secret:<ref>} it finds. Use it wherever you only have the dumped dict, not the validated model: a workflow stored as an opaque blob by whatever is persisting it, for example.

Resolving a secret at run time

Secret.resolve() looks up the real value lazily:

  1. HORUS_SECRET_<REF> (ref uppercased, non-alphanumerics replaced with _; see env_key_for_ref).
  2. Failing that, the file named by $HORUS_SECRETS_FILE: YAML, keyed by ref.

Neither is set: resolve() raises SecretResolutionError, naming the env var and file it checked.

export HORUS_SECRET_SSH_PASSWORD=hunter2
# or
export HORUS_SECRETS_FILE=./secrets.yaml   # { ssh_password: hunter2 }

Secrets in a task's environment

A task can declare credentials its command needs as environment variables:

tasks:
  - id: fetch_model
    kind: horus_task
    runtime:
      kind: command
      command: huggingface-cli download meta-llama/Llama-3.1-8B
    secrets:
      HF_TOKEN: "${secret:hf_token}"

An entry can also carry a display name and a description. The key stays the env var name:

    secrets:
      HF_TOKEN:
        value: "${secret:hf_token}"
        name: Hugging Face token
        description: Read access to gated model weights

Each entry is a TaskSecret whose value is a Secret, so it is redacted on export like any other secret field (tasks.<N>.secrets.<NAME>.value). When the task starts, every entry is resolved as described above; a missing one fails the task before its command runs. The resolved values are set on every command the task issues through BaseTarget.run_command, whichever executor and target run it, and the caller's own env wins on a name clash. They are not set on the next task.

Names

A name must match [A-Za-z_][A-Za-z0-9_]* and be at most 128 characters. Names starting with HORUS_ are reserved, and so are PATH, HOME, USER, LOGNAME, TMPDIR, TC_CACHE_DIR, LD_LIBRARY_PATH, LD_PRELOAD, LD_AUDIT, IFS, BASH_ENV, ENV, CDPATH, SHELLOPTS, BASHOPTS and PS4. A task with any other name fails validation, e.g. Secret 'PATH': PATH is reserved. horus_runtime.secrets.env_name_error(name) returns the same message, or None for a valid name.

Docker

The Docker executor passes each name as -e NAME, so the value is not on the docker run command line. It is still in the container config, visible with docker inspect while the container exists.

Detached local jobs

LocalTarget hands a detached job its environment through an env file in the job directory. The file has mode 0600, and the job reads it and then deletes it, so the values never appear on the job's command line.

Non-goals

Key management/rotation, encrypting the local secrets file, and per-run credential scoping are out of scope for this mechanism.

Follow-up

Lifting stored credentials out of an opaque workflow_data blob into a real credentials store, with re-injection at run dispatch, is a separate, not-yet-built piece. This mechanism only covers marking, redaction, and env/file resolution.

On this page