Lightning.Kickstart (Lightning v2.19.0-pre)
View SourceDeclarative, idempotent kickstarting of Lightning from a scenario file.
Given a plain map (typically decoded from a YAML/JSON scenario file), this
module creates users (with optional API tokens), credentials and projects,
and provisions each project's workflows through
Lightning.Projects.Provisioner — the same engine that backs the
/api/provision HTTP API.
It serves local development (bin/e2e --scenario) and external test
harnesses: boot Lightning into a known state, read the manifest, drive the
public APIs.
It is not a way to provision a live instance. There is no release entry
point, and run/1 refuses to run outside dev and test — see "Safety".
Deploying state to a running instance is what /api/provision and the CLI
are for.
Idempotency
Re-running the same scenario converges instead of duplicating:
- Users are matched by email, credentials by
{owner, name}, and both are reused when they already exist. - API tokens are signed JWTs and cannot be supplied; when
api_token: truethe user's oldest existing API token is reused, and one is generated only if none exists. Tokens are surfaced throughmanifest/1. - Projects, workflows, triggers, jobs and edges get deterministic ids
(UUIDv5-style, derived from a project/workflow's name and from a record's
key in the workflow spec) unless an explicit
idis given, so the provisioner upserts them on subsequent runs. - Project members are added or have their role updated, never removed.
Renaming a record changes its derived id, and what that costs depends on the record:
- Renaming a job, trigger or edge deletes the old row and creates a new
one -
Workflowdeclareshas_many :jobs, on_replace: :delete. A renamed webhook trigger answers at a new/i/<id>URL. - Renaming a workflow or collection fails, naming the record: the old one
is still on the project and the provisioner treats the document as the
complete set. Pin an explicit
idon a workflow you intend to rename and its jobs, triggers and edges keep their ids too. - Renaming a project creates a new one and leaves the old one alone.
The whole run happens in a single transaction — a failing scenario leaves the database untouched.
Safety
Kickstarting creates users (including superusers), and applies a scenario as
the desired state: records it declares are overwritten from the file, so a
job body edited in the editor is reverted on the next run. That is fine for
a database that exists to be thrown away, and wrong for one anybody relies
on — so it is confined to dev and test, and run/1 raises anywhere
else.
Mix tasks aren't shipped in a release, so mix lightning.kickstart cannot
be reached in production at all; the environment check exists to also stop
someone calling this module directly from a remote console.
Scenario shape
users:
- email: amy@openfn.org # required
first_name: Amy
superuser: true
api_token: true # generate/reuse a token, see manifest
# password defaults to "welcome12345"
credentials:
- name: dhis2-prod # required, unique per scenario
owner: amy@openfn.org # required, a user declared above
schema: dhis2
body:
password: ${env:DHIS2_PASSWORD} # explicit env interpolation
projects:
- name: my-project # required, url-safe
description: An example project # optional
members: # required, exactly one owner
- { email: amy@openfn.org, role: owner }
credentials: [dhis2-prod] # optional, exposed to this project
collections: # optional
- name: my-collection
workflows: # workflow-spec documents, see below
- name: my-workflow
jobs:
transform:
name: transform
adaptor: "@openfn/language-common@latest"
body: "fn(state => state);"
credential: dhis2-prod # optional, a credential above
triggers:
webhook:
type: webhook
enabled: true
edges:
webhook->transform:
source_trigger: webhook
target_job: transform
condition_type: always
enabled: trueEach entry under workflows is a workflow spec — the same
hand-writable format the collaborative editor imports and exports and that
workflow templates are written in, validated against the same JSON Schema
and converted by Lightning.Workflows.Spec. There is no kickstart-specific
workflow dialect: anything you can paste into the editor's YAML import works
here, and vice versa. The one extra key kickstart resolves is a job's
credential, which names a credential declared at the top level (the schema
already allows it; the editor ignores it).
Every other key — the scenario itself, and each user, credential, member and
project — is checked against an explicit allow-list, so a typo (usres:) or
an unsupported field (e.g. channels, not yet handled here) raises instead
of being silently ignored.
Keys are strings, as produced by the YAML/JSON parsers.
Env interpolation is explicit: only ${env:VAR} references are replaced
(and it is an error for VAR to be unset). Plain ${...} is left alone,
so JS template literals in job bodies (${id}, ${HOME}, ${state.data})
can never collide with interpolation.
Summary
Functions
Parse a scenario file into a map. Supports YAML and JSON.
Structured, JSON-encodable manifest of a run/1 result — everything an
external harness needs to drive the instance: user emails and API tokens,
record ids, and webhook paths.
Create or update everything described by scenario, atomically.
Load a scenario file (.yaml, .yml or .json) and run/1 it.
Human-readable one-line-per-record summary of a run/1 result.
Types
@type result() :: %{ users: %{required(String.t()) => user_result()}, credentials: %{required(String.t()) => Lightning.Credentials.Credential.t()}, projects: [map()] }
@type user_result() :: %{ user: Lightning.Accounts.User.t(), api_token: String.t() | nil }
Per-user result: the persisted user and any generated API token.
Functions
Parse a scenario file into a map. Supports YAML and JSON.
Structured, JSON-encodable manifest of a run/1 result — everything an
external harness needs to drive the instance: user emails and API tokens,
record ids, and webhook paths.
Create or update everything described by scenario, atomically.
Returns a result map describing the records; pass it to manifest/1 for a
JSON-encodable summary or summary/1 for a human-readable one.
Raises unless kickstarting is enabled (see the module docs), and rolls the whole run back on any error.
Load a scenario file (.yaml, .yml or .json) and run/1 it.
Options:
:manifest- path to write the JSON manifest to.
Human-readable one-line-per-record summary of a run/1 result.