Lightning.Workflows.YamlFormat.V2 (Lightning v2.19.0)

Copy Markdown View Source

v2 (CLI-aligned) YAML format for Lightning workflows.

See test/fixtures/portability/v2/canonical_workflow.yaml for the spec-by-example. New contributors should read that file before this module.

Spec source (pinned)

This module implements the OpenFn Portability Spec as defined at:

The links above pin specific commits so future readers can audit which version of the spec this code targets. The spec carries explicit // TODO markers (next-as-string deprecation, step.id required, condition.label vs name, credential as id-string) — those are pending upstream. Where Lightning's behavior diverges from the in-flight spec, see "Subject to upstream finalization" below.

Subject to upstream finalization

  • cron_cursor — Lightning emits cron_cursor: <step-id> flat at the trigger root (stateless, by-name). The kitchen-sink example uses cron_cursor_job_id: <uuid> flat at the trigger root, which contradicts the spec's statelessness principle and isn't defined in portability.d.ts at all; we use the step-id form pending upstream resolution.
  • next: form — portability.d.ts:60 carries a // TODO remove next: string (next should always be an object) note. Lightning emits verbose-only to align with the spec direction and to avoid the bare-string parsing bug in @openfn/project@0.15 where next: <step-id> gets misread as iterating over the target id's characters. Bare-string next: is no longer emitted.
  • name vs label on ConditionalStepEdge — portability.d.ts has // TODO this is probably the name. Lightning emits label: today; swap when upstream lands.

Shape

Workflow portability shape:

id: <string>
name: <string>
start: <step-id>          # entry trigger's step-id (spec: WorkflowSpec.start)
steps: [<step>, ...]      # one array — both jobs and triggers

Before emission, the canonical map splits the single steps: array into two sibling keys — :triggers and :steps — so the emitter can iterate triggers and jobs separately. Both keys are always present (empty list when there are none).

A trigger step has a type discriminator (webhook / cron). All Lightning extensions land flat at the trigger root — there is no openfn: wrapper:

- id: <string>
  name: <string>
  enabled: true | false
  type: webhook | cron
  cron_expression: "0 0 * * *"          # cron only (spec: flat field)
  cron_cursor: <step-id>                # cron only (Lightning ext, flat)
  webhook_reply: <string>               # webhook only (spec: flat field)
  webhook_response_config:              # webhook only (Lightning ext)
    success_code: <int>                 #   omitted entirely when unset
    error_code: <int>
  next:
    <step-id>:
      condition: always | on_job_success | on_job_failure | <js body>

A job step has no type field:

- id: <string>
  name: <string>
  adaptor: <string>
  expression: |
    fn(state => state)
  configuration: <credential-key>      # optional
  next:
    <step-id>:
      condition: always | on_job_success | on_job_failure | <js body>
      expression: |                    # only emitted alongside js body
        <js body>
      label: <string>                  # optional
      disabled: true                    # optional, defaults to false

Condition discrimination

Per lightning.d.ts:102, condition is the union 'always' | 'on_job_success' | 'on_job_failure' | string. Lightning emits the named literals (always, on_job_success, on_job_failure) verbatim for those condition_type values; for :js_expression the field carries the user-supplied JS body string. The literal is always present (matches the kitchen-sink example), so parsers don't need to assume a default for a missing condition: field.

Field-name table

conceptv2 field name
workflow steps array (YAML)steps: (jobs + triggers)
workflow start stepstart: (workflow head)
trigger discriminatortype:
trigger enabledenabled:
step expression / bodyexpression:
step adaptoradaptor:
step credentialconfiguration:
cron expression (flat on trig)cron_expression:
cron cursor (flat on trig, ext)cron_cursor:
webhook reply (flat on trig)webhook_reply:
webhook response codes (trig)webhook_response_config:
project channels (Lightning ext)channels: (array)
channel destination credentialdestination_credential:
outgoing edges from a nodenext: (string or object)
edge conditioncondition:
edge js bodyexpression: (with JS cond)
edge labellabel:
edge disabled (inverted)disabled:

Summary

Functions

Serialize a project to v2 YAML.

Serialize a workflow struct to v2 YAML.

Functions

serialize_project(project, snapshots \\ nil)

@spec serialize_project(Lightning.Projects.Project.t(), [any()] | nil) ::
  {:ok, binary()} | {:error, term()}

Serialize a project to v2 YAML.

Produces a stateless project document — no UUIDs in the body. Stable hyphenated names are the join keys.

The snapshots argument is accepted for façade-compatibility with the v1 serializer but is not used for v2 (v2 always emits the project's current workflow set).

Expects the project to have its associations preloaded; if not, this function preloads them itself.

serialize_workflow(workflow)

@spec serialize_workflow(Lightning.Workflows.Workflow.t()) ::
  {:ok, binary()} | {:error, term()}

Serialize a workflow struct to v2 YAML.

Expects workflow.jobs, workflow.triggers, workflow.edges to be loaded.