Lightning.DataclipScrubber (Lightning v2.19.0-pre)

View Source

Handles scrubbing of dataclips

Summary

Functions

Returns an Ecto query for credentials (with project env) used in the same run or earlier steps.

Scrubs a batch of dataclip bodies, resolving what to scrub with a fixed number of queries.

Returns webhook auth methods for an http_request dataclip by traversing: dataclip -> work_order -> trigger -> webhook_auth_methods

Returns webhook auth methods for a run by traversing: run -> work_order -> trigger -> webhook_auth_methods

Returns webhook auth methods for a step by traversing: step -> run_step -> run -> work_order -> trigger -> webhook_auth_methods

Functions

credentials_for_step(step_id, started_at)

Returns an Ecto query for credentials (with project env) used in the same run or earlier steps.

Uses a self-join on RunStep to leverage existing indexes.

scrub_dataclip_bodies!(dataclips)

@spec scrub_dataclip_bodies!([
  %{body: String.t() | nil, type: atom(), id: Ecto.UUID.t()}
]) :: %{
  optional(Ecto.UUID.t()) => String.t() | nil
}

Scrubs a batch of dataclip bodies, resolving what to scrub with a fixed number of queries.

scrub_dataclip_body!/1 looks up the step behind a dataclip, then its credentials, then its webhook auth methods - three queries per dataclip, two of them many-table joins. That is fine for the one dataclip a controller renders and an N+1 for the hundreds a page of the history export touches.

Takes the same %{id:, type:, body:} maps as scrub_dataclip_body!/1 and returns the scrubbed bodies keyed by dataclip id.

scrub_dataclip_body!(dataclip)

@spec scrub_dataclip_body!(%{body: String.t() | nil, type: atom(), id: Ecto.UUID.t()}) ::
  String.t() | nil

webhook_auth_methods_for_dataclip(dataclip_id)

Returns webhook auth methods for an http_request dataclip by traversing: dataclip -> work_order -> trigger -> webhook_auth_methods

webhook_auth_methods_for_run(run_id)

Returns webhook auth methods for a run by traversing: run -> work_order -> trigger -> webhook_auth_methods

webhook_auth_methods_for_step(step_id)

Returns webhook auth methods for a step by traversing: step -> run_step -> run -> work_order -> trigger -> webhook_auth_methods