# `Lightning.Adaptors`
[🔗](https://github.com/OpenFn/lightning/blob/main/lib/lightning/adaptors.ex#L1)

The adaptor catalogue: which npm adaptors exist, their versions, schemas
and icons.

A scheduler fetches the catalogue from the configured source and
persists it. Reads check an in-memory cache first and the database
second, and answer immediately: a read that finds nothing in a catalogue
which has never loaded is `{:error, :not_ready}`. Waiting for that first
load is opt-in, through `ensure_loaded/2` and `fetch_adaptor/3`.

## Adaptor specs

A spec is the `name@version` string a job stores, where the version is
a semver or one of two sentinels: `latest` resolves to the catalogue's
current version when the job runs, and `local` points the worker at an
adaptor on its own filesystem. The version may also be omitted.

## Configuration

Under `config :lightning, Lightning.Adaptors`:

  * `:strategy` - the module that fetches from the source
  * `:refresh_interval` - how often the scheduler re-checks the source,
    in milliseconds; `0` disables the periodic tick
  * `:first_load_timeout` - how long a read waits for the initial load
  * `:cache_timeout_ms` - how long a read waits for a cache fill
  * `:icon_path` - where fetched icons are written

Defaults are in `Lightning.Adaptors.Config`.

## Testing

Every function that talks to a running process takes the supervisor
name as an optional first argument, defaulting to
`Lightning.Adaptors.Config.default_instance/0`. Start a
`Lightning.Adaptors.Supervisor` under another name and pass that name
to run a test against its own catalogue and cache, or stub
`default_instance/0` to make it the default for the test. The
recommended way to get a private instance is `setup :isolated_adaptors`
from `Lightning.AdaptorTestHelpers`, which does both.

# `error`

```elixir
@type error() :: :not_found | :not_ready | :timeout | :unavailable
```

Why a catalogue read could not answer.

  * `:not_found` - the catalogue has loaded and holds no such adaptor
  * `:not_ready` - the catalogue has never loaded
  * `:timeout` - the first load, or a cache fill, did not finish in time
  * `:unavailable` - no Scheduler process is reachable

# `icon_error`

```elixir
@type icon_error() ::
  {:icon_sha_mismatch, keyword()} | {:ext_mismatch, keyword()} | term()
```

What an icon read can fail with on top of `t:error/0`: the fetched bytes
disagreeing with the row that described them, or the strategy failing to
fetch them at all, which it reports in its own terms.

# `catalogue`

```elixir
@spec catalogue(atom()) ::
  {:ok,
   {{DateTime.t() | nil, non_neg_integer()},
    [Lightning.Adaptors.Store.rendered_entry()]}}
  | {:error, error()}
```

Returns `{:ok, {{latest_updated_at, count}, entries}}`: every adaptor
with its full version list and icon URLs, rendered once per change
rather than per request, alongside the ETag basis for it.

One read, so the stamp always describes the entries it comes with.

Returns `{:error, reason}` on a backing-store failure; see `t:error/0`.

# `dump_to_file`

Writes the catalogue to a JSON snapshot file. See
`Lightning.Adaptors.Dump.dump_to_file/2`.

# `ensure_loaded`

```elixir
@spec ensure_loaded(atom(), keyword()) :: :ok | {:error, error()}
```

Waits until the catalogue has loaded at least once, triggering the
first load if needed, bounded by `:timeout` (default
`Lightning.Adaptors.Config.first_load_timeout/0`).

Returns `:ok`, or one of the `fetch_adaptor/3` errors other than
`:not_found`.

# `fetch_adaptor`

```elixir
@spec fetch_adaptor(atom(), String.t(), keyword()) ::
  {:ok, Lightning.Adaptors.Package.t()} | {:error, error()}
```

Returns `{:ok, adaptor}` for the adaptor named `name`, waiting for the
catalogue's first load if it has never loaded, bounded by `:timeout`
(default `Lightning.Adaptors.Config.first_load_timeout/0`).

Takes a bare package name, not a spec; see `parse_spec/1`.

Errors:

  * `{:error, :not_found}` - the loaded catalogue has no such adaptor
  * `{:error, :timeout}` - the first load did not finish in time
  * `{:error, :unavailable}` - no Scheduler process is reachable
  * `{:error, :not_ready}` - the load ran but left the catalogue empty

# `icon`

```elixir
@spec icon(atom(), String.t(), :square | :rectangle) ::
  {:ok, Path.t()} | {:error, error() | icon_error()}
```

Returns the on-disk path of the adaptor's `:square` or `:rectangle`
icon, fetching it on the first request. `{:error, :not_ready}` against a
catalogue that has never loaded.

# `icon_meta`

```elixir
@spec icon_meta(atom(), String.t()) ::
  {:ok, Lightning.Adaptors.Store.icon_meta()}
  | {:error, :not_found | :not_ready}
```

Returns the stored extension and sha256 of each icon shape for the
adaptor named `name`, without touching disk. See
`t:Lightning.Adaptors.Store.icon_meta/0`.

# `packages`

```elixir
@spec packages(atom()) :: {:ok, [Lightning.Adaptors.Package.t()]} | {:error, error()}
```

Returns every adaptor in the catalogue as `Package` structs, or
`{:error, :not_ready}` against a catalogue that has never loaded.

# `parse_spec`

```elixir
@spec parse_spec(String.t()) ::
  {:ok, {String.t(), String.t() | nil}} | {:error, :invalid_format}
```

Splits an adaptor spec into `{:ok, {name, version}}`, with `version` `nil`
when the spec carries none, or `{:error, :invalid_format}` for a spec that
is not a package name plus an optional `@version`.

# `refresh`

Starts a catalogue refresh, or joins one already running.

Returns `:ok` as soon as the refresh is underway. With `await: true`,
blocks until the cycle completes, bounded by `:timeout` (default
`Lightning.Adaptors.Config.first_load_timeout/0`), and returns
`{:ok, counts}` or `{:error, reason}`; see
`Lightning.Adaptors.Scheduler.await_refresh/2` for the counts.

# `refresh`

```elixir
@spec refresh(atom(), keyword()) ::
  :ok | {:ok, Lightning.Adaptors.Scheduler.refresh_counts()} | {:error, term()}
```

# `refresh_icons`

```elixir
@spec refresh_icons(atom()) ::
  {:ok, %{updated: non_neg_integer(), unchanged: non_neg_integer()}}
  | {:error, term()}
```

Refetches every adaptor's icons and updates those whose bytes changed.
Returns `{:ok, %{updated: n, unchanged: m}}`.

# `refresh_package`

```elixir
@spec refresh_package(atom(), String.t()) :: :ok | {:error, :not_found | term()}
```

Refetches the adaptor named `name` from the source and persists it,
whether or not its version changed.

# `resolve_name`

```elixir
@spec resolve_name(atom(), String.t()) :: {:ok, String.t()} | {:error, error()}
```

Resolves a legacy short adaptor name such as `"http"` to its full npm
package name, `"@openfn/language-http"`, when the full name is in the
catalogue. A name the catalogue already knows comes back unchanged. So
does a name it knows in neither form, because a loaded catalogue that
has never heard of it is a real answer.

`"raw"` and `"oauth"` are sentinels, not adaptor names, and are returned
unchanged without consulting the catalogue.

Does not wait for a first load. A catalogue that has never loaded is
`{:error, :not_ready}`.

# `schema`

```elixir
@spec schema(atom(), String.t()) :: {:ok, String.t()} | {:error, error()}
```

Returns the credential schema of the adaptor named `pkg`, as a JSON
binary. An adaptor with no schema yields `"{}"` and an unknown name
is `{:error, :not_found}`, or `{:error, :not_ready}` against a catalogue
that has never loaded.

# `seed_from_file`

Populates the catalogue from a JSON snapshot file. See
`Lightning.Adaptors.Seed.seed_from_file/2`.

# `subscribe_to_updates`

```elixir
@spec subscribe_to_updates(atom()) :: :ok | {:error, term()}
```

Subscribes the calling process to catalogue update broadcasts.

Updates arrive as `%{event: "adaptors_updated", payload: %{names: [...]}}`
messages.

# `to_wire`

```elixir
@spec to_wire(atom(), String.t() | nil) ::
  {:ok, String.t()}
  | {:error, :not_found | :timeout | :unavailable | :not_ready}
```

Renders an adaptor spec for the worker: `latest` becomes the
catalogue's current version, `local` is kept, and a `:local` source
forces `name@local`. A `nil` spec renders as `""`.

# `valid_format?`

```elixir
@spec valid_format?(String.t()) :: boolean()
```

Returns whether `spec` is a well-formed adaptor spec: a package name
plus an optional `@version`, with no newlines or shell metacharacters.

---

*Consult [api-reference.md](api-reference.md) for complete listing*