View Source Running Lightning Locally

This guide provides instructions for running Lightning locally, either by installing dependencies on your machine or using Docker.

By Installing Dependencies

Setup

Postgres

Requires postgres 15. When running in dev mode, the app will use the following credentials to authenticate:

  • PORT: 5432
  • USER: postgres
  • PASSWORD: postgres
  • DATABASE: lightning_dev

This can however be overridden by specifying a DATABASE_URL env var. e.g. DATABASE_URL=postgresql://postgres:postgres@localhost:5432/lightning_dev

We recommend that you use docker for running postgres as you'll get an exact version that we use:

docker volume create lightning-postgres-data

docker create \
  --name lightning-postgres \
  --mount source=lightning-postgres-data,target=/var/lib/postgresql/data \
  --publish 5432:5432 \
  -e POSTGRES_PASSWORD=postgres \
  postgres:15.3-alpine

docker start lightning-postgres

Elixir, NodeJS

We use asdf to configure our local environments. Included in the repo is a .tool-versions file that is read by asdf in order to dynamically make the specified versions of Elixir, Erlang and NodeJs available. You'll need asdf plugins for Erlang, NodeJs Elixir and k6.

Libsodium

We use libsodium for encoding values as required by the Github API. You'll need to install libsodium in order for the application to compile.

For Mac Users:

brew install libsodium

For Debian Users:

sudo apt-get install libsodium-dev

You can find more on how to install libsodium here

Compilation and Assets

asdf install  # Install language versions
mix local.hex
mix deps.get
mix local.rebar --force
[[ $(uname -m) == 'arm64' ]] && CPATH=/opt/homebrew/include LIBRARY_PATH=/opt/homebrew/lib mix deps.compile enacl # Force compile enacl if on M1
[[ $(uname -m) == 'arm64' ]] && mix compile.rambo # Force compile rambo if on M1
mix lightning.install_runtime
mix lightning.install_schemas
mix lightning.install_adaptor_icons
mix ecto.create
mix ecto.migrate
npm install --prefix assets

In case you encounter errors running any of these commands, see the troubleshooting guide for known errors.

Running the App

To start the lightning server:

mix phx.server

Once the server has started, head to localhost:4000 in your browser.

By default, the worker is started when run mix phx.server in dev mode. In case you don't want to have your worker started in dev, set RTM=false:

RTM=false mix phx.server

Using Docker

There is an existing docker-compose.yaml file in the project's root which has all the services required. To start your services:

docker compose up

There 2 docker files in the root, Dockerfile builds the app in prod mode while Dockerfile-dev runs it in dev mode. It is important to note that mix commands do not work in the prod images.

For example, to run migrations in dev mode you run:

docker compose run --rm web mix ecto.migrate

While in prod mode:

docker compose run --rm web /app/bin/lightning eval "Lightning.Release.migrate()"

Configuring the Worker

By default, lightning starts the worker when running in dev. This can also be configured using RTM env var. In case you don't want the hassle of configuring the worker in dev, you can just remove/comment out the worker service from the docker-compose.yaml file because lightning will start it for you.

Learn more about configuring workers

Using Local Adaptors

You can force lightning to use adaptor builds from your local adaptors repo.

Note that this is a global toggle: ALL runs will use local adaptor versions, and the adaptor picklist in the Workflow Editor will only suggest adaptors present in the monorepo.

Remember to re-build your adaptors after making changes (use pnpm build --watch in the monorepo).

To start, set up the following environment variables:

  • LOCAL_ADAPTORS: Used to enable or disable the local adaptors mode. Set it to true to enable.
  • OPENFN_ADAPTORS_REPO: This should point to the adaptors monorepo. This is the same variable used when you pass -m to the CLI.

Example configuration:

export LOCAL_ADAPTORS=true
export OPENFN_ADAPTORS_REPO=/path/to/repo/

You can also run the server directly in local mode with:

LOCAL_ADAPTORS=true mix phx.server

Ensure that the OPENFN_ADAPTORS_REPO directory is correctly set up with the necessary packages subdirectory, otherwise the app wont start

Problems with Apple Silicon

You might run into some errors when running the docker containers on Apple Silicon. We have documented the known ones here