Development
We use Poetry to handle all of the dependencies we use for this
system. We also provide the poetry.lock so that we have a consistent set of dependencies for any
build we do.
For local development we provide a docker-compose.yaml file for easy setup, and extra make
commands for ease of use.
There are 2 main ways of developing, developing inside docker, and directly on the host.
Docker Development
For developing directly inside docker you can run make build and then make run which will
start up all the services required.
For the API side of the development, changes will be picked up on file save, and the API will restart to pick up those changes, however the background tasks will not do this, and the docker container will need to be rebuilt and restarted for changes to take place.
The way to rebuild the cron image would be to run docker compose build cron and then
docker compose up -d cron.
For this reason it is usually a lot simpler to rather develop directly on the host machine.
Host Development
For developing directly on the host machine you will still need to have Kafka and Redis running. To
accomplish this there is a helper command which can be used to start these services up which can be
run with make run-services.
Once the above services are running the API can be run with make run-display-api (this will also
get poetry dependencies installed, and run inside a new shell).
For running any of the cron commands you need to be in a poetry shell which you can do by running
poetry shell. And then to install the new commands run poetry install and then you can run
them directly.
For a list of tasks check pyproject.toml and look at the tool.poetry.scripts section.
A word of warning, you will need to set the Kafka and Redis env variables. To set them add
BROKER_INSTANCE=localhost:29092 for Kafka and REDIS_HOST=localhost for Redis before any
command that you need to run.
As an example to run the Visibility Receive task run the following:
BROKER_INSTANCE=localhost:29092 REDIS_HOST=localhost signal-task-process-visibility-receive --verbose
The standard arguments that can be added is:
--verboseor-v- to change the logging to outputINFO--debugor-d- to change the logging to outputDEBUG