Skip to content

Arkindex setup for developers

Following this documentation allows you to set up a locally running Arkindex instance (Community Edition) for development, where you can make any changes to the code and immediately see how they affect the backend Rest API and frontend display.

Warning

This setup is NOT suitable for production use. To find out how to deploy Arkindex to production (using Docker images), you should refer to the deployment documentation.

Requirements

We recommend you use a Linux distribution such as Debian or Ubuntu to follow these instructions. We at Teklia exclusively use Ubuntu LTS 22.04.

Minimal software requirements are:

Cloning the source code

In order to set up your local Arkindex instance, you first need to clone two projects: Arkindex backend and frontend.

You should clone them in the same parent folder, as some commands will assume specific relative paths.

Anonymous clone

You can clone the repositories anonymously using HTTPS clone:

git clone https://gitlab.teklia.com/arkindex/backend.git
git clone https://gitlab.teklia.com/arkindex/frontend.git

Authenticated clone

If you want to contribute to Arkindex and submit patches, we recommend you use SSH cloning. To do so, you need to create an account on our own Gitlab instance at gitlab.teklia.com.

In order to use SSH cloning, you need:

  • a working git client installation on your computer
  • an SSH key to authenticate yourself on a GitLab instance

You can find more information here on how-to setup an SSH key for git access.

Add your SSH key

Once you are connected to gitlab.teklia.com, you need to add your SSH public key to your user account on Teklia’s GitLab instance. To do so, you can follow the official GitLab instructions.

Activate 2FA

To gain and keep access to Arkindex’s GitLab repositories as a contributor, your GitLab account must be protected with Multiple Factor Authentication (generally using one time passwords generated from your phone).

You will need to setup an app like Google Authenticator or Duo Mobile on your phone, and then follow these steps.

If you do not complete the 2FA initial activation within 72 hours of creating your account, it will be automatically disabled.

Cloning with SSH

Finally you can clone the projects:

git clone git@gitlab.teklia.com:arkindex/backend.git
git clone git@gitlab.teklia.com:arkindex/frontend.git

Start up the instance

A few steps are required to get your local development instance up and fully functional. In the end, you should have four separate terminals open and running in order to be able to access your instance and use it to import documents, run machine learning workflows etc.

Run the third-party services

Arkindex relies on several third-party open-source services to store data and communicate across components:

  • PostgreSQL
  • Redis
  • MinIO
  • Cantaloupe IIIF server

All these services run through docker, and are configured in a docker-compose.yml file stored in the backend repository.

A convenient Makefile shortcut is available to start all these services in a dedicated shell:

cd backend
make services

You should see a few services start and log their output in the shell.

On a first start, mkcert will generate a Certificate Authority for your instance so that it’s only one exposed using HTTPS.

Run the backend

In order to setup the Arkindex backend, you must first (in a different terminal than the one where the services are running) create a Python virtual environment and populate it:

cd backend
mkvirtualenv -a $(pwd) -p /usr/bin/python3.10 arkindex
pip install -e .

Next you need to configure your Arkindex instance, by copying the sample config file to the expected path:

cp config.yml.sample arkindex/config.yml
Later on, you’ll be able to edit arkindex/config.yml to further configure your Arkindex instance.

After a few moment of Python dependencies installation, you should be able to run commands using the arkindex script.

First, you need to build the initial empty database structure:

arkindex migrate

Then populate it with some common settings for developers:

arkindex bootstrap

And create your own administrator account:

arkindex createsuperuser

Finally, you can run the development webserver:

arkindex runserver

You should be able to login on the admin portal through http://localhost:8000/admin/

Run a RQ worker for asynchronous tasks

We use rq, integrated via django-rq, to run tasks without blocking an API request or causing timeouts. The RQ worker is required to perform a number of asynchronous tasks, such as initializing worker activities, sending emails, deleting elements or projects, moving or linking elements, exporting data…

Info

Without the RQ worker running, you will never be able to start any workers process, as workers processes all use worker activities.

In a new terminal, activate the same virtual environment as for running the backend. In this terminal, start the RQ worker:

make worker

Run the frontend

The frontend installation only requires npm:

cd frontend
npm install

From there, you can run the development server:

npm run start

And access the whole interface at http://localhost:8080.

Info

The backend and services must be running while using the frontend interface, otherwise all you’ll get is a “server unreachable” error page.

You can also run the test suite with:

npm run test

The other utilities can be found in the Makefile of the project.