Skip to content

Arkindex 1.6.3

A new Arkindex release is available.

To upgrade a development instance, follow this documentation.

To upgrade a production instance, you need to:

  • Deploy this release’s Docker image: registry.gitlab.teklia.com/arkindex/backend:1.6.3
  • Run the database migrations: docker exec ark-backend arkindex migrate
  • Update the system workers: docker exec ark-backend arkindex update_system_workers

The release notes for Arkindex 1.6.3 are available here.

The main changes impacting developers and system administrators are detailed below.

Simpler management of system workers

System workers, or workers that provide specific Arkindex features like Workers processes and S3 ingest processes, are no longer managed using the YAML configuration. This means the following keys can be removed from your configuration files:

  • docker.tasks_image
  • docker.init_elements_image
  • docker.ingest_image

Previously, these values had to be manually set to the correct tag of the correct Docker image, then a worker version needed to be created manually with the same tag and specific settings. Upgrading required restarting the backend once or twice to apply and validate the necessary changes.

The assignation of a Docker image as providing a specific feature is now done through the worker versions themselves. Worker versions now have a feature attribute, which lets them be assigned to specific Arkindex features:

  • init_elements: The elements initialisation worker used in Workers processes to list the elements to process and distribute them between chunks.
  • s3_ingest: The worker that executes S3 ingest processes. This is not used when the ingest feature flag is disabled.
  • file_import: Reserved. In a future version, this worker will execute file import processes.

The feature attribute is unique: only one worker version at a time can provide each of the features.

The recommended way to manage features is to use the new arkindex update_system_workers command, which will in most cases be able to automatically create or update all of the necessary worker versions and assign features, using a set of Docker images that our team recommends for a specific Arkindex release. We also recommend that this command be run on each Arkindex instance upgrade, so that future updates of those workers are still reflected in your database.

Features can also be manually configured through the administration interface.

The W014 and W015 system checks have been updated to mention this new feature system and recommend the use of arkindex update_system_workers.

Arkindex developers also need to be aware of the new E012 system check, which could fail when releasing a new version of Arkindex if the recommended Docker images that update_system_workers relies on have not been manually approved for this new release.

S3 ingest feature flag

To simplify deployments, the S3 ingest feature is now disabled by default, and can be toggled using the new features.ingest feature flag.

When this feature flag is disabled:

  • The S3 ingest options in the frontend are not visible.
  • The ListBuckets and CreateS3Import endpoints are not available.
  • Existing S3 ingest processes cannot be retried.
  • There will be no system check warnings if no worker version exists to execute S3 import processes.

Docker container management

When running tasks with Community Edition, the Docker containers now have names starting with ponos- and ending in the UUID of each task, instead of the default names provided by Docker.

In addition, a new ponos.auto_remove_container option has been introduced. When enabled, each container is automatically removed once the task finishes.