Virtuoso Enterprise Edition Reference Guide for Docker Deployment

[hwilliams]

OpenLink Virtuoso Enterprise Edition 8.3 Docker Reference

Copyright (C) 2022 OpenLink Software

Downloading the image

To pull the latest Virtuoso 8.3 docker image to your local system, you can use the following command:

$ docker pull openlink/virtuoso-closedsource-8

To check the version of the Virtuoso binary, you can use the following command:

$ docker run openlink/virtuoso-closedsource-8 version

This Docker image is using the following version of Virtuoso:

OpenLink Virtuoso Universal Server (Enterprise Edition)
Version 08.03.3323-pthreads as of Apr 19 2022 (ef269d8965)
Compiled for Linux (x86_64-ubuntu_bionic-linux-gnu-glibc2.27)
Hosted Runtime Environments:  VDB
Copyright (C) 1998-2022 OpenLink Software

Configurable resources

The OpenLink Virtuoso Enterprise Edition docker image exposes a number of resources that can be configured or mapped to the local machine.

Ports

Port 1111

This port can be mapped to provide external access to the docker image to connect using any of the
Virtuoso client providers such as ODBC, JDBC, ADO.Net, OLE-DB, iSQL from either the host
system or from another docker image.

In case you want to run multiple docker images on the same host system you can use the -p or
--publish command-line option to map a free port on the local system (e.g. 1112) to the default
isql port 1111 on the docker system.

$ docker create .....
  ...
  --publish 1112:1111 \
  ...

Port 8890

This port can be mapped to provide external access to the Virtuoso HTTP server.

In case you want to run multiple docker images on the same host system you can use the -p or
--publish command-line option to map a free port on the local system. E.g. if you want this image
to be the default web-server, you can map port 80 on the host to the default HTTP port 8890 on
the docker system.

$ docker create .....
  ...
  --publish 80:8890 \
  ...

As soon as the docker image has been started you can connect to Virtuoso using the following URL in
your browser:

http://localhost/

Volume(s)

/database

This volume contains the virtuoso.ini, virtuoso.lic and all the files used by the Virtuoso database.

When creating or running any docker container there are two parameters that play an important role in data persistence:

• Named vs Unnamed containers (using the -n or --name argument)

• Mapped or Unmapped volumes (using the -v or --volume argument)

If you create a docker image without specifying either a name for the resulting (unnamed) container and a volume mapping, this docker image will be destroyed the moment the container is stopped and while the directory is not immediately removed from the host filesystem, it may not be easy to locate and recover any data inside.

As long as you at least use the --name argument, the docker container can be stopped and restarted without any loss of data.

If you create a docker image without mapping a volume to a local directory on your hard drive, the docker system will create a directory within its own filesystem which maybe on your root filesystem (on Linux docker images are located under /var/lib/docker/containers). This can cause problems if the filesystem is not large enough or when you want to change to a newer version of the image without losing your existing data. Using an unnamed volume also causes problems when you want to upgrade to a later version of the docker image while keeping your existing database.

While there are circumstances where you may prefer an unnamed container with mapped volume, we recommend you use both --name and --volume parameters on all virtuoso containers you create.

/initdb.d

This directory can contain a mix of shell (.sh) and Virtuoso PL (.sql) scripts that can perform functions such as:

• Installing additional Ubuntu packages
• Loading data from remote locations such as Amazon S3 buckets, Google Drive, or other locations
• Bulk loading data into the Virtuoso database
• Installing additional VAD packages into the database
• Adding new Virtuoso users
• Granting permissions to Virtuoso users
• Regenerating free-text indexes or other initial data

The scripts are run only once, during the initial database creation; subsequent restarts of the docker image will not cause these script to be re-run.

The scripts are run in alphabetical order, so we suggest starting the script names with sequence numbers, so the ordering is explicit and obvious.

For security purposes, Virtuoso will run the .sql scripts in a special mode, and will not respond to connections on its SQL (1111) and/or HTTP (8890) ports.

At the end of each .sql script, Virtuoso automatically performs a checkpoint, to make sure the changes are fully written back to the database. This is very important for scripts that use the bulk loader function rdf_loader_run() or manually change the ACID mode of the database for any reason.

After all the initialization scripts have run to completion, Virtuoso will be started normally, and start listening to requests on its SQL (1111) and HTTP (8890) ports.

Environment variables

DBA_PASSWORD_FILE

This environment variable should be set in combination with one of the secure options to pass
sensitive information to the docker image. The startup script will read the content from this file
on the internal docker filesystem and assign it to the DBA_PASSWORD environment variable
internally. In this case the DBA_PASSWORD setting is not recorded in the meta data of this docker
instance.

Example:

$ echo -n 'verysecretpw' | docker secret create dba_secret -

$ docker service create .... \
    ...
    --secret dba_secret \
    --env DBA_PASSWORD_FILE=/run/secrets/dba_secret \
    ...

See also:

• docker secrets (docker swarm only)
• kubernetes secrets

DAV_PASSWORD_FILE

See DBA_PASSWORD_FILE for more details.

DBA_PASSWORD

If this environment variable is set via either the command-line options -e, --env, --env-file
or via the environment section in a .yml file, the value will be assigned as the initial
password of the dba account when creating a new database.

If this variable is set to ami-id the docker image will attempt to get this value from the EC2
registry on the Amazon AMI. If this fails or if this docker container is not started on an
AMI the variable will be regarded as unset.

If this variable is unset, a random password will be generated and assigned as the initial
password of the dba account when creating a new database.

The password will be stored inside the docker container in /settings/dba_password and we recommend
you remove this file after you have changed the dba password to some other value, either via the
conductor web interface, or via the isql command.

Example:

$ docker create .....
  ...
  --env DBA_PASSWORD=mysecret \
  ...

NOTE: As environment variables are stored in the meta information of the docker image, the
initial password is easily visible to all users on the same machine that are able to run the docker inspect command. In cases where this could be an issue we recommend using the ‘docker secrets’
option in combination with the DBA_PASSWORD_FILE option.

DAV_PASSWORD_FILE

See DBA_PASSWORD_FILE for more details.

DAV_PASSWORD

If this environment variable is set via either the command-line options -e, --env, --env-file
or via the environment section in a .yml file, the value will be assigned as the initial
password of the dav account when creating a new database.

If this variable is not set, the value of the DBA_PASSWORD setting will be used for both dba
and dav accounts.

The password will be stored inside the docker container in /settings/dav_password and we recommend
you remove this file after you have changed the dav password to some other value, either via the
conductor web interface, or via the isql command.

VIRTUOSO_LIC_FILE

This environment variable is used in combination with one of the secure options to pass sensitive
information to the Virtuoso closed source docker images. The startup script will read the content
from this file on the internal docker filesystem and copy it to the /database/virtuoso.lic file.

See also:

• docker secrets (docker swarm only)
• kubernetes secrets

VIRTUOSO_INI_FILE

This environment variable is used in combination with one of the following options to pass
information to the docker image. The startup script will read the content of this file on the
internal docker filesystem and copy it to the /database/virtuoso.ini file, instead of installing
the default virtuoso.ini file from the virtuoso installation.

Please note that all path settings in virtuoso.ini should be based on the internal
filesystem structure of the docker image, and not refer to directory paths on the host system. It is
possible to bind additional directories from the host system into the docker instance using bind mounts using command-line options such as --mount

After installing a virtuoso.ini file, the startup script will parse the environment variables to
allow additional updates to the virtuoso.ini file as detailed in the next paragraph. The startup script will also rewrite the [Plugin] section to enable all the plugins installed in the hosting directory.

See also:

• docker config
• docker secrets (docker swarm only)
• kubernetes secrets
• docker bind mount

Updating virtuoso.ini via environment settings

Using environment variables when creating the Virtuoso docker instance via the command-line options
-e, --env, --env-file or via the environment section in a .yml file, you can add or
overrule any parameter within the virtuoso.ini file.

These environment variables must be named like:

VIRT_SECTION_KEY=VALUE

where

• VIRT is common prefix to group such variables together (always in upper case)
• SECTION is the name of the [section] in virtuoso.ini (case insensitive)
• KEY is the name of a key within the section (case insensitive)
• VALUE is the text to be written into the key (case sensitive)

The SECTION and KEY parts of these variable names can be written in either uppercase (most
commonly used) or mixed case, without having to exactly match the case used inside the virtuoso.ini
file. There is no validation on the SECTION and KEY name parts, so any mistake may result in a
non-functional setting.

The following names all refer to the same setting:

• VIRT_Parameters_NumberOfBuffers (case matching the section and key name in virtuoso.ini)
• VIRT_PARAMETERS_NUMBEROFBUFFERS (recommended way to name environment variables)
• VIRT_parameters_numberofbuffers (optional)

The VALUE is case sensitive string and should be surrounded by either single (`) or double (")
quotes in case there are spaces or other special characters in the string.

If the VALUE is set to ‘-’, the KEY entry will be removed from the SECTION.

Example:

$ docker create .... \
  ... \
  -e VIRT_PARAMETERS_NUMBEROFBUFFERS=1000000 \
  -e VIRT_HTTPSERVER_MaxClientConnections=100 \
  -e VIRT_HTTPSERVER_SSL_PROTOCOLS="TLS1.1, TLS1.2" \
  ...

Please note that all path settings in virtuoso.ini should be based on the internal
filesystem structure of the docker image, and cannot directly refer to directory paths on the host
system. It is possible to bind additional directories from the host system into the docker instance
using bind mounts using command-line options such as --mount.

See also:

• docker bind mount

Related

• Virtuoso Enterprise Edition Docker – Quick Start Guide
• Using the OpenLink Virtuoso Docker container as part of a service or swarm
• How to Bulk Load Data into a Virtuoso Docker Instance