Using the OpenLink Virtuoso Docker container as part of a service or swarm

docker
virtuosordbms

#1

OpenLink Virtuoso Enterprise Edition 8.2 Docker Image

Copyright © 2018 OpenLink Software

Running Virtuoso as part of a Docker Service

Introduction

This document describes various ways you can use the Virtuoso Docker images as part of a set of micro services.

Micro services normally consist of two or more components – each of them located in their own small container – that work together to deliver a service or application. A prime example is a LAMP (Linux, Apache, Mysql, and PHP) stack that normally would be installed on a single host. With docker we can split these components into separate containers such that one (or more) containers run the Apache + PHP + the web application we want to host, where another container runs the MySQL backend database.

While this can be accomplished using individual commands to start each container using standard docker create commands, we can also use commands like docker-compose and docker stack to orchestrate creation of multiple containers that are connected internally using a separate private network with options to start multiple instances, build in redundancy and monitoring of services, and even add computing power by adding extra computers to split the workload.

Preamble

Before you can use either docker-compose, docker service, or docker stack commands to create services, you will need to initialize your host system as a swarm manager using the docker swarm init command as shown below:

$ docker swarm init

Swarm initialized: current node (ucruXXXXXXXXX) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-XXXXXXXXXXXX 192.168.1.100:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

Note: Docker Community Edition for macOS initializes itself as a swarm manager by default, thus the above command does not typically have to be run on macOS.

Using docker-compose to create a standalone container

Here is a quick example of how to use docker-compose to start a standalone virtuoso docker instance.

First we create a directory that will contain the configuration file and Virtuoso database documents. All subsequent docker-compose commands should be run from this directory.

$ mkdir v8test
$ cd v8test

Create a file called docker-compose.yml and paste in the following content. Please note that .yml are whitespace sensitive, and you should use SPACE characters instead of TAB characters for indentation.

#  
#  Example of a docker config file to instantiate a standalone Virtuoso docker container
#
#  Copyright (C) 2018 OpenLink Software
#
version: "3"

services:

    virtuoso:

        container_name:
            virt8test

        image:
            openlink/virtuoso-closedsource-8:latest

        environment:
            DBA_PASSWORD: mysecret

        ports:
            - "1111:1111"
            - "8890:8890"

        volumes:
            - ./db:/database

#
# vim: tabstop=4 shiftwidth=4 softtabstop=4 expandtab:
#

Creating the container

Next we can create and start the instance by running the following command:

$ docker-compose up -d

WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Creating network "standalone_default" with the default driver
Creating virt8test ... 
Creating virt8test ... done

This command will create a subdirectory called db where the Virtuoso database documents will be stored.

The -d option will start the container in a detached state. If you leave out this flag, you will start the container in the foreground (similar to a docker run --interactive --tty), and you will have to use CTRL-C to stop the container and return to the prompt.

Checking the state of the container

$ docker-compose ps
  Name              Command           State                             Ports                          
-------------------------------------------------------------------------------------------------------
virt8test   /virtuoso-entrypoint.sh   Up      0.0.0.0:1111->1111/tcp, 60001/tcp, 0.0.0.0:8890->8890/tcp

Checking the logs

$ docker-compose logs
...
...
virt8test   | 18:27:45 HTTP/WebDAV server online at 8890
virt8test   | 18:27:45 Server online at 1111 (pid 1)
virt8test   | 18:27:46 ZeroConfig registration virtuoso (541879C56FD2)

Executing commands in the container

To start a bash shell inside the container, we can run the following command:

$ docker exec --interactive --tty virt8test bash

Stopping the container

To stop the container, we can run the following command:

$ docker-compose stop

Starting the container

To restart the container, we can run the following command:

$ docker-compose start

Connecting to the HTTP port

If you have a browser running on the host machine, you can put the following URL into your address bar:

https://localhost:8890/

Removing the container

To stop the container and remove its definition, we can use:

$ docker-compose down
Stopping virt8test ... done
Removing virt8test ... done
Removing network standalone_default

NOTE: As we mapped the database to the ./db subdirectory, the content of the database is saved and will be reused as long as the container is recreated with the same volumes mappings.

Running virtuoso as a standalone service

Creating and starting a service

In the following example, we will create a simple docker service that creates a single replica of a Virtuoso Docker container that exposes the HTTP port only to the host machine that is managing the container:

$ mkdir v8service
$ cd v8service
$ mkdir db

$ docker service create \
    --replicas 1 \
    --name virt8-service \
    --publish 8890:8890 \
    --mount type=bind,source=`pwd`/db,destination=/database \
    openlink/virtuoso-closedsource-8

vd14oguo4y04ekq6lkafox34z
overall progress: 1 out of 1 tasks 
1/1: running   [==================================================>] 
verify: Service converged 
$

Checking the logs

$ docker service logs virt8-service
...
...
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:35 PL LOG: DB.DBA.SYS_PROJ4_SRIDS now contains 8650 spatial reference systems
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:35 Checkpoint started
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:36 Checkpoint finished, log reused
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:38 HTTP Server threads exceed the number of licensed connections. Setting to 1
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:38 HTTP/WebDAV server online at 8890
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:38 Server online at 1111 (pid 1)
virt8-service.1.68pm30j0c9or@dagobert    | 14:55:38 ZeroConfig registration virtuoso (ABA1850141BB)

Check the status of the service

We can list all the services that are currently defined using the following command:

$ docker service ls

ID                  NAME                MODE                REPLICAS            IMAGE                                     PORTS
kxdfug9djcv7        virt8-service       replicated          1/1                 openlink/virtuoso-closedsource-8:latest   *:8890->8890/tcp              

We can check the state of the created micro-service(s) using:

$ docker service ps virt8-service

ID                  NAME                IMAGE                                     NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
68pm30j0c9or        virt8-service.1     openlink/virtuoso-closedsource-8:latest   dagobert            Running             Running 9 minutes ago                     

Finally, we can check the state of the underlying docker container:

$ docker ps

CONTAINER ID        IMAGE                                     COMMAND                  CREATED             STATUS              PORTS                           NAMES
aba1850141bb        openlink/virtuoso-closedsource-8:latest   "/virtuoso-entrypoin…"   12 minutes ago      Up 12 minutes       1111/tcp, 8890/tcp, 60001/tcp   virt8-service.1.68pm30j0c9orjau0kbzcqycua

Starting a shell in the container

As we can see from the above docker ps command, the container id is aba1850141bb, so we can use the following command to start a bash shell:

$ docker exec --interactive --tty aba1850141bb bash

Connecting to the HTTP port

When using a docker service command, the docker subsystem creates an internal private network for the underlying containers and only maps the published ports to the IP address of the host machine. This means that you can use either the IP address or the hostname of your machine to connect to the port, but not localhost.

In this case, you can use the following in your browser’s address bar to contact the Virtuoso HTTP server:

http://hostname:8890/

Removing the service

To stop and remove the service from the host, use:

$ docker service rm virt8-service

virt8-service

NOTE: As we mapped the database to the ./db subdirectory, the content of the database is saved and will be reused as long as the container is recreated with the same bind mount mappings.

Running virtuoso as part of a docker swarm

Creating an instance

In this example, we first create a couple of secrets which will contain the passwords we want to pass to the Virtuoso Docker instance:

$ echo -n 'dbasecret' | docker secret create virt_dba -
$ echo -n 'davsecret' | docker secret create virt_dav -

Next, we edit a file called swarm-compose.yml and paste in the following content. Remember that .yml are whitespace sensitive, and you should use SPACE characters instead of TAB characters for indentation.

#  
#  Example of a docker config file to instantiate a standalone Virtuoso docker container in a swarm
#
#  Copyright (C) 2018 OpenLink Software
#

version: "3.1"
services:

    virtuoso:

        image:
            openlink/virtuoso-closedsource-8:latest

        environment:
            DBA_PASSWORD_FILE: /run/secrets/virt_dba
            DAV_PASSWORD_FILE: /run/secrets/virt_dav

        ports:
            - 8890:8890
            - 1111:1111

        volumes:
            - db-data:/database

        networks:
            - frontend

        secrets:
            - virt_dba
            - virt_dav

        deploy:
            replicas: 1
            placement:
                constraints: [node.role == manager]


networks:
    frontend:

volumes:
    db-data:


secrets:
    virt_dba:
        external: true

    virt_dav:
        external: true


#
# vim: tabstop=4 shiftwidth=4 softtabstop=4 expandtab:
#

We can now use the following command to deploy this service:

$ docker stack deploy --compose-file swarm-compose.yml virt8-swarm

Check the status of the swarm

Check which stacks are currently defined:

$ docker stack ls
NAME                SERVICES
virt8-swarm         1

Check the state of all the tasks in the stack:

$ docker stack ps virt8-swarm
ID                  NAME                     IMAGE                                   NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
tu6qmh35eauq        virt8-swarm_virtuoso.1   openlink/virtuoso-closedsource-8:latest   dagobert            Running             Running 28 seconds ago                       

List the services in the stack:

$ docker stack services virt8-swarm
ID                  NAME                   MODE                REPLICAS            IMAGE                                   PORTS
7ukvfpznisyj        virt8-swarm_virtuoso   replicated          1/1                 openlink/virtuoso-closedsource-8:latest   *:1111->1111/tcp,*:8890->8890/tcp

Connecting to the HTTP port

When using a docker service command the docker subsystem creates an internal private network for the underlying containers and only maps the published ports to the IP address of the host machine. This means that you can use either the IP address or the hostname to connect to the port, but not localhost .

In this example you can use the following in your browser’s address bar to contact the Virtuoso HTTP server:

http://hostname:8890/