Azure OpenLink Virtuoso Ubuntu PAGO VM

[hwilliams]

Azure OpenLink Virtuoso Ubuntu PAGO VM

Introduction

This document details the process for deploying the OpenLink Virtuoso Pay-As-You-Go (PAGO) Edition Marketplace Offer for the Azure Cloud.

Prerequisites

• An Azure Cloud subscription account.

Deployment from Azure Marketplace Offer

1. From the Azure Marketplace search for the keywords OpenLink or Virtuoso to locate the available Virtuoso offerings and select the PAGO offer:
   
   
   
   
   
   
   
2. Click on the GET IT NOW button to start the subscription and deployment of the PAGO offer.
   
   
   
   
   
   
   
3. Select Continue to start the deployment process in the Azure Portal.
   
   
   
   
   
   
   
4. Select the Create button to start the VM configuration manually or select the Start with a pre-set configuration to start the a pre set configuration.
   
   
   
   
   
   
   
   
   
   
   
   
5. From the Basics tab choose the Resource group , which can be an existing or new group; set the Virtual machine name to call the deployment; set the Regionto deploy from; forSSH public key source` choose to either use and existing or public key to be used.
   
   
   
   
   
   
   
   
   
   
6. From the Disks tab, typically the defaults can be used, or additional disks can be added as required.
   
   
   
   
   
   
   
7. From the Networking tab, typically the defaults can be used.
   
   
   
   
   
   
   
8. From the Management tab, typically the defaults can be used.
   
   
   
   
   
   
   
9. From the Advanced tab, typically the defaults can be used.
   
   
   
   
   
   
   
10. From the Tags tab, optionally add any tags required.
    
    
    
    
    
    
    
11. Once the Validation passed message occurs, click on the Create button to start the deployment.
    
    
    
    
    
    
    
12. The Deployment is in progress message indicator will be displayed.
    
    
    
    
    
    
    
13. The Your deployment is complete message is display once successfully completed, at which point the Go to resource button can be clicked.
    
    
    
    
    
    
    
14. The Overview page of the successfully deployed VM is displayed, from which the Public IP address can be copied for use to access the VM via ssh and http.
    
    
    
    
    
    
    

Administering Virtuoso Instance via SSH

1. Make a ssh connection to the VM using the public key (pem-file) and username (azureuser by default) chosen when creating the deployment, and the Public IP address from the previous section as follows:

    ssh -i {pem-file} azureuser@{Public IP address}

2. Once connected it is strongly recommended to update the VM to get the latest operating system and Virtuoso updates with the command:

    sudo apt-get upgrade

3. The following commands can be used to Administer the Virtuoso server:

• Start the Virtuoso Server:

    sudo service virtuoso start

• Stop the Virtuoso Server:

   sudo service virtuoso stop

• Restart the Virtuoso Server:

    sudo service virtuoso restart

• Check status of Vrtuoso Server:

    sudo service virtuoso status

4. Check the Virtuoso server is automatically started post deployment with the command:

    sudo service virtuoso status

5. Determine the random password set for the dba user with the command:

    sudo cat /opt/virtuoso/database/.initial-password

6. A SQL connection can then be made Virtuoso with the isql command line tool with the command on port 1111:

    isql 1111

7. Typical output for running these steps are:

$ ssh -i certificates/VirtVM.pem azureuser@51.104.20.171
The authenticity of host '51.104.20.171 (51.104.20.171)' can't be established.
ECDSA key fingerprint is SHA256:UOFjyixA/U5r1s8Vev5T4zFQw+1V+8xglW2cDWss6Ns.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '51.104.20.171' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1031-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Fri Oct 23 14:11:11 UTC 2020

  System load:  0.42              Processes:           134
  Usage of /:   3.4% of 48.29GB   Users logged in:     0
  Memory usage: 4%                IP address for eth0: 10.1.5.4
  Swap usage:   0%

 * Introducing autonomous high availability clustering for MicroK8s
   production environments! Super simple clustering, hardened Kubernetes,
   with automatic data store operations. A zero-ops HA K8s for anywhere.

     https://microk8s.io/high-availability

0 packages can be updated.
0 updates are security updates.

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

azureuser@VirtUbuntuBYOLTestOffer:~$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  base-files ubuntu-server
The following packages will be upgraded:
  apport bcache-tools cifs-utils cloud-init cryptsetup cryptsetup-bin grub-common grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed
  grub2-common initramfs-tools initramfs-tools-bin initramfs-tools-core iproute2 kmod libcryptsetup12 libkmod2 libnss-systemd libpam-modules
  libpam-modules-bin libpam-runtime libpam-systemd libpam0g libpcap0.8 libsystemd0 libudev1 python3-apport python3-distupgrade
  python3-problem-report shim shim-signed snapd systemd systemd-sysv ubuntu-minimal ubuntu-release-upgrader-core ubuntu-standard udev
  virtuoso-8-commercial-azure virtuoso-8-commercial-base virtuoso-8-commercial-client-isql virtuoso-8-commercial-client-jdbc
  virtuoso-8-commercial-client-odbc virtuoso-8-commercial-docs-manpages virtuoso-8-commercial-plugin-evc virtuoso-8-commercial-plugin-geo
  virtuoso-8-commercial-plugin-im virtuoso-8-commercial-plugin-ods virtuoso-8-commercial-plugin-qrcode virtuoso-8-commercial-server
  virtuoso-8-commercial-vad-cartridges virtuoso-8-commercial-vad-conductor virtuoso-8-commercial-vad-ods virtuoso-8-commercial-vad-val
  virtuoso-8-commercial-vsp-startpage
56 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Need to get 50.6 MB of archives.
After this operation, 4516 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam0g amd64 1.1.8-3.6ubuntu2.18.04.2 [54.9 kB]
Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-modules-bin amd64 1.1.8-3.6ubuntu2.18.04.2 [36.7 kB]
Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-modules amd64 1.1.8-3.6ubuntu2.18.04.2 [242 kB]
.
.
.
Setting up virtuoso-8-commercial-plugin-evc (8.3-2.181~gcb9d9aaa0d) ...
Setting up virtuoso-8-commercial-vad-cartridges (8.3-1.101~git208) ...
Setting up cryptsetup-bin (2:2.0.2-1ubuntu1.2) ...
Setting up virtuoso-8-commercial-plugin-ods (8.3-2.181~gcb9d9aaa0d) ...
Setting up virtuoso-8-commercial-base (8.3-2.181~gcb9d9aaa0d) ...
Setting up initramfs-tools-bin (0.130ubuntu3.11) ...
Setting up virtuoso-8-commercial-plugin-qrcode (8.3-2.181~gcb9d9aaa0d) ...
Setting up virtuoso-8-commercial-client-jdbc (8.3-2.181~gcb9d9aaa0d) ...
Setting up virtuoso-8-commercial-vad-ods (8.3-2.2.1~git719) ...
Setting up python3-distupgrade (1:18.04.40) ...
Setting up virtuoso-8-commercial-vsp-startpage (8.3-2.181~gcb9d9aaa0d) ...
Setting up cloud-init (20.3-2-g371b392c-0ubuntu1~18.04.1) ...
Installing new version of config file /etc/cloud/cloud.cfg.d/05_logging.cfg ...
Installing new version of config file /etc/cloud/templates/hosts.freebsd.tmpl ...
Setting up libpcap0.8:amd64 (1.8.1-6ubuntu1.18.04.2) ...
Setting up libpam-systemd:amd64 (237-3ubuntu10.42) ...
Setting up grub-efi-amd64-bin (2.02-2ubuntu8.18) ...
Setting up initramfs-tools-core (0.130ubuntu3.11) ...
Setting up cryptsetup (2:2.0.2-1ubuntu1.2) ...
update-initramfs: deferring update (trigger activated)
Setting up grub2-common (2.02-2ubuntu8.18) ...
Setting up virtuoso-8-commercial-server (8.3-2.181~gcb9d9aaa0d) ...

- Configuring: database
Setting up initramfs-tools (0.130ubuntu3.11) ...
update-initramfs: deferring update (trigger activated)
Setting up ubuntu-release-upgrader-core (1:18.04.40) ...
Setting up python3-apport (2.20.9-0ubuntu7.18) ...
Setting up snapd (2.46.1+18.04) ...
Installing new version of config file /etc/apparmor.d/usr.lib.snapd.snap-confine.real ...
Installing new version of config file /etc/profile.d/apps-bin-path.sh ...
snapd.failure.service is a disabled or a static unit, not starting it.
snapd.snap-repair.service is a disabled or a static unit, not starting it.
Setting up apport (2.20.9-0ubuntu7.18) ...
apport-autoreport.service is a disabled or a static unit, not starting it.
Setting up ubuntu-minimal (1.417.5) ...
Setting up virtuoso-8-commercial-azure (8.3-2.181~gcb9d9aaa0d) ...
Setting up ubuntu-standard (1.417.5) ...
Setting up grub-efi-amd64 (2.02-2ubuntu8.18) ...
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/50-cloudimg-settings.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.4.0-1031-azure
Found initrd image: /boot/initrd.img-5.4.0-1031-azure
Found linux image: /boot/vmlinuz-5.3.0-1034-azure
Found initrd image: /boot/initrd.img-5.3.0-1034-azure
done
Setting up grub-efi-amd64-signed (1.93.20+2.02-2ubuntu8.18) ...
Setting up shim-signed (1.37~18.04.8+15+1552672080.a4a1fbe-0ubuntu2) ...
Processing triggers for systemd (237-3ubuntu10.42) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for dbus (1.12.2-1ubuntu1.2) ...
Processing triggers for rsyslog (8.32.0-1ubuntu4) ...
Processing triggers for ufw (0.36-0ubuntu0.18.04.1) ...
Rules updated for profile 'oplmgr'

Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for install-info (6.5.0.dfsg.1-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
Processing triggers for initramfs-tools (0.130ubuntu3.11) ...
update-initramfs: Generating /boot/initrd.img-5.4.0-1031-azure
azureuser@VirtUbuntuPAGOTestOffer:~$ sudo service virtuoso status
● virtuoso.service - OpenLink Virtuoso Database
   Loaded: loaded (/lib/systemd/system/virtuoso.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-10-23 14:09:25 UTC; 2min 13s ago
  Process: 1310 ExecStart=/opt/virtuoso/bin/virtuoso-start.sh $VIRTUOSO_DB_NAMES (code=exited, status=0/SUCCESS)
 Main PID: 1906 (virtuoso)
    Tasks: 19 (limit: 4915)
   CGroup: /system.slice/virtuoso.service
           └─1906 ./virtuoso

Oct 23 14:09:05 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:   - Initializing the new database
Oct 23 14:09:22 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:   - Generating a random password
Oct 23 14:09:23 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:   - Securing database
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]: NOTE: The new database has been initialized with a randomly generated
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:       password for both dba and dav accounts. Make sure you change it
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:       at your earliest opportunity.
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:       The password has been saved to disk in:
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:             database/.initial-password
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer virtuoso-start.sh[1310]:   - Starting the database
Oct 23 14:09:25 VirtUbuntuPAGOTestOffer systemd[1]: Started OpenLink Virtuoso Database.
azureuser@VirtUbuntuPAGOTestOffer:~$ sudo cat /opt/virtuoso/database/.initial-password 
a6ea3012-aad9-4e10
azureuser@VirtUbuntuPAGOTestOffer:~$ which isql
/opt/virtuoso/bin/isql
azureuser@VirtUbuntuPAGOTestOffer:~$ isql 1111
OpenLink Virtuoso Interactive SQL (Virtuoso)
Version 08.03.3319 as of Sep  1 2020
Type HELP; for help and EXIT; to exit.

Enter password for dba :
Connected to OpenLink Virtuoso
Driver: 08.03.3319 OpenLink Virtuoso ODBC Driver
SQL> status('');
REPORT
VARCHAR
_______________________________________________________________________________

OpenLink Virtuoso VDB Server
Version 08.03.3319-pthreads for Linux as of Sep  1 2020 
Registered to OpenLink Virtuoso on Azure (PAGO) (Personal Edition, unlimited connections)
Started on: 2020-10-23 14:09 GMT+0
CPU: 0.00% RSS: 173MB PF: 0
 
Database Status:
  File size 69206016, 8448 pages, 6000 free.
  20000 buffers, 1125 used, 3 dirty 0 wired down, repl age 0 0 w. io 0 w/crsr.
  Disk Usage: 1105 reads avg 0 msec, 0% r 0% w last  10455 s, 319 writes flush      5.208 MB/s,
    34 read ahead, batch = 17.  Autocompact 21 in 20 out, 4% saved.
Gate:  68 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap. 
Log = virtuoso.trx, 6555 bytes
VDB: 0 exec 0 fetch 0 transact 0 error
2379 pages have been changed since last backup (in checkpoint state)
Current backup timestamp: 0x0000-0x00-0x00
Last backup date: unknown
Clients: 1 connects, max 1 concurrent
RPC: 6 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0 second 0M large, 14M max
Checkpoint Remap 38 pages, 0 mapped back. 0 s atomic time.
    DB master 8448 total 6000 free 38 remap 0 mapped back
   temp  256 total 251 free
 
Lock Status: 0 deadlocks of which 0 2r1w, 0 waits,
   Currently 1 threads running 0 threads waiting 0 threads in vdb.

25 Rows. -- 3 msec.
SQL> 

Accessing the Virtuoso Instance via HTTP

1. The Virtuoso http server can be access on http://{Public IP Address}:8890:
   
   
   
   
   
   
   
2. Click on the Conductor link in the left frame to access the Virtuoso Conductor Admin UI.
   
   
   
   
   
   
   
3. Using the Virtuoso dba user password determines as in Step 17 to login to the Conductor
   
   
   
   
   
   
   

Performance Tuning

There are a range of Azure VM instance types with different system memory and CPU combinations. Collectively, the factors above affect the performance of your Virtuoso instance. Thus, use Azure VM Instance Type s with more memory and CPU cores for best performance.

Note: This VM is configured to use minimal system memory. For the instance type chosen, the NumberOfBuffer and MaxDirtyBuffers parameters in the /opt/virtuoso/database/virtuoso.ini configuration file should be increased to match the available memory, as detailed in the Virtuoso Performance Tuning Guide, for example –

VM Instance Type	System RAM	Number Of Buffers	Max Dirty Buffers
B2MS	8 GB	680000	500000
B4MS	15 GB	1360000	1000000
M32S	32 GB	2720000	2000000
M64LS	64 GB	5450000	4000000

– and the Virtuoso server restarted as detailed above.

Extrapolate the NumberOfBuffer and MaxDirtyBuffers parameters accordingly for different sized VMs.

Trouble Shooting

If the Virtuoso server fails to start:

• Run the command sudo service virtuoso status to see if the Virtuoso server is running
• Check the /opt/virtuoso/database/virtuoso.log file to see why the server might have failed to start
• Ensure there file /opt/virtuoso/database/virtuoso.lck does not exist before starting the server
• Attempt to start the Virtuoso server with the command sudo service virtuoso start
• Run the command sudo service virtuoso status again to see if the Virtuoso server is running
• If it is now running attempt a connect via the SQL or HTTP interfaces are detailed above

Related

• Virtuoso Bring-Your-Own-License (BYOL) Edition for Azure
• Virtuoso Home Page
• Virtuoso Weblog
• OpenLink Software Home Page
• OpenLink Software Community Forum
• Twitter Page
• LinkedIn Page