Updating a Virtuoso Graph with Jena

[iesnaola]

(replicates virtuoso-opensource issue #830 on github)

Hi,

I am trying to update the content of a given Graph in a Virtuoso Open Source 07.20.3217 version. This is the code I am using:

//Connect to Virtuoso
VirtModel virtualModel = VirtModel.openDatabaseModel(GRAPH_NAME, endpoint, USERNAME, PASSWORD);
long modelInitialSize = virtualModel.size();

//Add model
virtualModel.add(model);
long modelNewSize = virtualModel.size();

virtualModel.close();

Where the endpoint parameter is something like jdbc:virtuoso://10.1.22.xxx:1111

And this is the content of the pom.xml file:

<!-- https://mvnrepository.com/artifact/org.apache.jena/jena-tdb -->
    <dependency>
        <groupId>org.apache.jena</groupId>
        <artifactId>jena-tdb</artifactId>
        <version>3.9.0</version>
    </dependency>


    <!-- https://mvnrepository.com/artifact/org.apache.jena/apache-jena-libs -->
    <dependency>
        <groupId>org.apache.jena</groupId>
        <artifactId>apache-jena-libs</artifactId>
        <version>3.9.0</version>
        <type>pom</type>
    </dependency>

Besides, I am using the virt_jena3.jar and virtjdbc4.jar dependencies from the Virtuoso webpage.

The function executes well until I reach the command: long modelNewSize = virtualModel.size();. It produces the following error org.apache.jena.shared.JenaException: virtuoso.jdbc4.VirtuosoException: Problem during serialization : Software caused connection abort: socket write error.

Once this error happens, it seems like the Virtuoso server goes down, and I need to restart the Virtual Machine in order to make it available again.

Anyone dealing with similar issues?

[TallTed]

07.20.3217 dates from 2016-04-25.

Are you able to update to 07.20.3229 (2018-08-15, stable/7) or 07.20.3230+ (2019-01-16, develop/7)?

I am not aware of any specific patches that would impact this issue, but it’s usually best to test with the latest possible binary before digging further.

[iesnaola]

Hi @TallTed,

I will try that one. Finger crossed it will help fixing the issue…

[iesnaola]

Hi,

I tried updating my Virtuoso Open Source 07.20.3217 to version 07.20.3229 but it seems there was some problem… so I started from scratch.

I have a new Ubuntu Server 18.04 where I am installing the Virtuoso version 07.20.3229. I follow the instructions as indicated in the README. However, when I try to run Virtuoso, with the command virtuoso-t -f &, I get the following error:

Command ‘virtuoso-t’ not found, but can be installed with
sudo apt install virtuoso-opensource-6.1-bin

Does this mean that I need to install Virtuoso version 6.1 previously? Or is is that I am missing some package not included in the Virtuoso 07.20.3229 I am installing?

[TallTed]

Note – the README you followed includes several choices to be made by the user, and without knowing what your choices were, we cannot provide specific commands that involve fully-qualified paths (among other things) at this point.

(For the future, it’s usually better to provide textual information, such as the command and error shown above, as text, rather than [or as well as] screenshots.)

Ubuntu knows that Virtuoso exists, but only knows about the very old apt packaging of v6.1, so when Ubuntu notices that someone unsuccessfully tries to run virtuoso-t, it “helpfully” suggests that v6.1 should be installed.

You should just need to set your environment such that the v7.20.3229 virtuoso-t binary is found within your active $PATH, or use the full path to the binary to start it, e.g., –

/usr/local/bin/virtuoso-t -f &

– or –

. /path/to/virtuoso-environment.sh
virtuoso-t -f &

[iesnaola]

Thanks for the help.

I think I am using the second approach you seggested: using the full path to the binary. This is where the binary was installed (correct me if I am misunderstanding it):

/usr/local/virtuoso-opensource/bin

As a matter of fact, when I execute the ls command in this directory, this is what I get:

I see that “virtuoso-t” file in this directory, so I try to execute the command:

virtuoso-t -f &

However, I get the same error:

Command 'virtuoso-t' not found, but can be installed with:
sudo apt install virtuoso-opensource-6.1-bin

Am I missing something? Thanks.

[TallTed]

@iesnaola -

It appears that your $PATH does not include . (the pwd shortcut), so the shell doesn’t look in your current directory for the command you’re trying to execute. You could also add /usr/local/virtuoso-opensource/bin to your $PATH such that virtuoso-t may be executed from any directory – such as the /usr/local/virtuoso-opensource/database @hwilliams suggested on github.

A “full path command”, which you could execute anywhere, without changing your $PATH, would be –

/usr/local/virtuoso-opensource/bin/virtuoso-t -f &

[iesnaola]

I think I am making some progresses thanks to your answers, but I am still not able to run this Virtuoso Open Source 07.20.3217 version.

As you suggested, I used the following “full path command”:

sudo /usr/local/virtuoso-opensource/bin/virtuoso-t -f &

I used sudo to avoid the error Unable to create file /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.lck (Permission denied).

Furthermore, I am execute this “full path command” from the directory where the virtuoso.ini file is (as suggested by @HughWilliams ).

Once I execute this command, I get the following response:
[1] 1013 (These numbers may vary).

However, when I try to access the Virtuoso SPARQL endpoint (at localhost:8890/sparql) or even the Virtuoso conductor (at localhost:8890), I cannot access to these pages (ERR_CONNECTION_TIMED_OUT ).

I just double checked whether the virtuoso.ini had other port by default, but it seems it is okay.

[HTTPServer]
ServerPort = 8890
ServerRoot = /usr/local/virtuoso-opensource/var/lib/virtuoso/vsp
MaxClientConnections = 10
(…)

Any ideas of what is going on here?

[TallTed]

The Unable to create file /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.lck (Permission denied). error should NOT be simply over-ridden with sudo – this message means that you should either change the permissions on the /usr/local/virtuoso-opensource/var/lib/virtuoso/db/ directory (such that the user who is launching Virtuoso, and under whose authority Virtuoso will therefore operate, has write permissions) – or address the fact that /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.lck already exists – which it may!

The [1] 1013 message comes from Linux, and simply means “background process 1 is running as Process ID 1013”. This is normal, and the numbers will indeed change with each execution.

The first “test” URL you should check is http://localhost:8890/ which will bring up a basic “splash” screen. The Conductor (if this VAD has been installed) would then be found at http://localhost:8890/conductor.

As to “any idea of what is going on here,” please look to the virtuoso.log file, which should be found alongside the virtuoso.ini and virtuoso.db (and virtuoso.lck) for this instance. Hopefully there are some useful messages written there, even though they are somehow not showing up in stdout/stderr as they should with the -f (a/k/a +foreground) command argument.

[iesnaola]

Hi @TallTed ,

Okay, so now the correct permissions are set in the /usr/local/virtuoso-opensource/var/lib/virtuoso/db/ directory. Then I execute the /usr/local/virtuoso-opensource/bin/virtuoso-t -f & command and I get the following in the log file:

  Mon Mar 25 2019

00:41:05 { Loading plugin 1: Type plain', file wikiv’ in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting’

00:41:05 WikiV version 0.6 from OpenLink Software
00:41:05 Support functions for WikiV collaboration tool
00:41:05 SUCCESS plugin 1: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/wikiv.so }
00:41:05 { Loading plugin 2: Type plain', file mediawiki’ in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting’

00:41:05 MediaWiki version 0.1 from OpenLink Software
00:41:05 Support functions for MediaWiki collaboration tool
00:41:05 SUCCESS plugin 2: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/mediawiki.so }
00:41:05 { Loading plugin 3: Type plain', file creolewiki’ in `/usr/local/virtuoso-opensource/lib/virtuoso/hosting’
00:41:05 CreoleWiki version 0.1 from OpenLink Software
00:41:05 Support functions for CreoleWiki collaboration tool
00:41:05 SUCCESS plugin 3: loaded from /usr/local/virtuoso-opensource/lib/virtuoso/hosting/creolewiki.so }
00:41:05 OpenLink Virtuoso Universal Server
00:41:05 Version 07.20.3229-pthreads for Linux as of Mar 12 2019
00:41:05 uses parts of OpenSSL, PCRE, Html Tidy
00:41:05 SQL Optimizer enabled (max 1000 layouts)
00:41:05 Compiler unit is timed at 0.000301 msec
00:41:07 Checkpoint started
00:41:07 Roll forward started
00:41:07 Roll forward complete
00:41:08 Checkpoint started
00:41:08 Checkpoint finished, log reused
00:41:08 Checkpoint started
00:41:08 Checkpoint finished, log reused
00:41:08 Checkpoint started
00:41:08 Checkpoint finished, log reused
00:41:08 Checkpoint started
00:41:09 Checkpoint finished, log reused
00:41:09 Checkpoint started
00:41:09 Checkpoint finished, log reused
00:41:09 PL LOG: Installing Virtuoso Conductor version 1.00.8783 (DAV)
00:41:09 PL LOG: Installing with dependencies Virtuoso Conductor version 1.00.8783/2019-03-12 08:24 (DAV)
00:41:09 Checkpoint started
00:41:09 Checkpoint finished, log reused
00:41:11 Checkpoint started
00:41:11 Checkpoint finished, log reused
00:41:11 PL LOG: Installation with dependencies complete
00:41:11 Checkpoint started
00:41:11 Checkpoint finished, log reused
00:41:11 HTTP/WebDAV server online at 8890
00:41:11 Server online at 1111 (pid 847)

As I see it, the server should be online, and I should be able to access a ‘splash screen’ in http://localhost:8890/ .

However, there is no page. Not even in “conductor” or “sparql endpoint”.

I double-checked my “localhost” address, and it seems it is fine, as I can connect to it via WinSCP for example.

I am not sure which is the correct way to stop the service and re-start it again (in case the problem just gets fixed when restarting it…)

[TallTed]

OK… The log entries indicate that the Conductor VAD has been installed successfully, so both http://localhost:8890/ and http://localhost:8890/conductor should load when the process is running (and the browser is running on the same host).

(Please note that localhost points to 127.0.0.1, which is the machine on which a process is running – so if you run WinSCP on Windows, localhost means that Windows host. Since Virtuoso is running on Linux, you need to either run your browser on Linux, or point to the IP address of the Linux environment, which you can usually discover by running ifconfig -a and/or netstat -i.)

1. Is the Virtuoso process still running? Check ps -ef | grep virtuoso or similar.

2. Do your firewall settings permit connections to port 8890? Note – you’ll also need to open up 1111, assuming default Virtuoso settings, for SQL (isql, ODBC, JDBC, ADO.NET, OLE DB, etc.) data connections.

[TallTed]

As commented on the parallel github issue, progress –

@iesnaola

Thanks @TallTed for the hint. It was the firewall which was not allowing me to connect to port 8890. Now that being solved, I will go back and try to update the Virtuoso Graph (which was the original reason of my post) and see how it goes.

– and success –

@iesnaola

Good news. Once the new Virtuoso has been installed, the addition of a new model to a graph via code is correctly done!! Thank you very much for your help throughout this process!