Multi-threaded on dumping graph

Virtuoso Universal Server
1

Hi,
I am trying to dump a hundred millions triples from Virtuoso, but the speed is not ideal enough and CPU cores seems not be efficiently used which is always 200% utilization.

Dump procedure is from 16.17.9. Dump and Reload Graphs .

Thank you very much!

2

The Dump Graphs function is a serial process thus you would not get full platform utilisation. We also have a Dump Graph to NQUADs function which is the preferred method of dumping Graphs in the Virtuoso QUAD Store.

3

I am sorry and I edited my mistake that I am trying to dump a hundred millions triples not graphs.
It costs an hour to dump 300+ millions triples from virtuoso , it that normal?
Or is there any higher efficient function?

Thank you very much!

4

The rate at which the triples can be dumped would be dependent on the hardware being run on, i.e., processor speed, memory, etc.

The state of the Virtuoso database would also be relevant in terms of memory allocated to Virtuoso for hosting the triples and dumping them. When the dump is being performed you should run the status(); command and in particular check the number of Buffers allocated and used, ensuring all the buffers are not being used which would result in swapping between memory and disk which will always significantly degrade performance:

SQL> status();
REPORT
VARCHAR
_______________________________________________________________________________

OpenLink Virtuoso VDB Server
Version 08.03.3314-pthreads for Mac OS X as of May 24 2019 
Registered to OpenLink Software (INTERNAL USE ONLY) (Personal Edition, 500 connections)
Started on: 2019-07-16 13:15 GMT+1
CPU%: 0.05
 
Database Status:
  File size 400556032, 48896 pages, 14588 free.
  20000 buffers, 3523 used,
.
.
.

Setting log_enable(2) to turn off transaction logging and turn on autocommit would also reduce the memory requirement and speed the dump process.