Virtuoso RDF Graph Replication log files Purge Function

What

By default Virtuoso RDF Graph Replication writes replication log and configuration files to the same directory as the replication master publisher node database files reside in.

Why

The replication logs files grow in number continuously overtime as more update activities take place on the replication master publisher node, consuming a significant amount of disk space. There is this the need for a function to purge these logs files and also and also write them to a different location if required.

How

A new function repl_purge_to_level ('__rdf_repl') is available in Virtuoso 08.03.3315+ and 07.20.3232+ builds for purging replication logs. This function will purge all existing replication logs up to the current log, unless second argument is given to specify to what replication level (also called sync anchor) to start the purge from ie repl_purge_to_level ('__rdf_repl', level).

After running the repl_purge_to_level ('__rdf_repl') function the purged logs will be renamed to xxx.log.bak and the repl.cfg file will be updated with active replications logs. The purged xxx.log.bak files can then be archived or deleted as required.

Example

SQL> repl_purge_to_level ('__rdf_repl');

Done. -- 17 msec.
SQL> quit;
hwilliams@madras:/openlink/support/virtuoso/v72$ ls -ltr database 
total 1505211
lrwxrwxrwx 1 hwilliams support        24 Sep 18 14:33 virtuoso -> ../bin/virtuoso-iodbc-t
-rw-r--r-- 1 hwilliams support     73076 Sep 18 14:33 php.ini
drwxr-sr-x 2 hwilliams support         2 Sep 18 14:33 backup
-rw-r--r-- 1 hwilliams support         0 Sep 18 14:33 virtuoso.pxa
-rw-r--r-- 1 hwilliams support      6651 Sep 18 14:33 virtuoso.ini
-rw-r--r-- 1 hwilliams support      3732 Sep 18 14:33 repl.cfg.bak
-rw-r--r-- 1 hwilliams support  15421021 Sep 18 14:33 __rdf_repl.log.bak
-rw-r--r-- 1 hwilliams support  15845745 Sep 18 14:33 __rdf_repl20181123160100236959.log.bak
-rw-r--r-- 1 hwilliams support  16538582 Sep 18 14:33 __rdf_repl20181123160108136316.log.bak
-rw-r--r-- 1 hwilliams support  13586224 Sep 18 14:33 __rdf_repl20181123160116631230.log.bak
.
.
.
-rw-r--r-- 1 hwilliams support  10096470 Sep 18 14:33 __rdf_repl20181123162012881755.log.bak
-rw-r--r-- 1 hwilliams support  10147865 Sep 18 14:33 __rdf_repl20181123162013702397.log.bak
-rw-r--r-- 1 hwilliams support   7860840 Sep 18 14:33 __rdf_repl20181123162014529454.log
-rw-r--r-- 1 hwilliams support        14 Sep 18 15:06 virtuoso.lck
-rw-r--r-- 1 hwilliams support   2097152 Sep 18 15:06 virtuoso-temp.db
-rw-r--r-- 1 hwilliams support 400556032 Sep 18 15:06 virtuoso.db
-rw-r--r-- 1 hwilliams support      1715 Sep 18 15:16 virtuoso.trx
-rw-r--r-- 1 hwilliams support    644696 Sep 18 15:18 virtuoso.log
-rw-r--r-- 1 hwilliams support        56 Sep 18 15:18 repl.cfg
hwilliams@madras:/openlink/support/virtuoso/v72$
hwilliams@madras:/openlink/support/virtuoso/v72$ cat database/repl.cfg
__rdf_repl           __rdf_repl20181123162014529454.log
hwilliams@madras:/openlink/support/virtuoso/v72$

Note when initially setting up the replication master publisher, the directory replication files are written to can also be specified, with the Virtuoso Configuration (virtuoso.ini) file LogDirectory parameter in the Replication section ie

[Replication]
LogDirectory = replication

Related

Hi hwilliams,
I try to enable replication process - Enable RDF Publishing . I get this error, Im using Virtuoso 8.3 with demo license.

An error has occurred when processing “/conductor/db_repl_trans.vspx” page.

SQL State 42000
SQL Message The replication module is not enabled

Thanks

@hrihan: Please provide a copy of the virtuoso.log file in the database directory for review, as looks as if the license file being used does not have the Replication module enabled and the log file will show what modules are enabled when the Virtuoso server starts …

Hi hwilliams,

This is log file,

Thanks

		Fri Nov 22 2019
19:11:35 { Loading plugin 1: Type `plain', file `wikiv' in `../hosting'
19:11:35   WikiV version 0.6 from OpenLink Software
19:11:35   Support functions for WikiV collaboration tool
19:11:35   SUCCESS plugin 1: loaded from ../hosting/wikiv.so }
19:11:35 { Loading plugin 2: Type `plain', file `mediawiki' in `../hosting'
19:11:35   MediaWiki version 0.1 from OpenLink Software
19:11:35   Support functions for MediaWiki collaboration tool
19:11:35   SUCCESS plugin 2: loaded from ../hosting/mediawiki.so }
19:11:35 { Loading plugin 3: Type `plain', file `creolewiki' in `../hosting'
19:11:35   CreoleWiki version 0.1 from OpenLink Software
19:11:35   Support functions for CreoleWiki collaboration tool
19:11:35   SUCCESS plugin 3: loaded from ../hosting/creolewiki.so }
19:11:35 { Loading plugin 4: Type `plain', file `im' in `../hosting'
19:11:35   IM version 0.63 from OpenLink Software
19:11:35   Support functions for Image Magick 6.9.9
19:11:35   SUCCESS plugin 4: loaded from ../hosting/im.so }
19:11:35 { Loading plugin 5: Type `plain', file `wbxml2' in `../hosting'
19:11:35   WBXML2 version 0.9 from OpenLink Software
19:11:35   Support functions for WBXML2 0.9.2 Library
19:11:35   SUCCESS plugin 5: loaded from ../hosting/wbxml2.so }
19:11:35 { Loading plugin 6: Type `attach', file `libphp5.so' in `../hosting'
19:11:35   SUCCESS plugin 6: loaded from ../hosting/libphp5.so }
19:11:35 { Loading plugin 7: Type `Hosting', file `hosting_php.so' in `../hosting'
19:11:35   Hosting version 3314 from OpenLink Software
19:11:35   PHP engine version 5.6.39
19:11:35   SUCCESS plugin 7: loaded from ../hosting/hosting_php.so }
19:11:35 { Loading plugin 8: Type `plain', file `qrcode' in `../hosting'
19:11:35   QRcode version 0.1 from OpenLink Software
19:11:35   Support functions for ISO/IEC 18004:2006, using QR Code encoder (C) 2006 Kentaro Fukuchi <fukichi@megaui.net>
19:11:35   SUCCESS plugin 8: loaded from ../hosting/qrcode.so }
19:11:35 { Loading plugin 10: Type `plain', file `proj4' in `../hosting'
19:11:35   plain version 1.0.3314 from OpenLink Software
19:11:35   Cartographic Projections support based on Frank Warmerdam's proj4 library
19:11:35   SUCCESS plugin 10: loaded from ../hosting/proj4.so }
19:11:35 { Loading plugin 11: Type `plain', file `geos' in `../hosting'
19:11:35   plain version 1.0.3314 from OpenLink Software
19:11:35   GEOS plugin based on Geometry Engine Open Source library from Open Source Geospatial Foundation
19:11:35   SUCCESS plugin 11: loaded from ../hosting/geos.so }
19:11:35 { Loading plugin 12: Type `plain', file `shapefileio' in `../hosting'
19:11:35   ShapefileIO version 0.1virt71 from OpenLink Software
19:11:35   Shapefile support based on Frank Warmerdam's Shapelib
19:11:35   SUCCESS plugin 12: loaded from ../hosting/shapefileio.so }
19:11:35 OpenLink Virtuoso Universal Server
19:11:35 Version 08.03.3314-pthreads for Linux as of May 24 2019
19:11:35 uses parts of OpenSSL, PCRE, Html Tidy
19:11:35 
19:11:35 License not found
19:11:35 Please make sure it is installed correctly
19:11:35 If you did not receive a license file,
19:11:35 contact sales@openlinksw.com or visit www.openlinksw.com
19:11:35 and a new license will be made available to you
19:11:35 
19:11:35 Demo license enabled
19:11:35 
19:11:35 Database version 3126
19:11:35 SQL Optimizer enabled (max 1000 layouts)
19:11:36 Compiler unit is timed at 0.000146 msec
19:11:38 Roll forward started
19:11:38     3 transactions, 189 bytes replayed (100 %)
19:11:38 Roll forward complete
19:11:39 Checkpoint started
19:11:39 Checkpoint finished, log reused
19:11:41 HTTP Server threads exceed the number of licensed connections. Setting to 1
19:11:41 HTTP/WebDAV server online at 8890
19:11:41 Server online at 1111 (pid 6389)
19:11:42 ZeroConfig registration virtuoso (VIRTUOSO-LICENSED-LINUX)
20:11:42 Checkpoint started
20:11:42 Checkpoint finished, log reused
21:11:43 Checkpoint started
21:11:43 Checkpoint finished, log reused
22:11:44 Checkpoint started
22:11:44 Checkpoint finished, log reused
23:11:45 Checkpoint started
23:11:45 Checkpoint finished, log reused

The log shows your license file has not been found, thus the built-in demo license has been enabled, which will only be active for 10mins.

Where did you place your license file, which should be placed in a location where it can be found by the OpenLink License Manager (oplmgr) as detailed here.