Hello,
We are encountering an issue with our Virtuoso setup and would greatly appreciate your assistance in diagnosing and resolving the problem.
Environment:
- Virtuoso Version: 07.20.3239 (the same issue was observed in previous versions)
- Apache Jena Libraries Version: 4.9.0
- Server Configuration:
- CPU: 2 cores
- RAM: 8 GB
Data Ingestion:
We are saving data to Virtuoso using the Jena library with the following code snippet:
private void saveWithConnection(String graphName, Model model, RDFConnection connection) {
try {
connection.load(graphName, model);
} catch (Exception e) {
log.error("Could not flush!", e);
if (e instanceof HttpException) {
HttpException httpException = (HttpException) e;
log.error("HttpException: {}", httpException.getResponse());
}
throw new TripleStoreRepositoryException(format("Could not save model to '%s'", graphName), e);
}
}
You can view the full code here.
Problem:
We’re experiencing issues when attempting to write a relatively large dataset (82 MB) from this TTL file.
During the write operation, Virtuoso appears to hang indefinitely without completing the write, eventually crashing after a significant delay.
Virtuoso Status Output during the Write Operation:
Virtuoso Server
Version 07.20.3239-pthreads for Linux as of Feb 13 2024 (d698f21712)
Started on: 2024-08-26 13:36 GMT+0 (up 20:32)
CPU: 0.05% RSS: 6249MB VSZ: 7139MB PF: 0
Database Status:
File size 174063616, 21248 pages, 10366 free.
680000 buffers, 6217 used, 3736 dirty 1 wired down, repl age 0 0 w. io 6 w/crsr.
Disk Usage: 76365 reads avg 0 msec, 0% r 0% w last 102 s, 2631 writes flush 30.12 MB/s,
113 read ahead, batch = 25. Autocompact 225 in 157 out, 30% saved.
Gate: 299 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
Log = ../database/virtuoso.trx, 65461820 bytes
682 pages have been changed since last backup (in checkpoint state)
Current backup prefix: bck_dev_
Current backup timestamp: 0x091C-0x02-0x00
Last backup date: Mon Aug 26 23:57:28 2024
Clients: 0 connects, max 0 concurrent
RPC: 1 calls, 0 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0 second 131M large, 293M max
Checkpoint Remap 0 pages, 0 mapped back. 1 s atomic time.
DB master 21248 total 10366 free 0 remap 0 mapped back
temp 1024 total 1016 free
Lock Status: 0 deadlocks of which 0 2r1w, 6 waits,
Currently 8 threads running 6 threads waiting 1 threads in vdb.
Pending:
...
(100 instances of "IER NO_CONN")
Memory Status during the Stall:
P.S.:The issue does not occur consistently. Sometimes the write operation completes successfully within a few minutes.
Has anyone encountered a similar issue or have any insights on what might be causing Virtuoso to hang and eventually crash during large data writes?
Any suggestions on what we could try to resolve this would be greatly appreciated.
Thank you!