Virtuoso instance killed when run a query

Hi,
Im using Virtuoso 8.3 on linux, when i run the query then virtuoso instance killed. I didn’t find any error details on virtuoso.log.
any idea
Thanks

Hi

What is the exact Virtuoso version details which can be obtained with the command ./virtuoso-iodbc-t -?, run from the bin directory of your installation ?

What is the actual query being run that is killing the Virtuoso server instance ?

Is core file generation enabled on the machine with the ulimit -c unlimited command such that a core file for analysis can be created if the kernel wants to ?

Are there any Virtuoso errors/messages in the kernel system log ie /var/log/messages or /var/log/syslog depending on the Linux distro being used ?

Hi
This is the version details:

OpenLink Virtuoso Universal Server (Enterprise Edition)
Version 08.03.3314-pthreads as of May 24 2019 (8463d866b4)
Compiled for Linux (x86_64-generic-linux-glibc25)
Hosted Runtime Environments:  VDB
Copyright (C) 1998-2019 OpenLink Software
Usage:
  virtuoso-iodbc-t [-fcnCbDARwMKrBd] [+foreground] [+configfile arg]
                   [+no-checkpoint] [+checkpoint-only] [+backup-dump]
                   [+crash-dump] [+crash-dump-data-ini arg]
                   [+restore-crash-dump] [+wait] [+mode arg] [+dumpkeys ar
g]
                   [+restore-backup arg] [+backup-dirs arg] [+debug]
                   [+pwdold arg] [+pwddba arg] [+pwddav arg]
  +foreground            run in the foreground
  +configfile            use alternate configuration file
  +no-checkpoint         do not checkpoint on startup
  +checkpoint-only       exit as soon as checkpoint on startup is complete
  +backup-dump           dump database into the transaction log, then exit
  +crash-dump            dump inconsistent database into the transaction l
og, then exit
  +crash-dump-data-ini   specify the DB ini to use for reading the data to
 dump
  +restore-crash-dump    restore from a crash-dump
  +wait                  wait for background initialization to complete
  +mode                  specify mode options for server startup (onbalr)
  +dumpkeys              specify key id(s) to dump on crash dump (default 
: all)
  +restore-backup        restore from online backup
  +backup-dirs           default backup directories
  +debug                 Show additional debugging info
  +pwdold                Old DBA password
  +pwddba                New DBA password
  +pwddav                New DAV password

There is no error in the log file:

20:34:53 OpenLink Virtuoso Universal Server
20:34:53 Version 08.03.3314-pthreads for Linux as of May 24 2019
20:34:53 uses parts of OpenSSL, PCRE, Html Tidy
20:34:53 Registered to sama
20:34:53 Personal Edition license for unlimited connections
20:34:53 Issued by OpenLink Software
20:34:53 This license will expire on Fri May 29 00:00:00 2020 GMT
20:34:53 Enabled Cluster Extension
20:34:53 Enabled Column Store Extension
20:34:53 Enabled Virtual Database Extension
20:34:53 Enabled Replication Extension
20:34:53 Enabled Scalable ACL Extension
20:34:53 Enabled Custom Reasoning & Inference Rules
20:34:53 Database version 3126
20:34:53 SQL Optimizer enabled (max 1000 layouts)
20:34:54 Compiler unit is timed at 0.000157 msec
20:34:57 Roll forward started
20:34:57     3 transactions, 189 bytes replayed (100 %)
20:34:57 Roll forward complete
20:35:00 Checkpoint started
20:35:00 Checkpoint finished, log reused
20:35:02 HTTP/WebDAV server online at 8890
20:35:02 Server online at 1111 (pid 18400)
20:35:02 ZeroConfig registration virtuoso (VIRTUOSO-INSTANCE-LINUX-SMALL)

This is error log message on /var/log/messages:

May 14 20:01:41 virtuoso-instance-linux-small kernel: [9266066.006318] virtuoso[31480]: segfault at 14 ip 000000000061c17d sp 00007f035cff1$
May 14 20:10:12 virtuoso-instance-linux-small kernel: [9266577.498988] virtuoso[17868]: segfault at 14 ip 000000000061c17d sp 00007fd93e2fc$
May 14 20:13:36 virtuoso-instance-linux-small kernel: [9266781.153540] virtuoso[18023]: segfault at 14 ip 000000000061c17d sp 00007f5328adc$
May 14 20:27:47 virtuoso-instance-linux-small kernel: [9267632.178156] virtuoso[18269]: segfault at 14 ip 000000000061c17d sp 00007f923eff5$

Thanks

Hi @hrihan,

Please paste the text of the SPARQL Query here as that would help with our analysis etc…

Hi
This query works well

prefix family:                  <http://family.com/ontology/>
prefix family-class:            <http://family.com/ontology/class/>
prefix family-card-item:     <http://family.com/ontology/card-item/>
prefix foaf:  			<http://xmlns.com/foaf/0.1/>
prefix family-code:         <http://family.com/ontology/code/>

 SELECT   * 
   WHERE
{
   ?ti a family-class:CardItem ;
      family:hasParent ?t ;
      family:haskeyId ?g1 ;
      family:hasCard ?s1 ;
      family:hasMoney ?c1 ;
      family:hascode ?e1 .
  FILTER(?e1 IN
        (family-code:102,
          family-code:104 )
  )
#  <<regex>>
  ?s1 family:hasName ?i1 .
  ?c1 family:hasName ?o1 .
  optional
  {
    ?c1  family:hasSubFamily ?sub .
    ?sub foaf:name ?subn .
  }
  BIND(concat("pri:", ?t) as ?id)
  BIND(str(?t) AS ?parent)
  BIND(str(?i1) as ?family_name)
  BIND(str(?o1) AS ?object_name)
  BIND(str(?g1) as ?keyid)
  BIND(str(?subn) as ?sub_family)
}
 GROUP BY ?parent
 ORDER BY ?parent
LIMIT 100

But when i used count(*) instead of * , Virtuoso instance is crashed.

below query cause crash for Virtuoso instance.

prefix family:                  <http://family.com/ontology/>
prefix family-class:            <http://family.com/ontology/class/>
prefix family-card-item:     <http://family.com/ontology/card-item/>
prefix foaf:  			<http://xmlns.com/foaf/0.1/>
prefix family-code:         <http://family.com/ontology/code/>

 SELECT  count(*)
   WHERE
{
   ?ti a family-class:CardItem ;
      family:hasParent ?t ;
      family:haskeyId ?g1 ;
      family:hasCard ?s1 ;
      family:hasMoney ?c1 ;
      family:hascode ?e1 .
  FILTER(?e1 IN
        (family-code:102,
          family-code:104 )
  )
#  <<regex>>
  ?s1 family:hasName ?i1 .
  ?c1 family:hasName ?o1 .
  optional
  {
    ?c1  family:hasSubFamily ?sub .
    ?sub foaf:name ?subn .
  }
  BIND(concat("pri:", ?t) as ?id)
  BIND(str(?t) AS ?parent)
  BIND(str(?i1) as ?family_name)
  BIND(str(?o1) AS ?object_name)
  BIND(str(?g1) as ?keyid)
  BIND(str(?subn) as ?sub_family)
}
 GROUP BY ?parent
 ORDER BY ?parent
LIMIT 100 

Thanks

Can you please include your queries in verbatim tags (as I have edited your post and done) such that any special characters do not get lost.

Have you enabled core file generation on the machine with the ulimit -c unlimited command before starting Virtuoso, then run the query that causes the crash and check if a core file is created by the kernel, which can then be analysed to try and determine the cause of the crash ?

Hi, This query is not working (Cause crash ),

prefix family:                  <http://family.com/ontology/> 
prefix family-class:            <http://family.com/ontology/class/>
prefix foaf:  			<http://xmlns.com/foaf/0.1/>
prefix family-code:         <http://family.com/ontology/code/>

 SELECT count(*)  WHERE
{?ti a family-class:CardItem ;
      family:hasParent ?t ;
      family:hascode ?e1 .
  FILTER(?e1 IN (family-code:102 )  )
  ?c1 family:hasName ?o1 .
  optional
  { ?c1  family:hasSubFamily ?sub .  }
  BIND(concat("pri:", ?t) as ?id)
  BIND(str(?t) AS ?parent) }
 GROUP BY ?parent
 ORDER BY ?parent
LIMIT 100 

How i can show the error which cause the crash.

Thanks

I did command ulimit -c unlimited and added it into /root/.bashrc but the same error, instance crashed when run query,

I am not expecting ulimit -c unlimited to resolve the problem, it enables core file creation if set for a running process, which can then be analysed. Thus having started Virtuoso with the setting in place is a core or core.xxx file generated in the Virtuoso database directory ?

Hi,
Now I have core file in Virtuoso database directory but it seems binary file (not readable) . is it right?

Indeed it is a binary for which a debugger tool like gdb has to be used to analyse it, as detailed at this document on using gdb to analyse Virtuoso core files. Except in your case as you are not using open source and have a prebuilt binary, you would only have to perform steps 6 & 7.

I note you are using a Virtuoso 08.03.3314 binary from May 2019, whereas a an update release 08.03.3316 build was release in Mar 2020. You can download a copy of this updated binary virtuoso-iodbc-t of this binary with debug symbols in tact to enable more details diagnostic on the case of the crash to be obtained.

To use gunzip the file, rename to virtuoso-iodbc-t , replace the existing file of the same name in the bin directory of your installation and restart Virtuoso. The run the problem query again to see if it still crashes and if it does provide a gdb stack trace as detailed in points 6 & 7 of the gdb post.