Virtuoso special Malloc Debug logs creation

Virtuoso Universal Server Tips, Tricks, and HowTos
,
1

What

Virtuoso special custom Malloc Debug binaries can be created by development on request to enable logs of the Virtuoso memory consumption to be created, which can be used for tracking down the source of memory consumptions issues in the code by development.

How

To use perform the following steps:

  1. Obtain copy of the special Virtuoso Malloc Debug binary (virtuoso-iodbc-t) and ensure the binary itself has execution permissions (chmod +x virtuoso-iodbc-t)

  2. Shutdown your existing Virtuoso instance ensuring to perform a checkpoint immediately prior to shutting down to commit all pending transaction in the virtuoso.trx ( transaction log) file to the database file.

  3. Replace the existing virtuoso-iodbc-t binary in the bin directory of your Virtuoso installation with the updated binary and restart Virtuoso checking the virtuoso.log file to confirm the new binary is being used, as the version and build id are reported in the log on startup, as well as the message:

*** THIS SERVER BINARY CONTAINS MEMORY DEBUG CODE! ***
  1. Create the following procedure in the Virtuoso database using the isql command line tool:
create procedure debug_memory(in doall integer := 0)
{ 
    declare path varchar;

    path := date_iso8601(curutcdatetime());
    if (doall)
      {
        path := '/tmp/mem_all_' || path || '.txt';
        mem_all_in_use (path);
      }
    else
      {
        path := '/tmp/mem_new_' || path || '.txt';
        mem_new_in_use (path);
      }
}
;
  1. Run the following command to obtain a full dump of memory usage:
debug_memory (1);  --- get a full log
  1. Then every few hours run the following command to get an incremental dump of memory usage over time:
debug_memory(); -- get incremental
  1. The incremental dump command debug_memory() can added to the Virtuoso scheduler (or a cron job or other if preferred) such that it is automatically run at a set interval with a query of the form:
INSERT INTO SYS_SCHEDULED_EVENT (SE_NAME, SE_SQL, SE_START, SE_INTERVAL) VALUES('<schedule_event_name>', 'debug_memory()', '<timestamp>', '<schedule_interval_in_minutes');
  1. When done the scheduled event can be removed with the query:
DELETE FROM SYS_SCHEDULED_EVENT WHERE SE_NAME='<schedule_event_name>'

The memory dumps will be created in the /tmp directory, or amend the debug_memory() procedure to place in a preferred location, and can then be provided for analysis by development …

Related