Perfomance metrics

Hi! Im trying to make Prometheus exporter for my OpenLink Virtuoso Server 07.20.3217-pthreads for Linux. I found 1.4.28. What performance metrics does Virtuoso offer?

It says I can get a lot of perfomance and usage metrics, but i found only several of them. I read 6.1.6. Performance diagnostics, but it is not very useful.

Cause status() documentation is absolutely non-informative, I have to write here((

I want to know how i can get:

  • CPU utilization
  • general concurrency (what means max N concurrent in client status string?)
  • interconnect traffic
  • clean and dirty cache pages (cache pages means buffers?)
  • virtual memory swapping warning
  • Statistics on memory usage for disk caching index-by-index
  • disk random and sequential read times

Thank you so much if you would help me to find a solution.

3 Likes

@sibilkov: I presume you are referring to the Prometheus systems and service monitoring system ?

Virtuoso also has a sys_stat() function that can be used to obtain statistical information about a running Virtuoso server for monitoring purposes, etc. This is what we use for integration with the Zabbix system monitoring tools we use in-house, as have other customers for monitoring tools they use.

Wow, thanks! It looks like what I was looking for!
Is there a more detailed description of stat_names?

UPD
And I can’t find metrics from first message in sys_stat parameter list, or something similar…

UPD2
Yes, I’m talking about Prometheus monitoring system, currently I parse status() output, and how I can see sys_stat() can’t give me much more info than status().

@sibilkov: The combination of the status() and sys_stat() functions is what we and other users have used to get server status monitoring statistics for their needs.

You can parse CPU usage from status() output as CPU%: 0.05.

General concurrency usage can be obtained from status() output Clients: 6 connects, max 1 concurrent, or sys_stat() for st_cli_connects and st_cli_max_connected.

Clean and dirty buffers can be obtained from status() output 200000 buffers, 1130 used, 109 dirty, or using sys_stat() for st_db_buffers, st_db_used_buffers, st_db_dirty_buffers. Note these are 8k in size.

For the other params you ask about, I will have to check with development if these can be obtained, as I am not aware they can within Virtuoso.

Thanks!

One more question:

I do not have a CPU parameter in my status() output ((
Why can it be?

@sibilkov: Development have confirmed that the CPU% usage is not available in the open source code for the the status() function, but they can and will add it in an update to the git develop/7 branch soon as this was an oversight.

Note that unix getrusage command is available in Virtuoso and can provide SWAP and other metrics that may be of interest like —

struct timeval ru_utime; /* user time used */
struct timeval ru_stime; /* system time used */
long ru_maxrss;          /* max resident set size */
long ru_ixrss;           /* integral shared text memory size */
long ru_idrss;           /* integral unshared data size */
long ru_isrss;           /* integral unshared stack size */
long ru_minflt;          /* page reclaims */
long ru_majflt;          /* page faults */
long ru_nswap;           /* swaps */
long ru_inblock;         /* block input operations */
long ru_oublock;         /* block output operations */
long ru_msgsnd;          /* messages sent */
long ru_msgrcv;          /* messages received */
long ru_nsignals;        /* signals received */
long ru_nvcsw;           /* voluntary context switches */
long ru_nivcsw;          /* involuntary context switches */

Thus, within Virtuoso, getrusage() returns a vector of results that can be indexed numerically as follows:

SQL> select getrusage()[1] as SYS_TIME, getrusage()[5] as SWAP;
SYS_TIME         SWAP
LONG VARCHAR     LONG VARCHAR
____________     ____________

295              0

1 Rows. -- 3 msec.
SQL>

@hwilliams
Hi! I have one additional question about metrics and I think will be better ask it in this topic.

I get a problem with db_activity() function. I trying to use db_activity(2) or db_activity(3), but always get a set of zeros:

db_activity
LONG VARCHAR
_______________________________________________________________________________

     0 rnd      0 seq      0 same seg       0 same pg      0 same par      0 disk      0 spec disk      0B /      0 messages      0 fork

or

vector(0,0,0,0,0,0,0,0,0)

Why? How can I fix it?

@sibilkov: Did you run the prof_enable(1) function to enable query logging to enable tracing of such database activity? As I can get it working when I enable it:

SQL> prof_enable(1);

Done. -- 2 msec.
SQL>
.
.
.
SQL> select db_activity(1);
db_activity
LONG VARCHAR
_______________________________________________________________________________

vector(0,0,0,0,15277,0,0,0,0)

1 Rows. -- 2 msec.
SQL> 

yes, query logging is enabled.

@sibilkov, @hwilliams -

Did you test all of
db_activity();
db_activity(0);
db_activity(1);
db_activity(2);
db_activity(3);
?

I ask because I see (1) in @hwilliams’s comment, and (2) and (3) in @sibilkov’s comment – so I’m wondering if the issue @sibilkov is seeing only occurs with the latter two arguments and if this is reproducible on @hwilliams’s instance.

Hi!
I tried all of them and always get a set of zeros :confused:

@hwilliams I still don’t see the CPU% usage in the status() function. Is the update still not ready?

The version I am using is 07.20.3237 released on Jun 7 2023.

It appears the CPU usage has not yet been added to the open source edition status() output, so I will have to check with development why this has not been done yet.

1 Like