Database Probe - full recording stopped

Questions about YourKit Java Profiler
Post Reply
davec
Posts: 3
Joined: Sun Aug 10, 2014 7:11 am

Database Probe - full recording stopped

Post by davec »

I'm running the lateset Yourkit 2014 (14090) release

I've been performing an analysis of an application which has a lot of database activity.
During the course of its execution I was analysing CPU Performance and taking periodic CPU Snapshots with the option to "Continue CPU Profiling selected"

I was then surprised to see in the later snaphots that the "SQL" tab under "J2EE Statistics" seemed to have stopped recording database activity.
I'd read about similar historical issues in forum messages, so had been careful to use "probe_on=com.yourkit.probes.builtin.Databases", but still this didn't change the situation when i repeated the test.

However I've now noticed in the "events" tab that the Database entry has "full recording stopped" marked against it after taking a snapshot.
I think this is probably the cause of my issue. Interesting this doesn't seem to effect other probes,

In earlier version of Yourkit, taking a 'continuing snapshot never used to exhibit this behavior, so is this a bug or am i missing something?
( all of course assuming my analysis is correct)

thanks
Dave
davec
Posts: 3
Joined: Sun Aug 10, 2014 7:11 am

Re: Database Probe - full recording stopped

Post by davec »

Actually some further input - as i 'd think I'd jumped to a conclusion.

When I use the "Clear Tables" option in the Events tab to clear the Database tab, yourkit starts recording information, but very quickly (within seconds) it then reports "full - recording stopped" again
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Database Probe - full recording stopped

Post by Anton Katilin »

Hi Dave

You are right.
As soon as the stored event count reaches its limit, newer events are no longer being added until the tables are cleared, either explicitly or, for probes in Auto mode, by (re)starting CPU profiling:
http://www.yourkit.com/docs/java/help/r ... probes.jsp

The same behavior was in version 2013 too. The change in version 204 is that the full tables are now explicitly indicated in the UI, and that the rule to calculate the limit has slightly changed. An excerpt from the change list:
Event capacity limit is now applied to the event count in each top level table (e.g. "File" and "Database") together with all events in its dependent tables (e.g. "Read" and "Query"). This removes result inconsistency by eliminating partially recorded events. The default limit has been increased to 100,000 events (customizable with the agent startup option probetablelengthlimit).

Tables that reached the limit are indicated in the UI.
In addition to the Events of which only the first N are recorded, please also consider the Performance charts which provide continuous statistics like other telemetry graphs do.
http://www.yourkit.com/docs/java/help/perf_charts.jsp
Events allow to debug the detail, such as where and what events happen, how they are nested, how they follow each other in time, etc. You don't need all events to investigate the behavior pattern: a limited subset of them is usually enough. Performance charts provide less detail, they show request counts and detail on the longest one in each telemetry sample, but the capacity is as large as the telemetry limit:
http://www.yourkit.com/docs/java/help/s ... metrylimit

You can increase the event count limit with
http://www.yourkit.com/docs/java/help/s ... engthlimit
... but if requests are very frequent any limit will be reached sooner or later.

Best regards,
Anton
davec
Posts: 3
Joined: Sun Aug 10, 2014 7:11 am

Re: Database Probe - full recording stopped

Post by davec »

Hi Anton - thanks very much - very helpful.

Just in case its of interest ....

I increased the size of the probe table to 1M per your instructions. It alleviated my scenario for a short time longer (30 seconds rather than 3 seconds) but then still went full. However my SQL Java EE Statistics was still showing relatively few application SQL statements, so was still mystified.

However at that point i noticed that Yourkit was helpfully showing the number of 'Prepared Statements' records in the Events tab and was shocked to see it showing >500000.

Likely some issue with the JDBC driver or the Vendor "application pool" sitting on top of the driver I think. Investigation work for us to folllow....

However problem explained from a Yourkit perspective, so again many thanks
Vladimir Kondratyev
Posts: 1624
Joined: Tue Aug 10, 2004 7:52 pm

Re: Database Probe - full recording stopped

Post by Vladimir Kondratyev »

davec, thank you for details.

If you find that probes ("Events" tab) show incorrect results, please share the snapshot file which demonstrates the problem. We will perform investigation on our side.

Best regards,
Vladimir Kondratyev
YourKit, LLC
http://www.yourkit.com
"Don't get lost in data, get information!"
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Database Probe - full recording stopped

Post by Anton Katilin »

An update:
However at that point i noticed that Yourkit was helpfully showing the number of 'Prepared Statements' records in the Events tab and was shocked to see it showing >500000.
Next build 14092 will show PreparedStatement open/close rate in Performance charts, as well as prepared vs non-prepared query counts separately. This will allow to detect such problems even if event storage runs out of capacity.
Post Reply