Problems with Yourkit + Java 11 + Native memory tracking

Questions about YourKit Java Profiler
Post Reply
cdietrich
Posts: 7
Joined: Wed Jul 08, 2020 12:55 pm

Problems with Yourkit + Java 11 + Native memory tracking

Post by cdietrich »

hi,

when i profile my app with yourkit and native memory tracking enabled i get
wrong/implausible numbers reported for the compiler memory

Total: reserved=15072248KB, committed=13810372KB
- Java Heap (reserved=12582912KB, committed=12582912KB)
(mmap: reserved=12582912KB, committed=12582912KB)
....
- Compiler (reserved=18014398509481965KB, committed=18014398509481965KB)
(malloc=195KB #2360)
(arena=18014398509481770KB #5)
...

here are the vm args i use
-XX:+UnlockDiagnosticVMOptions
-XX:+PrintNMTStatistics
-XX:NativeMemoryTracking=summary

i am on Java 11.0.7 on RHEL

is this a yourkit bug? a openjdk bug? am i holding it wrong?
unfortunately i cannot reproduce this with an arbitrary java program
and with my app only with a specific workload.

Thanks
Christian
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Problems with Yourkit + Java 11 + Native memory tracking

Post by Anton Katilin »

Hi Christian

I'm sure that numbers such as 18014398509481965KB are really invalid.

These statistic messages are printed by the JVM, the profiler agent does not interfere the numbers printed out. So I believe it may be a Java bug. Although the profiler agent may somehow indirectly affect its reproducibility by e.g. using some JVMTI API functions.

Could you please try running the profiler agent with the startup option "disableall". If it helps, please try individual options or their combinations instead:
https://www.yourkit.com/docs/java/help/ ... disableall

What garbage collector settings do you use? Perhaps changing the collector can help.

If possible, please check if the problem is reproducible with the newest Java 14 instead.

Best regards,
Anton
cdietrich
Posts: 7
Joined: Wed Jul 08, 2020 12:55 pm

Re: Problems with Yourkit + Java 11 + Native memory tracking

Post by cdietrich »

problem does not happen with disableall.
problem happens with default GC settings too. (we have adjusted the ratio only)
problem does not happen with Java 14 (did have to set the yourkit option to disable java version check for that)
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Problems with Yourkit + Java 11 + Native memory tracking

Post by Anton Katilin »

problem does not happen with disableall.
Great. Please also try the combinations
"disablealloc,disablenatives,disabletracing,probe_disable=*"
and
"disablealloc,disabletracing"
problem does not happen with Java 14 (did have to set the yourkit option to disable java version check for that)
Please consider version 2020.7 EAP, it works with Java 14 out of the box:
https://www.yourkit.com/eap/
cdietrich
Posts: 7
Joined: Wed Jul 08, 2020 12:55 pm

Re: Problems with Yourkit + Java 11 + Native memory tracking

Post by cdietrich »

disabletracing seems sufficient to make the problem go away
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Problems with Yourkit + Java 11 + Native memory tracking

Post by Anton Katilin »

Thank you for the feedback. "disabletracing" prevents bytecode instrumentation for CPU tracing, which apparently affects the compiler statistics. Anyway, it's great that the issue is solved in Java 14 and that there are known workarounds for Java 11.
Post Reply