Analysing 20GB Heap Dump

[opticyclic]

I took a heap dump with

Code: Select all

jmap -dump:format=b,file=/tmp/ds_19Oct2013.hprof 15985 

Due to it being a large enterprise application with badly configured JVM params the heap dump was 20GB in size.

Now when I try and load it in to YourKit I always get an error

Not enough Virtual Memory

Troubleshooting instructions will be shown in your browser after you close this dialog.

After reading the web page I decided to give at least twice the size of the heap dump of memory.
I tried running it on the server with the following options but the error still occurred.

Code: Select all

-Xmx100G
-XX:PermSize=256m
-XX:MaxPermSize=512m
-XX:+HeapDumpOnOutOfMemoryError
-XX:SurvivorRatio=16
-XX:NewRatio=1
-Xbootclasspath/a:./../lib/tools.jar

This gives both the Eden Space and Old Gen 50GB with about 2.8GB for the survivor spaces.

What is confusing is that I have run it with VisualGC attached and I can see that the memory spaces are not full.
Eden is about 1GB
Old Gen is about 5.5GB
Perm Gen is about 25MB

The log says:

Code: Select all

13044 - 0.927: Command line:
  ./../jre64/bin/java
  -Xmx100G
  -XX:PermSize=256m
  -XX:MaxPermSize=512m
  -XX:+HeapDumpOnOutOfMemoryError
  -XX:SurvivorRatio=16
  -XX:NewRatio=1
  -Xbootclasspath/a:./../lib/tools.jar
  -jar
  ./../lib/yjp.jar
13044 - 0.927: uname:
  sysname: Linux
  release: 2.6.18-348.4.1.el5
  version: #1 SMP Fri Mar 22 05:41:51 EDT 2013
  machine: x86_64
13044 - 0.927: Linux; fast_thread_cpu_time=1; 64-bit process
13044 - 0.927: Today is 20131022
13044 - 0.927: shmem 4234148E: OK
13044 - 0.930: shmem 42341490: OK
13044 - 1.581: uidCount: 212
13044 - 1.581: UID OK
13044 - 53.953: Loader for HPROF: /tmp/ds_19Oct2013.hprof
13044 - 53.962: pass1: starting
13044 - 185.617: pass1: finished
13044 - 185.621: pass2: starting
13044 - 365.621: pass2: finished
13044 - 434.603: OIDs converted
13044 - 434.607: createStorage: objectCount=371115306: process: CPU 455529 ms (kernel: 37293 ms); 110007 Mb
13044 - 448.831: createStorage: result=80730800: process: CPU 469989 ms (kernel: 41369 ms); 119917 Mb
13044 - 449.853: pass3: starting
13044 - 629.853: pass3: finished
13044 - 708.274: initMemPart: 0
13044 - 709.224: initMemPart: 1
13044 - 889.224: initMemPart: 2
13044 - 889.224: initMemPart: 3
13044 - 1054.824: initMemPart: 4
13044 - 1054.825: dc: 0: process: CPU 2533742 ms (kernel: 75663 ms); 136953 Mb
13044 - 1054.825: dc: 1: oc=371115307 rc=2775: process: CPU 2533742 ms (kernel: 75663 ms); 136953 Mb
13044 - 1054.825: exception caught at ObjectStorageNatives.cpp:373

So why is it failing?

[Anton Katilin]

After reading the web page I decided to give at least twice the size of the heap dump of memory.

Please carefully read the instructions. They say to do the opposite: DON'T increase -Xmx if you get the "No virtual memory" error.

Please see this thread:
http://forums.yourkit.com/viewtopic.php?t=4420

In particular, please try the -Dyjp.dominators=false workaround.

Best regards,
Anton

[Anton Katilin]

FYI: we've improved snapshot loading in v2014 EAP:
http://www.yourkit.com/eap

If you still have that or similar huge snapshots, please try opening them with the new build.