Not enough virtual memory

Questions about YourKit Java Profiler
Post Reply
ItsHarry
Posts: 7
Joined: Sat Feb 16, 2013 3:10 pm

Not enough virtual memory

Post by ItsHarry »

Hello,

I have a memory snapshot with a size of 17GBs, but I can't get it to open on my computer which has 8GBs of physical RAM.
I tried allocating 3, 4, 5, 6 and 7 GBs of RAM to YourKit using the -xmX option but I get the same error every time, saying that I don't have enough virtual memory.

Is there any way to fix this or do I need more physical RAM?

Thanks.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Not enough virtual memory

Post by Anton Katilin »

Hello,

Instead of increasing -Xmx please increase virtual memory size on the machine. What is the swap file size limit?

Anyway, I would recommend that you use a machine with more physical memory. The UI will work much faster then.

Best regards,
Anton
ItsHarry
Posts: 7
Joined: Sat Feb 16, 2013 3:10 pm

Re: Not enough virtual memory

Post by ItsHarry »

Hello,

I will try allocating more virtual memory and see if that fixes the issue.

I'll let you know
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Not enough virtual memory

Post by Anton Katilin »

In addition to increasing the virtual memory, please don't set too big -Xmx because the biggest part of the snapshot is directly mapped to memory outside the Java heap. 2G or 4G will likely be enough.
ItsHarry
Posts: 7
Joined: Sat Feb 16, 2013 3:10 pm

Re: Not enough virtual memory

Post by ItsHarry »

The snapshots loads now. However, it freezes on "Performing final tasks loading the snapshot :"
I can see the RAM usage changing the whole time, so it's definitely doing something, but it has been stuck for quite some time now.
ItsHarry
Posts: 7
Joined: Sat Feb 16, 2013 3:10 pm

Re: Not enough virtual memory

Post by ItsHarry »

It's been stuck on "Performing final tasks loading the snapshot" for 20 minutes now, this is the second time I'm trying it.

How do I fix this?
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Not enough virtual memory

Post by Anton Katilin »

As I wrote, try to find a machine with more physical memory available. Peak memory usage of the profiler opening a memory snapshot can be 3x of the snapshot size on disk. There is no wonder that a 17 GB snapshot opens very slowly on your machine which has 8 GB of RAM.

Alternatively, please try the -Dyjp.dominators=false option as explained here:
http://forums.yourkit.com/viewtopic.php ... 672#p15658
ItsHarry
Posts: 7
Joined: Sat Feb 16, 2013 3:10 pm

Re: Not enough virtual memory

Post by ItsHarry »

Okay I will try -Dyjp.dominators=false.

I understand that the best solution would be getting more physical RAM but I can't get any more than 8GBs of RAM at the moment.
ItsHarry
Posts: 7
Joined: Sat Feb 16, 2013 3:10 pm

Re: Not enough virtual memory

Post by ItsHarry »

Can't get it to work, I'll just give up for now
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Not enough virtual memory

Post by Anton Katilin »

Have you applied -Dyjp.dominators=false ?
If you have, "Performing final tasks loading the snapshot" should complete relatively fast.
kchan
Posts: 11
Joined: Fri Jan 11, 2013 11:31 pm

Re: Not enough virtual memory

Post by kchan »

I am having similar issue.

I am trying to open a hprof snanshot, which is 24GB in disk. The computer that I used to load this has 32GB of RAM. These are the different tries and the result:

-xmx6g pagefile=6G => Out of Virtual Memory
-xmx6g pagefile=32G => Out of Memory
-xmx12g pagefile=32G => Out of Virtual Memory
-xmx9g pagefile=32G => Out of Virtual Memory
-xmx9g pagefile=50G => Out of Virtual Memory, the pagefile was already at 32G, but never grew to the allowable 50G

I am now at a lost on what else to do. What do you suggest that I need to increase/decrease to get this file to load?

Also, since it takes a long time (30 to 60 minutes) to load the file to encounter the error, I wasn't always staring at the screen. Once the error occurred, the "status" dialog is gone, and replaced with the Out of Virtual memory dialog. So, I never know how far the load process went to. Is there anyway to get that information, perhaps in a log file?

Thanks

Kam
kchan
Posts: 11
Joined: Fri Jan 11, 2013 11:31 pm

Re: Not enough virtual memory

Post by kchan »

Sorry, I should also mention that I want the dominators information, so I don't want to turn that off.
kchan
Posts: 11
Joined: Fri Jan 11, 2013 11:31 pm

Re: Not enough virtual memory

Post by kchan »

Update:

I have now increased the system RAM to 64GB, and let the OS (Windows) to manage the page file. (-xmx9g)

Still "Not enough Virtual memory"! The page file size was at 50GB, and did not increase during and after the loading.

Now I am really stuck.

Kam
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: Not enough virtual memory

Post by Anton Katilin »

Hi Kam

Please try -Dyjp.dominators=false (edit: should be -Dyk.dominators=false since version 2017.02).
It should definitely help. Instead of using the dominator tree, you'd be able to use the Class list + calculate exact retained sizes.

Regarding the swap file. Is there enough disk space available to let it grow?

Is it possible to get your snapshot for testing?

Best regards,
Anton
kchan
Posts: 11
Joined: Fri Jan 11, 2013 11:31 pm

Re: Not enough virtual memory

Post by kchan »

Anton,

I gave in, and used the dominators=false option. It loaded fine. Previously, when I used the Class list, "Calculate exact retained sizes", it does not seem to work for long. Seems after a few minutes, it came back with "Continue calculate", the retained sizes did not change, if I recall. I now used the Ctrl-Q to get the retained sizes, but each request takes a long time.
Regarding the swap file. Is there enough disk space available to let it grow?
Definitely. The page file is now at 50GB, unchanged from the time I increased the size. There are still 100GB of diskspace.

I will upload the file for you to test.

Thanks

Kam
Post Reply