Why heap dump limit is always around 840~850mb?
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Why heap dump limit is always around 840~850mb?
Hi,
I am new to this board & yourkit profiler too. So please let me know if my way of writing is not appropriate for this board.
1.
https://imgur.com/a/9M3K5is
On image there we can see heap limit is around 840mb.
When I give -Xmx 128M, 1G, 2G, .. it's always 840mb. Something wrong in my configuration?
My full configuration is below;
-Xmx 1G -Xms 1G -J-agentpath:/usr/share/yourkit/bin/linux-x86-64/libyjpagent.so=exceptions=disable,disable_async_sampling,listen=all
2.
When I made a trigger, heap memory > 90% then, CaptureMemorySnapshot + CaptureHprofSnapshot, however, I usually get only hprof file. Is there any specific reason? How to ensure getting memory snapshot
Thanks
I am new to this board & yourkit profiler too. So please let me know if my way of writing is not appropriate for this board.
1.
https://imgur.com/a/9M3K5is
On image there we can see heap limit is around 840mb.
When I give -Xmx 128M, 1G, 2G, .. it's always 840mb. Something wrong in my configuration?
My full configuration is below;
-Xmx 1G -Xms 1G -J-agentpath:/usr/share/yourkit/bin/linux-x86-64/libyjpagent.so=exceptions=disable,disable_async_sampling,listen=all
2.
When I made a trigger, heap memory > 90% then, CaptureMemorySnapshot + CaptureHprofSnapshot, however, I usually get only hprof file. Is there any specific reason? How to ensure getting memory snapshot
Thanks
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Why heap dump limit is always around 840~850mb?
Hello,
1. It's strange. The limit should reflect the -Xmx value.
Do you really use spaces in "-Xmx 1G -Xms 1G"? Please specify the options without them: "-Xmx1G -Xms1G"
2. Could you please provide the full definition of your trigger.
It would help if you could share the profiler agent log file ~/.yjp/log/<session name>-<pid>.log
Best regards,
Anton
1. It's strange. The limit should reflect the -Xmx value.
Do you really use spaces in "-Xmx 1G -Xms 1G"? Please specify the options without them: "-Xmx1G -Xms1G"
2. Could you please provide the full definition of your trigger.
It would help if you could share the profiler agent log file ~/.yjp/log/<session name>-<pid>.log
Best regards,
Anton
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Why heap dump limit is always around 840~850mb?
P.S.
> CaptureMemorySnapshot + CaptureHprofSnapshot,
Please note that a memory snapshot supersedes a HPROF snapshot. You can capture memory snapshot only. If you need a HPROF snapshot per-se i.e. a .hprof file, you can extract it via the command
java -jar <path to>/lib/yourkit.jar -snapshot2hprof <source-file>.snapshot <target-file>.hprof
> CaptureMemorySnapshot + CaptureHprofSnapshot,
Please note that a memory snapshot supersedes a HPROF snapshot. You can capture memory snapshot only. If you need a HPROF snapshot per-se i.e. a .hprof file, you can extract it via the command
java -jar <path to>/lib/yourkit.jar -snapshot2hprof <source-file>.snapshot <target-file>.hprof
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Re: Why heap dump limit is always around 840~850mb?
Hi,
I am using scala so I forgot giving -J option to use java option.
Now I can see heap limit is increased when giving -J-Xmx1G
Thanks
I am using scala so I forgot giving -J option to use java option.
Now I can see heap limit is increased when giving -J-Xmx1G
Thanks
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Why heap dump limit is always around 840~850mb?
Hi,
Have you also resolved the issue with triggers? If you haven't, we need to see the trigger definition.
Best regards,
Anton
Have you also resolved the issue with triggers? If you haven't, we need to see the trigger definition.
Best regards,
Anton
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Re: Why heap dump limit is always around 840~850mb?
Thanks,
As I wrote above, with -J, it's resolved.
As I wrote above, with -J, it's resolved.
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Why heap dump limit is always around 840~850mb?
Thank you for confirmation.
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Re: Why heap dump limit is always around 840~850mb?
Hi, this is another problem.
Last time, I tried to use yourkit profiler on my local machine, and tried to reach test server. With -J option I could've seen it worked.
Now I applied options for prod server, and I couldn't get memory snapshot. I changed a little bit like below;
https://imgur.com/a/j9t5II4
And there were OOM problem, so my process restarted around 13:56 so why yourkit log restarted logging like the 2nd image.
The 3rd image is the tail of logging when my process died.
I checked the server health status, and there were OOM but I couldn't get memory snapshot. What's wrong in my jvm option even though I used "usedmem=90"?
Thanks
Last time, I tried to use yourkit profiler on my local machine, and tried to reach test server. With -J option I could've seen it worked.
Now I applied options for prod server, and I couldn't get memory snapshot. I changed a little bit like below;
https://imgur.com/a/j9t5II4
And there were OOM problem, so my process restarted around 13:56 so why yourkit log restarted logging like the 2nd image.
The 3rd image is the tail of logging when my process died.
I checked the server health status, and there were OOM but I couldn't get memory snapshot. What's wrong in my jvm option even though I used "usedmem=90"?
Thanks
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Why heap dump limit is always around 840~850mb?
"usedmem" option instructs the profiler agent to capture a snapshot when memory usage reaches the specified threshold. Memory usage is checked and compared periodically in the agent's telemetry collecting thread, by default once a second. According to the first log file, the telemetry thread pauses were actually much longer due to other thread activities.
It seems that memory grown too high (as high as causing an OOME) between the samples, so the trigger didn't have chance to work. The trigger is _not_ guarantied to always work in fast memory growing scenarios.
Heap dump on OOME is a different thing. It is triggered by the JVM itself, internally. Oracle documentation explains where the dump file is created:
https://docs.oracle.com/javase/8/docs/t ... ts001.html
"
By default the heap dump is created in a file called java_pid<pid>.hprof in the working directory of the VM <...>. You can specify an alternative file name or directory with the -XX:HeapDumpPath= option. For example -XX:HeapDumpPath=/disk2/dumps will cause the heap dump to be generated in the /disk2/dumps directory.
"
To reduce telemetry sample overhead please add the agent option "disablestacktelemetry". By the way, it is recommended by default in the EE server integration wizard. It is recommended to use the wizard instead of manually composing -agentpath because the wizard offers useful option presets: https://www.yourkit.com/docs/java/help/ ... remote.jsp
It seems that memory grown too high (as high as causing an OOME) between the samples, so the trigger didn't have chance to work. The trigger is _not_ guarantied to always work in fast memory growing scenarios.
Heap dump on OOME is a different thing. It is triggered by the JVM itself, internally. Oracle documentation explains where the dump file is created:
https://docs.oracle.com/javase/8/docs/t ... ts001.html
"
By default the heap dump is created in a file called java_pid<pid>.hprof in the working directory of the VM <...>. You can specify an alternative file name or directory with the -XX:HeapDumpPath= option. For example -XX:HeapDumpPath=/disk2/dumps will cause the heap dump to be generated in the /disk2/dumps directory.
"
To reduce telemetry sample overhead please add the agent option "disablestacktelemetry". By the way, it is recommended by default in the EE server integration wizard. It is recommended to use the wizard instead of manually composing -agentpath because the wizard offers useful option presets: https://www.yourkit.com/docs/java/help/ ... remote.jsp
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Re: Why heap dump limit is always around 840~850mb?
Ok, I executed integrate.sh, then applied options.
I will take a look at the server, and write more questions if necessary.
Thanks.
I will take a look at the server, and write more questions if necessary.
Thanks.
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Re: Why heap dump limit is always around 840~850mb?
Hi,
When I tested with and without yourkit profiler option like below, there are too much difference in memory usage. I made that option using yourkit/bin/integrate.sh
https://imgur.com/a/sszeQWX
Thanks,
Jun
When I tested with and without yourkit profiler option like below, there are too much difference in memory usage. I made that option using yourkit/bin/integrate.sh
This is memory status. Both are the same spec servers, and run the same application. Blue line from 11th to 13th, I applied yourkit option on daytimes. After I removed it, then blue and red goes similar in 14th.-J-Xmx7G -J-Xms7G -J-agentpath:/usr/share/yourkit/bin/linux-x86-64/libyjpagent.so=delay=10000,usedmem=90,listen=localhost
-J-Xmx7G -J-Xms7G
https://imgur.com/a/sszeQWX
Thanks,
Jun
-
- Posts: 6172
- Joined: Wed Aug 11, 2004 8:37 am
Re: Why heap dump limit is always around 840~850mb?
Hi Jun,
Please add "disablestacktelemetry" as earlier suggested.
Best regards,
Anton
Please add "disablestacktelemetry" as earlier suggested.
Best regards,
Anton
-
- Posts: 11
- Joined: Fri Feb 07, 2020 7:02 am
Re: Why heap dump limit is always around 840~850mb?
Dear Anton,
Thanks for your help, now it goes stable. very similar to another for which I didn't apply yourkit.
Thanks.
Thanks for your help, now it goes stable. very similar to another for which I didn't apply yourkit.
Thanks.