Fully featured low overhead profiler for Java EE and Java SE platforms.
Performance monitoring and profiling of Jenkins, TeamCity, Gradle, Maven, Ant, JUnit and TestNG.
Easy to use performance and memory profiler for .NET framework.

Determining app's peak heap memory usage

Moderators: Vladimir Kondratyev, Anton Katilin

Determining app's peak heap memory usage

Postby JimF » Wed Jul 24, 2019 10:57 pm

I want to determine my app's peak heap consumption over the span of about 3 hours, not including objects eligible for GC. Is there a way to do this with YourKit?

Without YourKit, I can think of two ways to do this, but both ways have problems:

1) Execute this code least a few times per second to sample memory used:

System.gc();
Runtime rt = Runtime.getRuntime();
long usedMemory = rt.totalMemory() - rt.freeMemory();
System.out.println(usedMemory);

Problems:
- System.gc doesn't force a GC, it only suggests a GC. Also cannot force a full GC.
- The peak could occur between sample points.

2) Run my app repeatedly. On each run, gradually lower the -Xmx max heap size setting until the app throws OutOfMemoryException.

Problem: will take days to complete. Each time I modify app code, I must repeat the run iterations.
JimF
 
Posts: 11
Joined: Tue Jun 19, 2018 1:21 am

Re: Determining app's peak heap memory usage

Postby Anton Katilin » Thu Jul 25, 2019 8:50 am

In the profiler you can see allocated heap memory which includes both reachable objects and objects eligible for GC. You can use it as an estimation.
https://www.yourkit.com/docs/java/help/ ... emetry.jsp
Peak allocated memory usage can be seen in the "Summary" tab too.

I think it is theoretically impossible to always know used heap size minus GC-able objects without actually performing the GC, hence some sort of estimation should be used instead.

Regarding your approach #1. Although System.gc() does not guarantee a GC, in practice several sequential calls (e.g. 3 times in a row) usually do with high probability.

You can improve your approach #1 with the profiler. The profiler allows to force GC via a JVMTI call which is described as such: "Force the VM to perform a garbage collection. The garbage collection is as complete as possible."
You can use a trigger to periodically force GC and output memory usage: https://www.yourkit.com/docs/java/help/triggers.jsp

But as you wrote, this approach is not perfect because the peak may be missed between GCs. Also, explicit GCs add overhead.

I think that as alternative approach you may try to tune with corresponding JVM options the garbage collector to make it as aggressive as possible, therefore reducing the number of non-collected objects at any time. It will never be zero, but your estimation will be more accurate.

Also, the question is how accurately you actually need to measure the peak. Usually, some rough estimate is practically sufficient. E.g., if your app fails with -Xmx8G just run it with -Xmx12G, etc.
Anton Katilin
 
Posts: 5823
Joined: Wed Aug 11, 2004 8:37 am

Re: Determining app's peak heap memory usage

Postby JimF » Thu Jul 25, 2019 6:36 pm

Anton, thanks... you are very helpful.

My goal is to determine the amount each particular code change I make to our very complex app reduces the app's overall peak memory usage. During my test runs to measure memory, slow performance is ok.

I like your idea of tuning GC to be as aggressive as possible during my tests. After reading a lot today about GC tuning, it's still unclear which settings will accomplish this -- do you have any advice or documents you recommend?

I didn't realize JVMTI's ForceGarbageCollection method actually forced GC, I assumed it just called System.gc(). Good to know!
JimF
 
Posts: 11
Joined: Tue Jun 19, 2018 1:21 am

Re: Determining app's peak heap memory usage

Postby JimF » Thu Jul 25, 2019 6:36 pm

Anton, thanks... you are very helpful.

My goal is to determine the amount each particular code change I make to our very complex app reduces the app's overall peak memory usage. During my test runs to measure memory, slow performance is ok.

I like your idea of tuning GC to be as aggressive as possible during my tests. After reading a lot today about GC tuning, it's still unclear which settings will accomplish this -- do you have any advice or documents you recommend?

I didn't realize JVMTI's ForceGarbageCollection method actually forced GC, I assumed it just called System.gc(). Good to know!
JimF
 
Posts: 11
Joined: Tue Jun 19, 2018 1:21 am

Re: Determining app's peak heap memory usage

Postby Anton Katilin » Fri Jul 26, 2019 6:37 am

Available garbage collectors and corresponding JVM settings depend on the Java version you're using. What is the version?
Anton Katilin
 
Posts: 5823
Joined: Wed Aug 11, 2004 8:37 am

Re: Determining app's peak heap memory usage

Postby JimF » Fri Jul 26, 2019 9:28 pm

Currently using Oracle 64bit 1.8.0_181-b13
JimF
 
Posts: 11
Joined: Tue Jun 19, 2018 1:21 am

Re: Determining app's peak heap memory usage

Postby Anton Katilin » Mon Jul 29, 2019 6:14 am

Basic information about available Java 8 collectors can be found here:
https://docs.oracle.com/javase/8/docs/t ... ctors.html

Parallel collector is likely used by default, so you may try instead CMS collector with -XX:+UseConcMarkSweepGC or the G1 collector with -XX:+UseG1GC. G1 is newer and is believed to be superior to CMS. Many good articles can be found by searching for something like "Java CMS tuning" or "Java G1 tuning".
Anton Katilin
 
Posts: 5823
Joined: Wed Aug 11, 2004 8:37 am


Return to Java Profiler

Who is online

Users browsing this forum: No registered users and 15 guests

cron