Fully featured low overhead profiler for Java EE and Java SE platforms.
Ultimate profiling and monitoring solution for Gradle, Maven, Ant, JUnit and TestNG.
Easy to use performance and memory profiler for .NET framework.

YourKit Java agent leaks memory when new threads are created

Moderators: Vladimir Kondratyev, Anton Katilin

YourKit Java agent leaks memory when new threads are created

Postby jeff.evans » Thu Mar 30, 2017 6:56 pm

It seems as though simply adding the 2017.02 Java agent causes the YourKit JNI code to leak memory anytime a Thread is created, on JDK 8, even if the application doesn't leak threads itself. Test case to demonstrate:

Code: Select all
public class TestManyThreads {
    public static void main(String[] args) throws InterruptedException {

        final long loopMax = 1000000;

        long loopCnt = 0;
        for (;;) {
            final long thisCnt = loopCnt;
            final Thread t = new Thread() {
                @Override
                public void run() {
                    if (thisCnt == loopMax) {
                        System.out.println("Another million!");
                    }
                }
            };
            t.start();
            t.join();
            loopCnt++;
        }
    }
}


Code: Select all
java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

javac TestManyThreads.java

# run with YourKit agent
java -agentpath:/Applications/YourKit-Java-Profiler-2017.02.app/Contents/Resources/bin/mac/libyjpagent.jnilib TestManyThreads &

# (observe memory usage in top, etc. and observe resident set size continues to grow)

kill %1

# run without YourKit agent
java TestManyThreads &

# (observe memory usage in top, etc. and observe resident set size remains relatively constant after a long time)


Can anyone else confirm this behavior?
jeff.evans
 
Posts: 2
Joined: Thu Mar 30, 2017 6:46 pm

Re: YourKit Java agent leaks memory when new threads are cre

Postby Anton Katilin » Fri Mar 31, 2017 7:18 am

Hello Jeff

We identified the problem and are working on its resolution.

By the way, I should note that any noticeable negative effect can be observed only in applications constantly creating huge number of short living threads. There is no problem in application using thread pools instead.

Best regards,
Anton
Anton Katilin
 
Posts: 5576
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Postby jeff.evans » Fri Mar 31, 2017 2:57 pm

Thanks for the confirmation, Anton. Would this also be the case, albeit much more slowly, for something like a cached thread pool executor, where threads are removed if left idle for a period of time, then recreated the next time a task is submitted?
jeff.evans
 
Posts: 2
Joined: Thu Mar 30, 2017 6:46 pm

Re: YourKit Java agent leaks memory when new threads are cre

Postby Anton Katilin » Fri Mar 31, 2017 7:10 pm

Yes, it would.

We're working on a fix now.
Anton Katilin
 
Posts: 5576
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Postby Anton Katilin » Sat Apr 01, 2017 2:30 pm

Hi Jeff

The fix for this issue will be included to the next build 56 of the released version 2017.02.

However, since we still believe this issue is negligible in most cases and its fix itself is not trivial, we decided not to enable it by default in this release, at least until it is thoroughly tested.

When v2017.02 build 56 is published, please do the following:

1. Specify the startup option "_enable_thread_gc" to enable the fix.

2. Additionally, you may disable the built-in probe Threads with the startup option "probe_disable=.Threads". The probe does not leak memory, but if you disable it you will slightly reduce the agent's memory footprint, as well as let the fix do its job slightly faster in an application actively creating short living threads.

Best regards,
Anton
Anton Katilin
 
Posts: 5576
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Postby Thorsten Goetzke » Tue May 02, 2017 7:58 am

Hello,

Can someone give a number how much memory per thread is actually leaking?
Some Thirdparty/Java Internals just create a lot of Threads and there is nothing you can do about it.
Thorsten Goetzke
 
Posts: 13
Joined: Mon Aug 27, 2012 8:58 am

Re: YourKit Java agent leaks memory when new threads are cre

Postby Anton Katilin » Tue May 02, 2017 9:08 am

Hello Thorsten

It's approximately 50 bytes per thread plus, depending on active profiling modes (e.g. whether you run CPU profiling or stack telemetry), the thread name may be remembered as well.

If you find these amounts important for your application and its life cycle please use the option specified in my previous post.

Best regards,
Anton
Anton Katilin
 
Posts: 5576
Joined: Wed Aug 11, 2004 8:37 am


Return to Java Profiler

Who is online

Users browsing this forum: No registered users and 11 guests