YourKit Java agent leaks memory when new threads are created

Questions about YourKit Java Profiler
Post Reply
jeff.evans
Posts: 2
Joined: Thu Mar 30, 2017 6:46 pm

YourKit Java agent leaks memory when new threads are created

Post by jeff.evans »

It seems as though simply adding the 2017.02 Java agent causes the YourKit JNI code to leak memory anytime a Thread is created, on JDK 8, even if the application doesn't leak threads itself. Test case to demonstrate:

Code: Select all

public class TestManyThreads {
    public static void main(String[] args) throws InterruptedException {

        final long loopMax = 1000000;

        long loopCnt = 0;
        for (;;) {
            final long thisCnt = loopCnt;
            final Thread t = new Thread() {
                @Override
                public void run() {
                    if (thisCnt == loopMax) {
                        System.out.println("Another million!");
                    }
                }
            };
            t.start();
            t.join();
            loopCnt++;
        }
    }
}

Code: Select all

java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

javac TestManyThreads.java

# run with YourKit agent
java -agentpath:/Applications/YourKit-Java-Profiler-2017.02.app/Contents/Resources/bin/mac/libyjpagent.jnilib TestManyThreads &

# (observe memory usage in top, etc. and observe resident set size continues to grow)

kill %1

# run without YourKit agent
java TestManyThreads &

# (observe memory usage in top, etc. and observe resident set size remains relatively constant after a long time)
Can anyone else confirm this behavior?
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Post by Anton Katilin »

Hello Jeff

We identified the problem and are working on its resolution.

By the way, I should note that any noticeable negative effect can be observed only in applications constantly creating huge number of short living threads. There is no problem in application using thread pools instead.

Best regards,
Anton
jeff.evans
Posts: 2
Joined: Thu Mar 30, 2017 6:46 pm

Re: YourKit Java agent leaks memory when new threads are cre

Post by jeff.evans »

Thanks for the confirmation, Anton. Would this also be the case, albeit much more slowly, for something like a cached thread pool executor, where threads are removed if left idle for a period of time, then recreated the next time a task is submitted?
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Post by Anton Katilin »

Yes, it would.

We're working on a fix now.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Post by Anton Katilin »

Hi Jeff

The fix for this issue will be included to the next build 56 of the released version 2017.02.

However, since we still believe this issue is negligible in most cases and its fix itself is not trivial, we decided not to enable it by default in this release, at least until it is thoroughly tested.

When v2017.02 build 56 is published, please do the following:

1. Specify the startup option "_enable_thread_gc" to enable the fix.

2. Additionally, you may disable the built-in probe Threads with the startup option "probe_disable=.Threads". The probe does not leak memory, but if you disable it you will slightly reduce the agent's memory footprint, as well as let the fix do its job slightly faster in an application actively creating short living threads.

Best regards,
Anton
Thorsten Goetzke
Posts: 13
Joined: Mon Aug 27, 2012 8:58 am

Re: YourKit Java agent leaks memory when new threads are cre

Post by Thorsten Goetzke »

Hello,

Can someone give a number how much memory per thread is actually leaking?
Some Thirdparty/Java Internals just create a lot of Threads and there is nothing you can do about it.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Post by Anton Katilin »

Hello Thorsten

It's approximately 50 bytes per thread plus, depending on active profiling modes (e.g. whether you run CPU profiling or stack telemetry), the thread name may be remembered as well.

If you find these amounts important for your application and its life cycle please use the option specified in my previous post.

Best regards,
Anton
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: YourKit Java agent leaks memory when new threads are cre

Post by Anton Katilin »

[Update]

Since version 2018.04, the described thread collection is enabled by default, and option "_enable_thread_gc" no longer has any effect. Instead, the collection can be disabled with an opposite option "_disable_thread_gc".
Post Reply