It seems as though simply adding the 2017.02 Java agent causes the YourKit JNI code to leak memory anytime a Thread is created, on JDK 8, even if the application doesn't leak threads itself. Test case to demonstrate:
public class TestManyThreads {
public static void main(String[] args) throws InterruptedException {
final long loopMax = 1000000;
long loopCnt = 0;
for (;;) {
final long thisCnt = loopCnt;
final Thread t = new Thread() {
@Override
public void run() {
if (thisCnt == loopMax) {
System.out.println("Another million!");
}
}
};
t.start();
t.join();
loopCnt++;
}
}
}
java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
javac TestManyThreads.java
# run with YourKit agent
java -agentpath:/Applications/YourKit-Java-Profiler-2017.02.app/Contents/Resources/bin/mac/libyjpagent.jnilib TestManyThreads &
# (observe memory usage in top, etc. and observe resident set size continues to grow)
kill %1
# run without YourKit agent
java TestManyThreads &
# (observe memory usage in top, etc. and observe resident set size remains relatively constant after a long time)
We identified the problem and are working on its resolution.
By the way, I should note that any noticeable negative effect can be observed only in applications constantly creating huge number of short living threads. There is no problem in application using thread pools instead.
Thanks for the confirmation, Anton. Would this also be the case, albeit much more slowly, for something like a cached thread pool executor, where threads are removed if left idle for a period of time, then recreated the next time a task is submitted?
The fix for this issue will be included to the next build 56 of the released version 2017.02.
However, since we still believe this issue is negligible in most cases and its fix itself is not trivial, we decided not to enable it by default in this release, at least until it is thoroughly tested.
When v2017.02 build 56 is published, please do the following:
1. Specify the startup option "_enable_thread_gc" to enable the fix.
2. Additionally, you may disable the built-in probe Threads with the startup option "probe_disable=.Threads". The probe does not leak memory, but if you disable it you will slightly reduce the agent's memory footprint, as well as let the fix do its job slightly faster in an application actively creating short living threads.
Can someone give a number how much memory per thread is actually leaking?
Some Thirdparty/Java Internals just create a lot of Threads and there is nothing you can do about it.
It's approximately 50 bytes per thread plus, depending on active profiling modes (e.g. whether you run CPU profiling or stack telemetry), the thread name may be remembered as well.
If you find these amounts important for your application and its life cycle please use the option specified in my previous post.
Since version 2018.04, the described thread collection is enabled by default, and option "_enable_thread_gc" no longer has any effect. Instead, the collection can be disabled with an opposite option "_disable_thread_gc".