Fully featured low overhead profiler for Java EE and Java SE platforms.
Performance monitoring and profiling of Jenkins, TeamCity, Gradle, Maven, Ant, JUnit and TestNG.
Easy to use performance and memory profiler for .NET framework.

Class objects are NOT roots

Moderators: Vladimir Kondratyev, Anton Katilin

Class objects are NOT roots

Postby tombrus » Wed Dec 01, 2004 9:09 pm

Hi, I am using YourKit 3.1 to figure out why our ant builds take so much memory.

YK considers objects of type java.lang.Class as roots, but in my opinion they are not. Specially in situations where ClassLoaders are created (and hopefully GCed) on the fly, they will become garbage as soon as they are not refed anymore.

Maybe somebody can enlighten me? :? Or is this a bug?

Thanks,

-Tom
tombrus
 
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm
Location: Netherlands

Postby Vladimir Kondratyev » Fri Dec 03, 2004 1:49 pm

Actually unloaded classes should not be presented in the dump. So all shown classeses are indeed roots.
Vladimir Kondratyev
 
Posts: 1519
Joined: Tue Aug 10, 2004 7:52 pm
Location: Düsseldorf, Germany

Postby tombrus » Fri Dec 03, 2004 2:07 pm

Unloaded classes are indeed garbage and they should not be shown. I fully agree with that.

What I want to argue about is the Class objects that are NOT garbage because they are referenced by some (or more) other object(s). I want to know what paths there are from the roots (real roots not Classes) to the Class object so that I can find out why my application keeps all the unneeded Classes in the heap. YourKit currently does not show me any paths, just a dialog saying the Class object **IS** a root.

In my opinion the 'rootness' of Class objects is not different from that of String objects or Hashmap objects or any other regular object. The VM and GC indeed do not handle them as roots, just as regular objects.

I hope my point is clear now.
tombrus
 
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm
Location: Netherlands

Postby Vladimir Kondratyev » Fri Dec 03, 2004 7:46 pm

If class doesn't have instances (and even its loader is collected) it doesn't mean that class is not root. Class instance is references by JVM internals. And it means that class is GC root, and it will be root until JVM issues "unload" event.

The VM and GC indeed do not handle them as roots, just as regular objects.


it's not true
Vladimir Kondratyev
 
Posts: 1519
Joined: Tue Aug 10, 2004 7:52 pm
Location: Düsseldorf, Germany

Postby tombrus » Fri Dec 03, 2004 10:20 pm

Vladimir Kondratyev wrote:If class doesn't have instances (and even its loader is collected) it doesn't mean that class is not root. Class instance is references by JVM internals. And it means that class is GC root, and it will be root until JVM issues "unload" event.

How could the VM possibly determine if the Class should be unloaded? My statement is that the VM knows this only because the GC determined that it is garbage. The GC will, by design, never regard roots as garbage. With your line of thought they will never be garbage. That can not be true. The exception here is the bootstrap ClassLoader and the system classes, they are indeed roots and will therefore never be GCed.

All other Class objects are collected by the regular GC. When it decides that a Class object is no longer referenced it, it initiates the unloading. So the fact that it is garbage controls the unloading.

Every Class references its ClassLoader through the internal <loader> variable. Every ClassLoader has a (private) list referencing all Class objects it has loaded. This is an excerpt from ClassLoader.java:
Code: Select all
public abstract class ClassLoader {
    // The classes loaded by this class loader.  The only purpose of this table
    // is to keep the classes from being GC'ed until the loader is GC'ed.
    private Vector classes = new Vector();
}
In the comment you can clearly read that classes are indeed GCed if they are not referenced. The 'classes' list and the <loader> references tie the ClassLoader and its Classes together in order to unload all or none.

A relevant section in the VMSpec at http://java.sun.com/docs/books/vmspec/2nd-edition/html/Concepts.doc.html#19147might be:
2.17.8 Unloading of Classes and Interfaces

A class or interface may be unloaded if and only if its class loader is unreachable. The bootstrap class loader is always reachable; as a result, system classes may never be unloaded.


Bottom line for me is that my application leaks memory enormously and it is because classes are not unloaded. That is caused by some bogus left over reference to some Class. YourKit refuses to search for root paths to this Class because it erroneously thinks it IS a root. Not very helpful.

I hope this makes things clearer. I need to find that bug soon...
tombrus
 
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm
Location: Netherlands

Postby Vladimir Kondratyev » Mon Dec 06, 2004 4:10 pm

We'll address this issue to the next version of the profiler. As temporary work-around please try to open incoming references to your class. It should help to define who retains class in memory.
Vladimir Kondratyev
 
Posts: 1519
Joined: Tue Aug 10, 2004 7:52 pm
Location: Düsseldorf, Germany

Postby tombrus » Mon Dec 06, 2004 10:24 pm

Thanks, I appreciate it.

There are a lot of incoming refs and cycles, I'll take my machete and hack away.... ;)

-Tom
tombrus
 
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm
Location: Netherlands

Postby Anton Katilin » Mon Dec 13, 2004 5:32 pm

The first EAP build of version 4.0 will contain the fix. The EAP will start soon, by the end of December.
Anton Katilin
 
Posts: 5966
Joined: Wed Aug 11, 2004 8:37 am

Postby tombrus » Mon Dec 13, 2004 7:59 pm

Good news! 8) Thanks for the info.
tombrus
 
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm
Location: Netherlands

Thanks!

Postby tombrus » Fri Jan 07, 2005 6:01 pm

Thanks!

I finally came around to using the EAP release on our program. I found the unintentional links in no time. Great tools, great service, compliments.

For those who want to know:
We had some (non-running) threads. They appear to be always entered in the global Thread tree. Therefore they are not garbage (even being Deamons), therefore their classes are not garbage, therefore thier ClassLoader is not garbage, therefore all the classes loaded by that ClassLoader are not garbage, therefore all static data of those classes are not garbage. Presto. It was easy to change the code so that the Thread is only constructed when it is actually run and not always created and sometimes run. That did the trick.

Thanks again. :D
tombrus
 
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm
Location: Netherlands

Postby attila » Sun Feb 13, 2005 2:52 pm

Hi there,

You've almost sold me your profiler by this feature alone. However, I have a very nasty case of a memory leak that I still can't track down. Like tombrus, I'm too having a system with code loaded through custom class loaders, and am seeing it being stuck in memory. Now, With YourKit 3.2 I could correctly see those class loaders as being stuck in memory, but I couldn't find what's anchoring them. With the latest 4.x EAP, they're no longer even shown, however the app is still leaking memory each time code is reloaded in a new class loader.

Now, there are few other strong references you still don't take into account. If you could add them, it'd be great - maybe turned off by default and only with ability to turn them on when such "advanced" memory leak hunting is happening. Notably these strong references are:

  • from each object to its Class object
  • from each Class object to its superclass Class object
  • from each Class object to its interfaces Class objects
  • from each Class object to its declaring class Class object (for inner classes)
  • from each Class object to its component type Class object (for array classes).


If you think these are far fetched, take a look the two-part diary of my current memory leak hunting chores (third installment unfortunately coming soon). In the first part, you'll see that I had a memory leak caused (transitively) by holding onto the component type of an array in a 3rd party library.

So, if you could allow searching for roots ("roots" here excluding classes and their static variables - static variables are merely just reachable through classes; the "real" roots are just JNI references and threads) taking into account the above dependencies as well, I might finally track down my leak. I set my watch and warrant on it that I'll buy a license if you can solve this.
attila
 
Posts: 15
Joined: Fri Feb 11, 2005 6:04 pm
Location: Szeged, Hungary


Return to Java Profiler

Who is online

Users browsing this forum: No registered users and 1 guest

cron

YourKit uses cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from.

By browsing our website, you consent to our use of cookies and other tracking technologies in accordance with the Privacy Policy.