Class objects are NOT roots

Questions about YourKit Java Profiler
Post Reply
tombrus
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm

Class objects are NOT roots

Post by tombrus »

Hi, I am using YourKit 3.1 to figure out why our ant builds take so much memory.

YK considers objects of type java.lang.Class as roots, but in my opinion they are not. Specially in situations where ClassLoaders are created (and hopefully GCed) on the fly, they will become garbage as soon as they are not refed anymore.

Maybe somebody can enlighten me? :? Or is this a bug?

Thanks,

-Tom
Vladimir Kondratyev
Posts: 1624
Joined: Tue Aug 10, 2004 7:52 pm

Post by Vladimir Kondratyev »

Actually unloaded classes should not be presented in the dump. So all shown classeses are indeed roots.
tombrus
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm

Post by tombrus »

Unloaded classes are indeed garbage and they should not be shown. I fully agree with that.

What I want to argue about is the Class objects that are NOT garbage because they are referenced by some (or more) other object(s). I want to know what paths there are from the roots (real roots not Classes) to the Class object so that I can find out why my application keeps all the unneeded Classes in the heap. YourKit currently does not show me any paths, just a dialog saying the Class object **IS** a root.

In my opinion the 'rootness' of Class objects is not different from that of String objects or Hashmap objects or any other regular object. The VM and GC indeed do not handle them as roots, just as regular objects.

I hope my point is clear now.
Vladimir Kondratyev
Posts: 1624
Joined: Tue Aug 10, 2004 7:52 pm

Post by Vladimir Kondratyev »

If class doesn't have instances (and even its loader is collected) it doesn't mean that class is not root. Class instance is references by JVM internals. And it means that class is GC root, and it will be root until JVM issues "unload" event.
The VM and GC indeed do not handle them as roots, just as regular objects.
it's not true
tombrus
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm

Post by tombrus »

Vladimir Kondratyev wrote:If class doesn't have instances (and even its loader is collected) it doesn't mean that class is not root. Class instance is references by JVM internals. And it means that class is GC root, and it will be root until JVM issues "unload" event.
How could the VM possibly determine if the Class should be unloaded? My statement is that the VM knows this only because the GC determined that it is garbage. The GC will, by design, never regard roots as garbage. With your line of thought they will never be garbage. That can not be true. The exception here is the bootstrap ClassLoader and the system classes, they are indeed roots and will therefore never be GCed.

All other Class objects are collected by the regular GC. When it decides that a Class object is no longer referenced it, it initiates the unloading. So the fact that it is garbage controls the unloading.

Every Class references its ClassLoader through the internal <loader> variable. Every ClassLoader has a (private) list referencing all Class objects it has loaded. This is an excerpt from ClassLoader.java:

Code: Select all

public abstract class ClassLoader {
    // The classes loaded by this class loader.  The only purpose of this table
    // is to keep the classes from being GC'ed until the loader is GC'ed.
    private Vector classes = new Vector();
}
In the comment you can clearly read that classes are indeed GCed if they are not referenced. The 'classes' list and the <loader> references tie the ClassLoader and its Classes together in order to unload all or none.

A relevant section in the VMSpec at http://java.sun.com/docs/books/vmspec/2 ... html#19147might be:
2.17.8 Unloading of Classes and Interfaces

A class or interface may be unloaded if and only if its class loader is unreachable. The bootstrap class loader is always reachable; as a result, system classes may never be unloaded.
Bottom line for me is that my application leaks memory enormously and it is because classes are not unloaded. That is caused by some bogus left over reference to some Class. YourKit refuses to search for root paths to this Class because it erroneously thinks it IS a root. Not very helpful.

I hope this makes things clearer. I need to find that bug soon...
Vladimir Kondratyev
Posts: 1624
Joined: Tue Aug 10, 2004 7:52 pm

Post by Vladimir Kondratyev »

We'll address this issue to the next version of the profiler. As temporary work-around please try to open incoming references to your class. It should help to define who retains class in memory.
tombrus
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm

Post by tombrus »

Thanks, I appreciate it.

There are a lot of incoming refs and cycles, I'll take my machete and hack away.... ;)

-Tom
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

The first EAP build of version 4.0 will contain the fix. The EAP will start soon, by the end of December.
tombrus
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm

Post by tombrus »

Good news! 8) Thanks for the info.
tombrus
Posts: 27
Joined: Wed Dec 01, 2004 8:53 pm

Thanks!

Post by tombrus »

Thanks!

I finally came around to using the EAP release on our program. I found the unintentional links in no time. Great tools, great service, compliments.

For those who want to know:
We had some (non-running) threads. They appear to be always entered in the global Thread tree. Therefore they are not garbage (even being Deamons), therefore their classes are not garbage, therefore thier ClassLoader is not garbage, therefore all the classes loaded by that ClassLoader are not garbage, therefore all static data of those classes are not garbage. Presto. It was easy to change the code so that the Thread is only constructed when it is actually run and not always created and sometimes run. That did the trick.

Thanks again. :D
attila
Posts: 15
Joined: Fri Feb 11, 2005 6:04 pm

Post by attila »

Hi there,

You've almost sold me your profiler by this feature alone. However, I have a very nasty case of a memory leak that I still can't track down. Like tombrus, I'm too having a system with code loaded through custom class loaders, and am seeing it being stuck in memory. Now, With YourKit 3.2 I could correctly see those class loaders as being stuck in memory, but I couldn't find what's anchoring them. With the latest 4.x EAP, they're no longer even shown, however the app is still leaking memory each time code is reloaded in a new class loader.

Now, there are few other strong references you still don't take into account. If you could add them, it'd be great - maybe turned off by default and only with ability to turn them on when such "advanced" memory leak hunting is happening. Notably these strong references are:
  • from each object to its Class object
  • from each Class object to its superclass Class object
  • from each Class object to its interfaces Class objects
  • from each Class object to its declaring class Class object (for inner classes)
  • from each Class object to its component type Class object (for array classes).
If you think these are far fetched, take a look the two-part diary of my current memory leak hunting chores (third installment unfortunately coming soon). In the first part, you'll see that I had a memory leak caused (transitively) by holding onto the component type of an array in a 3rd party library.

So, if you could allow searching for roots ("roots" here excluding classes and their static variables - static variables are merely just reachable through classes; the "real" roots are just JNI references and threads) taking into account the above dependencies as well, I might finally track down my leak. I set my watch and warrant on it that I'll buy a license if you can solve this.
Post Reply