Deep size & Profiler API

bertrand · Post by **bertrand** » Mon Dec 27, 2004 11:30 am

Hello,

I just found the YourKit Java Profiler and it seems to be very interesting to me. I really wanted to be able to run a Java application normally (without any performance impact), then connect to the JVM, get a snapshot of the memory, and analyze it (see which parts of the application uses more memory, walk into the objects, ...). The YourKit Profiler seems to do that.

I tried it but have some problems to get the information i want.
- In the graphical interface, i can walk into the objects of the snapshot (great !), but i really would want to be able to get the "deep size" of an object (size taken by all the objects that can be reached from this object, most profilers have this concept). It would be a lot easier for me to get the informations i search (objects are often referenced multiple times, and retained size is for many objects close to shallow size). Is it possible to get the deep size of an object ?

- I also found great the idea to get a Profiler API, and hoped that i can program all the analyzes i want.
But the only methods i found for the analyze of a memory dump are getObjectCount(set) and getShallowSize(set). It is insufficient for me to program something interesting. I would at least need methods to get the instances of a set, and to get outgoing and ingoing references to an instance. The best would be i can do with the API everything i can do with the graphical interface.
Does a more complete API exist ?
If not, do you plan to publish a more complete API ?

I really find the YourKit Profiler very useful and hope i will buy and use it. But i really need for that a solution for at least one of the points above.

Thanks you in advance for your answers !

Post by **Vladimir Kondratyev** » Mon Dec 27, 2004 1:56 pm

In the graphical interface, i can walk into the objects of the snapshot (great !), but i really would want to be able to get the "deep size" of an object (size taken by all the objects that can be reached from this object, most profilers have this concept). It would be a lot easier for me to get the informations i search (objects are often referenced multiple times, and retained size is for many objects close to shallow size). Is it possible to get the deep size of an object ?

"deep" size is a "retained" size in terms of YJP. It's possible to get retained size of any selected object(s) by running Ctl+Q (View | Quick Info) action.
"Class tree" view also has retained size column

I also found great the idea to get a Profiler API, and hoped that i can program all the analyzes i want.
But the only methods i found for the analyze of a memory dump are getObjectCount(set) and getShallowSize(set). It is insufficient for me to program something interesting. I would at least need methods to get the instances of a set, and to get outgoing and ingoing references to an instance. The best would be i can do with the API everything i can do with the graphical interface.
Does a more complete API exist ?
If not, do you plan to publish a more complete API ?

There is no low level API to traverse object hierarchy and such an API will not be implemented in 4.0 version. But we are going to implement this feature in the version next after 4.0. So you are welcome to formalize your requirement.

bertrand · Post by **bertrand** » Mon Dec 27, 2004 6:49 pm

Vladimir Kondratyev wrote: "deep" size is a "retained" size in terms of YJP. It's possible to get retained size of any selected object(s) by running Ctl+Q (View | Quick Info) action.
"Class tree" view also has retained size column

Thank you for your answer.

The retained size of YJP is NOT my concept of "deep size".
For me, deep size is the size of ALL referenced objects (including objects that are referenced from somewhere else).
For example, let's say we have objects A, B, C (with references to A & B) and D (with reference to B). The retained size of C will be the sum of shallow sizes of C and A (but not B, because also referenced by D). My concept of "deep size" would be sum of shallow sizes of A,B and C, because the exist a path from C to these objects (and the fact that B is also referenced by D has absolutely no importance).
Deep size of an object is always bigger (or equals) than the deep size of the objects it references.

Concepts of shallow size, retained size and deep size are all useful.

Do you have (or plan to have) my concept of deep size in YJP ?

bertrand · Post by **bertrand** » Mon Dec 27, 2004 7:43 pm

Vladimir Kondratyev wrote:There is no low level API to traverse object hierarchy and such an API will not be implemented in 4.0 version. But we are going to implement this feature in the version next after 4.0. So you are welcome to formalize your requirement.

In a perfect world, i would be very happy with all the primitives you use in the graphical interface. Every thing i can do by clicking in the interface could be done instead by calling methods in the Java API.
(If the graphical interface is written in Java, you already have everything i'm searching for

, but i cannot use it because this code is not published and anonymized

)

The main primitives i need are :
- Get starting objects of interests. For example, Instance[] getInstances(xmlSet). the object Instance would tell me the type of the object and the address in memory (not so important).
- Walk into the heap. From a starting point object, find references to/from this object. Instance[] getIncomingReferences(Instance) and Instance[] getOutcomingReferences(Instance).

With this 2 methods, i can walk into the heap and calculate everything i want. It would be easier to have more complete interface, but it is only to make easier the exploitation of the code (nice to have). For example :
- int getRetainedSize(Instance) or int getDeepSize(Instance). If it doesn't exist, i can walk into the outgoing references tree and sum the shallow sizes...
- Path[] getGCRootsPaths(Instance). If it doesn't exist, i can follow the incoming references...
- Every click available in the graphical interface : get objects sorted by
name, shallow size, retained size, ...

Vladimir Kondratyev wrote:But we are going to implement this feature in the version next after 4.0.

When is this release expected ? Will alpha/beta-releases be available soon ? I really like the approach of YKP but can't wait for months before finding a solution to my problems of profiling.

As you already have all the information i need (you can display it), i really would enjoy to get soon a way to access it (without any garanty).

Thanks in advance.

Post by **Anton Katilin** » Tue Dec 28, 2004 1:55 pm

Hello,

I think your "deep size" concept is covered by "reachable-objects" tag of the XML description langauge. Please find the detailed description in the Help.

Regarding release dates. Version 4.0 is planned to be released in January, 2005. We'll most likely start development of the next version soon after 4.0 release, and early access will be available (early access of version 4.0 is here: http://www.yourkit.com/eap ).

Best regards,
Anton