No way to explore *all* objects

Questions about YourKit Java Profiler
Locked
Andrei.L
Posts: 35
Joined: Thu Mar 12, 2009 5:23 am

No way to explore *all* objects

Post by Andrei.L »

In our application heap dump that leaks menu instances I've failed to see all objects (>400.000) of a specific type.
I've tried object explorer, objects by class - no way.
Only some few hundreds are shown in the table - which might be regular instances.
I want explore all instances and randomly pick some for closer inspection.
I haven't found a way to perform this task. Yourkit only shows some "and more..." text at the end of the table.
I can't use object ID either, because how I can know that for object that I can't see???

With the visualvm I can open the same dump and show all references as a tree that can be explored for every single tree item.
Similar to yourkit, some objects are shown immediately in the table, and at the table end is a parent not saying "another 400.000 references left" that contains sub-entries grouping every 1000 next elements.

Would be nice to get a hint how all objects of given type can be explored or to add this feature to the next Yourkit version.
Thanks.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: No way to explore *all* objects

Post by Anton Katilin »

Hi,

You can configure how many objects are shown in the explorer by setting -Dyk.max.objects.to.show=<limit> in ~/.yjp/ui.ini. The default value is 500.

However, it's unlikely effective to find the object of interest among 400,000 objects shown as a plain list by exploring objects one by one. Do you know any detail on the object which you want to find?

If it's the retained size, sort explorer by the retained size column.

If you know some field values, please try using the actions "Memory | Objects by Field Value" or "Memory | Strings by Pattern".

If you know where the objects might be referenced from, use the Merged paths dependent view.

Anyway, reducing the set of object by any means is preferred to manually browsing a huge set as a list.

Best regards,
Anton
Andrei.L
Posts: 35
Joined: Thu Mar 12, 2009 5:23 am

Re: No way to explore *all* objects

Post by Andrei.L »

You can configure how many objects are shown in the explorer by setting -Dyk.max.objects.to.show=<limit> in ~/.yjp/ui.ini. The default value is 500.
Good to know. I will change that. Could it be, it doesn't work for "Incoming instances"? I've set to 10 and 10.000 but it looks there are still max ~500 shown and a last node "and more from other objects...".
If it's the retained size, sort explorer by the retained size column.
No way, same size for bad and good objects (only native reference is either disposed or not).
That's the point: we have no clue who is leaking *native* memory related to the leaked instances, but we see that this always happens via specific library class. So we have ~400.000 instances that *inside* the object have an "id" which might shed the light to the origin of the leak.
But ~1000 of those ~400.000 instances are valid and not leaked!
So I might have no luck and see exact those 1000 valid instances in the tree shown by Yourkit...
If you know some field values, please try using the actions "Memory | Objects by Field Value" or "Memory | Strings by Pattern".
To know that field value I have first find & open the "bad" instance of leaked object to get the "id" and check if that id is referenced ~400.000 times :-)

"Memory | Strings by Pattern" works, shows however max 500 items (same limit). So if I would know the id of the "bad" object, I could verify that this object appears at least 500 times with this option.
If you know where the objects might be referenced from, use the Merged paths dependent view.
No way. With a huge UI application a menu object can be created in too many places.
Anyway, reducing the set of object by any means is preferred to manually browsing a huge set as a list.
Using "метод научного тыка" with the help pf visualvm I've quickly identified the offending code, just kind of "bisecting" the 400000 table lead pretty fast to the bad guy.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: No way to explore *all* objects

Post by Anton Katilin »

Could it be, it doesn't work for "Incoming instances"?
You are right. Incoming references have their own threshold -Dyjp.incoming-refs.referrer.limit=<limit> with the same 500 as the default.
Vladimir Kondratyev
Posts: 1624
Joined: Tue Aug 10, 2004 7:52 pm

Re: No way to explore *all* objects

Post by Vladimir Kondratyev »

I believe that manual exploration of 400.000 objects has no much sense. It is not possible for human, and showing such huge amount of data in UI is not scalable. So the first step should be a reducing of object set. Now it is possible to do via "set description language" https://www.yourkit.com/docs/java/help/language.jsp

May be additional filters on primitive and String fields are needed?
Andrei.L
Posts: 35
Joined: Thu Mar 12, 2009 5:23 am

Re: No way to explore *all* objects

Post by Andrei.L »

I believe that manual exploration of 400.000 objects has no much sense.
Of course this depends on the use case, and of course I haven't explored all 400000 elements. But being able to pick some random samples and inspect the values makes lot of sense.
It is not possible for human
Ergo I'm not a human being :-) As said before, using visualvm object browser I bisected the problem very quickly, in few clicks, not even using any automation scripts.
and showing such huge amount of data in UI is not scalable.
That is possible and thanks to visualvm I was able to find offending code quickly, unfortunately not with the tool of my choice (would be yourkit).
I can't attach screenshots here, so please download visualvm from https://visualvm.github.io, open any heap dump and explore any trivial object appearing ~100000 times.
I just want that the same possibility would exist in yourkit, so I can decide by myself what is the most suitable way to solve my current problem.
Note, both visualvm and yourkit are using same underlined UI toolkit (Swing), so it is clearly not UI toolkit limit, but only application limit we are talking here.
Now it is possible to do via "set description language"
That's great to have such capability, but again way complicated compared to clicking few times in the UI.
In my case an object browser that would allow me to randomly pick some nodes and inspect the content would be a great time saver.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: No way to explore *all* objects

Post by Anton Katilin »

Hi,

In the next major version we will improve the object explorer scalability. It will open much faster when a huge limit is specified with -Dyk.max.objects.to.show=<limit>.

Best regards,
Anton
Andrei.L
Posts: 35
Joined: Thu Mar 12, 2009 5:23 am

Re: No way to explore *all* objects

Post by Andrei.L »

Thanks Anton!
Locked