generations - clarification

Questions about YourKit Java Profiler
Post Reply
inger
Posts: 7
Joined: Mon Mar 21, 2011 1:27 pm

generations - clarification

Post by inger »

Even though I find the generations concept great and intuitive, am I actually a bit lost when it comes to practice..
As far as I understand, when incrementing the generator: (only) newly created objects from that point on will be marked to that new generation count. Also, taking a snapshot implicitly increments the generation.

So, a few questions:
1) when I take a memory snapshot, and look inside it, I see the last generation name is action "Capture snapshot, created by the snapshot itself..
But how come it's non-empty? (if it bumps the gen, and only newly created objects are affected, how come any of those newly created objects(after the snapshot) end up in the snapshot )

2) I had this little test program. I assumed the TryMain object would end up in the gen #3. is this right?
it seems to come in the gen #1 "agent startup.." at least sometimes..

Code: Select all

public class TryMain {
    Object m = null;
    public TryMain() {
        m = new byte[2500];
    }
    public static void main(String[] args) throws IOException {        
        System.out.println("please advance generation : gen #2, then hit enter");
        System.in.read();
        System.out.println("take a memory snapshot now: gen #3 , then hit enter");
        System.in.read();
        
        TryMain d = new TryMain();    // I'd assume this would end up Gen #3.
        
        System.out.println("allocated TryMain");
        System.out.println("please advance generation : gen #4, then hit enter");
        System.in.read();
        System.out.println("take a memory snapshot now: gen #5, then hit enter");
        System.in.read();
        
        System.err.println(d);
    }
}
3) Any way to use Eclipse debugger + YJP attach mode? It largely seems to work, but may break generation numbers?

thanks for listening,
Gergo
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: generations - clarification

Post by Anton Katilin »

Hi Gergo
(only) newly created objects from that point on will be marked to that new generation count.
Looks like the text in the tooltip and the help are not very correct, and we should rework them.

Please consider the following scenario which explains what actually happens:

1) JVM starts its initialization, creates N objects in core classes. They are yet unassigned to any generation.
2) JVM finishes its initialization. These N objects become marked as "#1: JVM initialization". The agent does it automatically.
3) JVM goes on, enters the application entry point, the application starts running... As the result, M objects are created. They are yet unassigned to any generation.
4) You explicitly advance the generation, or capture a snapshot. These M objects are assigned to "#2: Marked by user" or "#2. Captured snapshot ...", respectively.
5) JVM continues running the application, more Z objects are created, they are yet unassigned.
6) Steps 4 and 5 may repeat, giving #3, #4 and so on.
2)
I hope the explanation above covers this case.
3) Any way to use Eclipse debugger + YJP attach mode? It largely seems to work, but may break generation numbers?
Sorry, I don't understand your question. Could you please provide more detail.

Best regards,
Anton
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: generations - clarification

Post by Anton Katilin »

P.S.

I suggest to show the following description in toolbar:
Use to advance current object generation number.
Objects created since that moment will belong to
the next generation.

Note that generations are also automatically advanced
on capturing memory snapshots.
inger
Posts: 7
Joined: Mon Mar 21, 2011 1:27 pm

Re: generations - clarification

Post by inger »

Hi Anton,
thank you for the quick reply and the effort to explain this.
Please consider the following scenario which explains what actually happens:
....
3) JVM goes on, enters the application entry point, the application starts running... As the result, M objects are created. They are yet unassigned to any generation.
4) You explicitly advance the generation, or capture a snapshot. These M objects are assigned to "#2: Marked by user" or "#2. Captured snapshot ...", respectively.
...
So what you're saying: the newly bumped generation assigned to all existing objects that don't have any previous generation assigned to them?...
..which the only meaningful option I can imagine apart from the "bump the counter, all newly created objects will pick that one up", ie no-mass changing modification of records are necessary -- and this is definitely the way I understood from the tooltip and documentation..
Looks like the text in the tooltip and the help are not very correct, and we should rework them.
Yes, again I found them quite clearly and unambiguosly suggesting the other interpretation above.
2) I hope the explanation above covers this case.

Well, if I run the above program - actually a modified version:

Code: Select all

public class TryMain {
    Object data = new byte[2500];
    public static void main(String[] args) throws IOException {        
        System.out.println("please advance generation : gen #2, hit enter");   System.in.read();
        System.out.println("take a memory snapshot now: gen #3, hit enter");   System.in.read();
        System.out.println("please advance generation : gen #4, hit enter");   System.in.read();

        TryMain d = new TryMain();  
        System.out.println("allocated TryMain");
        
        System.out.println("please advance generation : gen #5, hit enter");   System.in.read();
        
        System.out.println("take a memory snapshot now: gen #6, hit enter");   System.in.read();
        System.out.println(d);
    }
}
If understand your version correctly (yet unassigned objects added to new gen), the data object should end up in the generation created just after the allocation: gen #5.
Now that I ran the above again, it's actually gen #4 which I originally assumed... but it seemed I got different results than that earlier, like it ended up in get#1 "starting the agent" which seemed definitely wrong. Not sure what I did differently there then but I cannot reproduce it right now.

So sorry but now it seems the help and the tooltip was right after all - new objects will belong to the generation last created..

In this case however, I don't really understand how the snapshot-created generations can contain any real objects in their respective snapshots (e.g "#2 snapshot" generation which is listed within the that same snapshot file).
3) Any way to use Eclipse debugger + YJP attach mode? It largely seems to work, but may break generation numbers?
Sorry, I don't understand your question. Could you please provide more detail.
[/quote]
Sure, I meant running my code in a debugger, then attaching the profiler (eg while it's stopping at a breakpoint), increment generations, step through code, take a snapshot, compare.
E.g take the above program, remove the System.in.read()-s, step through instead by debugger.

The reason I'm asking is, this seems basically working but I'm not sure how reliable it is (e.g if all threads are suspended, how is the agent executing profiler commands.. but maybe I'm misguided).
I was thinking of the strange behaviour like above(like when my object is assigned to the gen #1, even though there was definitely a later one before the object was created) was because the clash between the debug and profiler agent
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Re: generations - clarification

Post by Anton Katilin »

Hi Gergo

Before answering your questions, I'd like to clarify one thing which is perhaps not obvious, but is essential for understanding how the generations thing works.

In the UI, the label which stands next to the #<number> describes the event which _ended_ the time span in which the generation's objects have been created, but not the event which started that time span.

So, if you see:

#1 JVM initialization 0 sec - 1 sec
#2 Marked by user 1 sec - 10 sec
#3 Snapshot x captured 10 sec 20 sec

the plain list of events is:

(uptime =) 0 sec: JVM started
1 sec: JVM initialization finished; all existing objects assigned generation 1, i.e. JVM initialization.
10 sec: user used Advance generation; all objects created from second 1 until second 10 assigned to generation 2
20 sec: snapshot captured; all objects created from second 10 until second 20 assigned to generation 3
So what you're saying: the newly bumped generation assigned to all existing objects that don't have any previous generation assigned to them?...
..which the only meaningful option I can imagine apart from the "bump the counter, all newly created objects will pick that one up", ie no-mass changing modification of records are necessary -- and this is definitely the way I understood from the tooltip and documentation..
Advance generation action separates two generations.
Snapshot capture also separates two generation.

And there is no renumbering at all. Technically, once an object is created, it has no generation tag at all (or you can think that it has generation 0; it's like a map between objects and numbers; when there is no mapping assigned for a key, the map gives 0 for it). When generation is advanced or a snapshot is captured, all objects tagged with 0 are found, and marked with current generation number N. After all such objects are tagged with N, N is increased. This is an atomic operation, so no objects are created since we start tagging 0's and until we do N++.

So, again, objects are never renumbered. Generation is a persistent tag for each particular object. Object is born with no tag, may be tagged once in its life, if ever, and lives with this tag until GC kills it.

And unlike object allocation recording which must record each object creation event, we don't care about the moments when each object is created. Instead, we just need to find yet untagged ones from time to time; namely, when the generation is advanced or a memory snapshot is captured. This makes generations overhead almost negligible, which can't be said about object allocation recording.
The reason I'm asking is, this seems basically working but I'm not sure how reliable it is (e.g if all threads are suspended, how is the agent executing profiler commands.. but maybe I'm misguided).
I was thinking of the strange behaviour like above(like when my object is assigned to the gen #1, even though there was definitely a later one before the object was created) was because the clash between the debug and profiler agent
It should be reliable to use it this way, however please note that if debugger stop the agent request thread, the profiler won't be able to communicate until debugger resumes.

Please note that you can advance generations programmatically with the help of the profiler API:
http://www.yourkit.com/docs/java/help/api.jsp
or trigger:
http://www.yourkit.com/docs/java/help/triggers.jsp

Please also note that you can make the profiler agent loaded on startup instead of attaching to a running JVM by adding a single VM option -agentpath, as described here:
http://www.yourkit.com/docs/java/help/agent.jsp

Best regards,
Anton
Post Reply