JDialogs not GC'ed When Using IDEA

Questions about YourKit Java Profiler
Post Reply
gkedge
Posts: 7
Joined: Mon Oct 23, 2006 2:24 pm

JDialogs not GC'ed When Using IDEA

Post by gkedge »

Strange, but true. This problems doesn't occur using Eclipse, but if I debug an application that can launch a JDialog (simple sample below) from IDEA, the JDialog (and its contents) will not be freed up after that JDialog is dismissed (red 'X') and numerous GC's are requested. By launching the application to be profiled from the command line (or Eclipse) and attaching YourKit, YourKit reports the JDialog freeing up nicely after it is dismissed.

Since I prefer to debug and profile at the same time, I like to launch the application in debug mode with the yjpagent manualy included with VM options. But the problem occurs whether I Debug or use the Profile Plugin. Since it was a suprise to me that the Plugin didn't offer Debug when Profiling, is there any problem that you are aware of regarding the profiling of applications under debugger control, specifically IDEA debugger control? Even so, I should be able to use the Plugin and reclaim memory...

Environment:
* WinXP
* 1G RAM
* IDEA 6.0.1 (Build#: 5784)
* JDK 1.5.0_08
* YourKit 5.5.6 (Build#: 938)

Code: Select all

import java.awt.*;
import java.awt.event.*;

import javax.swing.*;

public class SuperSimpleApp extends JFrame {

    public SuperSimpleApp(String title) throws . {
        super(title);
        JButton button = new JButton("Launch Dialog");
        button.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                JDialog dialog = new JDialog(SuperSimpleApp.this, "Super Simple Dialog", true);
                dialog.setDefaultCloseOperation(JDialog.DISPOSE_ON_CLOSE);
                dialog.getContentPane().add(new JLabel("Howdy"));
                dialog.pack();
                dialog.setVisible(true);
            }
        });
        getContentPane().add(button);
    }

    public static void main(String[] args) {
        try {
            EventQueue.invokeAndWait(new Runnable() {
                public void run() {
                    SuperSimpleApp simp = new SuperSimpleApp("Super Simple");
                    simp.addWindowListener(new WindowAdapter() {
                        public void windowClosing(WindowEvent e) {
                            System.exit(0);
                        }
                    });
                    simp.pack();
                    simp.setVisible(true);
                }
            });
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
[/code]
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

Hello,

It is impossible to add profiling to debug because of the design of IDEA open API.

Instead, Run/Debug and Profile are independant, "same level" options.

If IDEA API (and launch UI structure) is changed such that plugins are able to add options to Debug as well, we'll be happy to use this opportunity.

By the way, AFAIK in Eclipse 3.3 it is planned to provide similar capabilities.
gkedge
Posts: 7
Joined: Mon Oct 23, 2006 2:24 pm

Post by gkedge »

Okay. I understand why you can't offer it within the Plugin, but suppose I bypass the Plugin and launch the app Debug with the yjpagent manually added to the VM args. Shouldn't I be able to attach to that running profile-enabled application and debug and profile at the same time?

It seem that when I do that, JDialogs don't clean up as expected.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

I have made experiments with your example.

I think it is rather a JDK-related issue than IDE's.

For example, if I run the example with Java 5 (1.5.0_09), JDialog instances remain [JNI Global] roots. This means they are hold somewhere in native code of AWT/Swing. It seems a memory leak in Swing.

With Java 6 only one (recent if many are opened/closed) JDialog leaks, because it is hold inside repaint manager internals. Also seems a leak in Swing.

Which Java version does your application runs when you launch it with Eclipse and from command line where as you say dialogs do not leak?
gkedge
Posts: 7
Joined: Mon Oct 23, 2006 2:24 pm

Post by gkedge »

Anton:

I am very grateful that you checked out the sample code, but I have to say you freaked me out :shock: a bit with the memory leak in Swing guess. So, I redoubled my efforts to check on your observations. I have rewritten the test to perhaps flush out as much confusion as possible. Any comments to further hone in on the problem or remove confusion would be most welcome! What I have added to the test is a ???GC??™ button to the right within the primary frame. This offers the ability minimize any affects caused by a simple window focus change to an attached YourKit to perform that task. The ???GC??™ button also saves a YourKit memory snap after a short delay after the GC. Prior to running GC, I null out all the nooks and crannies within Swing where strong references can misguide those searching for real memory leaks. Stdout will show when each prior launched dialog (SuperSimpleApp$FinalizeReportDialog) is finialized and will report any lingering dialogs still weakly referenced within the frame??™s ownedWindows. If your press 'Launch Dialog' 3x; I expect you to see 3 dialogs finalized when you press 'GC'.

Try using this program on the command line (remember to include the yjpagent). I think you will find that for every time that you have pressed ???Launch Dialog??™, a press of ???GC??™ will yield as many finalizations of dialogs and an empty list of weakly referenced ownedWindows. I and my fellow Eclipser are using JDK 1.5.0_06. Additionally, I have used 1.5.0_9 with the same observation. Not until I Run or Debug or even Profile via YourKit plugin from IDEA do they not clean up. The memory snap points out that all SuperSimpleApp$FinalizeReportDialog dialogs are indeed JNI Globals.

This seems to indicate to me that it isn??™t the JDK, but some relationship with IDEA 6.0 that causes the weirdness. Memory leaks within Swing??™s history are well documented in the Bug Parade, however a memory leak at this stage of the maturity of 1.5 for such a simple and oft used scenario as exercised in my example would really surprise me. I know you guys have history with the JB dudes so I am hoping that you could share your observations with them perhaps and see what they make of it? If not, I understand and will take it up with them...

Performing the test using JDK 6.0 shows that all SuperSimpleApp$FinalizeReportDialog dialogs are finalized with the exception of the one being held by the RepaintManager. This happens on the command line or using IDEA. Okay, a memory leak in Swing for an oft used feature, but it iiiiis still beta...:wink:

My Eclipser won??™t be able to retest my juiced up sample until tomorrow, but I will be surprised if she finds an Eclipse debug session creating unrecoverable JNI Global anchored dialogs...

Wish there was a better way to share code with you... :|

Code: Select all

import java.awt.*;
import java.awt.event.*;
import java.lang.reflect.*;

import javax.swing.*;

import com.yourkit.api.Controller;

public class SuperSimpleApp extends JFrame {

    public SuperSimpleApp(String title) throws . {
        super(title);
        getContentPane().add(new JButton(new AbstractAction("Launch Dialog") {
            public void actionPerformed(ActionEvent e) {
                new FinalizeReportDialog().setVisible(true);
            }
        }));
        getContentPane().add(new JButton(new AbstractAction("GC") {
            public void actionPerformed(ActionEvent ae) {
                // Clear out the Swing internals that could be holding onto
                // references...
                SuperSimpleApp.referenceSmackDown();
                System.gc();
                final Timer t = new Timer(500, new SnapMemoryAndReportOwnedDialogs());
                t.setRepeats(false);
                t.start();
            }
        }), BorderLayout.EAST);
    }

    // Get agresssive with removing every possible reference to the
    // FinalizeReportDialog...
    private static void referenceSmackDown() {
        try {
            final Field newFocusOwner = KeyboardFocusManager.class.getDeclaredField("newFocusOwner");
            newFocusOwner.setAccessible(true);
            if (newFocusOwner.get(null) instanceof FinalizeReportDialog) {
                System.out.println("Smack newFocusOwner...");
                newFocusOwner.set(null, null);
            }

            final Field realOppositeWindow = DefaultKeyboardFocusManager.class.getDeclaredField("realOppositeWindow");
            realOppositeWindow.setAccessible(true);
            if (realOppositeWindow.get(KeyboardFocusManager.getCurrentKeyboardFocusManager()) instanceof FinalizeReportDialog) {
                System.out.println("Smack realOppositeWindow...");
                realOppositeWindow.set(KeyboardFocusManager.getCurrentKeyboardFocusManager(), null);
            }

            final Method getTemporaryLostComponentMethod =
                    Window.class.getDeclaredMethod("getTemporaryLostComponent");
            getTemporaryLostComponentMethod.setAccessible(true);

            final Method setTemporaryLostComponentMethod =
                    Window.class.getDeclaredMethod("setTemporaryLostComponent", Component.class);
            setTemporaryLostComponentMethod.setAccessible(true);

            final Object[] nullAry = new Object[]{null};
            for (Frame frame : Frame.getFrames()) {
                if (getTemporaryLostComponentMethod.invoke(frame) instanceof FinalizeReportDialog) {
                    System.out.println("Setting Frame temporaryLostComponent...");
                    setTemporaryLostComponentMethod.invoke(frame, nullAry);
                }
                for (Window ownedWindow : frame.getOwnedWindows()) {
                    if (getTemporaryLostComponentMethod.invoke(ownedWindow) instanceof FinalizeReportDialog) {
                        System.out.println("Setting Frame's owned window temporaryLostComponent...");
                        setTemporaryLostComponentMethod.invoke(ownedWindow, nullAry);
                    }
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private class FinalizeReportDialog extends JDialog {
        public FinalizeReportDialog() throws . {
            super(SuperSimpleApp.this, "Dialog", true);
            setDefaultCloseOperation(JDialog.DISPOSE_ON_CLOSE);
            getContentPane().add(new JLabel("Dismiss with red 'X'"));
            pack();
        }

        protected void finalize() throws Throwable {
            super.finalize();
            System.out.println("Dialog Finalized. Yahoo!");
        }
    }

    private class SnapMemoryAndReportOwnedDialogs implements ActionListener {
        public void actionPerformed(ActionEvent ae) {
            try {
                new Controller().captureMemorySnapshot();
                System.out.println("Frame owned dialogs: ");
                for (Window window : SuperSimpleApp.this.getOwnedWindows())
                    System.out.println('\t' + window.getClass().getName() +
                                       "@" + Integer.toHexString(window.hashCode()));
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

    public static void main(String[] args) {
        try {
            EventQueue.invokeAndWait(new Runnable() {
                public void run() {
                    final SuperSimpleApp simp = new SuperSimpleApp("Super Simple");
                    simp.addWindowListener(new WindowAdapter() {
                        public void windowClosing(WindowEvent e) {
                            System.exit(0);
                        }
                    });
                    simp.pack();
                    simp.setVisible(true);
                }
            });
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

Hello,

I have tested you new example with Java 5, launching from IDEA or from the command line.

In either case, JDialogs remain [JNI Global] roots.

So I cannot see difference running from IDEA or not from IDEA.

Maybe I'm missing something making this experiment.
Performing the test using JDK 6.0 shows that all SuperSimpleApp$FinalizeReportDialog dialogs are finalized with the exception of the one being held by the RepaintManager. This happens on the command line or using IDEA. Okay, a memory leak in Swing for an oft used feature, but it iiiiis still beta...Wink
Could you please post this bug to Sun? I don't think they'll fix it in Java release (too few time left), but possibly they can fix it in one of update releases (6.1 or however they finally decided to number them). And I'm absolutely sure they won't fix it if they have no bug submitted :)

Actually, we also fight with Swing leaks by nulling some known fields in Swing classes. Also, on dialog dispose a good measure is to detach content pane. Even if dialog itself leaks, its contents will be freed. This technics works for us because we don't subclass JDialog but rather make classes (with fields that can retain memory) for content panes.
gkedge
Posts: 7
Joined: Mon Oct 23, 2006 2:24 pm

Post by gkedge »

In either case, JDialogs remain [JNI Global] roots.
So I cannot see difference running from IDEA or not from IDEA.
Wow! :shock: That's strange. I am wondering if there is platform sensitivity here. I am using WinXP. You?

Here is what I get running the app on the command line. I press 'Launch Dialog' 3x closing each in succession. I then press 'GC' button and get the desired finalizations x 3. A look at the generated snap shot shows no dialogs.

Code: Select all

% C:/[color=red]jdk1.5.0_06[/color]/bin/java -agentlib:/YourKit/bin/win32/yjpagent -cp "/YourKit/lib/yjp
-controller-api-redist.jar;SuperSimpleFrame.jar" SuperSimpleApp
[YourKit Java Profiler 5.5.6] Using JVMTI
[YourKit Java Profiler 5.5.6] Profiler agent is listening on port 10001...
[YourKit Java Profiler 5.5.6] *** HINT ***: To get profiling results, connect to the application from the profiler UI
Smack realOppositeWindow...
Dialog Finalized. Yahoo!
Dialog Finalized. Yahoo!
Dialog Finalized. Yahoo!
[YourKit Java Profiler 5.5.6] Memory snapshot is saved to C:\Documents and Settings\gkedge.PAYCHEX\SuperSimpleApp-2006-10-25(1).memory
Frame owned dialogs:
If I:
  • * press 'Launch Dialog' again and leave it up.
    * attach YourKit to the running app
    * snap memory from within YourKit
I get the the expected Stack Local designation for the dialog.

Image
Could you please post this bug to Sun?
Done. I neglected to copy the exact title prior to submission and the bug hasn't been assigned a number yet. I will pass on the reference when they post it.
Actually, we also fight with Swing leaks by nulling some known fields in Swing classes.
I notice that YourKit's UI 'Force garbage collection' is more thorough that what I have done. I have come up with my list of culprits through painstaking trial. Are there any more that you could add beyond:
  • *KeyboardFocusManager.newFocusOwner
    * DefaultKeyboardFocusManager.realOppositeWindow
    * DefaultKeyboardFocusManager.realOppositeComponent
    * Window.temporaryLostComponent
Also, on dialog dispose a good measure is to detach content pane.
This way?:

Code: Select all

public class MyDialog extends JDialog {
    public void dispose() {
      removeAll();
or
      remove(getContentPane());
      super.dispose();
    }
}
OR

Code: Select all

dialog.addWindowListener(new WindowAdapter() {
  public final void windowClosed(WindowEvent e) {
     JDialog d = ((JDialog)e.getSource());
     d.removeAll();
or
     d.remove(getContentPane());
  }
}
This technics works for us because we don't subclass JDialog but rather make classes (with fields that can retain memory) for content panes.
I'm sorry, but I don't understand what you mean by that?
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

Regarding the roots. I'll repeat the experiments once again. I'll write about the results later.
Done. I neglected to copy the exact title prior to submission and the bug hasn't been assigned a number yet. I will pass on the reference when they post it.
Great. Thank you.
I notice that YourKit's UI 'Force garbage collection' is more thorough that what I have done.
Do you mean that we do something special in the profiled application when you press 'Force garbage collection'?
Actually, we don't. We simply invoke garbage collection in the profilee, nothing special.

Actually, writing about fixing Swing leak, I was talking about the UI of a particular application - the UI of the profiler itself.
I have come up with my list of culprits through painstaking trial. Are there any more that you could add beyond:

*KeyboardFocusManager.newFocusOwner
* DefaultKeyboardFocusManager.realOppositeWindow
* DefaultKeyboardFocusManager.realOppositeComponent
* Window.temporaryLostComponent
We also handle KeyboardFocusManager.permanentFocusOwner and BasicPopupMenuUI.menuKeyboardHelper.lastFocused

Possibly there are other things to look at, but we don't have such problems with our UI.

Unfortunately, Swing internals is real mess if we talk about leaking components. IMHO, these parts should be rewritten, e.g. using weak references if it's too difficult to manage hard references correctly.
Also, on dialog dispose a good measure is to detach content pane.
We use this:

Code: Select all

    public void dispose() {
      getRootPane().remove(getContentPane());
      super.dispose();
    }
Works fine in our case.
gkedge
Posts: 7
Joined: Mon Oct 23, 2006 2:24 pm

Post by gkedge »

My Eclipser concurs with my expectation: JDialogs do not create JNI Global roots.

Anton: Have you had a chance to try the new program from the command line to see if you experience JNI Globals?
jacobt
Posts: 2
Joined: Wed Feb 14, 2007 4:19 pm

Post by jacobt »

Hi Greg,

I've seen this same problem when profiling with IDEA using JProfiler.
So apologies if this is off-topic here but...

I also get memory leak for JDialogs on Windows.
JProfiler shows a "JNI Global Reference" even when dispose() is called on the dialog and all references are cleared.

I'm using Java 1.5.0_08.
Does anyone know if this is known java bug?
Or are you saying that this is not a real leak and is somehow caused by running the profiler from IDEA?
gkedge
Posts: 7
Joined: Mon Oct 23, 2006 2:24 pm

Post by gkedge »

jacobt wrote: Or are you saying that this is not a real leak and is somehow caused by running the profiler from IDEA?
I am only observing this when using YourKit from IDEA. The problem is that I don't know which of the two are causing the retention. I am inclined to believe that it has something to do with IDEA, but I never took it up with them because I don't know how to prove it.

But, Antone is blaming the JDK. My bug report was rejected from Sun because I wasn't using the latest and greatest at the time. We are using 5.0 and if I recall correctly, they didn't insist that I upgrade my rev of 5.0, they wanted me to go to 6.0! That ain't happening soon... So, I don't know were to go with this.

Since Antone has a close relationship with the IDEA folks, I would hoping that they could come to grips with what the real problem is.

ANTONE: any chance that this can be followed up on?
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

Hello,

I still think it is rather a JDK problem.

I'm absolutely sure we don't do any operations in YourKit profiler agent that can cause JDialog instances to become JNI globals.

As I wrote before, according to my experiments, the problem existed whenever the test was launched from IDEA or not, so I either wouldn't blame IDEA.

Anyway, I have just contacted some IDEA folks. Possibly they have some ideas :)

Could you please make the following experiment: run the test with Java 6 (with Run, not Profile) and capture a snapshot via jmap utility and open it with YourKit. If the objects are still roots, this will proof that the agent is not guilty. Unfortunately, jmap cannot make heap snapshots with Java 5.
Anton Katilin
Posts: 6172
Joined: Wed Aug 11, 2004 8:37 am

Post by Anton Katilin »

The IDEA folks tell me there's no code in production IDEA that creates such roots (namely, this is JNI call NewGlobalRef()).
Anyway, they recommend trying to launch IDEA without all the *.dll in the bin directory to see if this makes a change (e.g. by moving the files temporarily away from the bin directory).
jacobt
Posts: 2
Joined: Wed Feb 14, 2007 4:19 pm

Post by jacobt »

Thanks for your responses and for getting the feedback from the IDEA guys.

Greg - I'd like to see what the response was to your bug report from Sun - do you happen to remember the bug ID?

I've been doing a bit more investigation and I'm inclined to agree that this has not been introduced by IDEA or by the profiler (either YourKit or JProfiler).

I ran the original example in JProfiler, without using IDEA.
No leaks were reported using java 1.4.2_04.
The JNI Global Reference was reported using java 1.5.0_08
I have not yet tried a Java 6 build.

I did wonder if the difference was somehow due to the fact that we are using JVMTI (presumably?) in both of these profiler environments when we move to 1.5, so I decided to adapt your demo to check for leakage without the use of a profiler (and hence without having to initialize JVMTI?).

Here is the original example, which I've adapted a little:

Code: Select all

import java.awt.*;
import java.awt.event.*;
import java.util.ArrayList;
import java.lang.ref.WeakReference;

import javax.swing.*;

public class SimpleApp extends JFrame {
	WeakReference dialogRef = new WeakReference(null);

    public SimpleApp(String title) {
        super(title);

		// button to launch the leaky dialog
		JButton launchButton = new JButton("Launch Dialog");
        launchButton.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                JDialog dialog = new SimpleDialog(SimpleApp.this, "Super Simple Dialog2", true);
				dialog.pack();
                dialog.setVisible(true);
				dialogRef = new WeakReference(dialog);
				// dialog should now only be weakly referenced when disposed
			}
        });
        getContentPane().add(launchButton, BorderLayout.WEST);

		// button to check for leakage via WeakReference
		JButton checkButton = new JButton("Check Leakage");
        checkButton.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
				System.out.println("Dialog is Referenced: " + isReferenced(dialogRef));
			}
        });
        getContentPane().add(checkButton, BorderLayout.EAST);
	}

    public static void main(String[] args) {
        try {
            EventQueue.invokeAndWait(new Runnable() {
                public void run() {
                    SimpleApp simp = new SimpleApp("Simple");
                    simp.addWindowListener(new WindowAdapter() {
                        public void windowClosing(WindowEvent e) {
                            System.exit(0);
                        }
                    });
                    simp.pack();
                    simp.setVisible(true);
                }
            });
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

	// ok so this doen't prove there is a strong ref - but it is pretty likely..
	private static boolean isReferenced(WeakReference ref) {
		for (int i = 0; i < 100; ++i) {
			System.gc();
			System.runFinalization();
			if (ref.get() == null) {
				return false;
			}
		}
		return true;
	}

}

class SimpleDialog extends JDialog
{
	// lots of data to leak with our dialog
	Data data = new Data();

	SimpleDialog(Frame owner, String title, boolean modal)
	{
		super(owner, title, modal);
		setDefaultCloseOperation(JDialog.DISPOSE_ON_CLOSE);
		// button to dispose() the dialog
		// doesn't really matter since default close op is DISPOSE_ON_CLOSE
		JButton disposeButton = new JButton("dispose() me");
		disposeButton.addActionListener(new ActionListener() {
			public void actionPerformed(ActionEvent e)
			{
				setVisible(false);
				dispose();
			}
		});
		getContentPane().add(disposeButton);
	}
}

class Data {
	private Character[] chars = new Character[0xfffff];
}

The dialog now has a button that causes dispose() to be called - I don't think this makes any difference since the DISPOSE_ON_CLOSE should do the same.

The "Check Leakage" button attempts to use a WeakReference to see if the dialog has been cleared. Interestingly, this is reported the dialog as not cleared with both the JDKs I tried.

Finally, I added a load of data to my dialog - if we invoke the dialog enough times, we should see the memory rise in Task Manager and eventually give the OutOfMemoryError that we are worried about here. Using a modest -Xmx64m, the OutOfMemoryError happens on the 15th invoke of the dialog for me with 1.5, but it gets much further (and the memory does not rise too steeply) with 1.4.

My conclusion: I think there is a regression introduced in Java 1.5 that *all* JDialogs and their contents leak. Also - and this is a bit of a guess based on the WeakReference check - I think that some (maybe 1?) references to JDialogs were kept in 1.4 - but profilers using JVMPI could not detect these.

I'd interested to hear what you think of this..

Best Regards
--
Jacob Tredinnick
Post Reply