This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
NB 5.0 (fcs) + JDK 1.5.0_06 I tried to investigate memory usage with NB projects from NB sources. I've checked out sources from trunk *all* : 1. Open *ant* module: - start IDE, GCs -> memory meter shows 12/32 MB - open ant project(and all required projects), GCs -> 26/52 MB - close all projects, GCs -> 20/52 MB - open ant project(and all required projects), GCs -> 26/52 MB 2. Open *nbbuild* : - start IDE, GCs -> memory meter shows 12/32 MB - open nbbuild project(and all required projects), GCs -> 88/127 MB - close all projects, GCs -> 76/127 MB ! -> after 5 minutes(no work in IDE, just reading emails out of the NB, GC forced 20x) it went down to 32/120 MB ! - open nbbuild project(and all required projects), GCs -> 86/127 MB - close all projects -> 80/127 MB -> after 5 minutes(no work in IDE, just reading emails out of the NB, GC forced ~20x) it went down to 55/120 MB , then slightly without any work it went up to 78 and held close to this value! -> memory meter showed the same values after 10 minutes(the same work) - open nbbuild project(and all required projects), GCs -> after 5 minutes(no work in IDE, just reading emails out of the NB, GC forced ~20x) it went down to 55/120 MB , then slightly without any work it went up to 78 and held close to this value! -> after 7 minutes (the same work as few minutes ago) it was only 25/89 MB
Doesn't seem to have anything to do with NBM projects, actually. But I am investigating and I think fixing.
Found memory leaks in several components which I will fix in the trunk. I have to say it's a little late to be looking for leaks in 5.0 now... oh well.
Found four problems and tried to correct them. Seems to work for me anyway. Module owners encouraged to review, esp. the JMManager patch which was not trivial. committed * Up-To-Date 1.127 java/javacore/src/org/netbeans/modules/javacore/JMManager.java committed * Up-To-Date 1.41 javacvs/cvsmodule/src/org/netbeans/modules/versioning/system/cvss/util/Utils.java committed * Up-To-Date 1.48 projects/projectui/src/org/netbeans/modules/project/ui/OpenProjectList.java committed * Up-To-Date 1.36 projects/projectui/src/org/netbeans/modules/project/ui/ProjectChooserAccessory.java
Hmm, some of these bugs were in fact pretty serious, so maybe I had better explain in detail, in the order in which I found them. Maybe others can evaluate how serious they are for 5.0. My estimates: P2-P3 for #1 and #2; P1-P2 for #3 and #4. BTW I used jhat on JDK 6 to find all of these. Pretty simple: just run IDE --jdkhome .../jdk1.6, then jps -lm | fgrep $userdir, jmap -dump:format=b,file=$map $pid, jhat -J-Xmx256m $map, and look for instances of Project with nonweak reference chains from roots. Fast and accurate. My test case: open IDE w/ no projects open; Open Project; open web/freeform and all req projects; close them all; wait 20 seconds or so; GC repeatedly until stable; and look for leaks. (After closing projects you must wait at least 15 seconds before trying a GC, so that project.TimedWeakReference becomes weak.) 1. cvss.util.Utils - was tracking global node selection and caching a Node[] -> Lookup<File> calculation based on it. The cache value was not problematic, but the (singleton) cache key was: it often contained a project root node with a reference to some Project. This Project was then held after it was closed. Fix is just to hold the cache key weakly, permitting it to be collected. I am not quite sure how long a key would be held without this fix; my guess is that it would go away if you just selected any other node (another project, something in Favorites, whatever), but I did not actually observe this release happening. And someone who just closes projects and then waits for the memory toolbar to go down will be disappointed. 2. ProjectChooserAccessory - subprojectsCache was holding onto projects which had been displayed in Open Project (either selected or listed in Required Projects). The question is why the PCA itself was still in memory, since it is a file chooser which is supposed to be used once and then discarded. It seems that the native (XAWT) Clipboard was holding a reference to this dialog, though why I am not sure (I didn't copy anything from it, I think). Presumably ref would have been cleared by choosing Open Project again on a different dir, though again I did not observe such a clearage and someone closing projects and expecting memory to go down at once would consider this a bug. Fix was to clear project references when the dialog is hidden. (Note: Required Projects list updater can continue to run after dialog is closed, which is why there needs to be a null check.) 3. OpenProjectList - I found that out of some 50 or so projects I was opening from nb.org, 4 were left in memory and referred to from ProjectManager.modifiedProjects. When running with -J-Dorg.netbeans.api.project.ProjectManager=0, I found that the editor/bookmarks module was modifying them as they were being closed! Apparently I had active bookmarks in sources in these modules, and it seems editor/bookmarks always stores the bookmarks list even if you do not change it (???). What makes this really bad is that it modifies the project *after* its close hook has been run, which is what normally saves the project if it happens to have any pending modifications. Thus the project is left modified and forcibly kept in memory until the IDE is shut down. There does not appear to be any workaround, so this is serious. The fix just makes the Close Project action explicitly save every project it closes *after* firing the change in the list of open projects, so that the editor/bookmarks module saves its bookmarks spuriously as usual and then the project gets saved and is eligible for collection. Could also probably be fixed in editor/bookmarks by (1) making it only save changes when you actually make changes; (2) having it explicitly call saveProject after making changes when a project is being closed. 4. JMManager - this is the weirdest one. Not technically a memory leak but just as bad. Sometimes (but not always - I could not tell exactly what was necessary to reproduce) after closing some projects with Java sources the projects would not be releases. Looking at ProjectManager logging again, I was surprised to see that after clicking the memory toolbar button to do a GC, many projects were getting *loaded* that had been collected before! After some tracing, I found that JMM.cleanUpCodebases (apparently called after GC of ClassPath objects or something?) was indirectly calling SourceLevelQuery on the source roots of these projects, which of course caused them to be loaded again since the default impl of this query delegates to the owner project. This caused a cycle which was sometimes broken with the project getting collected for good, but just as often meant the project would stay around for a long time (and a lot of activity besides GC was happening, of course, hurting overall performance). Now cleanUpCodebases only called SLQ to append e.g. "1.4" or "1.5" to a label which it was removing from a Set<String>; presumably the suffix was there to prevent some bug when source levels are changed on the fly (?). So I changed it to just remove any label which had the right main part with any plausible 1.x source-level suffix - which probably has the exact same effect, but much more cheaply. After that fix, I did not observe any further projects loads (or other queries to ProjectManager) during a GC, and closed projects were collected in a more or less timely manner.
Moving to projects, because the #3 is the most serious and we need to hold it under tracked bugcout.
I consider #4 to be potentially as bad as #3, if it is consistently reproducible. #3 only affects projects in which you have set bookmarks, whereas #4 can affect any project with Java sources, so it is likely to cause bigger leaks. Furthermore, it produces not just a leak but a cycle of repeated GC and allocation that drains CPU resources.
Additional patch for PCA in issue #72495.
Jesse, we would like to fix this also in NB 5.5 . Could you do that? If not, please reassigne. Thanks in advance.
I can do it, if someone from Java team approves the patch to JMManager and someone from CVS team approves the patch to Utils. Just let me know when to merge it. release55 branch? Do I need to do anything special, or just commit it anytime?
Utils 1.41 OK
Merged to release55 incl. issue #72495 refinement. Please verify if possible. Checking in java/javacore/src/org/netbeans/modules/javacore/JMManager.java; /shared/data/ccvs/repository/java/javacore/src/org/netbeans/modules/javacore/JMManager.java,v <-- JMManager.java new revision: 1.118.2.4.2.1; previous revision: 1.118.2.4 done Checking in javacvs/cvsmodule/src/org/netbeans/modules/versioning/system/cvss/util/Utils.java; /shared/data/ccvs/repository/javacvs/cvsmodule/src/org/netbeans/modules/versioning/system/cvss/util/Utils.java,v <-- Utils.java new revision: 1.20.2.3.2.1; previous revision: 1.20.2.3 done Checking in projects/projectui/src/org/netbeans/modules/project/ui/OpenProjectList.java; /shared/data/ccvs/repository/projects/projectui/src/org/netbeans/modules/project/ui/OpenProjectList.java,v <-- OpenProjectList.java new revision: 1.35.6.2.2.1; previous revision: 1.35.6.2 done Checking in projects/projectui/src/org/netbeans/modules/project/ui/ProjectChooserAccessory.java; /shared/data/ccvs/repository/projects/projectui/src/org/netbeans/modules/project/ui/ProjectChooserAccessory.java,v <-- ProjectChooserAccessory.java new revision: 1.31.12.1.2.1; previous revision: 1.31.12.1 done
v/c