Product Version = NetBeans IDE 7.2 (Build 201207171143)
Operating System = Linux version 3.5.0-10-generic running on amd64
Java; VM; Vendor = 1.7.0_06
Runtime = Java HotSpot(TM) 64-Bit Server VM 23.2-b09
After working with the IDE for several hours debugging a project, Netbeans complains about having no enough memory to compile the project and all sort of errors are shown in the editor (it doesn't recognize existing references to other classes, and things like that that basically make it show errors everywhere).
The only way to recover from the error is to shutdown the IDE and start over, so I guess there is a memory leak somewhere.
I have the dump file but its size is 840 Mb. If you need it, I could upload it to my dropbox public folder and attach a link here.
Created attachment 123502 [details]
Changing to Priority 1 because it leaves the source code with problems. Even after restarting the IDE I need to manually edit and save files in which errors are detected (introducing a new space, for example) for the IDE to be able to compile the project.
After a successful compilation (with no errors) most of my tests fail due to NoClassDefFound Errors (when they run perfectly well before the out of memory error was shown).
reassign to performance team for further evaluation
please upload the zipped heap dump via
heap dump uploaded as requested
Well, even after restarting my computer I cannot run the tests on this project. I can't imagine how the IDE ruined the source code...
Interesting, but I could open, compile AND run the test with no errors with 7.2RC1 without changing a single line of code, but everything keeps failing with 7.2!
Dismiss last comment, the project is not running in 7.2RC1 either
I finally got it working by copying the whole code of the class it couldn't be loaded into a text editor, save it empty, compiling to make it fail, putting the code back in, and compiling it again... Annoying...
Heap dump is available at <http://netbeans.org/projects/profiler/downloads/download/Heapdumps/heap_217339.zip>
There is nothing obvious in the heap dump, but the messages.log is full of
class file: file:/home/marcelo/.cache/netbeans/7.2/index/s3/java/14/classes/suncertify/gui/swing/config/ConfigDialog.sig
source file: file:/home/marcelo/Documents/NetBeansProjects/SJCD_Certification/src/suncertify/gui/swing/config/ConfigDialog.java
WARNING [org.netbeans.modules.java.source.TreeLoader]: Dump could not be written. Either dump file could not be created or all dump files were already used. Please check that you have write permission to '/home/marcelo/.netbeans/7.2/var/log/' and clean all *.dump files in that directory.
are you sure you have correct permissions for netbeans userdir?
assigning to java to evaluate this.
(In reply to comment #11)
> There is nothing obvious in the heap dump, but the messages.log is full of
There surely is something obvious in the heap dump: there are 27 instances of javac! The easiest way to look for them is to look for Symtab - each instance of Symtab maps to an instance of javac.
The instances are held by (in my personal order of importance):
-12 instances (#3, #4, #5, #6, #7, #8, #9, #10, #11, #12, #13, #14) held though refactoring (RenameRefactoring, SafeDelete, WhereUsed), through AbstractRefactoring.getContext(), which includes CompilationInfo. That is illegal. The instances of AbstractRefactoring are typically held from the Undo support, but I think I have seen one instance held differently. Will file a separate bug for that.
-9 instances (#15, #16, #17, #19, #21, #22, #23, #24, #25) that have no GC root. Not sure why these are still on heap, but tzezula mentioned he will try to make LowMemoryWatcher.free() more intensive.
-2 instances (#1, #2) held through code templates, in a way I am yet to understand.
-2 instances (#18, #20) held by eu.easyedu.netbeans.svuid.SerialVersionUidHint$FixImpl
-1 instance (#27) held through a TimedWeakReference ParserManager.cachedParsers.
-1 instance (#26) held through SourceCache (for the actually opened source code in the editor, presumably). This is the only instance which is held undoubtedly validly.
And two generic comments:
-we should consider clearing the content of the CompilationInfo (which then transitively holds the javac) after the task ends. I know Tomas did a lot on this, and everything is prepared, but there are still broken clients that export their CIs outside the parsing tasks. But maybe we should bite the bullet and more enforce the "no javac instances outside parsing tasks" constraint. Clearing the CIs would not prevent all javac-related leaks, but would prevent some of them (in the case of this dump, at least the refactoring-related ones).
-we should double check that marking the source root as dirty and rebuilding it works as intended when the IDE goes out of memory.
/home/marcelo/.netbeans/7.2/var/log/ -> uid=marcelo gid=marcelo permissions=rwxrwxr-w
/home/marcelo/.netbeans -> uid=marcelo gid=marcelo permissions=rwxr-xr-w
/home/marcelo/.netbeans/7.2 -> uid=marcelo gid=marcelo permissions=rwxrwxr-w
Let me know if you need more info about the folders.
One last piece of information I forgot to mention. After getting the error from Netbeans, if I click on the red X circle icon, the following warning message is shown: "Cannot create process. Check the browser configuration".
Depends on contains bugs created for refactoring. A bug for serialversionuid generator plugin:
The code templates are now separate in bug #218063.
I checked if the roots are marked correctly dirty and recompiled on start, and it worked fine, at least in my simple test case.
All the dependant bugs are marked as resolved. Is this bug resolved or do we need to track it as opened?
Also - is this a beta stopper? If not please make it P2....
tzezula made the LMW.free more aggressive, which should help with eviction of garbage from the memory:
Found memory leaks reported separately, see above.
Still present in NetBeans IDE Dev (Build 201209190001)
If the IDE screws up my code somehow, and I need to do the copy & save & paste & save trick to have it compile source code of random affected classes after the error shows up, then I think it is definitely a beta stopper.
(In reply to comment #18)
> Still present in NetBeans IDE Dev (Build 201209190001)
So is there a new heap dump I could analyze? Thanks.
Created attachment 124760 [details]
picture of htop showing lots of netbenas processes
Well, is it normal to have so many processes of Netbeans running? See attached screenshot.
Also, the IDE stays longer without giving me the error, but after long hours (+12) of work, I still get it.
I am attaching a new memory dump...
Memory Dump uploaded to Huston #65
Thanks for the heap dump. There are a few javac held through refactorings (appear to be mostly different paths than the last time), and two or three retained while in a weak-reference based cache (which should be ready for GC), but most of them don't have any GC root, and should have been GCed. I have no idea why these are still there, and how to ensure these are cleared at appropriate places (without actually forcing an OOME, which is obviously very dangerous, or risking a real OOME).
I will file bugs against the refactoring tomorrow, and will optimize some other places, e.g.:
One thing I noticed is that Unit test execution time increases each time I run the test suites. The performance hit is so bad, that a JDialog that opens in .5 seconds in the first run, would do it in about 2.5 seconds in the 4th run. This makes lots of timed tests to fail.
Product Version: NetBeans IDE Dev (Build 201209260001)
I got the error with no need to do any refactoring...
I am uploading a new thread dump. Maybe you can find something else there...
I fixed issue with TypeElement. Hope it helps...
I'll give it a try. But for the records, the memory problems still exists without the extension installed... So it is a bigger problem...
Is there any progress on this issue? It is really frustrating having to restart the IDE to avoid it ruining the source code...
(In reply to comment #27)
> Is there any progress on this issue? It is really frustrating having to restart
Leaks are being regularly fixed AFAIK (typically under the bug numbers under which the corresponding heap dump was uploaded). Some time ago I checked the code that should ensure re-indexing of all broken source roots on next startup after the OOME-like situation, and I was not able to find a problem (which does not mean there is not any). I/we have some ideas on how to improve the memory-is-tight check, but it is uncertain whether these present a viable option.
> the IDE to avoid it ruining the source code...
Just to be precise, I assume that the IDE's caches where broken, not your source code as such. I agree that is still pretty bad.
All sub issues are fixed --> closing as fixed.
This issue is not fixed, in fact in 7.3.1 is even worse than before
Without deeper analysis we can't be sure it is the same bug, and adding to this already long issue doesn't seem to be too good a choice, closing again, please file new bug and upload the heap dump there along with messages.log, thanks.