This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 151786 [details] Screenshot of a jvisualvm session with a crawling NetBeans Having 20 (Java only) small/medium projects open and doing typical development work: editing, debugging and running unit tests, the memory consumption of NetBeans steadily increases from typically 100MB after startup to 700MB, makeing the IDE completely unresponsible and forcing me to restart it many times during a regular day. Reproducible. Always. Not only sometimes. This issue is fatal. GC doesn't really reduce the memory consumption, neither as displayed by NetBeans (in the toolbar) nor the one which JvisualVM reports. Memory snapshots always are showing a excessive number of "com.sun.tools.jdi.Packet" instances, see attached screen shot with around 2.700.000 entries. The full heap dump is available (altough rather large and I'm somewhat concerned about the Java sources it includes). NetBeans is started with default memory options on a system with 16GB RAM. Increasing the available memory with -Xmx doesn't help: it just takes a few edit/debug/test cycles more to sonsume the additional memory and the GC is even slower with more memory. System Info: Product Version = NetBeans Platform 8.0.2 (Build 201411181905) (#8230772f5f18) Operating System = Linux version 3.12.36 running on i386 Java; VM; Vendor = 1.8.0_31; Java HotSpot(TM) Client VM 25.31-b07; Oracle Corporation Runtime = Java(TM) SE Runtime Environment 1.8.0_31-b13 Java Home = /opt/java/jdk1.8.0_31/jre System Locale; Encoding = de_DE (nb); UTF-8
Are you using 32bit Linux and 32bit JDK on the system with 16GM memory? Please try to use 64bit JDK. If it happen again, reopen this issue and attach VisualVM's application snapshot of NetBeans - this snapshot contains much more information than your screenshot. Did you get OutOfMemoryError? If so, it would be great if you can provide heap dump from this OutOfMemoryError.
No, no out of OutOfMemoryError involved so far, just a more or less unusable NetBeans. I would be willing to send you a snapshot by mail or provide it for download, but only for a 32Bit JDK (many virtual machines running each with 1GB or less fit very well on a hardware with 16GB). Please give me some feedback if you are interested in bugs related to the 32Bit version.
So in fact you are running NetBeans on 32bit Linux with 1GB RAM not 16GB. This is a big difference. We are interested in bugs on 32bit JDK, my point was that running 32bit JDK on machine with 16GB RAM does not make much sense. Please attach VisualVM's application snapshot - it would be useful to monitor NetBeans with VisualVM for a minute or two with VisualVM and than create application snapshot. Please also take several snapshots from memory sapling. It would be nice to have a heap dump, but I understand that you may have concern about heap dump content. In addition to application snapshot, please also attach messages.log from the same NetBeans session. Thanks.
I prepared a small environment consisting of only 2 projects (PDFRenderer and Substance), no local history or version control involved. I did some code changes and debugging, it took more time as in my real working environment, but the result was the same: NetBeans started with around 50MB and finally exhausted the available 1024MB just with this two projects open, see attached snapshot and log.
Created attachment 151840 [details] IDE log
Created attachment 151841 [details] Application snapshot with some memory snapshots included
Can't attach heap dump, error: The file you are trying to attach is 252305 kilobytes (KB) in size. Attachments cannot be more than 10240 KB.
(In reply to bleun from comment #7) > Can't attach heap dump, error: > The file you are trying to attach is 252305 kilobytes (KB) in size. > Attachments cannot be more than 10240 KB. Right, heap dump cannot be attached to this issue. It is too big. Please zip the heap dump and upload it at <http://deadlock.netbeans.org/job/upload/build>. Thanks a lot.
Heap dump uploaded ("build" 577?) - do you have sufficient informations now to reopen the ticket again? Thanks.
The heap dump has 4,760,280 instances of com.sun.tools.jdi.Packet. There are also 21 (!) instances of com.sun.tools.jdi.TargetVM, which have retained size 599,347,530 bytes, which is 81% of whole heap. com.sun.tools.jdi.TargetVM instances hold Packet instances in waitingQueue instance variable. It looks like duplicate of issue #243837, but it would be nice to investigate, why there are 21 instances of com.sun.tools.jdi.TargetVM. The problem is probably caused by repeated debugging sessions, but it would be great to have steps how to reproduce this problem. I was not able to reproduce it with simple j2se project and several consecutive debugging sessions. com.sun.tools.jdi.Packet was always correctly garbage-collected. Reassigning to debugger/java
The packet accumulation problem is hopefully fixed by issue #243837. Multiple TargetVM instances can indicate a memory leak in NB debugger, or they simply represent unfinished sessions for which the packets are accumulating.
To reproduce, it may help to debug the very same project (PDFRenderer 0.91 from https://java.net/projects/pdf-renderer/downloads) with this file https://www.adobe.com/devnet/pdf/pdf_reference.html. I had a lot of trouble and a frustrating experience while trying to debug the outline-tree reading in PDFFile.java (method getOutline), the debugger did freeze (100% CPU usage, no response at all) several times when trying to show method return values (just after positioning the mouse on a variable). I had to stop the debugger session multiple times. Very irritating and unusual. Maybe stuff for another ticket. The memory issues itself (with lots of lost JPDADebuggerImpl and jdi.Packet instances) is more or less unchanged since NetBeans 7.3 (and JDK 1.6) and needs no debugger freezes - it happens all the time for me. I would be willing to test nightly builds and provide prompt feedback on supposed fixes.
Out of curiosity I downloaded NetBeans IDE Dev (Build 201502060001) (#3c701cf9c7cb), which should inlcude the fix mentioned in #243837. I can easily reproduce the debugger issue: setting a breakpoint, hitting 3 or 4 time "step over" or "continue" and the debugger freezes: CPU monitor shows 100% utilisation of a single core, no further debugging actions possible, but the IDE is still resonsible. If I use the "Finish Debugger Session" button to stop the debugger in this situation, VisualVM immediately increases the number of "com.sun.tools.jdi.TargetVM" instances by one. Requesting a GC doesn't change them. This happens absolutely reliable, every single time I start the debugger. Might be related to the OS (Ubuntu 12.04) or desktop (KDE 4.8).
I've reproduced the problem with 100% CPU. It's a problem in the PDFRenderer implementation. The CPU is consumed by the debugged application, not by NetBeans. It's easy to reproduce, in getOutline() method step into the loop while (scan != null) { and hover the mouse over "scan" variable. This calls PDFObject.toString() and this call never returns. FYI I'm attaching the thread dump taken during this call. It looks like an infinite loop in the toString() implementation.
Created attachment 151870 [details] Thread dump of the debugged application.
I'll check the problem with "com.sun.tools.jdi.TargetVM" instances. This is possibly a memory leak in NetBeans.
Certainly the memory leak is more important and the subject of this ticket, but a debugger relying on well behaved target code isn't optimal either. I took the opportunity to question this in new bug 250305.
After some more experiments I'm sure to understand the reason of this bug, which is much worse than expected. As it turns out, _every_ debugger session leaves a new TargetVM behind, which represents a VirtualMachineImpl, easily having a retained size of 1MB or more. You might say: 1MB, that's nothing, it's conceded ... Well, sometimes those orphaned VMs are sort of alive, especially with all the JPDAThreadImpl instances still associated to them. In such situations the VM is causing low CPU load and NetBeans quickly runs out of memory, because all the JDI packets accumulate in the waitingQueue. The result: if I use the debugger, I have to restart NetBeans several times per day. After 5 debugger sessions I see 5 orphaned instances of VirtualMachineImpl as well as 5 JPDADebuggerImpl. Always. With some luck you can see that the number of packet instances increases even if the debugger session is long finished. You can have more heap dumps if you need some, but you should be able to reproduce this easily on your system. Just start and immediately finish a debugger session, no need to do anything special. The most trivial Java-SE project should cause it, or use the PDFRenderer as I did. Look at the TargetVM or VirtualMachineImpl instances in VisualVM after the session is stopped. They are neither released nor collected. This fatal memory leak seems to be caused by DebuggerManager and org.netbeans.api.debugger.Lookup, both holding all kinds of tree and table models, referencing JPDA objects representing com.sun.tools.jdi implementations having a VM reference. Just search the nearest GC root in VisualVM. The supposed fix - handling VM disconnect exceptions in the generated JDI wrappers - for Bug 243837, has nothing to do with this and doesn't prevent the packet accumulation at all. Tested with JDK 1.7 and 1.8, with NetBeans 7.3, 8.0.2 and the latest nightly builds, always the same.
I can reproduce the memory leak - I've submitted issue #250636.
(In reply to Tomas Hurka from comment #3) > my point > was that running 32bit JDK on machine with 16GB RAM does not make much > sense. I agree but a 64bit JDK cannot be run as client VM only as server VM. http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#64bit_compilers Its warm up time makes development slow where new VMs are started in the development cycle often. So I installed a 32bit JDK for faster development.
This should be finally fixed by fixing issue #250636. I've verified that TargetVM instance is released from memory after debugging finishes, therefore this should be fixed as well.