This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 161201 - Memory/CPU leak after imported key bindings from 6.5
Summary: Memory/CPU leak after imported key bindings from 6.5
Status: VERIFIED FIXED
Alias: None
Product: editor
Classification: Unclassified
Component: Settings (show other bugs)
Version: 6.x
Hardware: Macintosh Mac OS X
: P2 blocker (vote)
Assignee: Vitezslav Stejskal
URL:
Keywords: PERFORMANCE
: 161336 (view as bug list)
Depends on:
Blocks: 168536
  Show dependency tree
 
Reported: 2009-03-25 23:15 UTC by theshadow27
Modified: 2009-09-07 09:58 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
system log see middle around lines 500 (263.59 KB, text/plain)
2009-03-25 23:17 UTC, theshadow27
Details
Netbeans profile (12.82 KB, text/plain)
2009-03-26 19:08 UTC, theshadow27
Details
thread dump (60.27 KB, text/plain)
2009-03-26 19:34 UTC, theshadow27
Details
log_161201_2.txt terminal output (54.92 KB, text/plain)
2009-04-03 04:10 UTC, theshadow27
Details
threaddump during activity (23.51 KB, text/plain)
2009-04-03 04:10 UTC, theshadow27
Details
threaddump from 200904030200 (38.02 KB, text/plain)
2009-04-06 13:32 UTC, theshadow27
Details

Note You need to log in before you can comment on or make changes to this bug.
Description theshadow27 2009-03-25 23:15:02 UTC
Imported preferences from 6.5 into the nightly build from 2009-03-25, noticed CPU usage at 99% and memory peg at 512mb. Messages.log had many of 
these lines, without touching anything in the editor at all (just started it up)

3/25/09 5:56:00 PM [0x0-0x10ed0ec].org.netbeans.ide[61761] WARNING [org.netbeans.modules.editor.settings.storage.keybindings.KeyMapsStorage]: 
Duplicate shortcut 'D-SPACE' definition; rebound from 'all-completion-show' to 'completion-show' in (Editors/Keybindings/NetBeans/Defaults/org-
netbeans-modules-editor-keybindings.xml). 

from 5:55:41 - 5:59:30 so 3 minutes and 49 seconds, there were 405 of the same warning(1.7/second), with no other messages in between. No exceptions 
thrown. Setting as a P4 because it didn't break anything, but there is a leak somewhere. Not sure what stopped it, but the first log after is 

3/25/09 5:59:30 PM [0x0-0x10ed0ec].org.netbeans.ide[61761] INFO [org.netbeans.core.startup.NbEvents]: Warning: the module /modules/org-
netbeans-modules-vmd-kit.jar could not be found, ignoring...
Comment 1 theshadow27 2009-03-25 23:17:12 UTC
Created attachment 78867 [details]
system log see middle around lines 500
Comment 2 Vitezslav Stejskal 2009-03-26 14:55:14 UTC
Could you please reproduce it with an empty userdir (ie. --userdir C:\Temp) and when the CPU is at 99% generate several
threaddumps (http://wiki.netbeans.org/GenerateThreadDump) and attach them here. Thanks
Comment 3 theshadow27 2009-03-26 19:08:06 UTC
Created attachment 78928 [details]
Netbeans profile
Comment 4 theshadow27 2009-03-26 19:11:28 UTC
for some reason control \ doesn't work in terminal for a thread dump, unless I'm doing something wrong. Hard to get a good profile with an empty userdir 
because it's much shorter (1-10 logs, vs 400+)
Comment 5 theshadow27 2009-03-26 19:33:38 UTC
Okay, was able to reproduce the problem on an empty userdir AND get a thread dump. Steps to reproduce:
1) Configured NB6.5 instance with .netbeans directory in the default location (~/.netbeans), and modified key mappings
2) Removed old ~/.netbeans/dev directory 
2) Clean install of 200903250219 into default location (/Applications/NetBeans/NetBeans Dev 200903250219.app)
3) Launch, and say yes to "Import settings from netbeans 6.5
4) Same output in the log. 
Comment 6 theshadow27 2009-03-26 19:34:09 UTC
Created attachment 78932 [details]
thread dump
Comment 7 Vitezslav Stejskal 2009-03-30 14:46:59 UTC
Thanks for the threaddump.
Comment 8 theshadow27 2009-04-03 04:09:14 UTC
Got it again in M3(200904011705). Not after importing settings, on a fresh user directory and new install. Found after installing two plugins, at the same 
time:
1) Spellcheck
2) TODO highlighting
Memory usage 60mb -> 180mb, 
CPU usage at 100% (Got thread dump during this this time)

Same message:
WARNING [org.netbeans.modules.editor.settings.storage.keybindings.KeyMapsStorage]: Duplicate shortcut 'D-SPACE' definition; rebound from 'all-
completion-show' to 'completion-show' in (Editors/Keybindings/NetBeans/Defaults/org-netbeans-modules-editor-keybindings.xml).

Uploading: 
log_161201_2.txt -> from the terminal window, including the activation of the plugin(s)
threaddump-1238727663760.tdump -> Thread dump during the above log output from VisualVM

Comment 9 theshadow27 2009-04-03 04:10:15 UTC
Created attachment 79332 [details]
log_161201_2.txt terminal output
Comment 10 theshadow27 2009-04-03 04:10:49 UTC
Created attachment 79334 [details]
threaddump during activity
Comment 11 Vitezslav Stejskal 2009-04-06 13:21:25 UTC
*** Issue 161336 has been marked as a duplicate of this issue. ***
Comment 12 Vitezslav Stejskal 2009-04-06 13:23:10 UTC
See also the profiler snapshot in issue #161336.
Comment 13 theshadow27 2009-04-06 13:30:13 UTC
Installing spellcheck and TODO seems to be the easiest way to replicate. Found again on 200904030200. JSD
Comment 14 theshadow27 2009-04-06 13:32:00 UTC
Created attachment 79522 [details]
threaddump from 200904030200
Comment 15 Vitezslav Stejskal 2009-04-09 18:19:18 UTC
http://hg.netbeans.org/jet-main/rev/72793f0d45d8

The problem was too many events fired from filesystems. I'm not sure if something actually isn't broken in the
filesystems, but when a module is installed there is hundreds of fileAttributeChanged events fired and they all triggered:

org.netbeans.modules.editor.settings.storage.StorageImpl.refresh
org.netbeans.modules.editor.mimelookup.impl.CompoundFolderChildren.rebuild

StorageImpl was not handling filesystem events efficiently a must have been slow in other situations too, but
CompoundFolderChildren listens only on its closest parent folder and still receives heaps of events. I fixed the problem
by ignoring subsequent events based on their timestamp and FileObject path. But maybe somebody should investigate what
happens in filesystems when a module with a layer is installed.

Here are the steps that reliably reproduced the problem on my laptop:

1. clean build and comments out the ergonomics cluster in netbeans.clusters
2. [maybe not neccessary] prepare a userdir from 6.5 with a simple java project open and some keybindings modified (eg
Fast Import and Fix Imports)
3. download spellchecker plugin from http://plugins.netbeans.org/; unzip it somewhere, it should contain several nbms
4. turn on logging in CompoundCompoundFolderChildren
   (eg. -Dtryme.args="-J-Dorg.netbeans.modules.editor.mimelookup.impl.CompoundFolderChildren.level=FINE")

5. start IDE with empty userdir and import the 6.5 userdir from step 2; wait for the IDE to fully start
6. Tools -> Plugins and install downloaded spellchecker nmbs, all at once
7. look in the log file; there should be a lot of 'Filtering out ...' messages
Comment 16 theshadow27 2009-04-09 19:56:38 UTC
Is this resolved or just covered up? There are a few other big issues that seem to be file-system-listener related. It may be worth tracking to the root 
cause.
Comment 17 theshadow27 2009-04-10 02:19:04 UTC
200904091401 (reproduced via same steps :

WARNING [org.netbeans.modules.editor.settings.storage.keybindings.KeyMapsStorage]: Duplicate shortcut 'D-SPACE' definition; rebound from 'all-
completion-show' to 'completion-show' in (Editors/Keybindings/NetBeans/Defaults/org-netbeans-modules-editor-keybindings.xml).
WARNING [org.netbeans.modules.editor.settings.storage.keybindings.KeyMapsStorage]: Duplicate shortcut 'D-SPACE' definition; rebound from 'all-
completion-show' to 'completion-show' in (Editors/Keybindings/NetBeans/Defaults/org-netbeans-modules-editor-keybindings.xml).
WARNING [org.netbeans.modules.editor.settings.storage.keybindings.KeyMapsStorage]: Duplicate shortcut 'D-SPACE' definition; rebound from 'all-
completion-show' to 'completion-show' in (Editors/Keybindings/NetBeans/Defaults/org-netbeans-modules-editor-keybindings.xml).

I guess 72793f0d45d8 has not been pushed yet.
Comment 18 Quality Engineering 2009-04-10 07:37:33 UTC
Integrated into 'main-golden', will be available in build *200904100201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/72793f0d45d8
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #161201: filter out insignificant filesystem events
Comment 19 theshadow27 2009-04-22 04:22:11 UTC
Verified as of 200904210201
Comment 20 Vitezslav Stejskal 2009-09-03 21:42:31 UTC
I rewrote the original fix with using the brand new FileEvent.runWhenDeliveryOver(), which provides the best way for
coalescing multiple events fired from a single FileSystem.AtomicAction. This new technique should be even more effective
than the original fix. I'm not going to reopen this issue, but it might not be a bad idea if somebody re-verified the
fix in a 6.8 dev build with #a1cdcdd82037. I tested the fix with the steps from desc16 and it seemed to work ok. Thanks

http://hg.netbeans.org/jet-main/rev/a1cdcdd82037
Comment 21 Quality Engineering 2009-09-07 09:58:08 UTC
Integrated into 'main-golden', will be available in build *200909041634* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/a1cdcdd82037
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #161201: rewriting the original fix and using the new FileEvent.runWhenDeliveryOver() method