This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 239961 - maven.queries.Info & PackagingTypeDependentLookup leak project references
Summary: maven.queries.Info & PackagingTypeDependentLookup leak project references
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 7.4
Hardware: PC Linux
: P2 normal (vote)
Assignee: Tomas Zezula
URL:
Keywords: 8.0_HR_FIX, PERFORMANCE
Depends on:
Blocks: 227578
  Show dependency tree
 
Reported: 2014-01-03 15:16 UTC by Jesse Glick
Modified: 2014-03-03 00:47 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse Glick 2014-01-03 15:16:11 UTC
20131231-3d15e64e15b1. After using NB for a while, closing all projects, closing all tool windows (like Find Usages), and hitting GC several times, as usual I found my memory usage to be much higher than after startup. jmap -histo confirmed that there were 44 Maven projects still in heap. Around half had this root reference:

this     - value: org.netbeans.modules.maven.NbMavenProjectImpl #1
 <- project     - class: org.netbeans.modules.maven.queries.Info, value: org.netbeans.modules.maven.NbMavenProjectImpl #1
  <- this$0     - class: org.netbeans.modules.maven.queries.Info$3, value: org.netbeans.modules.maven.queries.Info #1
   <- [0]     - class: java.util.prefs.PreferenceChangeListener[], value: org.netbeans.modules.maven.queries.Info$3 #1
    <- prefListeners     - class: org.netbeans.core.startup.preferences.NbPreferences$UserPreferences, value: java.util.prefs.PreferenceChangeListener[] #664
     <- value     - class: java.util.HashMap$Entry, value: org.netbeans.core.startup.preferences.NbPreferences$UserPreferences #118
      <- [2]     - class: java.util.HashMap$Entry[], value: java.util.HashMap$Entry #119414
       <- table     - class: java.util.HashMap, value: java.util.HashMap$Entry[] #36310
        <- kidCache     - class: org.netbeans.core.startup.preferences.NbPreferences$UserPreferences, value: java.util.HashMap #8163
         <- parent     - class: org.netbeans.core.startup.preferences.NbPreferences$UserPreferences, value: org.netbeans.core.startup.preferences.NbPreferences$UserPreferences #6
          <- prefs     - class: org.netbeans.modules.uihandler.Installer, value: org.netbeans.core.startup.preferences.NbPreferences$UserPreferences #2

Your preferenceChangeListener (#227578) must use WeakListeners.

The other half also had a seemingly unrelated leak:

this     - value: org.netbeans.modules.maven.NbMavenProjectImpl #38
 <- project     - class: org.netbeans.modules.maven.api.NbMavenProject, value: org.netbeans.modules.maven.NbMavenProjectImpl #38
  <- watcher     - class: org.netbeans.modules.maven.NbMavenProjectImpl$PackagingTypeDependentLookup, value: org.netbeans.modules.maven.api.NbMavenProject #38
   <- proxy     - class: org.openide.util.lookup.ProxyLookup$WeakRef, value: org.netbeans.modules.maven.NbMavenProjectImpl$PackagingTypeDependentLookup #38
    <- result     - class: org.openide.util.lookup.ProxyLookup$WeakResult, value: org.openide.util.lookup.ProxyLookup$WeakRef #29400
     <- [38]     - class: org.openide.util.LookupListener[], value: org.openide.util.lookup.ProxyLookup$WeakResult #29400
      <- listeners     - class: org.openide.util.lookup.LookupListenerList, value: org.openide.util.LookupListener[] #502
       <- listeners     - class: org.openide.util.lookup.ProxyLookup$R, value: org.openide.util.lookup.LookupListenerList #446
        <- [0]     - class: org.openide.util.Lookup$Result[], value: org.openide.util.lookup.ProxyLookup$R #451
         <- results     - class: org.openide.util.lookup.ProxyLookup$WeakResult, value: org.openide.util.Lookup$Result[] #268
          <- weakL     - class: org.openide.util.lookup.ProxyLookup$R, value: org.openide.util.lookup.ProxyLookup$WeakResult #1419
           <- metaMergers     - class: org.netbeans.spi.project.support.DelegatingLookupImpl, value: org.openide.util.lookup.ProxyLookup$R #1489
            <- completeLookup     - class: org.netbeans.modules.maven.NbMavenProjectImpl, value: org.netbeans.spi.project.support.DelegatingLookupImpl #3
             <- project     - class: org.netbeans.modules.maven.classpath.SourceClassPathImpl, value: org.netbeans.modules.maven.NbMavenProjectImpl #3
              <- impl     - class: org.netbeans.api.java.classpath.ClassPath, value: org.netbeans.modules.maven.classpath.SourceClassPathImpl #2
               <- second     - class: org.openide.util.Pair, value: org.netbeans.api.java.classpath.ClassPath #32
                <- rootCache     - class: org.netbeans.modules.parsing.impl.indexing.errors.Utilities, value: org.openide.util.Pair #479

which I have trouble understanding but looks like a bug in the Parser API.
Comment 1 Jesse Glick 2014-01-03 15:16:50 UTC
Major memory leak → P2.
Comment 2 Jesse Glick 2014-01-03 15:36:32 UTC
Regarding the second leak, the occurrences match below LookupListener[] #502, which has 44 entries—the number of leaked projects! Something to do with the lookup on Projects/org-netbeans-modules-maven/Lookup/ I think: PackagingTypeDependentLookup.

The Parser API is leaking _one_ project (static cache of last-queried file/path), which is bad (it should use a WeakReference<Pair> to allow the cache to expire) but that is not the sole issue I guess. Note that the ProxyLookup.WeakRef leak is through its (strong) proxy field, not the (weak) referent. Bug in ProxyLookup.setLookups perhaps? It does not look like it is using weak listeners, though the logic is hard to follow. Maybe if a bunch of ProxyLookup’s all call setLookups with the same Lookup delegate, a single LookupListenerList is created which causes all the ProxyLookup’s to effectively refer to one another, leaking them all even if most of them could otherwise be collected? This would be easy enough to check with assertGC.
Comment 3 Milos Kleint 2014-01-04 11:22:59 UTC
http://hg.netbeans.org/core-main/rev/9fe43f675a88
Comment 4 Quality Engineering 2014-01-05 02:14:11 UTC
Integrated into 'main-silver', will be available in build *201401050002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/9fe43f675a88
User: Milos Kleint <mkleint@netbeans.org>
Log: #239961 weak listeners for preferences changes
Comment 5 Milos Kleint 2014-01-15 10:44:45 UTC
jglick, do you still have the heap dump? I've tried to reproduce (here's the dump - https://drive.google.com/file/d/0BxjBuKQiS9xkS2dlNUloekw0QVE/edit?usp=sharing)
but I've got a different nearest gc root - http://screencast.com/t/jZHtFSinv7T 
APTUtils in java.source as the most likely culprit. tzezula mentioned that the references should be removed on project close, but apparently that's not happening.
Comment 6 Milos Kleint 2014-01-15 11:50:02 UTC
another possible leak I've found is: http://screencast.com/t/LQtGZPhQ
JaxswNodeFactory.WsNodeList is adding a task to static RP and that way leaking whole project? cc mkuchtiak
Comment 7 Jesse Glick 2014-01-16 16:42:07 UTC
I do not still have the heap dump, sorry.

I have seen the APTUtils leak before. Of course existence of one leak does not mean that there is not another parallel leak for the same object; the profiler will show only the first it encounters.
Comment 8 Milos Kleint 2014-01-17 09:12:35 UTC
reassigning to java.source, once fixed, please reassign to webservices
Comment 9 Tomas Zezula 2014-02-25 15:24:40 UTC
The APTUtils.sourceRootUnregistered() which removes the M.E<root,APTUtils> from knownSourceRootsMap are called whenever the roots are removed from GPR.
I will  try to reproduce.
Comment 10 Jesse Glick 2014-02-25 16:16:08 UTC
Or is supposed to be removed, but maybe under some conditions this call gets skipped, for example if an exception interrupted the call stack a bit earlier and was not caught at an appropriate point. I suspect that you need to have some simple code double-checking the results of the much more complex code that manages “open” source roots: assertions, in other words. For example, if no projects are open and no editor tabs are open, yet there are some open roots, probably something went wrong and you should capture diagnostics.
Comment 11 Tomas Zezula 2014-02-25 16:26:01 UTC
I've verified in dev that the APTUtils are freed.
I've opened parent mvn project, then opened all the subprojects.
After scan I've closed all projects and I've done gc.
There was no APTUtils instance.
Comment 12 Tomas Zezula 2014-02-25 16:46:48 UTC
>For example, if no projects are open and no editor tabs are open, yet there are >some open roots, probably something went wrong and you should capture >diagnostics.
In fact it's there, you already complained about it in issue https://netbeans.org/bugzilla/show_bug.cgi?id=211163. :-)

I've found one case which may cause the problem I am fixing it.
A will also add a guard into APTUtils, if there are no opened projects but the knownRoots is non empty I will log a warning and clear the knownRoots.
Comment 13 Tomas Zezula 2014-02-26 11:58:15 UTC
Fixed jet-main 747472418edf
Comment 14 Tomas Zezula 2014-02-26 12:00:45 UTC
For web services I've created a new issue #242362.
Comment 15 Tomas Zezula 2014-02-28 09:41:28 UTC
Transplanted 73c04bd3a483
Comment 16 Quality Engineering 2014-03-01 06:14:38 UTC
Integrated into 'main-silver', will be available in build *201403010001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/747472418edf
User: Tomas Zezula <tzezula@netbeans.org>
Log: #239961:maven.queries.Info & PackagingTypeDependentLookup leak project references
Comment 17 Quality Engineering 2014-03-03 00:47:19 UTC
Integrated into 'releases/release80', will be available in build *201403022200* or newer. Wait for official and publicly available build.

Changeset: http://hg.netbeans.org/releases/rev/73c04bd3a483
User: Tomas Zezula <tzezula@netbeans.org>
Log: #239961:maven.queries.Info & PackagingTypeDependentLookup leak project references