Product Version: NetBeans IDE 6.7 Beta (Build 200904242137)
Java: 1.6.0_13; Java HotSpot(TM) Client VM 11.3-b02
System: Windows Vista version 6.0 running on x86; Cp1252; en_US (nb)

Cannot reproduce:

 1/ Created 5 Maven quickstart projects (fresh IDE installation, clean userdir)
 2/ Invoked Run on each project to generate some load
 3/ Taken heapdump: 5 NbMavenProject instances, ProfilingPointsManager in GC
roots chain
 4/ Closed all Maven projects
 5/ Invoked GC
 6/ Taken heapdump: 5 NbMavenProject instances, ProfilingPointsManager NOT in
GC roots chain


This really looks like a memory leak but the ProfilingPointsManager doesn't
seem to be involved.

Actually WORKSFORME from profiler viewpoint but leaving as INCOMPLETE for now -
please describe your setup and provide
exact steps to reproduce & display the problem.

I'll try to take a heap dump as well with my setup.

I was not able to reproduce the second time around, feel free to close.

actually I was able to reproduce the issue again. this time I have a heap dump
available..

to reproduce, checkout  
https://svn.apache.org/repos/asf/geronimo/server/trunk/

open some of the suprojects of the root project along with it's subprojects
(don't open the root project as it contains
200+ project and the waiting could be too long.) with about 50 projects opened,
close them again. I might have built the
genronimo projects before opening inthe IDE, cannot remember (it might have
inpact on which projects are correctly
loaded, it's dependencies etc, as the local repo is populated bit their bits)

GC repeatedly and wait for any tasks like scanning to finish..

PS: I'm about to push some performance improvements for loading projects -
http://hg.netbeans.org/main/rev/3bfc8f8551ae

Seems to be reproducible using Maven NetBeans Platform Application Archetype.

The problem with Maven NetBeans Platform Application is caused by incorrectly
reported subprojects for Maven projects.
The ProfilingPointsManager is listening for
OpenProjects.PROPERTY_OPEN_PROJECTS, this is my debug output when projects
are opened:

>>> Project opened:  Maven[E:/Projects/MavenProj/PlatformApp/pom.xml]
       subprojects: [Maven[E:/Projects/MavenProj/PlatformApp/module1/pom.xml],
                     Maven[E:/Projects/MavenProj/PlatformApp/branding/pom.xml],
                    
Maven[E:/Projects/MavenProj/PlatformApp/application/pom.xml]]
>>> Project opened:  Maven[E:/Projects/MavenProj/PlatformApp/module1/pom.xml],
       subprojects: []
>>> Project opened:  Maven[E:/Projects/MavenProj/PlatformApp/branding/pom.xml],
       subprojects: []
>>> Project opened:  Maven[E:/Projects/MavenProj/PlatformApp/application/pom.xml],
       subprojects: [Maven[E:/Projects/MavenProj/PlatformApp/module1/pom.xml],
                     Maven[E:/Projects/MavenProj/PlatformApp/branding/pom.xml]]

and this is a debug output when projects are closed:

>>> Project closed:  Maven[E:/Projects/MavenProj/PlatformApp/pom.xml],
       subprojects: [Maven[E:/Projects/MavenProj/PlatformApp/module1/pom.xml],
                     Maven[E:/Projects/MavenProj/PlatformApp/branding/pom.xml],
                    
Maven[E:/Projects/MavenProj/PlatformApp/application/pom.xml]]
>>> Project closed:  Maven[E:/Projects/MavenProj/PlatformApp/module1/pom.xml], subprojects: []
>>> Project closed:  Maven[E:/Projects/MavenProj/PlatformApp/branding/pom.xml], subprojects: []
>>> Project closed:  Maven[E:/Projects/MavenProj/PlatformApp/application/pom.xml], subprojects: []

The problem is caused by application/pom.xml which reports to have subprojects
(using
SubprojectProvider.getSubprojects()) when opened but doesn't report to have any
subprojects when closed.

This works fine when using sample NetBeans platform application PaintApp,
subprojects are reported correctly when both
opening and closing the projects.

I assume the problem with projects for
https://svn.apache.org/repos/asf/geronimo/server/trunk/ will have the same root
cause. This should be fixed on the Maven support side.

there are 2 ways of populating the subprojects for maven projects. 
1. for "pom" packages ones, you get the <modules> section recursively.
2. for all projects the open project list is matched against the list of
dependencies.

I guess 2 is a problem from your perspective. I guess I could remove that but
there's no contract in the
SubprojectProvider that the list of subprojects will stay the same during time.
So your assumption about having the same
set of subprojects at open time and close time might not work in other project
types either.

From my viewpoint the project/subproject relation is defined by projects
configuration, not open/closed status. I think
as long as the configuration is unchanged the subprojects should stay the same
during time.

The API docs don't mention anything about the result being dependent on project
state, but I agree the definition is
quite vague - unfortunately, currently there's no other way for the
ProfilingPointsManager to implement the logic than
what's currently used.

Milos is correct that there is no guarantee that the subprojects list will be
constant between project open and close
time, and the Profiler must not assume that it is. If it needs some list
guaranteed to be constant then it could for
example cache the list it got at project open time and use that at project
close time.

"the project/subproject relation is defined by projects configuration, not
open/closed status" - ideally yes but this is
not always practical for Maven projects, which refer to one another by an
identifier rather than disk path; the only
reasonable way to discover the location of a checkout of a module is to wait
for it to be opened. (It may be desirable
to then _cache_ this information even after the dependency is closed, though
there could be problems introduced by such
a strategy if not implemented carefully.)

Jesse, I don't agree. This is what the SubprojectProvider.getSubprojects() API
documentation says:

"Get a set of projects which this project can be considered to depend upon
somehow. This information is likely to be
used only for UI purposes. Only direct subprojects need be listed, not all
recursive subprojects. There may be no direct
or indirect cycles in the project dependency graph but it may be a DAG, i.e.
two projects may both depend on the same
subproject."

The "there is no guarantee that the subprojects list will be constant between
project open and close
time" statement conflicts with the method description "Get a set of projects
which this project can be considered to
depend upon somehow". If project A is listed as a dependency of project B at
some point, it clearly is considered to be
a dependency and should be always listed as long as the projects configuration
isn't changed. There's no note about the
list of dependencies being dependent on project state.

Moreover, as an API user I don't know anything about the implementation so I
cannot estimate at which point I should
cache the result as suggested. Maybe for other project implementations it could
be when closing the project... If
caching the subprojects when opening the project is a suggested workaround,
then it should be implemented in the maven
project support and not by API users. As you can see the current inconsistency
already caused problems.

Fixed the leak on profiler side by removing some profiling points
functionality. Integrated into profiler-main:


changeset:   129838:0f82b440283d
user:        Jiri Sedlacek <jis@netbeans.org>
date:        Mon May 04 15:55:32 2009 +0200
summary:     Bugfix #164140 - only open subprojects are scanned for profiling
points

"as an API user I don't know anything about the implementation so I cannot
estimate at which point I should cache the
result as suggested" - I don't think this is true. Typical callers of
SubprojectProvider should be using the return
value "only for UI purposes" so they will not be significantly affected if the
result sometimes changes.

The Profiler module, by contrast, seems to make a very specific assumption -
which is not justified by anything in
Javadoc - that the return value of SP cannot change. (Remember that the result
could change from time to time for _any_
project if its configuration is changed.) Furthermore, a failure of this
assumption causes a memory leak, rather than
merely a minor inconsistency in some display. Any code which holds references
to some objects with the expectation of
releasing those references in the future must assure for itself that it is
releasing the exact same set of objects it
initially acquired.

In your case, the "[point] at which [you] should cache the result" is precisely
the point at which you are initially
acquiring these object references that could leak.

None of this is to say that it would not be _desirable_ for Maven projects to
produce a less variable return value from
SP - just that improving this behavior would not absolve callers of SP from
being written robustly.

The assumption that the return value of SP cannot change was incorrect, the
leak in profiler is now fixed. BTW using
SubprojectProvider.addChangeListener() is the right solution, not caching the
results obtained when project is opened.

I've filed separate Issue 164392 for the SubprojectProvider.getSubprojects(),
the implementation of maven project
doesn't match the API documentation or the documentation is incorrect. Without
mentioning that the result could change
(just based on project state without any configuration changes) nobody can
expect it and write robust code.

Integrated into 'main-golden', will be available in build *200905070201* on
http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/0f82b440283d
User: Jiri Sedlacek <jis@netbeans.org>
Log: Bugfix #164140 - only open subprojects are scanned for profiling points