This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Performance tests found huge regression in test case : Expand Source Packages node (expand Source Packages node in Projects view, testing project JEdit) on all machines for the first usage between builds 200703011900 & 200703041900 Expand Source Packages node - 1st usage 121 % 110 % 108 % 127 % 112 % 103 % ( Linux FC 3/1.5.0_11, Solaris 9/1.5.0_11, Solaris 10/1.5.0_11, Win 2000/1.5.0_11, Win XP/1.5.0_11, Win XP/1.6.0) Logs from repaint manager shows the same number of repaints, the only one difference is that all paints are taking more time : http://beetle.czech.sun.com/automatedtests/xtest/netbeans_dev/200703041900/qa-performance/qa-perf-xp2/testrun_070305-004946/testbag_3/user/gui.action.ExpandNodesProjectsView/Expand%20Source%20Packages%20node/ActionTracker.xml http://beetle.czech.sun.com/automatedtests/xtest/netbeans_dev/200703011900/qa-performance/qa-perf-xp2/testrun_070302-012246/testbag_11/user/gui.action.ExpandNodesProjectsView/Expand%20Source%20Packages%20node/ActionTracker.xml Subsequent invocation is fine
Created attachment 39177 [details] Graph - history of measured values - Expand Source Packages node
Nothing has changed recently in the nodes+explorer, so It must be either some new and really slow loader added (profiler?) or some change in DataSystems/other loader. We need to fix this for 6.0, but I don't have time now to analyze this.
I'm going to look at this.
One note about the test: perf.dont.resolve.java.badges system property is no-op after Retouche merge.
I cannot reproduce this regression. Or if I can it is not bigger than 25%. Trying with the same project. The only difference is that it is not in same folder ignored by versioning (sharability query is visible when called from PackageView.findNonExcludedPackages in profiler results). My suspect was includes/excludes but this is not confirmed.
I already tried to put these project into directory ignored by versioning. Passing back to Marian for further investigation.
Radim, I measured both builds locally : 200703011900 Expand Source Packages node [1] 671 ms Expand Source Packages node [2] 28 ms 200703041900 Expand Source Packages node [1] 2114 ms Expand Source Packages node [2] 25 ms I was watching the tests and I think that in the build 200703011900 nodes are painted(added under Source Packages node) in Explorer by group of 3-5, but in build 200703041900 each node is painted(added under Source Packages node) separately. Logs from Repaint Manager shows that each paint of org.openide.explorer.view.TreeView$ExplorerTree, 0,0,1115,753 takes longer.
OK, now I see the difference between two these builds both in regular usage and in profiler. It is not that big on my notebook or my Ultra-20 desktop (it is like 50%). Slower machine running our tests can be more affected. The biggest diffference that I see is use of SharabilityQuery. This means that we convert FO to file and unfortunately SQ impls like ProjectSharabilityQuery convert this back to FO and deep in stack called from MasterURLMapper.geFOs you can see also FU.normalizeFile. FileOwnerQuery is now also called more often - from SourcesHelper$SourceRoot$Group.contains as well as from ProjectSharability query. Another problem that I see is PackageViewChildren.refreshKeysAsync that post to EDT request to call setKeys() so we refresh keys with each added node/package and it is almost always followed by repaint. This is probably the same as before integration of includes/excludes but still something what worked differently than in past where we processed larger batches IIRC. I'm reassigning to Jesse G. who integrated excludes/includes.
PackageViewChildren.refreshKeysAsync has been used for a few months. It makes absolute times worse but responsiveness much better. Could probably be massaged to batch up key refreshes with some timeout ~ 2-3sec (i.e. need to show even single refreshes within the timeout), but would need to play with it interactively to see the effect. Package expansion is sufficiently fast on my computer that anything I try will probably look fine. Regarding ProjectSharabilityQuery etc. - sounds plausible; various methods could probably be optimized to keep a cache of recent arguments, or something like this. (For example it is likely that FileUtil.toFile{,Object} could be made substantially faster when called many times, by keeping a LRU cache of arguments and near ancestors.) My test case is a simple j2seproject with JDK 7 (Subversion checkout) j2se/src/share/classes, running on JDK 6 -Xmx512m (although it in fact never uses >40Mb heap). I have the Retouche cache built in advance since that is slow to create the first time. In an uninstrumented VM, Source Packages expands on my Tecra M5 in a few seconds, with smooth progression (new packages appear steadily and progress bar moves along). I am trying to profile classes from several NBM projects, since the whole point is that it is unclear exactly what is the culprit: openide{fs,masterfs} projects/{queries,projectapi} java/{project,j2seproject} ant/project Unfortunately my attempts to get profiling results have failed badly. The profiler begins to collect results but then quickly eats up more and more memory (in the development IDE, not the target VM), leading to OOME even with -Xmx400m, and I eventually have to killall -9 java. Heap dumps show the devel IDE full of ~150k strings and a lot of WeakReference's.
I am still unable to run the profiler (see issue #107753), so I am not going to be able to work on this unless someone can either assist me in setting up the profiler, or tell me where the problem spots are (and verify the effectiveness of proposed patches).
See if these changes helped any. Checking in ant/project/src/org/netbeans/spi/project/support/ant/SourcesHelper.java; /shared/data/ccvs/repository/ant/project/src/org/netbeans/spi/project/support/ant/SourcesHelper.java,v <-- SourcesHelper.java new revision: 1.20; previous revision: 1.19 done Checking in projects/queries/src/org/netbeans/api/queries/VisibilityQuery.java; /shared/data/ccvs/repository/projects/queries/src/org/netbeans/api/queries/VisibilityQuery.java,v <-- VisibilityQuery.java new revision: 1.8; previous revision: 1.7 done Checking in java/project/src/org/netbeans/spi/java/project/support/ui/PackageView.java; /shared/data/ccvs/repository/java/project/src/org/netbeans/spi/java/project/support/ui/PackageView.java,v <-- PackageView.java new revision: 1.24; previous revision: 1.23 done Checking in java/project/src/org/netbeans/spi/java/project/support/ui/PackageViewChildren.java; /shared/data/ccvs/repository/java/project/src/org/netbeans/spi/java/project/support/ui/PackageViewChildren.java,v <-- PackageViewChildren.java new revision: 1.81; previous revision: 1.80 done Checking in projects/projectapi/src/org/netbeans/modules/projectapi/SimpleFileOwnerQueryImplementation.java; /shared/data/ccvs/repository/projects/projectapi/src/org/netbeans/modules/projectapi/SimpleFileOwnerQueryImplementation.java,v <-- SimpleFileOwnerQueryImplementation.java new revision: 1.16; previous revision: 1.15 done
verified with latest builds