This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 114155 - Apisupport should not scan universe during IDE startup
Summary: Apisupport should not scan universe during IDE startup
Status: RESOLVED WONTFIX
Alias: None
Product: apisupport
Classification: Unclassified
Component: Project (show other bugs)
Version: 6.x
Hardware: All All
: P3 blocker (vote)
Assignee: Jesse Glick
URL:
Keywords: PERFORMANCE
: 116720 (view as bug list)
Depends on: 105324
Blocks:
  Show dependency tree
 
Reported: 2007-08-29 19:41 UTC by Jan Becicka
Modified: 2007-10-24 22:47 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Becicka 2007-08-29 19:41:04 UTC
Todays build. JDK 1.6.0_02, Windows XP

NB startup is horrible due to this bug. It might be bug in editor.

main" prio=6 tid=0x209d2800 nid=0xa04 runnable [0x20d3f000..0x20d3fc94]
  java.lang.Thread.State: RUNNABLE
       at java.io.WinNTFileSystem.getBooleanAttributes(Native Method)
       at java.io.File.isFile(File.java:778)
       at org.netbeans.modules.apisupport.project.universe.ModuleList.scanPossibleProject(ModuleList.java:328)
       at org.netbeans.modules.apisupport.project.universe.ModuleList.scanNetBeansOrgStableSources(ModuleList.java:246)

       at org.netbeans.modules.apisupport.project.universe.ModuleList.createModuleListFromNetBeansOrgSources(ModuleList
java:214)
       at org.netbeans.modules.apisupport.project.universe.ModuleList.findOrCreateModuleListFromNetBeansOrgSources(Modu
eList.java:204)
       at org.netbeans.modules.apisupport.project.universe.ModuleList$1.run(ModuleList.java:151)
       - locked <0x02ab5068> (a java.util.HashMap)
       at org.netbeans.modules.apisupport.project.universe.ModuleList$1.run(ModuleList.java:122)
       at org.openide.util.Mutex.readAccess(Mutex.java:269)
       at org.netbeans.modules.apisupport.project.universe.ModuleList.getModuleList(ModuleList.java:121)
       at org.netbeans.modules.apisupport.project.NbModuleProject.getModuleList(NbModuleProject.java:485)
       at org.netbeans.modules.apisupport.project.Evaluator$2.run(Evaluator.java:203)
       - locked <0x02abd1e0> (a org.netbeans.modules.apisupport.project.Evaluator)
       at org.netbeans.modules.apisupport.project.Evaluator$2.run(Evaluator.java:198)
       at org.openide.util.Mutex.readAccess(Mutex.java:230)
       at org.netbeans.modules.apisupport.project.Evaluator.reset(Evaluator.java:197)
       at org.netbeans.modules.apisupport.project.Evaluator.access$100(Evaluator.java:74)
       at org.netbeans.modules.apisupport.project.Evaluator$1.run(Evaluator.java:185)
       - locked <0x02abd1e0> (a org.netbeans.modules.apisupport.project.Evaluator)
       at org.netbeans.modules.apisupport.project.Evaluator$1.run(Evaluator.java:182)
       at org.openide.util.Mutex.readAccess(Mutex.java:230)
       at org.netbeans.modules.apisupport.project.Evaluator.delegatingEvaluator(Evaluator.java:181)
       at org.netbeans.modules.apisupport.project.Evaluator.evaluate(Evaluator.java:140)
       at org.netbeans.modules.apisupport.project.NbModuleProject.getModuleJarLocation(NbModuleProject.java:391)
       at org.netbeans.modules.apisupport.project.queries.ClassPathProviderImpl.findClassPath(ClassPathProviderImpl.jav
:90)
       at org.netbeans.modules.apisupport.project.queries.ClassPathProviderImpl.getProjectClassPaths(ClassPathProviderI
pl.java:361)
       at org.netbeans.modules.apisupport.project.NbModuleProject.open(NbModuleProject.java:675)
       at org.netbeans.modules.apisupport.project.NbModuleProject$OpenedHook.projectOpened(NbModuleProject.java:628)
       at org.netbeans.spi.project.ui.ProjectOpenedHook$1.projectOpened(ProjectOpenedHook.java:59)
       at org.netbeans.spi.project.ui.support.UILookupMergerSupport$OpenHookImpl.projectOpened(UILookupMergerSupport.ja
a:163)
       at org.netbeans.spi.project.ui.ProjectOpenedHook$1.projectOpened(ProjectOpenedHook.java:59)
       at org.netbeans.modules.project.ui.OpenProjectList.notifyOpened(OpenProjectList.java:648)
       at org.netbeans.modules.project.ui.OpenProjectList.getDefault(OpenProjectList.java:176)
       at org.netbeans.modules.project.ui.OpenProjectsTrampolineImpl.getOpenProjectsAPI(OpenProjectsTrampolineImpl.java
44)
       at org.netbeans.api.project.ui.OpenProjects.getOpenProjects(OpenProjects.java:81)
       at org.netbeans.modules.editor.bookmarks.EditorBookmarksModule.restored(EditorBookmarksModule.java:75)
       at org.netbeans.core.startup.NbInstaller.loadCode(NbInstaller.java:356)
       at org.netbeans.core.startup.NbInstaller.load(NbInstaller.java:275)
       at org.netbeans.ModuleManager.enable(ModuleManager.java:918)
       at org.netbeans.core.startup.ModuleList.installNew(ModuleList.java:380)
       at org.netbeans.core.startup.ModuleList.trigger(ModuleList.java:316)
       at org.netbeans.core.startup.ModuleSystem.restore(ModuleSystem.java:260)
       at org.netbeans.core.startup.Main.getModuleSystem(Main.java:149)
       at org.netbeans.core.startup.Main.start(Main.java:300)
       at org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:88)
       at java.lang.Thread.run(Thread.java:619)
Comment 1 Jesse Glick 2007-08-29 19:52:30 UTC
What makes you think this is a regression? Nothing about universe scanning has changed very recently.
EditorBookmarksModule.restored is asking to load open projects, which forces apisupport to do some computation to
register its classpaths. AFAIK it has been this way for a long time.
Comment 2 Jan Becicka 2007-08-29 21:24:59 UTC
It is regression against last NB release (against NB 5.5)
Comment 3 Jesse Glick 2007-08-29 22:12:19 UTC
Did editor behave differently in NB 5.5 than in NB 6? I don't see anything in apisupport's behavior that would be any
different. When someone asks to load and then open a NB project, as editor is doing here, apisupport must register its
classpaths in GPR, and this has always required finding a list of universe modules so it can determine the correct
classpath.
Comment 4 Jan Becicka 2007-08-30 06:16:52 UTC
Well, I'm not sure if this bug is regression or not. But I don't remember, that NB 5.5 startup took that long.
Comment 5 David Simonek 2007-08-30 11:22:21 UTC
jbecicka: do you mean big IDE 6.0? Than slower startup is natural...
Editor bookmark guys, can you comment? Is it possible to do things differently in EditorBookmarksModule.restored ?

While I agree this is a problem, for P2 we should prove exactly where the code is worse IMO.
Comment 6 Jan Becicka 2007-08-30 12:06:39 UTC
No I don't. I mean Basic IDE, which has less modules than NB 5.5 and long startup is not natural.

God save the IDE, where bug reporter is required to prove where exactly in code is the bug.

Startup of NB takes several minutes, without any UI indication. It is P3. OK. Great.
Comment 7 Miloslav Metelka 2007-09-11 15:14:15 UTC
I'm not aware of any change regarding bookmarks loading since 5.5. Is it really the culprit? I've looked into the
bookmarks module and we could possibly make the loading more lazy in case there is no opened editor yet. Would it help
to resolve this issue?
Comment 8 Vitezslav Stejskal 2007-09-11 16:57:45 UTC
IMO editor/bookmarks could use ProjectOpenedHook to intercept the project close operation, which it needs for saving
bookmarks in the project's files. It would be more efficient than asking for the list of opened projects and listening
on its changes and computing what projects had been closed. Feel free to reassign.
Comment 9 Jesse Glick 2007-09-13 22:39:25 UTC
You can only implement ProjectOpenedHook in the project itself. Foreign code has no access to it.
Comment 10 Jesse Glick 2007-10-10 14:41:59 UTC
*** Issue 116720 has been marked as a duplicate of this issue. ***
Comment 11 Petr Nejedly 2007-10-10 14:56:57 UTC
There's one path that we should explore, in future, regardless of resolution of this issue, and that is, split the
"universe". Most of the NetBeans developers are wasting their time updating the whole CVS and building the big IDE.

I believe we should come up with a reasonable granularity and a process (guideline) on how to work on your module while
keeping your development platform up-to-date enough. We'll be facing the granularity issue anyway if we decide to move
onto  a different version control system for NB sources.
Comment 12 Jan Becicka 2007-10-10 15:04:44 UTC
I don't understand, why this issue is P3, and not P2.
Definition of P2 according to bug priority guidelines:
# UI responsiveness of a feature breaking UI guidelines (100ms, 1s)
# Significant scalability problem
# AWT thread blocked by a task which could run on background
# No or inappropriate indication of progress of a running task 

This issue meets all 4 criteria.
Comment 13 Jesse Glick 2007-10-10 15:20:19 UTC
Only affects nb.org modules, i.e. only a handful of nb.org module developers, not general IDE users, so not a priority
relative to bugs affecting anyone.

VCS granularity != build granularity != IDE performance. If we move to e.g. Hg we would likely want a single repo for
all (stable) sources; Hg is much faster anyway. Build granularity could be changed independently of VCS, but would
require a substantial rewrite of our build system - certainly not an option for the short term. IDE performance of
project loading depends on the algorithm used to compute certain important things like the classpath; this is
independent of the VCS and only loosely related to build granularity. Project loading optimizations generally fall into
simple optimizations (delay unnecessary work, speed up basic operations) or cache creation and maintenance (with
accompanying risk of not detecting stale cache conditions). A recent build optimization based on a cache made a big
difference, though it introduced at least two regressions I know of (both now fixed).
Comment 14 Jesse Glick 2007-10-18 17:03:42 UTC
*** Issue 119004 has been marked as a duplicate of this issue. ***
Comment 15 Jesse Glick 2007-10-24 22:47:26 UTC
I don't think it's possible to avoid loading the module universe, but I did put in an optimization which uses the scan
cache created by a build if there is one and it is up to date (see issue #105324). When that cache is usable, the result
is dramatically faster than a usual universe scan. Otherwise the IDE should quietly fall back to a full scan.

Checking in nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java;
/shared/data/ccvs/repository/nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java,v  <--  ModuleListParser.java
new revision: 1.47; previous revision: 1.46
done
RCS file:
/shared/data/ccvs/repository/apisupport/project/src/org/netbeans/modules/apisupport/project/universe/NetBeansOrgCachedEntry.java,v
done
Checking in apisupport/project/src/org/netbeans/modules/apisupport/project/universe/NetBeansOrgCachedEntry.java;
/shared/data/ccvs/repository/apisupport/project/src/org/netbeans/modules/apisupport/project/universe/NetBeansOrgCachedEntry.java,v
 <--  NetBeansOrgCachedEntry.java
initial revision: 1.1
done
Checking in apisupport/project/src/org/netbeans/modules/apisupport/project/universe/ModuleList.java;
/shared/data/ccvs/repository/apisupport/project/src/org/netbeans/modules/apisupport/project/universe/ModuleList.java,v 
<--  ModuleList.java
new revision: 1.35; previous revision: 1.34
done