This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Currently at startup: 1. Everything in Modules/*.xml must be parsed using a plain SAX parser and the information loaded. 2. All JARs opened to check their manifests. 3. Dependencies etc. calculated. Step #3 is probably fast enough as it is (TBD). #1 could easily be improved: 99% of the time, the XML files are in a well-known byte pattern and could be "parsed" without using an XML parser - fall back to using a real parser if they were e.g. hand-edited. #2 is more complicated - would require keeping some kind of cache of module manifests separate from the JARs themselves. Issue #21675's proposal for a general cache of module JAR contents would be one solution, but there might be simpler ways.
Copied from #21675 (thanks for info Petr!): ---%<--- I've measured starting with 1000 dumb modules (empty layer, no ModuleInstall, no manifest sections). The additional time spent in startup (~11s) was caused by: - 8s ModuleList.readInitial - ~4s spent in parsing separate Modules/modname.xml files - ~4s spent in opening the jars needed for reading manifests. - 3s ModuleManager.enable/module preparation - probably mostly spent in creating the classloaders/reopening the module jars ---%<--- Will need to additionally check whether inter-module dependencies consume much time (i.e. finding dependencies, calculating install order, etc.) if there are a lot of dependencies or there are deeply nested dependencies. I know the current partial ordering algorithms are extremely dumb. I finally have copies of proper algorithm books at my house so I can improve them. :-)
Updates: My measurements roughly agree with Petr's. I get an initial 7ms per module, more or less, to parse Modules/, open JARs to check their manifests, and so construct the initial module list. Can take the time down to around 4ms by using a hand-optimized "parser" and falling back to the XML parser only if that fails. I also see around 3ms overhead for computing dependencies, bringing up the classloader (meaning opening the JAR a second time), and skimming over nonexistent module installers and sections. Under 5k per module heap overhead. Parsing XML layers is more - assuming caching is on, adding a mid-sized layer to a module adds about 5ms to startup. It also adds a bunch of memory: say 10-15k heap. Opening a medium-sized JAR rather than a small one adds a little overhead; maybe a millisecond or two, plus say 1-2k heap.
Committed improved parser: committed Up-To-Date 1.35 core/src/org/netbeans/core/modules/ModuleList.java
I tried caching manifests. Hardly seemed to help - 95ms improvement, less than one standard deviation. Will investigate with OptimizeIt! to see what is going on.
Created attachment 7388 [details] Patch being tested
I did some more precise testing using ModuleInitTest. It seems the improvement is real, but not dramatic. If you have module JARs with some interdependencies containing about 200 files apiece, and empty layers, the per-module startup time overhead is about 6.86ms without caching, vs. 6.32ms with caching. So, a half-millisecond speedup per module.
committed * Up-To-Date 1.37 core/src/org/netbeans/core/modules/Module.java committed * Up-To-Date 1.6 core/src/org/netbeans/core/modules/ModuleInstaller.java committed * Up-To-Date 1.42 core/src/org/netbeans/core/modules/ModuleManager.java committed * Up-To-Date 1.47 core/src/org/netbeans/core/modules/NbInstaller.java
Probably the second jar opening was faster and it is slower now (because it became first-time opening).
"Probably the second jar opening was faster and it is slower now (because it became first-time opening)." - yes, probably so. May still be helpful for an installation with a large number of disabled modules, though.