As elaborated at http://core.netbeans.org/proposals/db/index.html
in order to improve scalability of the system it is necessary to store
information recalculated each time the IDE is started (content of XML
filesystems, lookup, etc.) in a persistent storage.
This issue should track the progress of such effords.
Cool, but please don't assign to me until we have agreed what is to be
done and when... I use IZ's priority rank to decide what to do every
I guess before we go designing some caching system, our first goal is
to understand what precisely we are trying to avoid. Specifically,
which of the following are significant startup time and/or memory
hogs, and which are minor:
- physically opening JAR files and getting the ZIP file list
- defining packages from JARs (probably no more since JarClassLoader
is more efficient, thanks Petr)
- searching for things (e.g. classes) in a ZIP file list after it is
- loading classes needed by modules to customize "entry-point"
functionality (e.g. action enablement logic), incl. any classes they
recursively load due to compile-time references, if those classes are
resolved normally when the module is not actively in use
- loading (but not resolving) classes needed as marker interfaces or
- parsing XML layers
- merging XML layers
- creating InstanceDataObject's without the instances: e.g. parsing
*.settings format and reading the instanceOf attrs
- loading and parsing bundles
If we understand which of these things are actually expensive, we can
know what it is useful to cache. For example without support from new
APIs we might want to:
- cache merged XML layer structure as one big XML layer, or as some
- cache instanceOf info
- cache info about what resources/classes are present in which JAR
- gather commonly-loaded classes & resources and place them all in one
"primer" JAR for quick access
API extensions or redesigns would be needed for:
- do not load classes at all
Definitively we can improve the behaviour by persiting the storage of
xml-layers. The issue 20168 shows that at least by 9% at startup.
Jesse, please take care of this issue
Begun, I guess.
I guess Yarda is next in line with issue #20190.
Target milestone was changed from '3.4' to TBD.
Petr, you were thinking about have a "cache" of resources loaded
during startup. Do you think that it belongs under this issue?
Partially I think. The title of this issue is a bit confusing to me
as I was going to write "No" here originally.
I was thinking about caching some early needed module resources
(this is *not* module state) to be accessible during the startup
without touching the modlue's jars. It has nice side effect of
the IDE looking smaller because of no mmap()s.
With the better structured JarClassLoader since v1.10
it is possible to have lazy jar openning and even simpler Jar
releasing (if I/somebody implement mod:// url handler instead
of the jar:file:// urls currently used).
There is another issue discussing caching of the manifests
and not parsing zillions of the module-state.xml files which could
fall into this category.
We'd also need to cache the jar's index out of the jar
for the above scenario to work.
Now back to the original topic:
The title should probably read "IDE state", not "module state".
I've fould that with current stable IDE, it takes ~2s to parse the
module states (parse .xml, open .jar, parse manifest), but it takes
~4s to initialize the lookup, where >2.2s is spent only in parsing
tons of .settings files so issue 20190 plays major role here
unless we introduce different (no-init) scheme for looking
up the services (I do not want to push the smart naming again ;-)
See issue 21675 - Scalability in number of initialized modules