This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 209263

Summary: Pregenerate NetBeans Caches in Installer
Product: installer Reporter: Jaroslav Tulach <jtulach>
Component: CodeAssignee: Jiri Rechtacek <jrechtacek>
Status: VERIFIED FIXED    
Severity: normal CC: anebuzelsky, apireviews, issues, jglick, jrechtacek, jtulach
Priority: P1 Keywords: API_REVIEW_FAST, PLAN
Version: 7.2   
Hardware: Other   
OS: Linux   
Issue Type: TASK Exception Reporter:
Bug Depends on: 210048    
Bug Blocks:    
Attachments: Introducing the org.netbeans.core.WindowSystem.show property

Description Jaroslav Tulach 2012-03-07 10:23:18 UTC
The NetBeans caches are now empty of absolute file paths and as such they can be "shared" between multiple users. It would make sense to pregenerate them during installation. The expected speed up is about 40-50% on start immediately after installation and about 60% on start after restart (e.g. admin installs, users use later).

The way to generate the caches is to invoke NetBeans as:
-J-Dnetbeans.close=true --nosplash -J-Dorg.netbeans.core.WindowSystem.show=false


The start sequence seems to write only to user directory, except in following cases:
/tmp/jna9159852664935453925.tmp (by JNA)
$INSTALL/platform/writtableArea5007905555762727776.dll (by Equinox)
The first write is likely OK, the second should be eliminated anyway.

The org.netbeans.Stamps infrastructure needs to be modified to find existing caches in the nb cluster (nb/var/cache/all-resources.dat, etc.). It also needs to be modified to pre-extract var/cache/netigso into the cache dir (when not present).
Comment 1 Jaroslav Tulach 2012-03-07 10:24:25 UTC
Created attachment 116420 [details]
Introducing the org.netbeans.core.WindowSystem.show property
Comment 2 Jaroslav Tulach 2012-03-07 12:45:09 UTC
I have created branch http://hg.netbeans.org/core-main/rev/SharedCaches209263
Comment 3 Jaroslav Tulach 2012-03-07 14:56:57 UTC
To observe and verify the system works run with

-J-Dorg.netbeans.Stamps.level=FINE

That should be useful for QE. Btw. it might be good run with this logging on in installer as well and include the output in logs installer produces.
Comment 4 Jaroslav Tulach 2012-03-07 15:03:05 UTC
I've just verified that with revision 9eb0cc81d394 and if I run the IDE once and then copy the var directory to nb cluster, cd to it and:

$ zip -r populate.zip netigso
$ rm -r netigso
$ rm -r lastModified
$ rm splash* # if any

I can then start the IDE with empty user directory and it reuses these caches automatically. Passing to installer guys to implement the installer part.

Here is the ls netbeans/nb/var/cache that I have:
$ ls -l netbeans/nb/var/cache/
celkem 14628
-rw-r--r-- 1 jarda jarda    42549 2012-03-07 15:53 all-files.dat
-rw-r--r-- 1 jarda jarda   601499 2012-03-07 15:53 all-layers.dat
-rw-r--r-- 1 jarda jarda        0 2012-03-07 15:53 all-local-layers.dat
-rw-r--r-- 1 jarda jarda  1204523 2012-03-07 15:53 all-manifest.dat
-rw-r--r-- 1 jarda jarda   612952 2012-03-07 15:53 all-modules.dat
-rw-r--r-- 1 jarda jarda 12244997 2012-03-07 15:53 all-resources.dat
-rw-r--r-- 1 jarda jarda    45635 2012-03-07 15:53 localeVariants
-rw-r--r-- 1 jarda jarda    26851 2012-03-07 15:53 netigso-bundles
-rw-r--r-- 1 jarda jarda   183471 2012-03-07 15:53 populate.zip
Comment 5 dlipin 2012-03-07 15:22:05 UTC
Make sure installation would work in headless environment (running installer with --silent option), or, at least, not fail overall installation in that case.
Comment 6 Jesse Glick 2012-03-08 13:02:36 UTC
Typo "checkPopuplateCache"
Comment 7 Jiri Rechtacek 2012-03-08 17:34:41 UTC
I continued in implementation on installer side - core-main/rev/e8a7baf73a05
Not final yet.
Comment 8 Jiri Rechtacek 2012-03-09 19:50:54 UTC
(In reply to comment #6)
> Typo "checkPopuplateCache"
core-main/rev/a7b87e868ca5
Comment 9 Jiri Rechtacek 2012-03-09 19:53:51 UTC
The installer side is ready to integration, hand over to Jarda to complete review.
Comment 10 Jaroslav Tulach 2012-03-09 20:17:42 UTC
The review only introduces org.netbeans.core.WindowSystem.show systemproperty which seems harmless. Let's integrate on Monday.
Comment 11 Jiri Rechtacek 2012-03-12 10:00:24 UTC
(In reply to comment #10)
> The review only introduces org.netbeans.core.WindowSystem.show systemproperty
> which seems harmless. Let's integrate on Monday.

http://hg.netbeans.org/core-main/rev/71171a7258ca
Comment 12 Quality Engineering 2012-03-13 14:06:49 UTC
Integrated into 'main-golden', will be available in build *201203130400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/e8a7baf73a05
User: Jiri Rechtacek <jrechtacek@netbeans.org>
Log: #209263: cont. impl on installer side
Comment 13 jacobd 2012-03-13 22:30:44 UTC
I noticed a couple of Typo's in the changeset noted below
Changeset: http://hg.netbeans.org/main-golden/rev/e8a7baf73a05

As noted already
private static class PupolateCacheAction extends WizardAction
should probably be
private static class PopulateCacheAction extends WizardAction
Although this is not a big issue.

However, I think the following line may have an affect on functionality
File populateCacheDir = new File(nbInstallLocation, "nb"/*nb clsuter*/ + File.separator + "var" + File.separator + "cache");
              
as I think clsuter should be cluster shouldn't it?
Comment 14 Petr Cyhelsky 2012-03-14 07:54:39 UTC
> private static class PupolateCacheAction extends WizardAction
> should probably be
> private static class PopulateCacheAction extends WizardAction
> Although this is not a big issue.

It is only private class, it doesn't matter so much how it is called.

> However, I think the following line may have an affect on functionality
> File populateCacheDir = new File(nbInstallLocation, "nb"/*nb clsuter*/ +
> File.separator + "var" + File.separator + "cache");
> 
> as I think clsuter should be cluster shouldn't it?

Comments don't affect anything, only the readability of the code.
Comment 15 Jiri Rechtacek 2012-03-14 08:35:35 UTC
(In reply to comment #13)
> I noticed a couple of Typo's in the changeset noted below
> Changeset: http://hg.netbeans.org/main-golden/rev/e8a7baf73a05
> 
> As noted already
> private static class PupolateCacheAction extends WizardAction
> should probably be
It was already fixed immediately after merge:
http://hg.netbeans.org/core-main/rev/47461eebf992

> private static class PopulateCacheAction extends WizardAction
> Although this is not a big issue.
> 
> However, I think the following line may have an affect on functionality
> File populateCacheDir = new File(nbInstallLocation, "nb"/*nb clsuter*/ +
> File.separator + "var" + File.separator + "cache");
The same, was fixed before merging.

Anyway, thanks for your comments.

> as I think clsuter should be cluster shouldn't it?