Bug 27151

1. System.getProperty("netbeans.home") is unfortunately used in a lot
of places, so this will not be trivial.

2. Is this really the best solution? If you just want to share
modules, why not put them all into (e.g.) /usr/lib/netbeans, and use a
session manager to enable and disable modules as desired? Seems
simpler, especially from the perspective of AU.

I would not mind having some kind of -profile
<path-to-layer.xml:another.xml> argument in the launcher that would
make it easy to select a particular set of modules, editor bindings,
etc. at startup. Could also be switchable at runtime, perhaps. More
tricky to do the UI than the impl, I suspect.

Ad1. "netbeans.home" would, for compatibility reasons point to the
first item in "netbeans.dirs"

Ad2. the session manager needs to be shared. So the info about what to
enable/disable cannot be in user dir. Seems to me that this is could
result in conflict with session manager used by user. But on the other
hand - the user could easily change the default set of modules to suit
his needs. So maybe this is the simplest solution, if S1S and NetBeans
agrees on use of one shared directory. This also implies that disabled
modules should not affect startup at all otherwise any eddition of Nb
suffers from the amount of installed (and disabled) modules.

Re. "netbeans.home" would, for compatibility reasons point to the
first item in "netbeans.dirs" - this does not guarantee compatibility.
It means that code expecting to find everything it needs in
netbeans.home would be broken. Yes, there are such cases; I have
written at least one, sorry to tell.

Re. sharing sessions - I did not realize sessions were supposed to be
shared between users, but anyway you can keep your master module
config in your user dir, and a session might contain only certain
changes, e.g. "make sure module XXX is turned on, but I do not care
about any others".

Re. impact of disabled modules - sure, you pay a few milliseconds
each; not a whole lot but a little. I think I can reduce that by
caching manifests from JARs (cf. issue #26786).

We are grouping sets of modules into "consolidation kits" that can be
versioned and installed anywhere on the user's system.

Like the NetBeans Platform, we want these kits to be reusable in
different product configurations.

Current thinking is that a 'productized IDE' would consist of a launch
script (eg. runj2ee, runwebsvcs, runmyapp) that invokes the Platform
along with a list of directories where contributing modules reside.
The user should not have to run a session manager to select or
configure an IDE product.

The solution to install all modules into /usr/lib/netbeans is
problematic if two versions of a given kit (delivering different
versions of the same module jar file) are installed on the same
system.

We should also have the ability to maintain more than one version of
the NB Platform, but use the same contributing modules from the other
kits, without having to duplicate them in the 2nd version of the
Platform.


I understand that this may be stretching the assumptions about where
modules are installed. However, I'm not sure I understand the issue
with {netbeans.home}. My assumption is that {netbeans.home} would
continue to be the directory where the NB platform is installed. So in
the above launch script example, run2jee would simply find the NB
Platform installation and instantiate an IDE from there, adding a list
of module directories to the command line.

Having duplicated kits would indeed require multiple installation
dirs. First I had heard of this requirement. Be very careful here -
for example, a user could not safely use the same user directory for
two different versions of the same kit, since settings from the newer
one would in general be unreadable by the older one.

I can imagine Auto Update becoming confused - AU in one VM session
thinks it has modules A and B available, and happily lets you install
C which depends on them, and then in another session A is available
but B is not, so C gets disabled... the implications for AU need to be
considered. (Would it let you install NBMs into an arbitrary module dir?)

"The user should not have to run a session manager to select or
configure an IDE product." - well, if each inst dir contains its own
system/Modules/*.xml, and the user has not overridden any of these in
$userdir/system/Modules/, then this will be true. Userdir-specific
customizations to the enablement state of a module will apply to any
config on which that userdir is used, though. Or if you are running
sessions, you would probably not want to share a given session with
more than one inst dir.

I suggest that the launcher not be changed; that netbeans.home
continue to point to the platform home (i.e. core); and that
additional directories be configured using Java code during startup.
Mostly what needs to change is that
org.netbeans.core.modules.ModuleSystem needs to produce several more
autoscan folders than it currently does. One implication: modules
installed in non-platform dirs *cannot* use lib/ext/ to add things to
the classpath. This is already true for modules in netbeans.user,
which is why AU forces you to install such NBMs in "global" mode. Same
for lib/ directory generally, and bin/.

Re. compatibility issue with netbeans.home - I understand that
netbeans.home would stay the same. The point is that there are various
places in the NB code base where it is assumed that *all
normally-installed modules* are located in either netbeans.home or
netbeans.user. This would cease to be true; such code would need to be
changed in every case. I don't think it's necessarily a lot of work,
just pointing it out.

Hrebejku seems more and more that this issue needs to be addressed in
4.0 timeframe. Needs staffing. Please think about that when planning.

We definitely need this for 4.0.

Probably makes sense for me to handle this, since it primarily affects
the module system, which I am the main developer of.

I am working on this. However there are dozens of uses of
${netbeans.home} throughout the NB source base, many of which need
either to be removed somehow, or edited to understand ${netbeans.dirs}
as well.

This issue affects XTest harness and IDE testability.
We should know exact behaviour of IDE startup scripts and
${netbeans.dirs} convention before integration to assure testability.

There should be no T9Y implications (unless you want to test #27151
itself of course): by default ${netbeans.dirs} will be unset, in which
case all will behave as before. The property would not be set by
NetBeans installations at all, probably; more likely used by
installers for S1S, IFDEF, etc. Any launcher changes would be purely
convenience additions to set the sysprop, and would be additions, not
changes to the existing logic. So I am removing the T9Y keyword for
now (re-add it if you disagree of course).

So when netbeans.dirs will not be set, will there still exist property
called netbeans.home or not ?

Concering XTest, because of planned changes XTest part depending on
IDE, I think XTest can be designed even without netbeans.home or
similar system properties.

"So when netbeans.dirs will not be set, will there still exist
property called netbeans.home or not?" - yes; even if it *is* set. The
platform (core*.jar etc.) will continue to be found under
${netbeans.home}. ${netbeans.dirs} if set would refer to additional
"installation" directories with other modules. As a test developer you
would only care about this at all if you were testing a split
installation, which I don't suppose you plan to do. (Perhaps for S1S
tests in the future.)

OK, now have it working pretty well with moduleconfigs:

platform:

autoupdate,core,core/javahelp,core/output,core/settings,core/term,core/ui,core/windows,openide,openide/io

ide:

ant,core/compiler,core/deprecated,core/execution,core/ide,diff,editor,extbrowser,html,httpserver,httpserver/servletapi,image,java/srcmodel,javacvs,libs/jaxp,libs/xalan,libs/xerces,openide/compiler,openide/deprecated,openide/execution,openidex,projects,properties,text,ui/welcome,usersguide,utilities,utilities/group,vcscore,vcscvs/compat,vcsgeneric,vcsgeneric/profiles/cvsprofiles,vcsgeneric/profiles/pvcs,vcsgeneric/profiles/vss,xml/api,xml/catalog,xml/core,xml/css,xml/schema,xml/tax,xml/text-edit,xml/tools,xml/tree-edit,xml/xsl

java:

beans,classfile,clazz,debuggercore,debuggerjpda,form,i18n,jarpackager,java,java/api,javadoc,scripting

nbdevel:

apisupport,apisupport/ant,apisupport/apidocs,apisupport/lite

web:

applet,j2eeserver,jndi,rmi,schema2beans,tomcatint/bundledtomcat,tomcatint/tomcat40,web/core,web/jspparser,web/jspsyntax,web/servletapi23

I.e. platform is ${netbeans.home} and each of the rest may optionally
be added to ${netbeans.dirs}. So you can switch module configs with
the startup param, as requested.

RMI gets broken here because it has something in lib/ext/*.jar. But,
that is RMI's fault - it must be installed in ${netbeans.home}, and I
forgot this.

AU can update things to either ${netbeans.home} or ${netbeans.user}
without problems. It does not know about ${netbeans.dirs} but it knows
that the modules from them are enabled and available as dependencies.

I patched various modules to use InstalledFileLocator or other fixes;
will file those separately for individual evaluation. Until those are
fixed, these modules will not work well in split-dir mode.

Implemented at least for common modules.

committed   * Up-To-Date  1.45       
apisupport/src/org/netbeans/modules/apisupport/beanbrowser/WrapperKids.java
committed   * Up-To-Date  1.357      
core/src/org/netbeans/core/Bundle.properties
committed   * Up-To-Date  1.6        
core/src/org/netbeans/core/NbRepository.java
committed   * Up-To-Date  1.83       
core/src/org/netbeans/core/NonGui.java
committed   * Up-To-Date  1.29       
core/src/org/netbeans/core/Plain.java
committed   * Up-To-Date  1.40       
core/src/org/netbeans/core/modules/ModuleList.java
committed   * Up-To-Date  1.28       
core/src/org/netbeans/core/modules/ModuleSystem.java
committed   * Up-To-Date  1.21       
core/src/org/netbeans/core/projects/ModuleLayeredFileSystem.java
committed   * Up-To-Date  1.17       
core/src/org/netbeans/core/projects/SessionManager.java
committed   * Up-To-Date  1.34       
core/src/org/netbeans/core/projects/SystemFileSystem.java
committed   * Up-To-Date  1.11       
core/src/org/netbeans/core/ui/ProductInformationPanel.form
committed   * Up-To-Date  1.15       
core/src/org/netbeans/core/ui/ProductInformationPanel.java
committed   * Up-To-Date  1.3        
core/test/unit/src/org/netbeans/core/modules/NbInstallerTest6.java

I'm unable to get the netbeans.dirs property to work
correctly. It brings up netbeans if I give a single
directory in the property but fails if I give multiple
directories. In the multiple directory case I get the
splash screen to flash and then netbeans exits
immediately. When I look at my log file its printed
some of the startup info but nothing else. Here is a
script of what I've done and my log file:

>> rm -rf ~/.ffj/rtest
>> nbplatform=/export/pucci1/rtest/opt/netbeans
>> nbnative=/export/pucci1/rtest/opt/SUNWspro
>> ls $nbplatform/modules
ant.jar             bin                 core-ui.jar         editor.jar
         externaleditor.jar  java.jar            usersguide.jar
autoload            clazz.jar           docs                ext      
          form.jar            patches             utilities.jar
autoupdate.jar      core-ide.jar        eager              
extbrowser.jar      jarpackager.jar     text.jar            welcome.jar
>> ls $nbnative/modules
cpp.jar     dbxGUI.jar  docs        locale
>> runide.sh -J-Dnetbeans.dirs="$nbplatform:$nbnative" -userdir
~/.ffj/rtest
>> cat ~/.ffj/rtest/system/ide.log
-------------------------------------------------------------------------------
>Log Session: Friday, May 16, 2003 12:10:31 PM PDT
>System Info: 
  Product Version       = NetBeans IDE 3.5 RC2 (Build 20030516)
  IDE Versioning        = IDE/1 spec=3.42.1 impl=20030516
  Operating System      = SunOS version 5.9 running on sparc
  Java; VM; Vendor      = 1.4.1_02; Java HotSpot(TM) Client VM
1.4.1_02-b06; Sun Microsystems Inc.
  Java Home             = /usr/j2sdk1.4.1_02/jre
  System Locale; Encod. = en; ISO646-US
  Home Dir; Current Dir = /home/gordonp;
/export/pucci1/rtest/opt/netbeans/bin
  IDE Install; User Dir = /export/pucci1/rtest/opt/netbeans;
/home/gordonp/.ffj/rtest
  CLASSPATH             =
/export/pucci1/rtest/opt/netbeans/lib/ext/boot.jar:/export/pucci1/rtest/opt/netbeans/lib/ext/crimson-1.1.3.jar:/export/pucci1/rtest/opt/netbeans/lib/ext/regexp-1.2.jar:/export/pucci1/rtest/opt/netbeans/lib/ext/xerces-2.0.2.jar:/export/pucci1/rtest/opt/netbeans/lib/ext/xml-apis-1.0b2.jar:/usr/java/lib/dt.jar:/usr/java/lib/htmlconverter.jar:/usr/java/lib/tools.jar
  Boot & ext classpath  =
/usr/j2sdk1.4.1_02/jre/lib/rt.jar:/usr/j2sdk1.4.1_02/jre/lib/i18n.jar:/usr/j2sdk1.4.1_02/jre/lib/sunrsasign.jar:/usr/j2sdk1.4.1_02/jre/lib/jsse.jar:/usr/j2sdk1.4.1_02/jre/lib/jce.jar:/usr/j2sdk1.4.1_02/jre/lib/charsets.jar:/usr/j2sdk1.4.1_02/jre/classes:/usr/j2sdk1.4.1_02/jre/lib/ext/sunjce_provider.jar:/usr/j2sdk1.4.1_02/jre/lib/ext/dnsns.jar:/usr/j2sdk1.4.1_02/jre/lib/ext/localedata.jar:/usr/j2sdk1.4.1_02/jre/lib/ext/ldapsec.jar
>> 

As you can see, I've got modules in both locations.

It works. You should not include the platform in the list of dirs - it
is redundant. Sample test run in NB 3.5 sources:

cd /space/src/r35

modules1="core core/output core/settings core/term core/ui
core/windows openide openide/io"
modules2="html"

ant -f nbbuild/build.xml `for i in $modules1 $modules2; do echo
all-$i; done`

mkdir /tmp/netbeans1
for i in $modules1; do cp -r $i/netbeans/* /tmp/netbeans1; done

mkdir /tmp/netbeans2
for i in $modules2; do cp -r $i/netbeans/* /tmp/netbeans2; done

# Build process normally does this:
chmod a+x /tmp/netbeans1/bin/runide.sh 

# Every dir must have a system subdir:
mkdir /tmp/netbeans2/system

/tmp/netbeans1/bin/runide.sh -J-Dnetbeans.dirs=/tmp/netbeans2 -userdir
/tmp/testme -J-Dnetbeans.logger.console=true

I found the problem. There is a hidden dependency.
The directory $nbnative/system must exist or netbeans
will silently exit. Whats more, when I run with
-Dnetbeans.logger.console=true I get the following
error message:

    The path /export/pucci1/rtest/opt/SUNWspro/system
    in netbeans.dirs does not point to an existing
    directory.

When I run without the console logger this message
does not go into my ide.log file.

"The directory $nbnative/system must exist" - yes, did you read the
comment in my last message? $x/system/ is added to the system file
system for every $x in ${netbeans.dirs}, so they need to exist.

"or netbeans will silently exit." - it exits with an error message.

"When I run without the console logger this message does not go into
my ide.log file." - currently the log file is buffered, and flushed on
exit; but only due to a normal exit via LifecycleManager, not an
abrupt startup error such as this. If it is not flushed, the tail end
could be missing.

BTW, I committed to trunk in NbTopManager.java & TopLogging.java:

"Flush the ide.log file when the VM exits, regardless of how.
Previously, shutdowns coming very soon after startup, e.g. due to a
bad ${netbeans.dirs} arg, would leave no trace in the log file."