Jesse, can you please evaluate this issue?  How much work does it mean
for developers and who in nb-core (besides you :-( would be able to do
the impl?  QA is very interested in having this RFE done.  Thanks

Re. #1 - $userdir/lib/ext/ is not used currently and I do not plan to
use it. Choice of userdir should not affect early startup; want to
leave open the option of selecting a userdir at runtime after early
initialization. Rather you should use the -cp:a switch.

Re. #2 - does not sound difficult. Basically a startup hook. Perhaps
included in org.netbeans.Main.

Re. #3 - I imagine you can do this already, what specifically are you
asking for?

Re. #4.1 - yes this is easy and already supported using the
netbeans.systemclassloader.patches system property, for example. I
dislike your suggestion to mount test sources in the Repository and
use internal execution. This is using an inappropriate tool for the
problem and is more work than you need, as well as being potentially
unreliable. Simply create a URLClassLoader parented to
systemClassLoader and load what you want, that is much easier.

Re. #4.2 and #4.3 - you can use netbeans.patches.code.name.base.here
and -cp:a to run tests from whatever loader you desire. If they need
access to the testing libraries, then I guess the testing libraries
will need to be in the startup classpath - which means no access to
openide, note, which may mean you have to clean up XTest to not
require openide classes at runtime (I don't see why they should be
needed anyway).

So so far the only thing I see core needing to provide that it does
not already provide is #2, which should be quite trivial (a few lines
of code), plus core or xtest developers need to document which hooks
are used by XTest and maybe have some kind of stripped-down test which
just checks that all those hooks are still working.

Re #1: OK, XTest can use -cp:a for jdbc drivers and jndi providers,
but where actually these jars should an ordinary user put? Ideally, I
would like to use the same way for this in XTest, as users will use
when using JDBC or JNDI for development in IDE.

#2: OK. Where will have to be stored the class containing the static
method? In lib/? Would it be possible to store it somewhere else than
directly in netbeans.home/? 

#3: yes, XTest currently uses netbeans.mainclass system property to
supply it's own Main class to core. But this class has to be loaded by
the boot classloader, therefore it has to be copied to lib/ directory.
I wanted to ask if something like this can be done when XTest is a
module (i.e. nothing is copied to lib/ and the module is stored in
modules/). When you look at #2, I just need to perform some settings
ASAP some code of IDE is running. This code should not contain any
openide/core calls (since most of the stuff is not yet initialized).
Then when IDE is fully initialized, I will need to perform some other
actions, this time with have usage of openide/core and possible of
projects/java/... as well.  By combination of #2 and #3 I was thinking
I could divide XTest into two parts - one is IDE independent, but
needs to be loaded ASAP, the other is IDE dependent and the best place
for it is a module. But if this will cause problems, I can hapilly
stay with the current solution (keeping in mind, that ideally nothing
should be changed in/copied to ${netbeans.home}/ when running XTest
tests).

#4.1 - so if I create my own URLClassloader, which will be
reposonsible for loading tests and testing libraries (e.g. JUnit,
Jemmy, Jelly), will I see all the necessary stuff from the test
classes and testing libraries (openide/core, public API of installed
modules)? Just one more thing, what about security policy - will I be
able to load tests from somewhere else than from ${netbeans.home} or
will I need to switch off the netbeans security? Also by which
classloader the custom classloader should be loaded ? I suppose it
will have to be in lib/, right?

#4.2 in the case of accessing package private stuff from ordinary
modules, I don't see any problem. These tests just must not use
anything like Jemmy or Jelly and IMHO this is fair assumption
(functional test shoudn't never access package private stuff).

#4.3 I'm not sure if the netbeans.patches.code.name.bese property
works also for openide and core. Does it really work for core classes
loaded by org.netbeans.JarClassLoader?  How should the
netbeans.patches... property look like in this case?

Re. #1 - users would normally put these things in lib/ext/, I *think*
- not sure what the current status is. (Depends on how JRE classes
find user-supplied classes: simple Class.forName, using a specific
ClassLoader, using Thread.contextClassLoader, etc.)

Re. #2 - would have to be in classpath (lib/ext/), unless the property
contained not only a class name but also a classpath, or was given as
a JAR file with Main-Class, etc.

Re. #3 - not sure what you should do exactly, but I don't think you
need anything from core. Generic init stuff can go into the startup
hook. All else should probably be done when tests are first run?
meaning part of some test infra module, I guess.
ModuleInstall.restored() could be used if you cannot do it on demand
for some reason. If it has soft dependencies on other modules,
consider using eager modules to factor out the dependencies, or you
may prefer to use reflection on the system class loader to access any
module class.

Re. #4.1 - make a URLCL parented to the NB system class loader, then
it access any public or protected class or member in openide, core, or
any module. Re. security policy, give it full permissions (see e.g.
core/src/org/netbeans/core/modules/Module.java for an example, it is
easy). Re. "by which class loader it should be loaded", I don't think
I understand the question. The custom CL is not "loaded" by anything,
it is created by some code you supply, which might live anywhere -
hopefully in a module.

Re. #4.2 - as you like.

Re. #4.3 - currently you can use the netbeans.classpath property to
insert things into the loader used by openide & core.

Re: #1 and #2 - and here comes the original request. I would like
XTest not to change anything in ${netbeans.home} directory during
testing (well, at least when running in a standard mode), so my idea
is to copy all the required stuff rather to ${netbeans.userdir}, since
XTest normally runs netbeans IDE with a clean userdir every time IDE
is started. So do you think it would be possible to make netbeans
check for ${netbeans.userdir}/lib and ${netbeans.userdir}/lib/ext for
mentioned jars?


Re #3 - when IDE is initialized, I just need to prepare IDE for
testing process (e.g. mount necessary jars user  specifies, or in the
future open specifed project if projects are available, etc ...) and
then start the tests. Anyway I'll try the things you suggested. Just
one more question - when ModuleInstall.restored() is called, after all
modules are initialized or immediately my module is loaded? In XTest I
think I need to wait until the whole IDE is initialized before I start
the additional setup and testing process. 

#4.1 - well, I'm not a classloader expert, but the classloader class
has to be loaded by another classloader to be present in VM, right? So
I thought it matters by which classloader is loaded, because the
loading classloader might restrict access rights of the loaded
permissions (or not)? Anyway I'll try to experiment with it a bit to
get more familiar with the classloader magic :-).

Re #4.3 - OK, when the classes specified in this property are loaded?
Before openide/core stuff or after ? (I mean can it be used for
patching ?)

Re. #1: "So do you think it would be possible to make netbeans check
for ${netbeans.userdir}/lib and ${netbeans.userdir}/lib/ext for
mentioned jars?" - I don't think this is a good idea. It seems to me
that the scope of what this XTest "module" does is getting pretty
confused. Just because you install a test suite does not mean it
should dump all kinds of JDBC drivers into your classpath. The user
should have the option to create a new IDE installation just for
testing - *not* their own installation - with some set of drivers etc.
If this is unacceptable, force the test tools NBM to be installed in
netbeans.home. It is not a "real" module anyway, I don't think anyone
would care.

Re. #3 - "when ModuleInstall.restored() is called, after all modules
are initialized or immediately my module is loaded?" - when your
module is loaded. "I need to wait until the whole IDE is initialized
before I start the additional setup and testing process" - fine,
though what precisely you mean by "initialized" is the hard part.

Re. #4.1 - "the loading classloader might restrict access rights of
the loaded permissions" - I think you are getting confused a bit. The
custom class loader defines permissions for all code loaded from it,
and has full authority for this. To *make* the custom class loader you
need to run some code which will need permissions to create class
loaders, create permission collections, etc. etc. Fortunately all
module code is always given AllPermissions so it can do anything it
likes. I presume the custom class loader would be created from some
module.

Re. #4.3 & "netbeans.classpath" - "when the classes specified in this
property are loaded? Before openide/core stuff or after ?" - before,
of course. The property is consulted when creating the class loader
which will load core.jar and openide.jar. This "classpath" is
prepended, so things in it can override core/openide classes if you
need to do that.

Re. #4.3 - my mistake, you can in fact still use lib/patches/ for
patching core.jar and openide.jar. But you may prefer to use
netbeans.classpath to avoid modifying the installation.

Re. #1: OK, now when users are not supposed to copy drivers and othe
3rd party libraries to lib/ext, XTest does not have to copy anything
in ${netbeans.home} - I just need to disable this option in the
buildscripts.  The static initialization method for getting IDE PID
can be loaded from a jar specified via -cp:a setting (the jar, if I
understand it correctly, can be stored outside netbeans.home), the
rest of the XTest classes required for running tests in IDE can  be
loaded as a XTest module, which creates it's own classloader to load
tests, runs the testing process and when done it exits IDE in a nice way.

Re. #1 to explain "It seems to me that the scope of what this XTest
"module" does is getting pretty confused" - no, it's not. XTest is not
going yo be just an IDE module. XTest as a module for IDE will be just
a couple of classes reponsible for the stuff written above. But I
still need something, which prepares testing environment for test
execution (e.g. add JDBC drivers to the place where IDE expects them,
the same thing applies to JNDI providers, and more 3rd party stuff not
being distrbuted with IDE). This is also XTest, but not a module part
of it. Furthermore, in the setup part (before running tests) I need to
ensure I don't add/change anything in ${netbeans.home} for let's say
testbag1, which might negatively affect testbag2, which is so
unfortunate to being running just after testbag1. So I thought I could
solve this by copying the stuff to userdir (which is usually  created
everytime new, when IDE is started ->  each testbag has it's own
userdir). It also helps test developers to use IDE from the same
directory for test development as well as for test execution (2 IDEs
are running at the same time) - something, which I was asked for many
times.

Re. #3: by "fully initialized" I mean state, when all visual
components are properly displayed, AWT queue is empty and IDE is ready
for user's input. I'm afraid I'm not able to reliably detect such a
state from ModuleInstall.restored() method (or am I?). On the other
hand, am I able to detect such a state in IDE at all (even is
openide/core able to do that)? 

Re. #4.1 - OK, thanks for explanation, now it's clear.

Re. #4.3 - OK, will implement this using netbeans.classpath

Well, after discussion with Radko, it looks like I'm going to have
item #5:

It doesn't seem to me there exist any standardized way how to add 3rd
party libraries (JDBC, JNDI, CORBA, ...) to IDE. Previously,
everything added to lib/ext could be used as these drivers, providers,
orbs, .... This was quite easy and XTest was able to prepare testing
enviroment with these 3rd party libraries with almost no effort.
Unfortunately, it looks like this is going to change ineavitably, but
I'm afraid there does not exist any standard way (either from the user
point of view or openide API) how to add the libraries. Unfortunately
this can make XTest's test setup process very complicated, because for
every module using 3rd party libaries it would have to support the
module's own way  of adding the libraries -> something which I'm not
interested in implementing in XTest :-(. 

So what I want is that I'd like to see a public openide API, which
would be be implemented by modules like db explorer, and which could
be used by XTest to install the libraries. Do you think something like
this is possible?

One can always use nbexec -cp:a to append anything to the classpath.
This is a standard API of the launcher. I hope it satisfies your
requirements.

Adding Jirka Skrivanek to CC, because he is currently the owner of XTest.

Yarda was responding only to #5, not #1-4.

Martin Brehovsky redesigned XTest that all requested features are now
available. I think this issue can be closed.