I am able to reproduce using build 2009-11-03_02-22-15 (installed from the installer, Java download bundle, Mac OS X
10.5.8).

Confirmed that this is ergonomics-related - after deleting the ergonomics cluster, the problem disappears. Reassigning.

GlassFish registration is flawed:
1. it runs premature registration of during startup: glassfish.common.Installer 
2. the Installer writes down(!) some files into config directory (why, can't this be done by installer?)
3. the Installer does not listen on changes in enabled modules

As a result when web project is created, nothing is written down into the config directory, until the system restarts. 
Flawed design, the registrations shall not be written down, but rather declared in the layer of some modules or 
written down by installer during installation.

Reassigning to glassfish to fix the design. Let me know if you need code and design review.

It looks like the init behavior of the gf common module was was radically changed in this commit...  http://hg.netbeans.org/main-golden/rev/2354f4681771 
This issue appears to be fallout from that.
oh well.

I just confirmed that reverting the change to ide.ergonomics/manifest.mf makes this issue go away...  oh well.

JTulach: I have some questions about your comments in http://qa.netbeans.org/issues/show_bug.cgi?id=175932#desc3

If the current registration is premature, when do you think a server plugin should register its 'default' server and not
trigger a regression of issue 173740?

It seems like you think the default server registration strategy that has been in place for many releases is flawed.  It
looks like you think the ide installer should initialize the various files that implement the server registries.  That
seems like a pretty big change at this point of the release cycle. Do you think we need new APIs to support that?

What is the API that will allow a module to listen for changes in the content of the set of activated module?

If something is flawed then it is FoD!

Yarda, either fix this on your/FoD side or suggest some simple solution for GF side. For next release we could/should
consider moving registration to NetBeans Installer but right now we need a quick fix. I do not know what GF code does
but the fact it has worked prior FoD and there were no associated performance problems makes you look like you are just
FUD-ing. And FoD-ing. :-)

I remembered a similar issue, from back in the April time frame, that I was able to find and study. It was issue 162064.

Once I found it, finding the modification necessary to fix this issue became pretty simple.

http://hg.netbeans.org/web-main/rev/69609de41168

For release 6.8 I indeed suggest to seek some easy fix. I was thinking the problem is in missing listener, but I was 
not sure where. Thanks for finding out that it was just missing folder definition in the base module.

For any future release I prefer change in the way installer and j2eeservers communicate. As far as I can tell, right 
now the installer sets a property in etc/netbeans.conf and various j2eeservers, when started, use this property and 
write something into $userdir/config/GlassFish, etc. This is flawed from performance point of view (the modules do 
some work on start, sooner then user needs to use them) as well as unnecessary complicated. I'd rather see the 
installer to create the file directly in nb6.9/config/GlassFish directory. I'll report this as separate issue when 6.8 
is over.

*** Issue 176005 has been marked as a duplicate of this issue. ***

Integrated into 'main-golden', will be available in build *200911041401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/69609de41168
User: Vince Kraemer <vkraemer@netbeans.org>
Log: #175932: this hasn't been necessary in the past...

Reproduced again in 200911171401,Win Vista,JDK 1.6.0_16,b72

any interesting or useful data in the IDE log file?

*** Bug 176534 has been marked as a duplicate of this bug. ***

cannot reproduce with 200911171401 on Mac OS X

Well,I'm not able to reproduce it anymore in that or latest build,so maybe it was caused by some old data in my userdir.Closing as fixed and will reopen if appears again.

Unfortunately still valid on:

Product Version: NetBeans IDE Dev (Build 200911191401)
Java: 1.6.0_16; Java HotSpot(TM) Client VM 14.2-b01
System: Linux version 2.6.31-14-generic running on i386; UTF-8; en_US (nb)

Reopening.
Use the steps from bug 176534:
"...
-start IDE with clean userdir
-open a java application (activates java se)
-open a java web application (activates java ee)
And there it is, the glassfish server is not available in the projects.
..."

yup. the steps described in that issue make it easy to reproduce this issue...

Created attachment 91443 [details]
initial proposed patch

This patch resolves the InstanceCreationException that users run into if they follow the work-flow described in https://netbeans.org/bugzilla/show_bug.cgi?id=175932#c16.

What is going on prior to the patch integration....

Here is what is happening after each user action...

1. Start IDE
   the usual stuff.  It is a clean userdir. none of the modules (other than Base IDE) are initialized.

2. create a Java SE app or library
   The modules that implement the Java SE features initialize.  
i. The J2eeserver module is also initialized at this time. I am not exactly sure 'why' it initializes.This includes the ServerRegistry.  There are no modules that provide plugins, so the registry is empty... 0 handlers.

ii. The glassfish.common module is also initialized at this time.  That module initializes a lookup listener for the path 'Servers/GlassFish'.  The module also registers the default server with the 'common server' registry.

3. The user attempts to create a web app project.
   The modules that implement the web and Java EE features are activated.  This includes the glassfish.javaee module.  The gf.javaee module has an entry for Servers/GlassFish in its layer file.  

i. The listener associated with the lookup Servers/GlassFish fires in gf.common code and the two modules start to work together to synchronize the 'common server' registry (has an instance of glassfish) and the j2eeserver registry (needs to have an instance of glassfish), by calling InstanceProperties.createInstancePropertiesWithoutUI.  That code is in glassfish.javaee/.../JavaEEServerModuleFactory.java (method createModule).

ii. The createInstancePropertiesWithoutUI method, checks to see that the registry is initialized... (it is).

iii. Since the registry is initialized, the method reads through all the currently available deployment factory objects that are in the registry to see if a server should be handled by a plugin, by checking the deployer URL.  Note, there are no serversplugins registered at this point.

iv. at some point AFTER the call to createInstancePropertiesWithoutUI has failed, the listener associated with path /J2EE/DeploymentPlugins fires as each of the server plugins is detected by the FileChangeListener that is associated with that path.  The list of servers grows and later calls to createInstanceProperties and createInstancePropertiesWithoutUI will be successful.


This patch resolves the issue by delaying when the initialization of server registration is considered to be done...

With initialization delayed, the DeploymentFactory objects are found, by finding the children of J2EE/DeploymentPlugins (which is populated before the file listeners that are hung on J2EE/DeploymentPlugins have had a chance to fire)

Is there another way to get around this problem... YES.

While I was writing up this summary, I thought about the question 'how does the Tomcat and SJSAS plugin avoid this issue' and I found the answer...

Those two plugins register their 'default instance' by avoiding the InstanceProperties api completely.  You can see a sample of this by looking at:

http://hg.netbeans.org/web-main/file/tip/tomcat5/src/org/netbeans/modules/tomcat5/TomcatFactory.java (see method registerServerInstanceFO)

http://hg.netbeans.org/web-main/file/tip/j2ee.sun.appsrv81/src/org/netbeans/modules/j2ee/sun/ide/j2ee/PluginProperties.java (see method registerDefaultDomain)

This avoidance tactic also appears to be used for the JBoss server, too...

http://hg.netbeans.org/web-main/file/tip/j2ee.jboss4/src/org/netbeans/modules/j2ee/jboss4/JBDeploymentFactory.java (see the method register)

So... it appears the root cause of this issue is in the ServerRegistry.

We can attempt to fix ServerRegistry now or I can use a similar hack to avoid using it (or actually avoid using InstanceProperties.createInstanceWithouUI.

My initial attempt at fixing server registry is about 9 lines changed.

The avoiding the call to createInstancePropertiesWithoutUI will require about 70 lines of code... most of which I can copy and adapt from InstanceProperties.  That code would be all 'in the plugin', but would be another hack that replicates code and is likely to fail in big ways if the ServerRegistry evolves.

did a bit more digging.

Creating an file object doesn't get me an InstanceProperties object.... and that is what we have to have/get at this point.

After being so deep in this issue, I kept asking myself...

Self, why doesn't tomcat exhibit the same behavior...  Since I did not know, I decided to try it out.

Here is what I did

1. build a very recent pull of web-main (without the patch)
2. used the following command to start the IDE

nbbuild/netbeans/bin/netbeans 
-J-Dorg.netbeans.modules.tomcat.autoregister.catalinaHome=
/Applications/NetBeans/apache-tomcat-6.0.20 
-J-Dorg.netbeans.modules.tomcat.autoregister.token=20091118082642 --userdir ~/nb68ud/tctestd


Note: tctestd was a new new userdir

3. immediately created a Java application project.  (don't do anything else)
4. waited for the activity level of my machine to drop back to its 'idle load'.  It took about 20 seconds on my machine.
5. attempted to create a Java Web App project.
  the wizard opens but when I get to page 3 (titled Server and Settings) the Server combo box is empty.

If I execute steps 1, 2 (with a different new userdir) and then 5, the Server combobox has an entry for Apache... the expected behavior.

Based on this result, I am going to transfer this issue to Serverplugins/infrastructure.

Can't reproduce. I've tried at least 10 times. Works just fine for me with tomcat & recent build (couple of minutes ago). Both with and without wait in step 4. Are you sure you have no changes locally? Any further details?

Regarding desc #19 I don't think the root cause is ServerRegistry. In steps 3i and 3iv there is now particular ordering of events in SFS, but the initialization of GF depends on such assumption.

The usage of FO for registration is not an avoidance it is just an API and creation of such FileObjects by installer is also solution jtulach would like to see in future imo.

If the ordering of your initialization is really the problem I would rather use the FO approach as 70 lines of code in serverplugin is better than 9 lines in infrastructure at this stage of development.

We can get rid of the code once installer will write such instance files...

Well.  After trying the create a FileObject directly hack, I can still replicate this issue for GF... so there must be something else I am missing.  I will keep looking.

PetrH: one quick question... do you still see an exception between step three and step 5?

No I don't see any exception for Tomcat. I'll try with GF.

I see an exception with GFv3 using steps in desc19.

I must have had some local change that was interfering with the TC initialization, because I just verified that I had no local changes and rebuilt.  I could not reproduce my earlier results.

Something like this fixes issue with GF for me.

Created attachment 91473 [details]
FO based fix

This is just rough fix. I would need to ensure myself that change in addPlugin(FO) does not break anything.

I don't know whether returning null from JavaEEServerModuleFactory.createModule() has any significant effect. This happens during default instance initialization, perhaps initialization should be done outside of createModule(), not sure...

For steps in desc19 I don't see any exception and GF is available.

I have been experimenting with something similar to the fix you have proposed....

I think returning null from createModule is probably a bad thing.  I am just about to finish my check on that.

Created attachment 91487 [details]
yet another patch

I think this could be a correct fix.

*** Bug 176044 has been marked as a duplicate of this bug. ***

Fixed in web-main 90a4a1a6f396. Please verify.

Created attachment 91530 [details]
stacktrace

This code is required as well: web-main 4436d56d3233

Working on a fix - there is still missing case :/

http://hg.netbeans.org/web-main/rev/90a4a1a6f396

it looks like you will put a server into the j2eeserver registry, even if the domain did not get created successfully.
  b/glassfish.common/src/org/netbeans/modules/glassfish/common/GlassfishInstanceProvider.java

copy/paste errors -- "Cannot register the default Tomcat server"

I am still uncomfortable with the use of the FO strategy... especially when the reference to J2EE/InstalledServers is getting added to code that will execute in the Ruby IDE.

I have also noticed that the ServerRegistry usually has an invariant that there isn't an InstalledServer that doesn't have a corresponding DeploymentPlugin...
 see ServerRegistry.addInstanceImpl

This batch of patches seems to violate that constraint... though I have to admit that I do not have a patch that satisfies the following constraints:

1. only changes 'glassfish.*' code
2. doesn't violate the ServerRegistry invariant.

> I am still uncomfortable with the use of the FO strategy

So would you prefer to the the proper fix involving installer?

Invariant is not broken.

There is still issue with server plugin initialized later than the call to addinstanceImpl in my patch.

re: https://netbeans.org/bugzilla/show_bug.cgi?id=175932#c41

Is there a j2eeserver API that we could use to determine when a plugin has finished being added to the ServerRegistry?

I'll attach 2 patches. Both should solve plugin/instance ordering once forever.

The first is simple solution that will try to load all not-yet-loaded plugins when creating instance for which we don't have plugin available.

The second patch is more complicated and is trying to actively shorten the list of loaded plugins for such situation.

Both patches reverts my previous changes. In both patches when the execution order is plugin->instance new code is not called at all (so the execution path remains the same).

Vince feel free to apply the patch (I would go with the first one) and test it. I'll read emails till midnight (approx.) CET.

Created attachment 91554 [details]
simple solution

Created attachment 91555 [details]
complicated solution

re : https://netbeans.org/bugzilla/show_bug.cgi?id=175932#c43

just saw these... will let you know before 10:00 pm CET...

simple solution was not effective in the 

start, create java app, create java web app case.

I ended up with a deep stack that had the v1 initialization in it... over and over again...

Petr: what are the specific issues that you have with the patch I proposed a couple days ago?  https://netbeans.org/bugzilla/attachment.cgi?id=91443

Can you share that stacktrace? I tried start-java-web and start-web test cases and both worked ok with gfv3.

I was bit scared of initialization anytime somebody touches the registry. There is also a chance race condition again in case another plugin will jump in during the whole FoD stuff. But we can keep it as backup plan, although I would be more happy with code executing only for the broken case.

I had been trying some different code after I added the comment.

I am doing a clean rebuild to make sure there isn't cruft from those experiments

back in about 20 minutes

here is the status...

I did the start,web-app case against a clean userdir.  It appeared to be fine.

I then tried to do the start,java app, web app case against a different, clean userdir.

I get to the 'Activating Java Web and EE' page of the web app wizard and then the pulse starts but does not stop.  I should have said thread dump in my previous message.

After a few minutes, I trigger a thread dump, which I will attach.

Created attachment 91559 [details]
the thread dump with simple solution

The command-line that I am using...

nbbuild/netbeans/bin/netbeans -J-Dorg.glassfish.v3ee6.installRoot=/export/home/vkraemer/glassfishv3_b72 --userdir /export/home/vkraemer/nb68ud/nov23i -J-Dcom.sun.aas.installRoot=/export/home/vkraemer/glassfishv2.1.1

Look like GFv2 plugin is writing instance in plugin init which in turn is trying to instantiate plugin etc.

I think it could be fixed by changing
addInstanceImpl(url, username, password, displayName, withoutUIFlag, null, true);
to
addInstanceImpl(url, username, password, displayName, withoutUIFlag, null, false);
in
ServerRegistry.addInstance(FO)

I'm thinking about that. There are two methods how to register instance
1) writing FO to SFS
2) by calling InstanceProperties.create()

The first one is in fact asynchronous and depends on the time where the file gets noticed. The second is synchronous and instance is created directly. I think it is enough when only the second case will try to load the missing plugins.

Can you test/confirm that? The change in patch is true->false in mentioned method.

I'm thinking about the case where somebody would call InstanceProperties.create from Server plugin initialization, but that would always cause InstanceCreationException even without the proposed change.

I was about to send you this message, but had hit a mid-air collision...

I made the following change to the 'simple solution' and cannot reproduce the situation that I hit with the thread dump...

in public void addInstance(FileObject fo), I changed 

addInstanceImpl(url, username, password, displayName, withoutUIFlag, null,true);

TO

addInstanceImpl(url, username, password, displayName, withoutUIFlag, null,false);


So, I confirm your change as effective.

What should the next steps be and their order...

a. commit the 'revert' 
b. commit the new simple solution to web-main
c. get folks around here to try it from the hudson build of web-main
d. get additional testing from qa folks there tomorrow.
e. send out the integration request tomorrow, if testing on multiple platforms and systems seems to show that the change is effective...

I agree. Vince can you do a & b? I'll ask Martin Schovanek to test it tomorrow morning.

BTW With false flag in addInstance(FO) I think it is more correct and less intrusive as new code is executed only when InstanceProperties.create... is called in situation when there is no plugin (yet).

will do.

currently running some tests from the TC, JBoss and other plugins

commit the revert: http://hg.netbeans.org/web-main/rev/f48506f10e99
commit modified 'simple solution': http://hg.netbeans.org/web-main/rev/3866fd400349

I missed an intermediate diff in the back and forth with Petr.

I am getting it ready to integrate now.

commit the fully modified 'simple solution'... http://hg.netbeans.org/web-main/rev/114e8c2a418d

Created attachment 91564 [details]
the diff to apply to release68

I could not reproduce the issue on Solaris with web-main and jdk 6 update 16.

I could not reproduce the issue on Mac OS X with web-main and jdk 6 update 15.

Prior to applying the changes to web-main, I had been able to reproduce the missing gfv3 combo box entry and the '0 handler' exception at will on both systems where I have been doing testing and development.

windows vista, jdk 6 update 16, http://bertram.netbeans.org/hudson/job/web-main/2155/artifact/nbbuild/NetBeans-dev-web-main-2155-on-091124-full.zip

I could not reproduce the issue using the either of the test scenerios.

I also try with windowXP 64bits and was not able to reproduce the problem. Was able to select gf v3 or v2.1 from the Server and Setting Panel
Build used:
http://bertram.netbeans.org/hudson/job/web-main/2155/artifact/nbbuild/NetBeans-dev-web-main-2155-on-091124-full.zip
jdk used:
  1.6.0_16

The fix looks fine.

Verified by QE in web-main build. Please integrate into the release68 branch.

Fixed in release68 c61e26142156.

Please verify.

added some test code for this: http://hg.netbeans.org/web-main/rev/ba20d8b51128 and http://hg.netbeans.org/web-main/rev/1f07d0d21301

Integrated into 'main-golden', will be available in build *200911241400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/90a4a1a6f396
User: phejl@netbeans.org
Log: #175932 GF v3 is missing in new Web Project wizard after activation

Verified at the release68.