Bug 179249

For tomcat:
It should be enough to create instance FO in config/J2EE/InstalledServers. This means empty tomcat_autoregistered_instance file and .nbattrs file with following fragment:

<!DOCTYPE attributes PUBLIC "-//NetBeans//DTD DefaultAttributes 1.0//EN" "http://www.netbeans.org/dtds/attributes-1_0.dtd">
<attributes version="1.0">
    <fileobject name="tomcat_autoregistered_instance">
        <attr name="admin_port" stringvalue="8025"/>
        <attr name="autoregistered" stringvalue="true"/>
        <attr name="displayName" stringvalue="Apache Tomcat 6.0.20"/>
        <attr name="httpportnumber" stringvalue="8084"/>
        <attr name="is_it_bundled_tomcat" stringvalue="true"/>
        <attr name="monitor_enabled" stringvalue="true"/>
        <attr name="registeredWithoutUI" stringvalue="false"/>
        <attr name="timestamp" stringvalue="1242256504000"/>
        <attr name="url" stringvalue="tomcat60:home=/home/sickboy/apache-tomcat-6.0.20:base=apache-tomcat-6.0.20_base"/>
    </fileobject>
</attributes>

Catalina base dir should be generated as well as it is the case in current and previous versions (.netbeans/6.8/apache-tomcat-6.0.20_base)

Sorry - not generated just configured.

I think the similar applies to GFv3? Can you describe the requirements Vince? I suppose it is config/GlassFishEE6/Instances/*

There are a couple use cases for auto-registration:

1. The user that is running the IDE owns (can write) into the GF v3 default domain directory (domains/domain1)

In that case, we need to create data in the config/GlassFishEE6/Instances directory and the config/J2EE/InstalledServers directory.

2. The user that is running the IDE does not own (cannot write) into the GF v3 default domain directory (domains/domain1).

In this case, we need to create a domain that the user CAN use and then fill the config/GlassFishEE6/Instances and config/J2EE/InstalledServers with the corresponding data.

We can specify the ports that the server will use at create time, so that multiple users/domains can be on 'the same box'.  I have had a long standing RFE to support port configuration at domain startup...  https://glassfish.dev.java.net/issues/show_bug.cgi?id=6151... but that hasn't gotten any traction, so it is difficult to create a domain that will use ports that are dynamically determined.

The support for multiple users on one box kind of requires that we support a per-user domain option....  

It also seems to require that we have to create a domain at some point after the IDE install.

(In reply to comment #5)
> There are a couple use cases for auto-registration:
> 
> 1. The user that is running the IDE owns (can write) into the GF v3 default
> domain directory (domains/domain1)
> 
> In that case, we need to create data in the config/GlassFishEE6/Instances
> directory and the config/J2EE/InstalledServers directory.
Is config/J2EE/InstalledServers really needed? I thought config/GlassFishEE6/Instances is "master" and information is propagated automatically to config/J2EE/InstalledServers if Java EE modules are enabled.

> 2. The user that is running the IDE does not own (cannot write) into the GF v3
> default domain directory (domains/domain1).
> 
> In this case, we need to create a domain that the user CAN use and then fill
> the config/GlassFishEE6/Instances and config/J2EE/InstalledServers with the
> corresponding data.
Same question ;)
 
> It also seems to require that we have to create a domain at some point after
> the IDE install.

I would suggest this to be part of the startup process. At least Tomcat's base dir is created at that time afaik.

You are right... config/GlassFishEE6/Instances is the master copy... and the j2eeserver registry info is added when the java ee features are enabled...

Hmmm.

The TC base dir is created as the IDE starts up or as TC starts up?

(In reply to comment #7)
> The TC base dir is created as the IDE starts up or as TC starts up?

Actually it is created when needed (the last point where it has to be ready is server startup) - there is a function TomcatManager.ensureCatalinaBaseReady(). Nothing is done on IDE startup.

Looks like there are two separate subissues 
- making the registration more robust by involving the installer
- avoiding loading the plugins 

Sounds like the first part should be quite easily doable on the installer side, but I am afraid the second part could be quite time consuming, so I'd say it is out of scope for NB 6.9.

"could be quite time consuming" - can we at least spend two hours to find out how time consuming it would be? Better do things correctly for the first time than patch them multiple times.

This is enhancement.

dlipin can you prepare the file for Tomcat in the installer? Vince is it doable for you to support the same mechanism in GF?

Thanks.

Can I support this change in the GF serverplugins... sure.

How will the installer write data into the userdir, since the value of the userdir cannot be calculated correctly at install time...

Here are a couple install/start-up combos that seem like they will be problematic:

1. install NB on a system used by multiple people.
2. NB started by the user with a '--userdir' argument

Installer can write the tomcat_autoregistered_instance and .nbattrs in installdir but not in userdir (unfortunately it is not clear from the comments where it is required to be created). 

Vince is correct, there are a bunch of problems with multiuser environment. That`s the main reason why we need to to store information about tomcat installdir into the NetBeans install dir, not NB userdir. The same for GlassFish.

Installer does not need to modify any userdir. Rather it shall create the file as

nb/config/J2EE/InstalledServers/tomcat_autoregistered_instance

This file will be visible to all users, with any userdir, as soon as they launcher this installation of NetBeans.

(In reply to comment #16)
> Installer does not need to modify any userdir. Rather it shall create the file
> as
> 
> nb/config/J2EE/InstalledServers/tomcat_autoregistered_instance
> 
> This file will be visible to all users, with any userdir, as soon as they
> launcher this installation of NetBeans.

And then the j2eeserver code that maintains the ServerRegistry needs to merge this 'global' data with the per-user data to preserve the API?

Plugins that use the Common Server API and have to maintain their own registry, like the v3 plugin, will also need to be extended to merge their 'per-install' and 'per-user' data into a common merged registry....

No merge is needed. Merge of layer.xml, cluster config/ and userdir/config is basic building block of NetBeans Platform. Just use FileUtil.getConfigFile(...) and you don't need to care where the data comes from.

Dmitry wrote:
> Can it be implemented in tomcat plugin as the separate class with main()
> method so that installer only need to call java with a few arguments? like
> java -cp
> platform/core/org-openide-filesystems.jar:nb/modules/org-netbeans-modules-t
> omcat5.jar org.netbeans.modules.tomcat5.DoRegisterTomcat /path/to/tomcat

You probably need to include org-openide-util.jar and org-openide-lookup.jar on the classpath too. Probably you need to pass a path to the nb cluster as additional parameter to the DoRegisterTomcat class. But yes, this can work.

I like that it keeps the registration logic tight with the server plugin's code and minimizes the interface between installer and bundle server plugins.

(In reply to comment #18)
> No merge is needed. Merge of layer.xml, cluster config/ and userdir/config is
> basic building block of NetBeans Platform. Just use FileUtil.getConfigFile(...)
> and you don't need to care where the data comes from.

OK.

So, it sounds like the installer will need to write something different to create the default instances, since it sounds like the registry will use a different persistence strategy (so it can leverage the config file 'merge' capabilities of FileUtil).

Do we know what the config file will look like?

(In reply to comment #20)
> So, it sounds like the installer will need to write something different to
> create the default instances, since it sounds like the registry will use a
> different persistence strategy (so it can leverage the config file 'merge'
> capabilities of FileUtil).
> 
> Do we know what the config file will look like?

Vince, I'm not sure I fully understand what you mean. I'll try to answer your questions:

- registry (j2eeserver) won't be changed at all
- there won't be any parameters (server related) in command line (netbeans.conf)
- installer will call my (Tomcat) or your (Glassfish) code (class with main method) which will write a FO to the cluster dir. It is enterprise/config/J2EE/InstalledServers for Tomcat. For GF I suppose it is enterprise/config/GlassFishEE6/Instances.
- the magic done by NetBeans' FS will make it visible to GF plugin the same way as it would be registered by user

I'm not sure how and whether you can handle GF domain creation somehow. I'll be able to send a prototype code for Tomcat on Monday.

(In reply to comment #21)
> [snip]
> - installer will call my (Tomcat) or your (Glassfish) code (class with main
> method) which will write a FO to the cluster dir. It is
> enterprise/config/J2EE/InstalledServers for Tomcat. For GF I suppose it is
> enterprise/config/GlassFishEE6/Instances.

What will be passed into this 'DoRegister' main class?

> - the magic done by NetBeans' FS will make it visible to GF plugin the same way
> as it would be registered by user
> 
> I'm not sure how and whether you can handle GF domain creation somehow. I'll be
> able to send a prototype code for Tomcat on Monday.

How: not sure at this point, too.
Whether: I will have to figure that out, too.

(In reply to comment #22)
> (In reply to comment #21)
> > [snip]
> > - installer will call my (Tomcat) or your (Glassfish) code (class with main
> > method) which will write a FO to the cluster dir. It is
> > enterprise/config/J2EE/InstalledServers for Tomcat. For GF I suppose it is
> > enterprise/config/GlassFishEE6/Instances.
> 
> What will be passed into this 'DoRegister' main class?

So far I need cluster dir and catalina home. You'll probably need cluster dir, GF home...

So far, my command line look like this:
java -cp platform/core/org-openide-filesystems.jar:platform/lib/org-openide-util.jar:platform/lib/org-openide-util-lookup.jar:platform/modules/org-netbeans-modules-masterfs.jar:enterprise/modules/org-netbeans-modules-j2eeapis.jar:enterprise/modules/org-netbeans-modules-j2eeserver.jar:enterprise/modules/org-netbeans-modules-tomcat5.jar org.netbeans.modules.tomcat5.registration.AutomaticRegistration /cluster/dir/installer/should/pass /catalina/home/installer/should/pass

I suppose installer can pass basically any info that is written to netbeans.conf. Am I right, dlipin?

Tomcat fixed in web-main c1e1c0a95625.

dlipin can you fix the installer?

Command is:
java -cp platform/core/core.jar:platform/lib/boot.jar:platform/lib/org-openide-modules.jar:platform/core/org-openide-filesystems.jar:platform/lib/org-openide-util.jar:platform/lib/org-openide-util-lookup.jar:enterprise/modules/org-netbeans-modules-j2eeapis.jar:enterprise/modules/org-netbeans-modules-j2eeserver.jar:enterprise/modules/org-netbeans-modules-tomcat5.jar org.netbeans.modules.tomcat5.registration.AutomaticRegistration <clusterDir> <catalinaHome>

dlipin: please give me a day to get the GF side of this ready, before you start to change the installer...

Integrated into 'main-golden', will be available in build *201002230200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/c1e1c0a95625
User: phejl@netbeans.org
Log: #179249 More robust and performant registration of GlassFish & co.

I have committed http://hg.netbeans.org/web-main/rev/ebc31e5dc5fd...

It is pretty much a copy/paste of Petr's code for Tomcat

the command is 

java -cp ./platform/core/core.jar:./platform/lib/boot.jar:./platform/lib/org-openide-modules.jar:\
       ./platform/core/org-openide-filesystems.jar:./platform/lib/org-openide-util.jar:\
       ./platform/lib/org-openide-util-lookup.jar:\
       ./enterprise/modules/org-netbeans-modules-j2eeapis.jar:\
       ./enterprise/modules/org-netbeans-modules-j2eeserver.jar:\
       ./ide/modules/org-netbeans-modules-glassfish-common.jar  \
             org.netbeans.modules.glassfish.common.registration.AutomaticRegistration \
                ./ide /export/home/vkraemer/Glassfish_v3/glassfish/

Hi Vince,
I noticed that with your latest changes methods defined by ModuleInstall class are not implemented at all. Please consider also following change.

P.

Created attachment 94455 [details]
suggested change

dlipin can you integrate the new way of the registration into the installer? Please let us know if you find any problems.

Integrated into 'main-golden', will be available in build *201002250200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ebc31e5dc5fd
User: Vince Kraemer <vkraemer@netbeans.org>
Log: #179249 More robust and performant registration of GlassFish

Working on that.

Can there be 2 more modes in AutomaticRegistration (e.g. controlled by command-line switches) which do the following:
1) removes server registration
2) gets the currently registered server location, if any

BTW, what will happen now if I do call this class twice with different GFv3 instances (location)? Would they both be registered or only the first/second one? The same question about Tomcat.


FYI. I`m also thinking about calling registration using URLCLassLoader().loadClass() and then calling main method through Reflection to get more control. Not sure that I would finally use this method ...

BTW, in case of Ruby bundle (NetBeans IDE with Ruby modules + GlassFish v3) "enterprise" cluster is not available in the distribution. 
Are org-netbeans-modules-j2eeapis.jar and org-netbeans-modules-j2eeserver.jar required to be on classpath or that is just wrong copy-past from tomcat and can be omitted?

(In reply to comment #33)
> Can there be 2 more modes in AutomaticRegistration (e.g. controlled by
> command-line switches) which do the following:
> 1) removes server registration
> 2) gets the currently registered server location, if any

It would be possible for Tomcat AFAIK. What is the usecase for this?

> BTW, what will happen now if I do call this class twice with different GFv3
> instances (location)? Would they both be registered or only the first/second
> one? The same question about Tomcat.

Tomcat:
Different instances would be registered both. Same instance would be registered only once. Subsequent calls trying to register the same instance are noops with exit status 0.

>> 1) removes server registration
>> 2) gets the currently registered server location, if any

> It would be possible for Tomcat AFAIK. What is the usecase for this?
Use case for 1: Tomcat is been uninstalled, IDE is still on disk. Integration should be removed.

Use case for 2: In the IDE uninstaller we check whether this IDE is registered with any Tomcat/GlassFish servers. If servers are registered and were installed using the installer (not by unpacking zip), we propose them for removal.

(In reply to comment #36)
> >> 1) removes server registration
> >> 2) gets the currently registered server location, if any
> 
> > It would be possible for Tomcat AFAIK. What is the usecase for this?
> Use case for 1: Tomcat is been uninstalled, IDE is still on disk. Integration
> should be removed.

Could be a bit complicated as server instance info might be spread on FS (cluster dir and user dir). I'll look into this. Jardo any suggestions/comments?
 
> Use case for 2: In the IDE uninstaller we check whether this IDE is registered
> with any Tomcat/GlassFish servers. If servers are registered and were installed
> using the installer (not by unpacking zip), we propose them for removal.

Servers registered by user and by installer are indistinguishable.
Could be done by setting some flag for the instance. So should be doable.

> Servers registered by user and by installer are indistinguishable.
> Could be done by setting some flag for the instance. So should be doable.

I would be quite happy if there is a way to get the list of *all* registered servers. I would be able to filter out those that are not installer-related by myself. Thanks.

(In reply to comment #34)
> BTW, in case of Ruby bundle (NetBeans IDE with Ruby modules + GlassFish v3)
> "enterprise" cluster is not available in the distribution. 
> Are org-netbeans-modules-j2eeapis.jar and org-netbeans-modules-j2eeserver.jar
> required to be on classpath or that is just wrong copy-past from tomcat and can
> be omitted?

Probably...  have you tried it?

not yet.

(In reply to comment #36)
> >> 1) removes server registration

all or just the installer installed?

> >> 2) gets the currently registered server location, if any
> 
> > It would be possible for Tomcat AFAIK. What is the usecase for this?
> Use case for 1: Tomcat is been uninstalled, IDE is still on disk. Integration
> should be removed.

I can see the utility of this.
> 
> Use case for 2: In the IDE uninstaller we check whether this IDE is registered
> with any Tomcat/GlassFish servers. If servers are registered and were installed
> using the installer (not by unpacking zip), we propose them for removal.

You already have this feature in the installer... why will the current strategy for this become unusable?

> all or just the installer installed?
Not all, and not installed via installer. I`d say we need a way to remove integration for specific server location regardless the way it is registered (manually/by installer/etc):
publiс void removeIntegration(File serverLocation) {...}



>> Use case for 2: In the IDE uninstaller we check whether this IDE is registered with any Tomcat/GlassFish servers. If servers are registered and were installed using the installer (not by unpacking zip), we propose them for removal.

> You already have this feature in the installer... why will the current strategy for this become unusable?

That is not exactly true. As of now, in installation database we don`t store the integration pair info(server<->ide), but we know which servers/IDEs are installed and which not. In other words, there is no way now in installer to check whether server X was integrated in the IDE without checking the IDE files hierarchy/contents.

In case of using system property (org.glassfish.v3ee6.installRoot) in netbeans.conf, it was enough to read netbeans_default_options in etc/netbeans.conf. That gave us the location of the server which is integrated in the IDE. It is very desired to have the same functionality in AutomaticRegistration.

Certainly, I can read ide/config/GlassFishEE6/Instances/.nbattrs and check for attributes/fileobject*/homefolder but I`d like to keep implementation details and exact path to .nbattrs hidden in AutomaticRegistration.

Maybe have main() handle specific cases like:
--register <cluster> <server-path>
--unregister <cluster> <server-path>
--list <cluster>

Re. list of servers - I think that all user defined servers end in $HOME/.netbeans/x.y/config/... as such they cannot be manipulated or even listed by installer. 

You can for sure design some contract to list whether tomcat or glassfish has already been installed in nb/config/... directory, but this is guaranteed to contain only installer registered servers (users don't have write access to this cluster from the IDE). As for the CLI contract I like Dmitry's parameter format.

Integrated into 'main-golden', will be available in build *201002260200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ebc31e5dc5fd
User: Vince Kraemer <vkraemer@netbeans.org>
Log: #179249 More robust and performant registration of GlassFish

> --register <cluster> <server-path>
> --unregister <cluster> <server-path>
> --list <cluster>

or, for clarity, --add, --remove, --list with the same parameters.

Installer part:
http://hg.netbeans.org/main-silver/rev/57232b4ba238

Integrated into 'main-golden', will be available in build *201003020200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/57232b4ba238
User: Dmitry Lipin <dlipin@netbeans.org>
Log: Issue #179249 More robust and performant registration of GlassFish & co.

Integrated into 'main-golden', will be available in build *201003120200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/09c735afd4a1
User: vince kraemer <vkraemer@netbeans.org>
Log: some code cleanup related to #179249 and suggested changes from Petr Hejl.

Add, remove, list for tomcat in web-main 6a1e67a98d54.

Dlipin, please fix the installer contract.

There one possible issue with the code in following scenario (I don't know if installer allow that):
1) tomcat instance registered by installer
2) user removed such instance from the ide
3) user removed tomcat installation via installer
4) user added tomcat installation via installer
5) tomcat instance is not visible (there is still info in userdir that such instance was removed)

Fixed using "--add" for Tomcat registration in main #9944938e462e.

Integrated into 'main-golden', will be available in build *201003160201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/6a1e67a98d54
User: Petr Hejl <phejl@netbeans.org>
Log: #179249: More robust and performant registration of GlassFish & co.