Bug 179249 - More robust and performant registration of GlassFish & co.
More robust and performant registration of GlassFish & co.
Status: STARTED
Product: serverplugins
Classification: Unclassified
Component: Infrastructure
6.x
Other Linux
: P2 (vote)
: TBD
Assigned To: dlipin
issues@serverplugins
i
:
Depends on: 181368
Blocks:
  Show dependency treegraph
 
Reported: 2010-01-06 05:39 UTC by Jaroslav Tulach
Modified: 2010-03-16 05:17 UTC (History)
4 users (show)

See Also:
Issue Type: ENHANCEMENT
:


Attachments
suggested change (3.52 KB, patch)
2010-02-24 01:05 UTC, Petr Hejl
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jaroslav Tulach 2010-01-06 05:39:39 UTC
We need effective way for registering servers from installer, that will not trigger any code of (not installed) server plugin until a server of its type is used.

I've created a wiki page to discuss what is needed and possible:
http://wiki.netbeans.org/FitnessForServers
Comment 1 Petr Hejl 2010-01-27 07:12:39 UTC
For tomcat:
It should be enough to create instance FO in config/J2EE/InstalledServers. This means empty tomcat_autoregistered_instance file and .nbattrs file with following fragment:

<!DOCTYPE attributes PUBLIC "-//NetBeans//DTD DefaultAttributes 1.0//EN" "http://www.netbeans.org/dtds/attributes-1_0.dtd">
<attributes version="1.0">
    <fileobject name="tomcat_autoregistered_instance">
        <attr name="admin_port" stringvalue="8025"/>
        <attr name="autoregistered" stringvalue="true"/>
        <attr name="displayName" stringvalue="Apache Tomcat 6.0.20"/>
        <attr name="httpportnumber" stringvalue="8084"/>
        <attr name="is_it_bundled_tomcat" stringvalue="true"/>
        <attr name="monitor_enabled" stringvalue="true"/>
        <attr name="registeredWithoutUI" stringvalue="false"/>
        <attr name="timestamp" stringvalue="1242256504000"/>
        <attr name="url" stringvalue="tomcat60:home=/home/sickboy/apache-tomcat-6.0.20:base=apache-tomcat-6.0.20_base"/>
    </fileobject>
</attributes>
Comment 2 Petr Hejl 2010-01-27 07:18:30 UTC
Catalina base dir should be generated as well as it is the case in current and previous versions (.netbeans/6.8/apache-tomcat-6.0.20_base)
Comment 3 Petr Hejl 2010-01-27 07:25:38 UTC
Sorry - not generated just configured.
Comment 4 Petr Hejl 2010-01-27 07:34:45 UTC
I think the similar applies to GFv3? Can you describe the requirements Vince? I suppose it is config/GlassFishEE6/Instances/*
Comment 5 Vince Kraemer 2010-01-27 16:22:36 UTC
There are a couple use cases for auto-registration:

1. The user that is running the IDE owns (can write) into the GF v3 default domain directory (domains/domain1)

In that case, we need to create data in the config/GlassFishEE6/Instances directory and the config/J2EE/InstalledServers directory.

2. The user that is running the IDE does not own (cannot write) into the GF v3 default domain directory (domains/domain1).

In this case, we need to create a domain that the user CAN use and then fill the config/GlassFishEE6/Instances and config/J2EE/InstalledServers with the corresponding data.

We can specify the ports that the server will use at create time, so that multiple users/domains can be on 'the same box'.  I have had a long standing RFE to support port configuration at domain startup...  https://glassfish.dev.java.net/issues/show_bug.cgi?id=6151... but that hasn't gotten any traction, so it is difficult to create a domain that will use ports that are dynamically determined.

The support for multiple users on one box kind of requires that we support a per-user domain option....  

It also seems to require that we have to create a domain at some point after the IDE install.
Comment 6 Petr Hejl 2010-01-28 08:11:33 UTC
(In reply to comment #5)
> There are a couple use cases for auto-registration:
> 
> 1. The user that is running the IDE owns (can write) into the GF v3 default
> domain directory (domains/domain1)
> 
> In that case, we need to create data in the config/GlassFishEE6/Instances
> directory and the config/J2EE/InstalledServers directory.
Is config/J2EE/InstalledServers really needed? I thought config/GlassFishEE6/Instances is "master" and information is propagated automatically to config/J2EE/InstalledServers if Java EE modules are enabled.

> 2. The user that is running the IDE does not own (cannot write) into the GF v3
> default domain directory (domains/domain1).
> 
> In this case, we need to create a domain that the user CAN use and then fill
> the config/GlassFishEE6/Instances and config/J2EE/InstalledServers with the
> corresponding data.
Same question ;)
 
> It also seems to require that we have to create a domain at some point after
> the IDE install.

I would suggest this to be part of the startup process. At least Tomcat's base dir is created at that time afaik.
Comment 7 Vince Kraemer 2010-01-28 08:43:15 UTC
You are right... config/GlassFishEE6/Instances is the master copy... and the j2eeserver registry info is added when the java ee features are enabled...

Hmmm.

The TC base dir is created as the IDE starts up or as TC starts up?
Comment 8 Petr Hejl 2010-01-29 06:09:25 UTC
(In reply to comment #7)
> The TC base dir is created as the IDE starts up or as TC starts up?

Actually it is created when needed (the last point where it has to be ready is server startup) - there is a function TomcatManager.ensureCatalinaBaseReady(). Nothing is done on IDE startup.
Comment 9 Petr Jiricka 2010-02-01 06:49:22 UTC
Looks like there are two separate subissues 
- making the registration more robust by involving the installer
- avoiding loading the plugins 

Sounds like the first part should be quite easily doable on the installer side, but I am afraid the second part could be quite time consuming, so I'd say it is out of scope for NB 6.9.
Comment 10 Jaroslav Tulach 2010-02-03 11:12:21 UTC
"could be quite time consuming" - can we at least spend two hours to find out how time consuming it would be? Better do things correctly for the first time than patch them multiple times.
Comment 11 Petr Hejl 2010-02-10 07:33:24 UTC
This is enhancement.
Comment 12 Petr Hejl 2010-02-15 03:22:40 UTC
dlipin can you prepare the file for Tomcat in the installer? Vince is it doable for you to support the same mechanism in GF?

Thanks.
Comment 13 Vince Kraemer 2010-02-15 08:00:17 UTC
Can I support this change in the GF serverplugins... sure.
Comment 14 Vince Kraemer 2010-02-15 08:07:41 UTC
How will the installer write data into the userdir, since the value of the userdir cannot be calculated correctly at install time...

Here are a couple install/start-up combos that seem like they will be problematic:

1. install NB on a system used by multiple people.
2. NB started by the user with a '--userdir' argument
Comment 15 dlipin 2010-02-15 08:51:18 UTC
Installer can write the tomcat_autoregistered_instance and .nbattrs in installdir but not in userdir (unfortunately it is not clear from the comments where it is required to be created). 

Vince is correct, there are a bunch of problems with multiuser environment. That`s the main reason why we need to to store information about tomcat installdir into the NetBeans install dir, not NB userdir. The same for GlassFish.
Comment 16 Jaroslav Tulach 2010-02-15 09:17:53 UTC
Installer does not need to modify any userdir. Rather it shall create the file as

nb/config/J2EE/InstalledServers/tomcat_autoregistered_instance

This file will be visible to all users, with any userdir, as soon as they launcher this installation of NetBeans.
Comment 17 Vince Kraemer 2010-02-15 10:12:44 UTC
(In reply to comment #16)
> Installer does not need to modify any userdir. Rather it shall create the file
> as
> 
> nb/config/J2EE/InstalledServers/tomcat_autoregistered_instance
> 
> This file will be visible to all users, with any userdir, as soon as they
> launcher this installation of NetBeans.

And then the j2eeserver code that maintains the ServerRegistry needs to merge this 'global' data with the per-user data to preserve the API?

Plugins that use the Common Server API and have to maintain their own registry, like the v3 plugin, will also need to be extended to merge their 'per-install' and 'per-user' data into a common merged registry....
Comment 18 Jaroslav Tulach 2010-02-16 01:55:41 UTC
No merge is needed. Merge of layer.xml, cluster config/ and userdir/config is basic building block of NetBeans Platform. Just use FileUtil.getConfigFile(...) and you don't need to care where the data comes from.
Comment 19 Jaroslav Tulach 2010-02-16 01:58:28 UTC
Dmitry wrote:
> Can it be implemented in tomcat plugin as the separate class with main()
> method so that installer only need to call java with a few arguments? like
> java -cp
> platform/core/org-openide-filesystems.jar:nb/modules/org-netbeans-modules-t
> omcat5.jar org.netbeans.modules.tomcat5.DoRegisterTomcat /path/to/tomcat

You probably need to include org-openide-util.jar and org-openide-lookup.jar on the classpath too. Probably you need to pass a path to the nb cluster as additional parameter to the DoRegisterTomcat class. But yes, this can work.

I like that it keeps the registration logic tight with the server plugin's code and minimizes the interface between installer and bundle server plugins.
Comment 20 Vince Kraemer 2010-02-16 10:40:38 UTC
(In reply to comment #18)
> No merge is needed. Merge of layer.xml, cluster config/ and userdir/config is
> basic building block of NetBeans Platform. Just use FileUtil.getConfigFile(...)
> and you don't need to care where the data comes from.

OK.

So, it sounds like the installer will need to write something different to create the default instances, since it sounds like the registry will use a different persistence strategy (so it can leverage the config file 'merge' capabilities of FileUtil).

Do we know what the config file will look like?
Comment 21 Petr Hejl 2010-02-19 09:22:53 UTC
(In reply to comment #20)
> So, it sounds like the installer will need to write something different to
> create the default instances, since it sounds like the registry will use a
> different persistence strategy (so it can leverage the config file 'merge'
> capabilities of FileUtil).
> 
> Do we know what the config file will look like?

Vince, I'm not sure I fully understand what you mean. I'll try to answer your questions:

- registry (j2eeserver) won't be changed at all
- there won't be any parameters (server related) in command line (netbeans.conf)
- installer will call my (Tomcat) or your (Glassfish) code (class with main method) which will write a FO to the cluster dir. It is enterprise/config/J2EE/InstalledServers for Tomcat. For GF I suppose it is enterprise/config/GlassFishEE6/Instances.
- the magic done by NetBeans' FS will make it visible to GF plugin the same way as it would be registered by user

I'm not sure how and whether you can handle GF domain creation somehow. I'll be able to send a prototype code for Tomcat on Monday.
Comment 22 Vince Kraemer 2010-02-19 11:45:21 UTC
(In reply to comment #21)
> [snip]
> - installer will call my (Tomcat) or your (Glassfish) code (class with main
> method) which will write a FO to the cluster dir. It is
> enterprise/config/J2EE/InstalledServers for Tomcat. For GF I suppose it is
> enterprise/config/GlassFishEE6/Instances.

What will be passed into this 'DoRegister' main class?

> - the magic done by NetBeans' FS will make it visible to GF plugin the same way
> as it would be registered by user
> 
> I'm not sure how and whether you can handle GF domain creation somehow. I'll be
> able to send a prototype code for Tomcat on Monday.

How: not sure at this point, too.
Whether: I will have to figure that out, too.
Comment 23 Petr Hejl 2010-02-19 12:02:06 UTC
(In reply to comment #22)
> (In reply to comment #21)
> > [snip]
> > - installer will call my (Tomcat) or your (Glassfish) code (class with main
> > method) which will write a FO to the cluster dir. It is
> > enterprise/config/J2EE/InstalledServers for Tomcat. For GF I suppose it is
> > enterprise/config/GlassFishEE6/Instances.
> 
> What will be passed into this 'DoRegister' main class?

So far I need cluster dir and catalina home. You'll probably need cluster dir, GF home...

So far, my command line look like this:
java -cp platform/core/org-openide-filesystems.jar:platform/lib/org-openide-util.jar:platform/lib/org-openide-util-lookup.jar:platform/modules/org-netbeans-modules-masterfs.jar:enterprise/modules/org-netbeans-modules-j2eeapis.jar:enterprise/modules/org-netbeans-modules-j2eeserver.jar:enterprise/modules/org-netbeans-modules-tomcat5.jar org.netbeans.modules.tomcat5.registration.AutomaticRegistration /cluster/dir/installer/should/pass /catalina/home/installer/should/pass

I suppose installer can pass basically any info that is written to netbeans.conf. Am I right, dlipin?
Comment 24 Petr Hejl 2010-02-22 03:20:36 UTC
Tomcat fixed in web-main c1e1c0a95625.

dlipin can you fix the installer?

Command is:
java -cp platform/core/core.jar:platform/lib/boot.jar:platform/lib/org-openide-modules.jar:platform/core/org-openide-filesystems.jar:platform/lib/org-openide-util.jar:platform/lib/org-openide-util-lookup.jar:enterprise/modules/org-netbeans-modules-j2eeapis.jar:enterprise/modules/org-netbeans-modules-j2eeserver.jar:enterprise/modules/org-netbeans-modules-tomcat5.jar org.netbeans.modules.tomcat5.registration.AutomaticRegistration <clusterDir> <catalinaHome>
Comment 25 Vince Kraemer 2010-02-22 11:21:04 UTC
dlipin: please give me a day to get the GF side of this ready, before you start to change the installer...
Comment 26 Quality Engineering 2010-02-22 22:02:38 UTC
Integrated into 'main-golden', will be available in build *201002230200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/c1e1c0a95625
User: phejl@netbeans.org
Log: #179249 More robust and performant registration of GlassFish & co.
Comment 27 Vince Kraemer 2010-02-23 22:42:52 UTC
I have committed http://hg.netbeans.org/web-main/rev/ebc31e5dc5fd...

It is pretty much a copy/paste of Petr's code for Tomcat

the command is 

java -cp ./platform/core/core.jar:./platform/lib/boot.jar:./platform/lib/org-openide-modules.jar:\
       ./platform/core/org-openide-filesystems.jar:./platform/lib/org-openide-util.jar:\
       ./platform/lib/org-openide-util-lookup.jar:\
       ./enterprise/modules/org-netbeans-modules-j2eeapis.jar:\
       ./enterprise/modules/org-netbeans-modules-j2eeserver.jar:\
       ./ide/modules/org-netbeans-modules-glassfish-common.jar  \
             org.netbeans.modules.glassfish.common.registration.AutomaticRegistration \
                ./ide /export/home/vkraemer/Glassfish_v3/glassfish/
Comment 28 Petr Hejl 2010-02-24 01:03:53 UTC
Hi Vince,
I noticed that with your latest changes methods defined by ModuleInstall class are not implemented at all. Please consider also following change.

P.
Comment 29 Petr Hejl 2010-02-24 01:05:01 UTC
Created attachment 94455 [details]
suggested change
Comment 30 Petr Hejl 2010-02-24 01:07:25 UTC
dlipin can you integrate the new way of the registration into the installer? Please let us know if you find any problems.
Comment 31 Quality Engineering 2010-02-24 22:05:48 UTC
Integrated into 'main-golden', will be available in build *201002250200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ebc31e5dc5fd
User: Vince Kraemer <vkraemer@netbeans.org>
Log: #179249 More robust and performant registration of GlassFish
Comment 32 dlipin 2010-02-25 05:45:06 UTC
Working on that.
Comment 33 dlipin 2010-02-25 05:56:28 UTC
Can there be 2 more modes in AutomaticRegistration (e.g. controlled by command-line switches) which do the following:
1) removes server registration
2) gets the currently registered server location, if any

BTW, what will happen now if I do call this class twice with different GFv3 instances (location)? Would they both be registered or only the first/second one? The same question about Tomcat.


FYI. I`m also thinking about calling registration using URLCLassLoader().loadClass() and then calling main method through Reflection to get more control. Not sure that I would finally use this method ...
Comment 34 dlipin 2010-02-25 06:01:51 UTC
BTW, in case of Ruby bundle (NetBeans IDE with Ruby modules + GlassFish v3) "enterprise" cluster is not available in the distribution. 
Are org-netbeans-modules-j2eeapis.jar and org-netbeans-modules-j2eeserver.jar required to be on classpath or that is just wrong copy-past from tomcat and can be omitted?
Comment 35 Petr Hejl 2010-02-25 06:23:50 UTC
(In reply to comment #33)
> Can there be 2 more modes in AutomaticRegistration (e.g. controlled by
> command-line switches) which do the following:
> 1) removes server registration
> 2) gets the currently registered server location, if any

It would be possible for Tomcat AFAIK. What is the usecase for this?

> BTW, what will happen now if I do call this class twice with different GFv3
> instances (location)? Would they both be registered or only the first/second
> one? The same question about Tomcat.

Tomcat:
Different instances would be registered both. Same instance would be registered only once. Subsequent calls trying to register the same instance are noops with exit status 0.
Comment 36 dlipin 2010-02-25 06:37:13 UTC
>> 1) removes server registration
>> 2) gets the currently registered server location, if any

> It would be possible for Tomcat AFAIK. What is the usecase for this?
Use case for 1: Tomcat is been uninstalled, IDE is still on disk. Integration should be removed.

Use case for 2: In the IDE uninstaller we check whether this IDE is registered with any Tomcat/GlassFish servers. If servers are registered and were installed using the installer (not by unpacking zip), we propose them for removal.
Comment 37 Petr Hejl 2010-02-25 06:43:35 UTC
(In reply to comment #36)
> >> 1) removes server registration
> >> 2) gets the currently registered server location, if any
> 
> > It would be possible for Tomcat AFAIK. What is the usecase for this?
> Use case for 1: Tomcat is been uninstalled, IDE is still on disk. Integration
> should be removed.

Could be a bit complicated as server instance info might be spread on FS (cluster dir and user dir). I'll look into this. Jardo any suggestions/comments?
 
> Use case for 2: In the IDE uninstaller we check whether this IDE is registered
> with any Tomcat/GlassFish servers. If servers are registered and were installed
> using the installer (not by unpacking zip), we propose them for removal.

Servers registered by user and by installer are indistinguishable.
Could be done by setting some flag for the instance. So should be doable.
Comment 38 dlipin 2010-02-25 06:59:59 UTC
> Servers registered by user and by installer are indistinguishable.
> Could be done by setting some flag for the instance. So should be doable.

I would be quite happy if there is a way to get the list of *all* registered servers. I would be able to filter out those that are not installer-related by myself. Thanks.
Comment 39 Vince Kraemer 2010-02-25 07:16:16 UTC
(In reply to comment #34)
> BTW, in case of Ruby bundle (NetBeans IDE with Ruby modules + GlassFish v3)
> "enterprise" cluster is not available in the distribution. 
> Are org-netbeans-modules-j2eeapis.jar and org-netbeans-modules-j2eeserver.jar
> required to be on classpath or that is just wrong copy-past from tomcat and can
> be omitted?

Probably...  have you tried it?
Comment 40 dlipin 2010-02-25 07:25:10 UTC
not yet.
Comment 41 Vince Kraemer 2010-02-25 07:50:36 UTC
(In reply to comment #36)
> >> 1) removes server registration

all or just the installer installed?

> >> 2) gets the currently registered server location, if any
> 
> > It would be possible for Tomcat AFAIK. What is the usecase for this?
> Use case for 1: Tomcat is been uninstalled, IDE is still on disk. Integration
> should be removed.

I can see the utility of this.
> 
> Use case for 2: In the IDE uninstaller we check whether this IDE is registered
> with any Tomcat/GlassFish servers. If servers are registered and were installed
> using the installer (not by unpacking zip), we propose them for removal.

You already have this feature in the installer... why will the current strategy for this become unusable?
Comment 42 dlipin 2010-02-25 08:11:30 UTC
> all or just the installer installed?
Not all, and not installed via installer. I`d say we need a way to remove integration for specific server location regardless the way it is registered (manually/by installer/etc):
publiс void removeIntegration(File serverLocation) {...}



>> Use case for 2: In the IDE uninstaller we check whether this IDE is registered with any Tomcat/GlassFish servers. If servers are registered and were installed using the installer (not by unpacking zip), we propose them for removal.

> You already have this feature in the installer... why will the current strategy for this become unusable?

That is not exactly true. As of now, in installation database we don`t store the integration pair info(server<->ide), but we know which servers/IDEs are installed and which not. In other words, there is no way now in installer to check whether server X was integrated in the IDE without checking the IDE files hierarchy/contents.

In case of using system property (org.glassfish.v3ee6.installRoot) in netbeans.conf, it was enough to read netbeans_default_options in etc/netbeans.conf. That gave us the location of the server which is integrated in the IDE. It is very desired to have the same functionality in AutomaticRegistration.
Comment 43 dlipin 2010-02-25 08:18:53 UTC
Certainly, I can read ide/config/GlassFishEE6/Instances/.nbattrs and check for attributes/fileobject*/homefolder but I`d like to keep implementation details and exact path to .nbattrs hidden in AutomaticRegistration.

Maybe have main() handle specific cases like:
--register <cluster> <server-path>
--unregister <cluster> <server-path>
--list <cluster>
Comment 44 Jaroslav Tulach 2010-02-25 10:17:57 UTC
Re. list of servers - I think that all user defined servers end in $HOME/.netbeans/x.y/config/... as such they cannot be manipulated or even listed by installer. 

You can for sure design some contract to list whether tomcat or glassfish has already been installed in nb/config/... directory, but this is guaranteed to contain only installer registered servers (users don't have write access to this cluster from the IDE). As for the CLI contract I like Dmitry's parameter format.
Comment 45 Quality Engineering 2010-02-25 22:13:19 UTC
Integrated into 'main-golden', will be available in build *201002260200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ebc31e5dc5fd
User: Vince Kraemer <vkraemer@netbeans.org>
Log: #179249 More robust and performant registration of GlassFish
Comment 46 dlipin 2010-02-26 06:53:34 UTC
> --register <cluster> <server-path>
> --unregister <cluster> <server-path>
> --list <cluster>

or, for clarity, --add, --remove, --list with the same parameters.
Comment 47 dlipin 2010-03-01 13:17:39 UTC
Installer part:
http://hg.netbeans.org/main-silver/rev/57232b4ba238
Comment 48 Quality Engineering 2010-03-01 22:11:44 UTC
Integrated into 'main-golden', will be available in build *201003020200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/57232b4ba238
User: Dmitry Lipin <dlipin@netbeans.org>
Log: Issue #179249 More robust and performant registration of GlassFish & co.
Comment 49 Quality Engineering 2010-03-12 04:28:04 UTC
Integrated into 'main-golden', will be available in build *201003120200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/09c735afd4a1
User: vince kraemer <vkraemer@netbeans.org>
Log: some code cleanup related to #179249 and suggested changes from Petr Hejl.
Comment 50 Petr Hejl 2010-03-15 14:12:55 UTC
Add, remove, list for tomcat in web-main 6a1e67a98d54.

Dlipin, please fix the installer contract.

There one possible issue with the code in following scenario (I don't know if installer allow that):
1) tomcat instance registered by installer
2) user removed such instance from the ide
3) user removed tomcat installation via installer
4) user added tomcat installation via installer
5) tomcat instance is not visible (there is still info in userdir that such instance was removed)
Comment 51 dlipin 2010-03-15 14:27:03 UTC
Fixed using "--add" for Tomcat registration in main #9944938e462e.
Comment 52 Quality Engineering 2010-03-16 05:17:17 UTC
Integrated into 'main-golden', will be available in build *201003160201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/6a1e67a98d54
User: Petr Hejl <phejl@netbeans.org>
Log: #179249: More robust and performant registration of GlassFish & co.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo