Bug 113341 - Poor semantics of cross-cluster module overlap
Poor semantics of cross-cluster module overlap
Status: RESOLVED FIXED
Product: platform
Classification: Unclassified
Component: Module System
6.x
All All
: P4 (vote)
: 6.x
Assigned To: Jesse Glick
issues@platform
: API_REVIEW_FAST
: 113351 121252 (view as bug list)
Depends on: 36701
Blocks: 113340
  Show dependency treegraph
 
Reported: 2007-08-21 13:49 UTC by pzajac
Modified: 2010-09-22 13:24 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
Start of patch (38.01 KB, patch)
2009-12-21 16:17 UTC, Jesse Glick
Details | Diff
Near-final patch (apichanges still missing) (52.34 KB, patch)
2009-12-22 18:01 UTC, Jesse Glick
Details | Diff
System property to go back to "First module jar wins" strategy (658 bytes, patch)
2010-09-21 14:40 UTC, fvogler
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description pzajac 2007-08-21 13:49:54 UTC
NetBeans Module Projects - None of the modules providing the capability org.netbeans.api.javahelp.Help could be
installed.
   Java Project Support - None of the modules providing the capability org.netbeans.modules.project.uiapi.ActionsFactory
could be installed.
   User Utilities - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   XML Core - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   HTTP Monitor - None of the modules providing the capability org.openide.util.HttpServer$Impl could be installed.
   JSP Parser - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   JPDA Debugger UI - None of the modules providing the capability org.netbeans.api.debugger.jpda.JPDADebuggerEngineImpl
could be installed.
   JPDA Debugger UI - None of the modules providing the capability org.netbeans.spi.debugger.ui could be installed.
   CSS Support - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   Database Core - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   Ant - None of the modules providing the capability org.openide.execution.ExecutionEngine could be installed.
   Ant - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   Database Explorer - None of the modules providing the capability org.netbeans.api.javahelp.Help could be installed.
   Database Explorer - None of the modules providing the capability org.openide.windows.IOProvider could be installed.
   Task List Core Support - None of the modules providing the capability org.netbeans.api.javahelp.Help could be installed.
   XSL Support - None of the modules providing the capability org.openide.util.HttpServer$Impl could be installed.
   JPDA Debugger Ant Task - None of the modules providing the capability org.netbeans.modules.debugger.jpda.ui could be
installed.
   Java Language Model Implementation - None of the modules providing the capability org.netbeans.api.mdr.MDRManager
could be installed.
Comment 1 Jaroslav Tulach 2007-08-27 15:23:20 UTC
I guess the root problem is:
 
       Text API - The module Editor MIME Lookup was requested in version >= 1.5 but only 1.3.31 was found.
 
Comment 2 Jaroslav Tulach 2007-08-27 15:48:05 UTC
The problem seems to be that we moved mimelookup and options.api to platform cluster, while in 5.5.x these modules 
used to be in ide7 cluster. As a result the platform 6.0 modules require new versions of these modules, but the 
modules system does not see them - they are hidden by config/Modules/*xml files provided by 5.5's ide7 cluster.

I am not sure how to fix this. The fact that cluster can hide definitions in lower clusters like platform7 is 
essencial for many applications including BlueJ. So that cannot be changed. Maybe this is wontfix, if we provide some 
guidelines for QA to run 5.5 clusters on top of platform7?
Comment 3 Jaroslav Tulach 2007-08-27 16:17:38 UTC
The workaround to this problem is to delete mimelookup module from the ide7 cluster:

rm ide7/config/Modules/org-netbeans-modules-editor-mimelookup.xml 
ide7/modules/org-netbeans-modules-editor-mimelookup.jar 
ide7/update_tracking/org-netbeans-modules-editor-mimelookup.xml 
Comment 4 Jesse Glick 2007-08-27 17:55:47 UTC
So if I understand the issue correctly:

platform55 - text(1.0)
platform60 - mimelookup(1.1), text(1.1) > mimelookup(1.1)
ide55      - mimelookup(1.0), whatever > text(1.0)
ide60      - whatever > text(1.0)

where when loading ide55 + platform60, mimelookup(1.0) is loaded (just because ide cluster is preferred to platform
cluster?), so text(1.1) fails. Did I get this right?

The only fix I can think of would be for the module system to always preferentially load the newest version of modules
it finds duplicates of, rather than the first version it encounters. I am not sure what problems such a behavior change
could introduce.
Comment 5 Jaroslav Tulach 2007-08-28 09:03:58 UTC
Unimportant correction:
ide60      - whatever > text(1.1)
otherwise this is the case.

As far as the fix goes, yes, I was hoping the module system will always prefer the newer version. However this is 
easier to say that to do. Problems:
#1 InstalledFileLocator returns just the first JAR (from ideX) not the second (from platformX), it would need to be 
extended to return enumeration
#2 List of modules is read from SystemFileSystem and here the ideX's Modules/bla-bla-mimelookup.xml overrides 
Modules/bla-bla-mimelookup.xml from platformX. As a result the system may not even know that there is the platform's 
version.

I guess the issue is so complex, that I can reassign to owner of core/modules.
Comment 6 Jaroslav Tulach 2007-08-28 09:04:40 UTC
*** Issue 113351 has been marked as a duplicate of this issue. ***
Comment 7 Jesse Glick 2007-08-28 16:59:42 UTC
Downgrading to P3. I am not aware of any user-visible bug associated with this issue, i.e. it is an architectural issue;
no one that I know of is actually trying to use a new platform cluster with an old ide cluster, and if they are, there
is a known workaround. Given the risks of making a significant change like this, I do not propose to touch anything in 6.0.

Agreed that fixing InstalledFileLocator to return the file from the correct cluster is necessary. I had not thought of
this aspect of the problem. (It is not used by the module class loader, but if a module explicitly calls IFL for
associated files that could matter.)

Regarding Modules/*.xml, this is indeed a bit subtle:

1. The ide cluster can have a _hidden mask which prevents the module from appearing at all.

2. The ide cluster could have a copy of the XML file which makes a regular module be disabled.

3. Both clusters could specify the module as enabled, but the (overridden) platform version could show the newer spec
(or release) version.

I think the answer is to continue to read Modules/*.xml from the system filesystem, with normal mask & override
semantics, which would preserve #1 and #2. Regarding #3, I am actually not sure that the version info in Modules/*.xml
is used for anything (it duplicates information in the JAR); it would be odd but perhaps OK for the module system to
encounter the following situation:

platform/modules/x.jar: OIDE-M-S-V: 1.1
platform/config/Modules/x.xml: <param name="specversion">1.1</>
ide/modules/x.jar: OIDE-M-S-V: 1.0
ide/config/Modules/x.xml: <param name="specversion">1.0</>
SFS:config/Modules/x.xml: <param name="specversion">1.0</>

If the module system were changed to load platform/modules/x.jar preferentially (because it sees the newer 1.1 version
in it), then SFS:config/Modules/x.xml would show the wrong version. I think this would cause there to be written out:

userdir/config/Modules/x.xml: <param name="specversion">1.1</>

resulting in:

SFS:config/Modules/x.xml: <param name="specversion">1.1</>

which is not ideal but may be OK.
Comment 8 Jesse Glick 2007-11-06 09:47:36 UTC
*** Issue 121252 has been marked as a duplicate of this issue. ***
Comment 9 Jesse Glick 2008-11-19 00:47:09 UTC
I am unaware of any problem outside of our artificial compatibility tests.
Comment 10 Jesse Glick 2008-12-10 00:39:20 UTC
Later.
Comment 11 Quality Engineering 2009-11-02 10:58:02 UTC
NetBeans.org Migration: changing resolution from LATER to WONTFIX
Comment 12 Jesse Glick 2009-12-21 16:05:57 UTC
Looks fixable.
Comment 13 Jesse Glick 2009-12-21 16:17:55 UTC
Created attachment 92834 [details]
Start of patch
Comment 14 Jesse Glick 2009-12-22 18:01:25 UTC
Created attachment 92876 [details]
Near-final patch (apichanges still missing)
Comment 15 Jesse Glick 2009-12-22 18:04:07 UTC
Please review this patch (also for bug #36701). Patch header describes changes in detail, but briefly:

- adding IFL.locateAll
- looking up newest version of a module in any cluster
- deleting old compatibility code for ModuleInstall "upgrades" and serialization

I will plan to commit in January.
Comment 16 Jesse Glick 2010-01-06 11:15:30 UTC
core-main #1fcfef0f3d39
Comment 17 Quality Engineering 2010-01-06 23:38:36 UTC
Integrated into 'main-golden', will be available in build *201001070200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/ee38a52b0f40
User: Jesse Glick <jglick@netbeans.org>
Log: Test failure from #113341 patch.
Comment 18 fvogler 2010-09-21 14:40:41 UTC
Created attachment 102097 [details]
System property to go back to "First module jar wins" strategy
Comment 19 fvogler 2010-09-21 14:41:24 UTC
Today I try to update our rcp based applications and build environment to platform 6.9.1. In our development process we use the ability of the suite project to overlap platform modules. With this change it did not work any more. The "Pick the newest one available" strategy won't fit in our continuous integration environment in all cases.

Explanation:
Our build environment is similar to the netbeans maven project (binary based with module repositories). The modules are build using hudson and continuous integration.
Our developers modify and build existing modules locally and use the suite project as a "patch" suite against installations of our rcp based applications to test there changes. Do get this work our suite project has a small extension, which installs a list of modules (patch nbms) into the suite cluster.

A system property to switch back to "First module jar wins" strategy would be fine.
Comment 20 Quality Engineering 2010-09-22 03:09:05 UTC
Integrated into 'main-golden', will be available in build *201009220000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/77fd5b59bb90
User: Jesse Glick <jglick@netbeans.org>
Log: Bug #113341 comment #19: Conditionally disable "newest JAR wins" semantics from specialized development environments.
Comment 21 Jesse Glick 2010-09-22 13:24:58 UTC
fvogler, best to be on CC for issues you comment on, so in this case you would see comment #20.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo