Bug 197842

Created attachment 107819 [details]
Sample changes in nbbuild task to see the OSGi binaries

The reference to netbeans-mylyn-sample in the patch is taken from:

http://source.apidesign.org/hg/netbinox-mylyn-sample/

As discussed privately, the proposed solution suffers from the issue that the new cluster would have no clear meaning, and would not interact properly with cluster configs.

Assuming Netbinox were moved into some cluster (platform or ide) in its current form, then we would want particular bundles such as those constituting Mylyn to be placed in existing clusters, probably ide (to replace the current libs.bugtracking and libs.bugzilla I suppose). We probably want one source "project" (a dir with a build.xml at a minimum) for each such bundle, for smoothest interaction with the rest of the build: that dir would be listed in a cluster config like any other module, and would store the physical bundle in the external dir as usual. Then the open issues are:

1. What should the build.xml look like? It needs some pieces from projectized.xml, like config/Modules/*.xml and update_tracking/*.xml creation; but not others, like JAR creation (would just use release.external/something.jar=modules/something.jar or the equivalent). <parseprojectxml> may be unnecessary; just needs to define the CNB manually, or automatically according to Bundle-Name in the JAR. Maybe an nbbuild/templates/osgi.xml is needed to automate this (ideally could be reused in the external Ant harness too).

2. Is any project.xml (or the rest of nbproject/) needed? There is no strong requirement to open the containing dir in the IDE as a source project, but it would not hurt to able to do so. However some code in nbbuild, including ModuleListParser, does go through cluster configs and collect project.xml metadata from all encountered modules; somehow we need to get information from OSGi wrappers so that they appear in the module list with a certain amount of metadata. Simplest change might be to have a nearly empty project.xml just specifying <code-name-base>, and (in order that the actual JAR be parsed for various reasons) <class-path-extension>. ModuleListParser probably needs to populate runtimeDependencies, so that regular modules depending on the bundle could get its transitive dependencies; thus either need to explicitly configure <module-dependencies>, or have this be implied. Similarly, apisupport.project might work best if such stub project metadata were present.

3. Complementing #2, are any changes needed to code which scan the binary clusters (nbbuild/netbeans/*/modules/*.jar)? For example, <module-dependencies> currently has no support for bundles. Probably need to introduce a helper class which provides a uniform view of modules & bundles, offering CNB, display name, dependencies, etc.

Any solution to #2 will be a hack, because really these bundle wrappers are a rather different "project type" and ought to be treated as such. But in the Ant harness there is no easy way to do that, since nbproject/project.xml is consulted both during the project build and independently as part of a universe scan. (In a Maven-based system, every interaction between modules is either a generic Maven feature, or handled at the binary level as in #3; bundles would just be retrieved directly from a repository, or a simple packaging offered through a plugin could be used to add some NB metadata files to them.)

Some things working in the use_osgi_197842 branch; org.eclipse.jgit successfully separated into its own (degenerate) source project, and used as a bundle at runtime.


The major problem is that UC generation is broken:

$ ant -Dallmodules=libs.jsch,o.eclipse.jgit,libs.git,git build-nbms generate-uc-catalog
...
nbbuild/build.xml:1640: Could not parse dependency: com.jcraft.jsch;bundle-version="[0.1.37,0.2.0)"

MakeUpdateDescTest.testFakeOSGiInfoXml covers this functionality. The problem is that while org-eclipse-jgit.jar is copied unmodified into nbbuild/nbms/extra, something has to be put into nbbuild/nbms/updates.xml which would tell AU that this is an available module and that it requires org.netbeans.libs.jsch. But there are two problems with that:

1. org-eclipse-jgit.jar's manifest gives no indication that org.netbeans.libs.jsch is needed; it only mentions com.jcraft.jsch. So MUD has no way of knowing how to expose this dependency. (If org-netbeans-libs-jsch.jar happened to be included in the same UC it could be parsed for Netigso-{SymbolicName,Version}. But this trick would not work if the Git support were built by its own Hudson job and merely presupposed the IDE's regular update center included JSch.)

2. This version range is not of the standard type (based on a conventional mapping of major release version ranges) that could be translated precisely into an OIDE-M-Module-Dependencies header, even assuming the NB CNB could be reconstructed. What is needed is "org.netbeans.libs.jsch/1 > 1.18".

I do not see any clear solution to these issues. Even if we bypassed problem #1 by mandating that bundles may have only bundle dependencies, i.e. replacing our current org-netbeans-libs-jsch.jar with com.jcraft.jsch_0.1.44.v201101211721.jar, #2 would still be troublesome - we could produce the fake dependency "com.jcraft.jsch > 0.1.37" but this would correspond to an original constraint of bundle-version="[0.1.37,100)" and improperly permit a com.jcraft.jsch_0.2.0.jar to be downloaded as a dependency. (Not sure if Netigso would then forbid the chain of modules from being enabled - from tests it seems that it will enable them despite the bad dep, let Felix reports errors and refuse to start the bundles, then just swallow the errors.)


I also added a failing assertion to MakeOSGiTest, since that would need to be updated too.

Resolved to not use bug #197908 (perhaps back it out?); distribute bundle version of JSch; create a fake dep in updates.xml (too bad if it is not precise); produce ModuleAutoDeps.xml in e.g. ide.kit.

(In reply to comment #5)
> distribute bundle version of JSch

Looks like it will not be possible to use Eclipse's version, at least not the currently published one: https://bugs.eclipse.org/bugs/show_bug.cgi?id=343416#c8

(In reply to comment #6)
> Looks like it will not be possible to use Eclipse's version

Resolved by just creating a bundle with the same name (adding manifest attributes to the existing external/*.jar).

Still TBD for the Git part: fixing generate-uc-catalog; adding ModuleAutoDeps.

generate-uc-catalog now succeeds, but the resulting catalog shows bundles, whereas they probably ought to be invisible (in both Installed and Available tabs).

Also there is no license text; not clear if Bundle-License can be used for this purpose.

Additionally, com.jcraft.jzlib is not loaded, though com.jcraft.jsch is; not sure why.

(In reply to comment #9)
> the resulting catalog shows bundles,
> whereas they probably ought to be invisible (in both Installed and Available
> tabs).

Fixed in branch.

> Also there is no license text; not clear if Bundle-License can be used for this
> purpose.

Still outstanding.

> Additionally, com.jcraft.jzlib is not loaded, though com.jcraft.jsch is; not
> sure why.

Reproduced in EnabledAutoloadTest.testAutoloadBundles. Something is wrong with how bundles are presented to the regular module system.

(In reply to comment #10)
> Something is wrong with how bundles are presented to the regular module system.

I guess the issue is that NetigsoModule.getDependencies{,Array} is returning an empty list, so ModuleManager.maybeAddToEnableList ignores the dependencies of a bundle. So, when dlight.nativeexecution specifies a dep on com.jcraft.jsch, that causes it to be loaded, but the indirect dep on com.jcraft.jzlib does not cause that to be loaded.

The only observable effect is that com.jcraft.jzlib is not listed in the console as an enabled module. I presume Felix is in fact resolving it, so its classes can be used from JSch. I do not know if it is also getting started, which would be irrelevant in this example but important in case it had an activator or a plugin.xml etc.

Not sure what the proper fix is. NetigsoModule could override gD{,A} (assuming gDA were made nonfinal) to return fake dependencies translated from Require-Bundle, probably with no specification version information but just code names and perhaps major release versions.

Running diff: hg di -r 'ancestor(default,use_osgi_197842):use_osgi_197842'

(In reply to comment #11)
> (In reply to comment #10)
> > Something is wrong with how bundles are presented to the regular module system.
> 
> I guess the issue is that NetigsoModule.getDependencies{,Array} is returning an
> empty list, so ModuleManager.maybeAddToEnableList ignores the dependencies of a
> bundle. So, when dlight.nativeexecution specifies a dep on com.jcraft.jsch,
> that causes it to be loaded, but the indirect dep on com.jcraft.jzlib does not
> cause that to be loaded.
> 
> The only observable effect is that com.jcraft.jzlib is not listed in the
> console as an enabled module. I presume Felix is in fact resolving it, so its
> classes can be used from JSch. I do not know if it is also getting started,
> which would be irrelevant in this example but important in case it had an
> activator or a plugin.xml etc.
> 
> Not sure what the proper fix is. NetigsoModule could override gD{,A} (assuming
> gDA were made nonfinal) to return fake dependencies translated from
> Require-Bundle, probably with no specification version information but just
> code names and perhaps major release versions.

This is how the Netigso behaves. It starts only bundles that are to be started due to their config/Modules/*.xml or because a NetBeans module depends on them. The system does not care about the bundles transitive closure of bundle dependencies at all. This is left up to OSGi framework (which only resolves, but does not start) and thus not even printed in list of enabled modules. You can start new issue to discuss it, but this is how it was designed and so far I see no problem that would need some fix.

(In reply to comment #13)
> This is left up to OSGi framework (which only resolves, but does not start)

So that will be a problem for any indirect dependency which includes an activator.

> and thus not even printed in list of enabled modules.

This is very confusing and would be a functional regression if we really want to replace existing NB modules with bundles - you cannot see from the log file what versions of libraries you are loading! I do not think that will be acceptable.

Starting OSGi bundle != enabling NetBeans module. It just can't enabling NetBeans module is very time consuming (change layers, rebuild caches). The other IDE on top of Netbinox is starting bundles a lot during its execution. We cannot slow that down by fully initializing appropriate NetBeans peer module when a bundle is started.

True, we the Netigso can print some info about started OSGi bundles. But that is going to be different information than about the list of enabled NetBeans modules.

(In reply to comment #15)
> Netigso can print some info about started OSGi bundles.

Resolved bundles, I guess you mean - I do not think the bundle is even started.

> that
> is going to be different information than about the list of enabled NetBeans
> modules.

I think we still need an enabled Module in the ModuleManager for such bundles, even if it does not participate in resource caches etc.:

- NbEvents logging needs to mention it, for diagnosis

- Auto Update needs to know it is there

> - NbEvents logging needs to mention it, for diagnosis

done in f21946c8d7af

> - Auto Update needs to know it is there

The module is there and listed. It is just disabled.

Merged as d14326fa3207

Be it, https://dltutuapp.com/tutuapp-download/
https://showbox.run/
https://kodi.software/