The problem is in test driver.
Reassigning for evaluation...

SOA depends on the J2EE/Enterprise cluster.. It will not work with only J2SE installed

Tien, the problem here is not with J2EE dependency. UC understands the dependencies and installs the required J2EE
modules properly.

The real issue here is that the installed SOA modules are located in the user dir, not in NB install dir. See #127132. 

The following from compapp project's build-impl.xml requires all the dependent modules installed under NB install dir.

   <junit showoutput="true" fork="yes" dir="${basedir}" failureproperty="tests.failed" errorproperty="tests.failed">
         <classpath>
                <pathelement path="${netbeans.home}/../soa1/modules/org-netbeans-modules-compapp-manager-jbi.jar"/>
                <pathelement path="${netbeans.home}/../soa1/modules/ext/jbi/catd.jar"/>
                <pathelement path="${netbeans.home}/../soa1/modules/ext/jbi/xmlunit-1.0.jar"/>
                <pathelement path="${netbeans.home}/../soa1/modules/ext/jbi/httpunit-1.6.jar"/>
                <pathelement path="${netbeans.home}/../java1/modules/ext/junit-3.8.2.jar"/>
                <pathelement path="${netbeans.home}/../java1/modules/ext/jaxws21/api/saaj-api.jar"/>
                <pathelement path="${netbeans.home}/../java1/modules/ext/jaxws21/saaj-impl.jar"/>
                <pathelement path="${netbeans.home}/../java1/modules/ext/jaxws21/FastInfoset.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-xml-xdm.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-xml-xam.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-editor-lib.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-xml-text.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-editor.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-editor-util.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-xml-core.jar"/>
                <pathelement path="${netbeans.home}/../ide8/modules/org-netbeans-modules-editor-lib2.jar"/>
                <pathelement path="${netbeans.home}/../platform7/lib/org-openide-modules.jar"/>
                <pathelement path="${netbeans.home}/../platform7/lib/org-openide-util.jar"/>
                <pathelement path="${netbeans.home}/../platform7/modules/org-openide-options.jar"/>
                <pathelement path="${netbeans.home}/../platform7/modules/org-openide-text.jar"/>
                <pathelement path="${netbeans.home}/../platform7/modules/org-openide-loaders.jar"/>
                <pathelement path="${netbeans.home}/../platform7/modules/org-openide-nodes.jar"/>
                <pathelement path="${netbeans.home}/../platform7/modules/org-netbeans-modules-editor-mimelookup.jar"/>
                <pathelement path="${netbeans.home}/../platform7/core/org-openide-filesystems.jar"/>
         </classpath>

         ......

    </junit>

I can replace "${netbeans.home}/../soa1/modules" by "${module.install.dir}". But there is no easy way to find out other
clusters' installation dir, say, java1 cluster. I can probably use something like "${ant.home}/.." as java1's
installation dir but I consider this a hack.

CCing some people who could know how to fix this problem in project's build-impl.xml. ie if there is any property which
covers both locations. Regarding dependency on j2ee cluster you should define correct dependency so that if any SOA
plugin is selected in Plugin dialog everything is installed. You cannot rely that j2ee cluster is installed automatically.

It looks like dependency is set correctly. All necessary jars are installed but they are in different location.

SOA modules do not require the whole j2ee cluster to be installed. 

Currently, the only missing dependency is GlassFish. (SOA modules do not have a hard dependency on the Glassfish module.
However, without the Glassfish module, SOA modules are probably not that useful.) I think I can add that dependency.

Created attachment 56901 [details]
Location related properties in ANT

CCing sustaining folks. Fix must be delivered to 6.0.1 via autoupdate using the regular patch process owned by Sustaining.

For 6.0.1, do we care about building/deploying/testing CompApp project from command line? If we do, then it's a bigger
problem. 

The following fix (mapping cluster names to cluster locations) looks really ugly to me. (see the above attached snapshot)

  platform7:   ${netbeans.home}
  soa1:        ${module.instal.dir}
  java1:       ${ant.home}/..
  ide8:        ${libs.MySQLDriver.classpath}/..
  enterprise4: ${copyfiles.classpath}/../..
  xml1:        ?

Is there a better solution?

Extra code is required in this approach, but this sounds much better than the hack I proposed in 127717.

Thanks,
Jun

Tientien Li wrote:
> Jun,
>
> I think IDE base clusters have to be insallted in netbeans.home. For optional modules like xml and enterprise, you can
use the following code to figure out they are installed, e.g., in netbeans.home or netbeans.user dir:
>
>        Class wsdlmodel = org.netbeans.modules.xml.wsdl.model.WSDLModel.class;
>        URL url = wsdlmodel .getProtectionDomain().getCodeSource().getLocation();
>
>        wsdlmodel: file:/E:/darkhorse/main/nbbuild/netbeans/ide9/modules/org-netbeans-modules-xml-wsdl-model.jar
>
> -- 
> Tientien Li
>
> Jun Qian wrote:
>> Sure, we could do that. But we still need to figure out how to get the locations of various clusters.
>>
>> Jun
>>
>> Tientien Li wrote:
>>> Jun,
>>>
>>> I look at the build-impl.xsl briefly. In addition to "netbeans.home", we can use the "module.install.dir" from
project private properties to set up the correct class path needed for ant tasks when running within NB.
>>>
>>> -- 
>>> Tientien Li 

Fixed in trunk (C/B #572). Please verify it before I integrate a similar fix into 6.0.1.

changeset:   67867:eb8aa36ec9c7
user:        Jun Qian <jqian@netbeans.org>
date:        Wed Feb 20 15:22:06 2008 -0800
summary:     #127717 No longer require soa/java/xml/enterprise/ide clusters to be installed under NB install dir.

changeset:   67865:47f18f805e49
user:        Jun Qian <jqian@netbeans.org>
date:        Wed Feb 20 09:33:20 2008 -0800
summary:     #127717 Add dependency to GlassFish.

Sorry. I do not know how to verify it without nbms. Where can I get nbms so that I can install additional plugins using
Plugin manager? The same applies to NB 6.0.1.

Or of course you can suggest another way but IMO using nbms is closest to UC.

As latest daily build is 2 days old I had to build IDE and nbms from repository. I built small IDE, installed SOA, (I
had to make install dir read only otherwise plugins are installed into install dir even if it is not explicitly set).
Now during deploy of SynchronousSampleApplication I get error:
/home/mslama/NetBeansProjects/SynchronousSample/SynchronousSampleApplication/nbproject/build-impl.xml:211: Failed
execution of Deploy: /home/m
slama/NetBeansProjects/SynchronousSample/SynchronousSampleApplication/dist/SynchronousSampleApplication.zip
ERROR: (JBIMA1408) Required component sun-bpel-engine is not started on instance server.
        at
org.netbeans.modules.compapp.projects.jbi.anttasks.DeployServiceAssembly.deployServiceAssembly(DeployServiceAssembly.java:322)
        at org.netbeans.modules.compapp.projects.jbi.anttasks.DeployServiceAssembly.execute(DeployServiceAssembly.java:294)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
        at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:105)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:357)
        at org.apache.tools.ant.Target.performTasks(Target.java:385)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1329)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1298)
        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1181)
        at org.apache.tools.ant.module.bridge.impl.BridgeImpl.run(BridgeImpl.java:277)
        at org.apache.tools.ant.module.run.TargetExecutor.run(TargetExecutor.java:460)
        at org.netbeans.core.execution.RunClassThread.run(RunClassThread.java:151)
BUILD FAILED (total time: 0 seconds)

full log is attached.

More details:
1.Build basic IDE using:
ant -Dcluster.config=basic build
2.Copy it elasewhere.
3.Build full IDE using:
ant build
4.Build nbms:
ant build-nbms
5.Build catalog:
ant -Dbuildnum=$BUILDNUM -Dbuildnumber=$BUILDNUMBER -f build.xml generate-uc-catalog -Dnbms.location=nbms
-Dcatalog.file=nbms/catalog.xml -Dcatalog.base.url="."
6.Make basic IDE dir read only. It was created in step 1.
7.Run basic IDE and install SOA modules. (In Plugin dialog Settings tab set UC as file:///path to catalog.xml eg.:
file:/home/mslama/local/netbeans/hg-nbsrc/main/nbbuild/nbms/catalog.xml).
8.Create Synchronous sample. Build SynchronousSampleApplication. Deploy it and I got that error. Did I do anything wrong?

Created attachment 56986 [details]
Error when deploying SynchronousSampleApplication

Created attachment 56987 [details]
Log from glassfish

Attached log from server when I restarted it.

I guess it's OK. The latest NetBeans SOA is not compatible with the old version of runtime. If you want to verify the
fix on NB 6.1, you need to get the latest GlassFish and OpenESB.

Alternatively, I can prepare a 6.0.1 patch and send it to you for verification before I check it into release601_fixes
branch. 

Let me know which way you want to go.

Attaching patches to release601_fixes branch...

Created attachment 57022 [details]
Patch to declare GlassFish dependency

Created attachment 57023 [details]
Patch to allow clusters installed outside of the NB install dir.

The next step would be to: Apply the patches attached by Jun to release601_fixes branch, increment spec numbers and
build the nbm file(s) for testing. It can be done in some local workspace (or by checking in the changes and picking up
the nbm from BE build. Usually we checkin to fixes branch only if the fix has been verified in trunk, but in this case,
the fix cannot be verified in trunk because a) the runtimes are different and b) the fix itself is different and is not
a mere backport. We can still choose to checkin, of course; we can always roll back).

Is anyone on this thread (pgabauer , mslama...) planning to build a local-workspace nbm?

I am working now on verification in local workspace. Checkout takes long time.

The status is still "NEW". It is too late to get this fix into NB 6.0.1 Patch2. Feb/25 is the cut-off date for issues
nomination. Every candidate issue should be fixed and verified before the cut-off date.

I am changing the "Status whiteboard" keyword from "release601_fixes_candidate2" to "release601_fixes_candidate3".

Jun and mslama, please feel free to correct it if I am wrong.

On branch release60 GlassFish has version 1.4.1.1.1 but you have dependency on 1.4.2. So I cannot install SOA modules. I
will update again source from branch release601_fixes as they should be fixed. This branch was broken on Friday so I
could not test it.

Now it works. Still there is one remaining problem with test when using IDE installed from Ubuntu packages. I
incremented spec version of ide8/modules/org-apache-xml-resolver.jar (library wrapper for resolver-1.2) and
ide8/modules/org-netbeans-modules-xml-catalog.jar, placed new modules into user dir and verified in ide log that correct
module version are loaded. I also added log to JarClassLoader to verify that correct version of resolver-1.2 is loaded
from user dir. (ie. these jars are used instead of jar in install dir). SynchronousSampleApplication is built fine and
deploys but test fails. I will attach verbose log from test. Please look what could be wrong with test/why it fails.

Created attachment 57196 [details]
Test fails after patch is applied

If you need to reproduce this test:
1.All NetBeans Ubuntu packages are already in repository so you need not install them manually from deb files. Simply
use synaptic, apt-get or aptitude to install package 'netbeans'. 
2.Then run /usr/bin/netbeans --jdkhome <Path to openjdk6> to create fresh user dir. 
3.Build SOA nbms locally with applied patch as described above.
4.Increment module versions of ide8/modules/org-apache-xml-resolver.jar and
ide8/modules/org-netbeans-modules-xml-catalog.jar, builkd them and copy three mentioned jars manually from
nbbuild/netbeans into user dir. (ide8/modules/org-apache-xml-resolver.jar, ide8/modules/ext/resolver-1.2.jar,
ide8/modules/org-netbeans-modules-xml-catalog.jar). (We will create update nbm for UC.)
5.Install SOA nbm from local UC repository.
6.Build SynchronousSampleApplication, deploy it and run test.

It looks like test still uses resolver-1.2 from install dir. When I removed resolver-1.2 from install dir test passes.

From your comment, it looks like you had *two* copies of resolver-1.2.jar when the test failed, one under NB install dir
and the other under user dir, and the one under install dir (which is probably old) was interfering.

As to GlassFish version, maybe the version in release60 and release601_fixes are different?

Yes there are 2 resolver-1.2.jar. One in install dir and one in user dir. As library wrapper org-apache-xml-resolver.jar
for resolver has incremented version in user dir IDE uses wrapper from user dir and so also resolver-1.2.jar from user
dir. But it looks like test *still* uses resolver-1.2.jar from install dir. It is necessary to fix it otherwise issue
#127132 cannot be fully resolved. (Main reason behind this is that packages are owned by root so they cannot be updated
by normal user => everything from UC goes to user dir and user dir content must completely override install dir content.)

Yes in branch release60 glassfish plugin has lower spec version. So you patch can be tested only on branch release601_fixes.

Yes there are 2 resolver-1.2.jar. One in install dir and one in user dir. As library wrapper org-apache-xml-resolver.jar
for resolver has incremented version in user dir IDE uses wrapper from user dir and so also resolver-1.2.jar from user
dir. But it looks like test *still* uses resolver-1.2.jar from install dir. It is necessary to fix it otherwise issue
#127132 cannot be fully resolved. (Main reason behind this is that packages are owned by root so they cannot be updated
by normal user => everything from UC goes to user dir and user dir content must completely override install dir content.)

Yes in branch release60 glassfish plugin has lower spec version. So your patch can be tested only on branch
release601_fixes.

Our plan is to create Ubuntu specific update containing resolver-1.2.jar, its wrapper and xml-catalog module. As these
jars will be installed into user dir we need to make sure that everything will use these jars from user and NOT from
install dir. I hope it is clear now.

CompApp project test framework doesn't have direct dependency on ide8/modules/org-apache-xml-resolver.jar.

When installing NBM from UC, will the user get only the patched jar files (ide8/modules/org-apache-xml-resolver.jar,
ide8/modules/ext/resolver-1.2.jar, ide8/modules/org-netbeans-modules-xml-catalog.jar) or the whole ide8 cluster that
includes those patched jar files in the user dir?

BTW, when the test fails, what is the content of the *actual* test output file?

FYI, this is what I've got after the fix:

        <junit showoutput="true" fork="yes" dir="${basedir}" failureproperty="tests.failed" errorproperty="tests.failed">
            <classpath>
                <pathelement path="${soa.module.install.dir}/org-netbeans-modules-compapp-manager-jbi.jar"/>
                <pathelement path="${soa.module.install.dir}/ext/jbi/catd.jar"/>
                <pathelement path="${soa.module.install.dir}/ext/jbi/xmlunit-1.0.jar"/>
                <pathelement path="${soa.module.install.dir}/ext/jbi/httpunit-1.6.jar"/>
                <pathelement path="${java.module.install.dir}/ext/junit-3.8.2.jar"/>
                <pathelement path="${java.module.install.dir}/ext/jaxws21/api/saaj-api.jar"/>
                <pathelement path="${java.module.install.dir}/ext/jaxws21/saaj-impl.jar"/>
                <pathelement path="${java.module.install.dir}/ext/jaxws21/FastInfoset.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-xml-xdm.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-xml-xam.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-editor-lib.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-xml-text.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-editor.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-editor-util.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-xml-core.jar"/>
                <pathelement path="${ide.module.install.dir}/org-netbeans-modules-editor-lib2.jar"/>
                <pathelement path="${netbeans.home}/lib/org-openide-modules.jar"/>
                <pathelement path="${netbeans.home}/lib/org-openide-util.jar"/>
                <pathelement path="${netbeans.home}/modules/org-openide-options.jar"/>
                <pathelement path="${netbeans.home}/modules/org-openide-text.jar"/>
                <pathelement path="${netbeans.home}/modules/org-openide-loaders.jar"/>
                <pathelement path="${netbeans.home}/modules/org-openide-nodes.jar"/>
                <pathelement path="${netbeans.home}/modules/org-netbeans-modules-editor-mimelookup.jar"/>
                <pathelement path="${netbeans.home}/core/org-openide-filesystems.jar"/>
            </classpath>
            .......
        <junit>

The following are defined in the compapp project's private.properties file. These locations are determined by the
various modules' locations *at runtime*.

enterprise.module.install.dir=C:\\nb6.0.1\\nbbuild\\netbeans\\enterprise4\\modules
ide.module.install.dir=C:\\nb6.0.1\\nbbuild\\netbeans\\ide8\\modules
java.module.install.dir=C:\\nb6.0.1\\nbbuild\\netbeans\\java1\\modules
soa.module.install.dir=C:\\nb6.0.1\\nbbuild\\netbeans\\soa1\\modules
xml.module.install.dir=C:\\nb6.0.1\\nbbuild\\netbeans\\xml1\\modules

Created attachment 57322 [details]
Second try: Patch to allow clusters installed outside of the NB install dir (with unused resolver dependency removed).

I just attached a newer patch with an unused resolver dependency removed. Marek, Could you try this one and see if it
works for you?

It looks clear now: Some jars from ide cluster are in install dir, some are in user dir. IDE is able to pick up correct
module jar with higher spec version but current project/test infrastructure is not able to be so smart.
private.properties content (now after I run test):
enterprise.module.install.dir=/home/mslama/.netbeans/6.0/modules
ide.module.install.dir=/usr/share/netbeans/6.0.1/ide8/modules
java.module.install.dir=/usr/share/netbeans/6.0.1/java1/modules
netbeans.user=/home/mslama/.netbeans/6.0
project.SynchronousSample=/home/mslama/NetBeansProjects/SynchronousSample/SynchronousSample
soa.module.install.dir=/home/mslama/.netbeans/6.0/modules
xml.module.install.dir=/home/mslama/.netbeans/6.0/modules


In current I see only solution is to perform update of ide cluster jars as root so that they are updated in install dir.
(I mean xml-catalog, resolver library wrapper, resolver-1.2.)

I tested your last patch and test passes now! I do not know what magic is behind it....
I see 
<pathelement location="${{ide.module.install.dir}}/org-apache-xml-resolver.jar"/>
in your patch so it works in IDE but I am not sure if it will work outside IDE.

Where can I find *actual* test output file?

Created attachment 57389 [details]
The *actual* test output

If the user dir always takes precedence over NB installation dir when loading classes, then we shouldn't have this
issue, unless someone uses hard-coded classpath, like I had before. 

CompApp project had the following unused dependency declaration which is removed now: 

-                <dependency>
-                    <code-name-base>org.apache.xml.resolver</code-name-base>
-                    <build-prerequisite/>
-                    <compile-dependency/>
-                    <run-dependency>
-                        <specification-version>1.1.21</specification-version>
-                    </run-dependency>
-                </dependency>

Just curious, what is the spec version of the resolver wrapper module in install dir and the one in the user dir? Will
that affect which one gets loaded?

Test passes now. I created update for resolver too. Actual output from test is:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" x
si:schemaLocation="http://schemas.xmlsoap.org/soap/envelope/ http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Body>
    <typeA xmlns="http://xml.netbeans.org/schema/SynchronousSample">
      <ns2:paramA xmlns:ns2="http://xml.netbeans.org/schema/SynchronousSample">Hello World</ns2:paramA>
    </typeA>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

I will coordinate with sustaining further actions.

org-apache-xml-resolver.jar in install dir (NB 6.0.1) has spec ver 1.2.1.
Update of org-apache-xml-resolver.jar has pec version 1.2.2. I also incremented spec version of
org-netbeans-modules-ide-kit.jar (visible in Plugin UI) to 1.2.3 and org-netbeans-modules-xml-catalog.jar to 1.12.2.4.4.
I also update dependencies. I will attach diff.

The fix has been backported into the release601_fixes branch.

/cvs/enterprise/compapp/jbimanager/nbproject/Attic/project.xml,v  <--  project.xml
new revision: 1.15.6.1; previous revision: 1.15

/cvs/enterprise/compapp/projectjbi/nbproject/Attic/project.xml,v  <--  project.xml
new revision: 1.22.10.1; previous revision: 1.22

/cvs/enterprise/compapp/projectjbi/src/org/netbeans/modules/compapp/projects/jbi/Attic/JbiProject.java,v  <-- 
JbiProject.java
new revision: 1.14.6.1; previous revision: 1.14

/cvs/enterprise/compapp/projectjbi/src/org/netbeans/modules/compapp/projects/jbi/resources/Attic/build-impl.xsl,v  <-- 
build-impl.xsl
new revision: 1.17.6.1; previous revision: 1.17

Created attachment 57460 [details]
Diff of fix

The fix was verified by mslama on Feb 28. I am changing the status to Verified.