Bug 197065

Submit a patch.

Created attachment 107347 [details]
Patch which allows test configuration to accept startup arguments

I've attached a patch I developed which adds support for startup arguments in the unit test.  It should be pretty straightforward to use, but here is an example:

    public static Test suite() {
        Configuration config = NbModuleSuite.createConfiguration(TestSayHelloMenu.class);
        config = config.addTest("testClickFileSayHelloMenuItem")
            .clusters(".*").enableModules(".*")
            .gui(true)
            .addStartupArgument("--branding", "functionaltestsuite")
            .addStartupArgument("--fontsize", "18");

        return NbModuleSuite.create(config);
    }

I apologize that I do not have a unit test for this.  Several of the unit tests for NbModuleSuiteTest fail on my machine (before applying the patch) and so I cannot create a new unit test with confidence.  I have tested this manually using the code above in conjunction with the minimal suite from #197066 and i4t seems to work fine.

Y01 Missing @since tag on new method
Y02 Missing test
Y03 Need a change in apichanges.xml

Otherwise ok, and when fixed in new patch, it can be applied. The trunk is currently closed for bugfixes only. Marian, can Tom get this in or does he need to wait for 7.1?

I will make all the changes you request, but out of curiosity, do all the tests in NbModuleSuiteTest pass for you?  Several of them fail on my machine and so I am not sure whether the tests are broken, I have a bad checkout or something's not right about my setup.

(In reply to comment #4)
> do all the tests in NbModuleSuiteTest pass for you?

For me and for

http://deadlock.netbeans.org/hudson/job/NB-Core-Build/lastStableBuild/testReport/org.netbeans.junit/NbModuleSuiteTest/

> Several of them fail on my machine

Details?

> I have a bad checkout

You are first checking without your patch applied I hope?

hg version
java -version
ant -version
hg clean --all
hg up -c default
hg st
hg log -r .
ant -Dcluster.config=basic build-nozip
ant -f nbjunit/build.xml test

> or something's not right about my setup

If there is anything environmental known to cause test failures that ought to be caught by something other than the test failing.

Thanks, Jesse. I figured that the tests would have to pass on deadlock in order to not consider the build broken.  I did run the tests without my patch applied and they failed. I was in a hurry at the time and didn't write down any details.  

What I found was that the tests pass when run from the command line yet fail when run from the IDE.  I didn't want to pollute this issue with details about that, so I filed #197548 and documented them all in there.

I will make the three changes Jarda requested in the next few days and attach a patch.  I will verify my test works correctly by running it from the command line.

Created attachment 107619 [details]
Revised patch which satisfies Y01, Y02 and Y03

I have attached a revised patch which satisfies Y01, Y02 and Y03.  I am still having problems running tests, so I'd certainly recommend running the test before committing changes related to this patch.

While working on the test case, I found that command-line arguments meant for the JVM (i.e. -J-Xmx256m or -J-Dname=value) are not being properly interpreted when supplied for a test case using the addStartupArgument method.  Options like --branding or --userdir work properly, as evidenced by the test case.  A startup argument like "-J-Dname=value" yields a message "Ignored unknown option: -J-Dname=value" in the output, followed by "Post-initialization command-line options could not be run" and the system property is never set.

Although I don't currently need to specify JVM parameters at startup, it would be nice if that also worked.  If you can give me a hint as to what I should change to support that, I will try and create a new patch to handle it.  Otherwise, I'd be happy to have the attached patch integrated in the NB JUnit module if it meets with your approval.

Re. "-J parameters". The NbModuleSuite cannot change certain parameters of JVM as JVM is already running. To do that you need to influence the junit task. Luckily there is a way to incluence that: 

$ grep test.*arg */nbproject/project.properties 
o.n.core/nbproject/project.properties:test.run.args=-ea -XX:PermSize=32m -XX:MaxPermSize=400m -Xmx520m
performance/nbproject/project.properties:test.run.args=-da -Xmx384M

Otherwise the patch is OK (except missing change of version in manifest). I'll integrate it as soon as API changes are allowed.

Created attachment 107691 [details]
Revised patch which increments version number and updates documentation

Thanks, Jarda.  I have attached a new patch which has the same changes as before, plus an incremented version number.  I also updated the documentation slightly to mention the test.run.args property and to remove obsolete references to XTest.

Integrated as ergonomics#879d18f15152

*** Bug 197066 has been marked as a duplicate of this bug. ***