Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 208873 - Build script TestNG integration
Build script TestNG integration
Status: RESOLVED FIXED
Product: java
Classification: Unclassified
Component: Project
7.2
All All
: P3 (vote)
: 7.2
Assigned To: Tomas Zezula
issues@java
TestNG_Inclusion
: API_REVIEW
: 197734 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-27 09:00 UTC by Theofanis Oikonomou
Modified: 2012-03-23 10:34 UTC (History)
8 users (show)

See Also:
Issue Type: ENHANCEMENT
:


Attachments
Content originally inlined in http://wiki.netbeans.org/TestNG_Integration (7.80 KB, application/xml)
2012-02-27 20:21 UTC, Jesse Glick
Details
proposed patch (18.13 KB, text/plain)
2012-03-01 03:29 UTC, Lukas Jungmann
Details
full trunk/branch diff (42.72 KB, patch)
2012-03-13 14:42 UTC, Lukas Jungmann
Details | Diff
testng inclusion (41.85 KB, patch)
2012-03-19 19:18 UTC, Lukas Jungmann
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Theofanis Oikonomou 2012-02-27 09:00:49 UTC
Due to license issue with JUnit, the library was removed from the standard NetBeans distribution in version 7.0.1

TestNG is an alternative testing framework for Java, which could be used by NetBeans. With TestNG integrated in the standard IDE distribution, NetBeans will gain out-of-the-box experience for its users again while providing them an option to use JUnit (as it is now), an option to easily migrate their older projects to TestNG as well as an option to write tests for their projects using TestNG only.

I order to use TestNG as default testing framework in Java-based NetBeans projects TestNG related targets and params should be added to build-impl.xml for them to be better maintained.

An overview of the targets to be added can be found here: http://wiki.netbeans.org/TestNG_Integration
Comment 1 Jesse Glick 2012-02-27 20:21:59 UTC
Created attachment 116155 [details]
Content originally inlined in http://wiki.netbeans.org/TestNG_Integration
Comment 2 Lukas Jungmann 2012-03-01 03:29:16 UTC
Created attachment 116223 [details]
proposed patch

Attached patch tries to solve this issue:

-junit macro and comments were renamed to 'test'

-new 'test-debug' macro has been added (benefit of this is that now even build-script extension can override test-debug target - this wasn't possible before)

-junit task has been replaced by testng

-adding testng library to the project is not solved by this patch
Comment 3 Lukas Jungmann 2012-03-01 03:33:25 UTC
forgot to mention that 'test' and 'test-debug' macros in this patch are also able to handle Run/Debug focused test action when compile on save is disabled (for both JUnit and TestNG)

Disadvantage of this patch is that it expects TestNG library on the classpath even for running/debugging JUnit tests only.
Comment 4 Tomas Zezula 2012-03-01 12:06:53 UTC
*** Bug 197734 has been marked as a duplicate of this bug. ***
Comment 5 Lukas Jungmann 2012-03-01 13:00:01 UTC
added to branch: http://hg.netbeans.org/main/rev/3393ae4e71e5
Comment 6 Tomas Zezula 2012-03-01 13:15:56 UTC
In fact the build-impl.xsl changes are incompatible API change which brakes all scripts overriding or depending on -init-macrodef-junit. It also breaks the users of j2seproject3:junit which was removed.
The build-impl.xsl should contain junit and TestNG macros. When there is TestNG on the class path the TestNG should be used when when there is no TestNG but there is a JUnit the junit should be used. This can be achieved by having 2 generic test macro def targets one triggered by presence of junit and second by presence of TestNG.
Comment 7 _ gtzabari 2012-03-01 14:44:37 UTC
I'm not sure whether this point has been made yet...

By default, TestNG supports running JUnit 3 out-of-the-box. Lukas introduced "mixed-mode" (for running JUnit 4 under TestNG) by bundling JUnit with TestNG's JAR.

Whatever solution you end up with, I think that TestNG's JAR should not bundle JUnit. It should use JUnit if it's available, but ignore it otherwise. This way, if a user wants to run TestNG tests he needs to include the TestNG JAR. If he wants to run JUnit tests, he needs to include the JUnit JAR. If they both happen to exist on the classpath, they should coexist, but one shouldn't be coupled with the other.

It's my understanding the only way to achieve this is to move some code out of TestNG's "mixed mode" code into Netbeans.
Comment 8 Lukas Jungmann 2012-03-01 16:00:17 UTC
cc'ing David from JavaEE team...
Comment 9 Lukas Jungmann 2012-03-01 16:54:57 UTC
(In reply to comment #6)
> In fact the build-impl.xsl changes are incompatible API change which brakes all
> scripts overriding or depending on -init-macrodef-junit. It also breaks the
> users of j2seproject3:junit which was removed.

I wasn't sure about this - thanks for the pointers

> The build-impl.xsl should contain junit and TestNG macros. When there is TestNG
> on the class path the TestNG should be used when when there is no TestNG but
> there is a JUnit the junit should be used. This can be achieved by having 2
> generic test macro def targets one triggered by presence of junit and second by
> presence of TestNG.

I'll follow up this way...

(In reply to comment #7)
> I'm not sure whether this point has been made yet...
> 
> By default, TestNG supports running JUnit 3 out-of-the-box. Lukas introduced
> "mixed-mode" (for running JUnit 4 under TestNG) by bundling JUnit with TestNG's
> JAR.

This is not true[1]. JUnit has been bundled for a long time already and during my work testng.jar w/u JUnit classes has been made available. Feel free to review changes I contributed to TestNG[2]

> 
> Whatever solution you end up with, I think that TestNG's JAR should not bundle
> JUnit. It should use JUnit if it's available, but ignore it otherwise.

This is exactly how it is - TestNG library suggested to be used NB is the one w/o JUnit classes (the opposite would be unacceptable)

[snip/]

 If they both
> happen to exist on the classpath, they should coexist, but one shouldn't be
> coupled with the other.
> 
> It's my understanding the only way to achieve this is to move some code out of
> TestNG's "mixed mode" code into Netbeans.

I'm inclining to use testng mixed mode if both jars are on the test cp for given test source root but I plan to check how big effort it would be to alter and set testincludes/testexcludes properly to avoid using mixed mode in this particular case


[1]: "junit and mixed modes require JUnit library on the tests classpath" - documented in: https://github.com/cbeust/testng/pull/178

[2]: http://wiki.netbeans.org/TestNG_MixedMode#ChangeLog - https://github.com/cbeust/testng/commit/f5c4b35193a0c7cf2b1d0b45dd0570f6a948bb59 in particular
Comment 10 Tomas Zezula 2012-03-01 20:35:40 UTC
I can help with the build-impl.xsl if you want, but I am out of office next week.
Comment 11 Lukas Jungmann 2012-03-01 20:55:39 UTC
(In reply to comment #10)
> I can help with the build-impl.xsl if you want, but I am out of office next
> week.

That would be great! Thanks! I'll get to this on Mon/Tue, so if there would be some progress I will continue from there.
Comment 12 David Konecny 2012-03-05 00:18:59 UTC
In theory build scripts in J2EE project types should be very very close to the ones in J2SE projects. So it should be matter of just transplanting changes from J2SE project to web.project, j2ee.ejbjarproject and j2ee.clientproject. j2ee.earproject should not have any testing support as it does not have sources. I'm happy to review the changes or help making them if necessary. Just let me know what your plans are and whether you need a help. Thanks.
Comment 13 Jesse Glick 2012-03-08 02:18:23 UTC
Check whether AntBuildExtender can be used in this case - it would be declutter build-impl.xml a bit if nbproject/junit.xml and nbproject/testng.xml (the latter I guess supporting also mixed mode) could be inserted by the relevant modules. This might be too tricky to do retroactively, though.
Comment 14 Lukas Jungmann 2012-03-08 02:49:52 UTC
http://hg.netbeans.org/main/rev/c33d4ae9c1b4

-put back junit macro to be backward compatible

-rewrote test and test-single targets so one can use either junit, testng, testng in mixed mode or both at once - actual target depends on availability of particular library, tests and can be overriden by specific property

I'm going to do debugging part similarly but giving this part out earlier so I can change my path if needed - feedback appreciated

(In reply to comment #13)
> Check whether AntBuildExtender can be used in this case - it would be declutter
> build-impl.xml a bit if nbproject/junit.xml and nbproject/testng.xml (the
> latter I guess supporting also mixed mode) could be inserted by the relevant
> modules. This might be too tricky to do retroactively, though.

I agree that using AntBuildExtender would be better but there are at least two issues with it:
-extensions are not versioned, as I was told, so it makes it difficult to keep them up-to-date
-not all test related targets are easily extensible at this point (at least not debug-test)

But I'm trying to keep possible future exclusion of testing related targets from build-impl in mind - see new 'test' macro definition (I'm going to add 'test-debug' macro too) - one should be able to easily override it from an extension in the future
Comment 15 Lukas Jungmann 2012-03-13 14:27:45 UTC
debugging part: http://hg.netbeans.org/main/rev/3393ae4e71e5
Comment 16 Lukas Jungmann 2012-03-13 14:42:13 UTC
Created attachment 116670 [details]
full trunk/branch diff

updated patch per Tomas' comments

In this patch native execution is supported by checking test files for presence of specific testng imports directly in the build script. Unfortunately there are two issues with this solution

-performance - file scanning adds some overhead during test run/debug
-testng does not support empty set of tests to run - it will fail while junit will just continue - this makes it tricky and error prone to support native test execution/debugging for single test

Therefore I'd propose to only document a way to use native runners for tests in case there are both JUnit and TestNG tests in the same test root and just use native JUnit runner for projects with JUnit tests and TestNG for projects with TestNG tests and JUnit+TestNG (using mixed mode) tests.
Comment 17 Lukas Jungmann 2012-03-19 19:18:45 UTC
Created attachment 116883 [details]
testng inclusion

updated version of the patch to:

-implement test/debug single method if compile-on-save is disabled (for TestNG as well as JUnit)

-use ${property} notation in 'if' and 'unless' attributes
Comment 18 Lukas Jungmann 2012-03-20 15:02:57 UTC
If there are no further comments, I'd like to integrate this tomorrow. Thank you.
Comment 19 Quality Engineering 2012-03-23 10:34:42 UTC
Integrated into 'main-golden', will be available in build *201203230400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/3393ae4e71e5
User: Lukas Jungmann <jungi@netbeans.org>
Log: #208873: Build script TestNG integration (j2se.project)


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo