This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Due to license issue with JUnit, the library was removed from the standard NetBeans distribution in version 7.0.1 TestNG is an alternative testing framework for Java, which could be used by NetBeans. With TestNG integrated in the standard IDE distribution, NetBeans will gain out-of-the-box experience for its users again while providing them an option to use JUnit (as it is now), an option to easily migrate their older projects to TestNG as well as an option to write tests for their projects using TestNG only. I order to use TestNG as default testing framework in Java-based NetBeans projects TestNG related targets and params should be added to build-impl.xml for them to be better maintained. An overview of the targets to be added can be found here: http://wiki.netbeans.org/TestNG_Integration
Created attachment 116155 [details] Content originally inlined in http://wiki.netbeans.org/TestNG_Integration
Created attachment 116223 [details] proposed patch Attached patch tries to solve this issue: -junit macro and comments were renamed to 'test' -new 'test-debug' macro has been added (benefit of this is that now even build-script extension can override test-debug target - this wasn't possible before) -junit task has been replaced by testng -adding testng library to the project is not solved by this patch
forgot to mention that 'test' and 'test-debug' macros in this patch are also able to handle Run/Debug focused test action when compile on save is disabled (for both JUnit and TestNG) Disadvantage of this patch is that it expects TestNG library on the classpath even for running/debugging JUnit tests only.
*** Bug 197734 has been marked as a duplicate of this bug. ***
added to branch: http://hg.netbeans.org/main/rev/3393ae4e71e5
In fact the build-impl.xsl changes are incompatible API change which brakes all scripts overriding or depending on -init-macrodef-junit. It also breaks the users of j2seproject3:junit which was removed. The build-impl.xsl should contain junit and TestNG macros. When there is TestNG on the class path the TestNG should be used when when there is no TestNG but there is a JUnit the junit should be used. This can be achieved by having 2 generic test macro def targets one triggered by presence of junit and second by presence of TestNG.
I'm not sure whether this point has been made yet... By default, TestNG supports running JUnit 3 out-of-the-box. Lukas introduced "mixed-mode" (for running JUnit 4 under TestNG) by bundling JUnit with TestNG's JAR. Whatever solution you end up with, I think that TestNG's JAR should not bundle JUnit. It should use JUnit if it's available, but ignore it otherwise. This way, if a user wants to run TestNG tests he needs to include the TestNG JAR. If he wants to run JUnit tests, he needs to include the JUnit JAR. If they both happen to exist on the classpath, they should coexist, but one shouldn't be coupled with the other. It's my understanding the only way to achieve this is to move some code out of TestNG's "mixed mode" code into Netbeans.
cc'ing David from JavaEE team...
(In reply to comment #6) > In fact the build-impl.xsl changes are incompatible API change which brakes all > scripts overriding or depending on -init-macrodef-junit. It also breaks the > users of j2seproject3:junit which was removed. I wasn't sure about this - thanks for the pointers > The build-impl.xsl should contain junit and TestNG macros. When there is TestNG > on the class path the TestNG should be used when when there is no TestNG but > there is a JUnit the junit should be used. This can be achieved by having 2 > generic test macro def targets one triggered by presence of junit and second by > presence of TestNG. I'll follow up this way... (In reply to comment #7) > I'm not sure whether this point has been made yet... > > By default, TestNG supports running JUnit 3 out-of-the-box. Lukas introduced > "mixed-mode" (for running JUnit 4 under TestNG) by bundling JUnit with TestNG's > JAR. This is not true[1]. JUnit has been bundled for a long time already and during my work testng.jar w/u JUnit classes has been made available. Feel free to review changes I contributed to TestNG[2] > > Whatever solution you end up with, I think that TestNG's JAR should not bundle > JUnit. It should use JUnit if it's available, but ignore it otherwise. This is exactly how it is - TestNG library suggested to be used NB is the one w/o JUnit classes (the opposite would be unacceptable) [snip/] If they both > happen to exist on the classpath, they should coexist, but one shouldn't be > coupled with the other. > > It's my understanding the only way to achieve this is to move some code out of > TestNG's "mixed mode" code into Netbeans. I'm inclining to use testng mixed mode if both jars are on the test cp for given test source root but I plan to check how big effort it would be to alter and set testincludes/testexcludes properly to avoid using mixed mode in this particular case [1]: "junit and mixed modes require JUnit library on the tests classpath" - documented in: https://github.com/cbeust/testng/pull/178 [2]: http://wiki.netbeans.org/TestNG_MixedMode#ChangeLog - https://github.com/cbeust/testng/commit/f5c4b35193a0c7cf2b1d0b45dd0570f6a948bb59 in particular
I can help with the build-impl.xsl if you want, but I am out of office next week.
(In reply to comment #10) > I can help with the build-impl.xsl if you want, but I am out of office next > week. That would be great! Thanks! I'll get to this on Mon/Tue, so if there would be some progress I will continue from there.
In theory build scripts in J2EE project types should be very very close to the ones in J2SE projects. So it should be matter of just transplanting changes from J2SE project to web.project, j2ee.ejbjarproject and j2ee.clientproject. j2ee.earproject should not have any testing support as it does not have sources. I'm happy to review the changes or help making them if necessary. Just let me know what your plans are and whether you need a help. Thanks.
Check whether AntBuildExtender can be used in this case - it would be declutter build-impl.xml a bit if nbproject/junit.xml and nbproject/testng.xml (the latter I guess supporting also mixed mode) could be inserted by the relevant modules. This might be too tricky to do retroactively, though.
http://hg.netbeans.org/main/rev/c33d4ae9c1b4 -put back junit macro to be backward compatible -rewrote test and test-single targets so one can use either junit, testng, testng in mixed mode or both at once - actual target depends on availability of particular library, tests and can be overriden by specific property I'm going to do debugging part similarly but giving this part out earlier so I can change my path if needed - feedback appreciated (In reply to comment #13) > Check whether AntBuildExtender can be used in this case - it would be declutter > build-impl.xml a bit if nbproject/junit.xml and nbproject/testng.xml (the > latter I guess supporting also mixed mode) could be inserted by the relevant > modules. This might be too tricky to do retroactively, though. I agree that using AntBuildExtender would be better but there are at least two issues with it: -extensions are not versioned, as I was told, so it makes it difficult to keep them up-to-date -not all test related targets are easily extensible at this point (at least not debug-test) But I'm trying to keep possible future exclusion of testing related targets from build-impl in mind - see new 'test' macro definition (I'm going to add 'test-debug' macro too) - one should be able to easily override it from an extension in the future
debugging part: http://hg.netbeans.org/main/rev/3393ae4e71e5
Created attachment 116670 [details] full trunk/branch diff updated patch per Tomas' comments In this patch native execution is supported by checking test files for presence of specific testng imports directly in the build script. Unfortunately there are two issues with this solution -performance - file scanning adds some overhead during test run/debug -testng does not support empty set of tests to run - it will fail while junit will just continue - this makes it tricky and error prone to support native test execution/debugging for single test Therefore I'd propose to only document a way to use native runners for tests in case there are both JUnit and TestNG tests in the same test root and just use native JUnit runner for projects with JUnit tests and TestNG for projects with TestNG tests and JUnit+TestNG (using mixed mode) tests.
Created attachment 116883 [details] testng inclusion updated version of the patch to: -implement test/debug single method if compile-on-save is disabled (for TestNG as well as JUnit) -use ${property} notation in 'if' and 'unless' attributes
If there are no further comments, I'd like to integrate this tomorrow. Thank you.
Integrated into 'main-golden', will be available in build *201203230400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-golden/rev/3393ae4e71e5 User: Lukas Jungmann <jungi@netbeans.org> Log: #208873: Build script TestNG integration (j2se.project)