This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Product Version: NetBeans IDE Dev (Build 201201250600) Java: 1.7.0_02; Java HotSpot(TM) 64-Bit Server VM 22.0-b10 System: Windows 7 version 6.1 running on amd64; Cp1252; en_US (nb) User directory: C:\Users\Gili\.netbeans\dev Cache directory: C:\Users\Gili\.netbeans\dev\var\cache 1. Open an existing project containing TestNG tests. 2. Install the TestNG Ant plugin 1.1. 3. ALT+F6 4. Netbeans fails because it is looking for JUnit tests instead of TestNG: Testcase: initializationError(org.pill.repository.local.LocalRepositoryTest): Caused an ERROR No runnable methods java.lang.Exception: No runnable methods at java.lang.reflect.Constructor.newInstance(Constructor.java:525) This behavior contradicts http://wiki.netbeans.org/TestNG and bug 130605. Netbeans 7.1 does *not* support TestNG.
I got a bit further by adding the following to build.xml: <target name="-post-compile-test"> <testng verbose="2"> <classpath path="${run.test.classpath}"/> <classfileset dir="build/test/classes" includes="**/*Test.class"/> </testng> </target> But now I get two tabs under Test Results: 1. The "Ant Suite" tab reads "No tests executed" 2. The second tab has the name of the project. It contains the following tree: No test passed, 1 test failed \-> <test class name> FAILED \-> Warning FAILED As you can see, it fails to provide any useful information about the failure.
I forgot to mention, I also had to add the TestNG library to my project and add the following above -post-compile-test: <taskdef resource="testngtasks" classpath="lib/testng/testng-6.3.1.jar"/> All of this should be handled automatically by installing the plugin or documented in great detail.
I got rid of the second tab (the one containing WARNING) by replacing -post-compile-test with -do-test-run Now the "Test Results" contains a single "Ant suite" tab. The test output shows up on the right (good!) but the left hand side still reads "0% - No tests executed." I found an excellent article at http://www.oocities.org/nimarukan/netbeans/nbtargets/nbtargets-alt-test-runners.html that describes how to integrate TestNG into Netbeans. I'm going to attach an updated version of nbtargets-testng.xml that worked for me (but again, the "Test Results" tab claims no tests were executed). At this point I'm looking for some feedback: how do I get the Test Results tab to work?
Created attachment 115363 [details] nbtargets-testng.xml updated for testng 6.3.1
which version of the plugin do you use? the one from development UC or from stable UC for earlier NB version? Is compile on save enabled or disabled for your project? The former case is not supported yet and there's a RFE for this already and some parts may still not work properly. The latter case should work - any help with hunting bugs here would be really appreciated FYI: The plugin on dev UC contains TestNG 6.3.2beta (also note this is a slightly different version from the one provided by Cedric), so the version you used is really important info ... w/o that => incomplete (for now)
(In reply to comment #5) > which version of the plugin do you use? the one from development UC or from > stable UC for earlier NB version? I'm using "Ant TestNG support 1.1" from "Latest Development Build" (note I am using a NB 7.2 dev build). > Is compile on save enabled or disabled for your project? The former case is not > supported yet and there's a RFE for this already and some parts may still not > work properly. The latter case should work - any help with hunting bugs here > would be really appreciated Compile on Save is disabled for my project. > FYI: The plugin on dev UC contains TestNG 6.3.2beta (also note this is a > slightly different version from the one provided by Cedric), so the version you > used is really important info ... w/o that => incomplete (for now) I manually added TestNG 6.3.1 to the "Test Libraries". Could that break anything? Should I remove it? Assuming that I configure everything properly, should the Test Results window display my tests or should it show "No tests executed" as I mentioned?
Lukas, Can you reproduce this problem?
sorry, haven't had a chance to try to reproduce this yet :-( But I'll have to resolve this by Monday, so if some fix is necessary, you can expected it during Fri/Sat. One more question: is it java se project where you see this issue or is it another project type? as for test results - does setting 'useDefaultListeners="true"' make any difference? Or can you try to refresh the view using filtering buttons? I'm aware there's some sync issue but after refresh, you should be able to see the stacktrace etc
(In reply to comment #8) > sorry, haven't had a chance to try to reproduce this yet :-( But I'll have to > resolve this by Monday, so if some fix is necessary, you can expected it during > Fri/Sat. Excellent :) > One more question: is it java se project where you see this issue or is it > another project type? Yes, it's a JavaSE project. > as for test results - does setting 'useDefaultListeners="true"' make any > difference? Or can you try to refresh the view using filtering buttons? I'm > aware there's some sync issue but after refresh, you should be able to see the > stacktrace etc Where do I set useDefaultListeners?
> > as for test results - does setting 'useDefaultListeners="true"' make any > > difference? Or can you try to refresh the view using filtering buttons? I'm > > aware there's some sync issue but after refresh, you should be able to see the > > stacktrace etc > > Where do I set useDefaultListeners? <testng verbose="2" useDefaultListeners="true"> it's an attribute of testng task - you can verify that it is applied by running ant in verbose mode (see Tools -> Options-> Misc -> Ant -> Verbosity level=Verbose) - you should see a lot of messages starting with 'VerboseReporter' or sth like that
I figured it out. It looks like the plugin fails silently when interacting with different versions of TestNG it was compiled against. If you use TestNG 6.3.1 in your project, the results window will not work. If you use 6.3.2beta which ships with the plugin it'll work fine. You don't need to use "useDefaultListeners" either. I think the biggest thing you're missing now is good documentation. I've added a "Getting Started" section at http://wiki.netbeans.org/TestNG#Getting_Started:_Ant_plugin
first of all - thanks for the time you're spending with this. (In reply to comment #11) > I figured it out. It looks like the plugin fails silently when interacting with > different versions of TestNG it was compiled against. > > If you use TestNG 6.3.1 in your project, the results window will not work. If > you use 6.3.2beta which ships with the plugin it'll work fine. You don't need > to use "useDefaultListeners" either. clearly a regression, which I thought I already fixed. The cause of it is that in previous versions all results were presented in the UI after all tests finished and in current version results are presented as they come after each test/testcase > > I think the biggest thing you're missing now is good documentation. I've added > a "Getting Started" section at > http://wiki.netbeans.org/TestNG#Getting_Started:_Ant_plugin Thanks! I'll try to improve it ASAP.
(In reply to comment #12) > clearly a regression, which I thought I already fixed. The cause of it is that > in previous versions all results were presented in the UI after all tests > finished and in current version results are presented as they come after each > test/testcase I get the feeling that you'd have a hard time unit testing the unit tests :) If it is possible, you might try using http://code.google.com/p/fest/ to avoid such regressions in the future. Thanks for the support! I've been wanting to migrate to TestNG for a while now :)
http://hg.netbeans.org/main/contrib/rev/becd301cfcbf should help -as for build.xml modifications - this is sth what I simply cannot do from within the plugin (technically speaking I can but...)
(In reply to comment #14) > -as for build.xml modifications - this is sth what I simply cannot do from > within the plugin (technically speaking I can but...) Sorry, I didn't really understand what you meant about build.xml. Can you elaborate?
(In reply to comment #15) > (In reply to comment #14) > > -as for build.xml modifications - this is sth what I simply cannot do from > > within the plugin (technically speaking I can but...) > > Sorry, I didn't really understand what you meant about build.xml. Can you > elaborate? the file you attached comment #4 - overriding of target '-debug-start-debuggee-test' defined in build-impl.xml will work if it done in project's build.xml but it won't work if I do it in the customized build script - testng-build.xml - which is included from build-impl.xml
(In reply to comment #16) > the file you attached comment #4 - overriding of target > '-debug-start-debuggee-test' defined in build-impl.xml will work if it done in > project's build.xml but it won't work if I do it in the customized build script > - testng-build.xml - which is included from build-impl.xml Some features, like the Netbeans profiler, changes build.xml directly (adding an <include> statement for its sub-script). At the very least, you should offer your own testng-build.xml implementation and ask users to include it in their build.xml.
(In reply to comment #17) > Some features, like the Netbeans profiler, changes build.xml directly (adding > an <include> statement for its sub-script). they most likely have a very good reason why they are doing it At the very least, you should offer > your own testng-build.xml implementation and ask users to include it in their > build.xml. there already is nbproject/testng-impl.xml (included from build-impl.xml) - to get it you have to create at least one test or test suite using new file wizard while not having testng lib on the cp but if you already have a project which has TestNG on the cp the file is not created (based on an assumption that if a project has the lib on the cp then it either already has the script present or it is some older or custom project where user added the lib himself and therefore he knows what he is doing - ie he already modified build.xml or included some other file with testng task definition etc
(In reply to comment #18) > there already is nbproject/testng-impl.xml http://hg.netbeans.org/main/contrib/rev/b4619fb1e7af should fix debugging related issues if testng-impl.xml is included directly from build.xml Only note 2 things: -compile on save must be disabled for global IDE actions and shortcuts to work as expected -to get the build script working with arbitrary official testng distro you have to remove 'mixed' from java and testng tasks there
If you don't mind, I'd like to reopen this issue until we resolve the basics of getting this working in an Ant project. I took my project with existing TestNG tests, removed TestNG from the classpath and created a test using the New File wizard. build.xml was modified as you mentioned and TestNG was added to the classpath. I then removed the class I added using the New File wizard (only my old tests remained) and when I tried running with ALT+F6 I got: [TestNGAntTask] TESTNG PASSED @C:\Users\Gili\AppData\Local\Temp\testng8168646004527825901 WHICH CONTAINS: -log 2 -usedefaultlisteners true -d C:\Users\Gili\Documents\pill\core\build\test\results -listener org.testng.reporters.VerboseReporter -suitename Ant suite -testname Ant test Usage: <main class> [options] You need to specify at least one testng.xml, one class or one method I am expecting my existing tests to run. I don't think you can count on users using the file dialog for everything (I routinely create a new Java Class and turn it into a test class later on). Instead, I recommend you scan "Test Packages" for the @Test annotation. Let me know what you think. > (In reply to comment #18) > -to get the build script working with arbitrary official testng distro you have > to remove 'mixed' from java and testng tasks there Will this limitation ever go away? I see you have "remove junit from inside testng jar" scheduled. I ask because I don't think people will want to dig into "private" xml files.
(In reply to comment #20) > If you don't mind, I'd like to reopen this issue until we resolve the basics of > getting this working in an Ant project. you mean existing/older ant project with testng tests, right? If so, also update the summary please > > I took my project with existing TestNG tests, removed TestNG from the classpath > and created a test using the New File wizard. build.xml was modified as you > mentioned and TestNG was added to the classpath. I then removed the class I > added using the New File wizard (only my old tests remained) and when I tried > running with ALT+F6 I got: the best and fastest would be to attach the project to this issue (or send it to me offline) so I can reproduce the problem on my end... Have you tried the same with fresh new project with up-to date testng library? If no, can you try that and let me know? (just run IDE with new userdir) > Instead, I recommend you scan "Test > Packages" for the @Test annotation. Let me know what you think. wouldn't it be better if this could be done directly by TestNG itself? > > > (In reply to comment #18) > > -to get the build script working with arbitrary official testng distro you have > > to remove 'mixed' from java and testng tasks there > > Will this limitation ever go away? I see you have "remove junit from inside > testng jar" scheduled. I ask because I don't think people will want to dig into > "private" xml files. I hope that it will never go away and it gets included in the official TestNG distro. Why? Answer is at http://wiki.netbeans.org/TestNG_MixedMode
(In reply to comment #21) > the best and fastest would be to attach the project to this issue (or send it > to me offline) so I can reproduce the problem on my end... > > Have you tried the same with fresh new project with up-to date testng library? > If no, can you try that and let me know? (just run IDE with new userdir) I fixed the problem by uninstalling the plugin, manually removing changes to build-impl.xml and other files (yuck! this was very error-prone) and reinstalling the plugin. > > Instead, I recommend you scan "Test > > Packages" for the @Test annotation. Let me know what you think. > > wouldn't it be better if this could be done directly by TestNG itself? Agreed. > > > (In reply to comment #18) > > > -to get the build script working with arbitrary official testng distro you have > > > to remove 'mixed' from java and testng tasks there > > > > Will this limitation ever go away? I see you have "remove junit from inside > > testng jar" scheduled. I ask because I don't think people will want to dig into > > "private" xml files. > > I hope that it will never go away and it gets included in the official TestNG > distro. > > Why? Answer is at http://wiki.netbeans.org/TestNG_MixedMode Did you ask TestNG to integrate this contribution? Is there an official RFE I can vote for? We can leave this issue closed. I filed a separate bug report for debugging TestNG classes at issue 208105
(In reply to comment #22) > (In reply to comment #21) > I fixed the problem by uninstalling the plugin, manually removing changes to > build-impl.xml and other files (yuck! this was very error-prone) and > reinstalling the plugin. yes, this is known to be problematic with plugins from development UC :-( > > Why? Answer is at http://wiki.netbeans.org/TestNG_MixedMode > > Did you ask TestNG to integrate this contribution? Is there an official RFE I > can vote for? I've just announced it @ testng-dev list: https://groups.google.com/group/testng-dev/browse_thread/thread/2f67e742de14fbeb?hl=en so let's see > > We can leave this issue closed. I filed a separate bug report for debugging > TestNG classes at issue 208105 thanks, that's the best, I'll look at it
(In reply to comment #23) > (In reply to comment #22) > > (In reply to comment #21) > > I fixed the problem by uninstalling the plugin, manually removing changes to > > build-impl.xml and other files (yuck! this was very error-prone) and > > reinstalling the plugin. > > yes, this is known to be problematic with plugins from development UC :-( What do other plugins do? Shouldn't we have buttons for explicitly installing/uninstalling TestNG integration from a project? You could still implicitly install it if the user uses the File New wizard.
(In reply to comment #24) > (In reply to comment #23) > > (In reply to comment #22) > > > (In reply to comment #21) > > > I fixed the problem by uninstalling the plugin, manually removing changes to > > > build-impl.xml and other files (yuck! this was very error-prone) and > > > reinstalling the plugin. > > > > yes, this is known to be problematic with plugins from development UC :-( > > What do other plugins do? Shouldn't we have buttons for explicitly > installing/uninstalling TestNG integration from a project? You could still > implicitly install it if the user uses the File New wizard. basic trick here to update module's version after each change - I'm about to do that as soon as my proposed patch for testng will be either accepted or rejected - this should inform even released versions of IDE that there's updated version available
(In reply to comment #25) > basic trick here to update module's version after each change - I'm about to do > that as soon as my proposed patch for testng will be either accepted or > rejected - this should inform even released versions of IDE that there's > updated version available That would do it but I'd also like to have the ability to remove plugin integration with my project for good (say I no longer want to use TestNG). Web projects have a "Frameworks" tab in Project Properties. Couldn't we have the same thing for plugins?
You made a good point. But I'm thinking about bit different solution for this - w/o any UI - at least at the very beginning[*]. Removing TestNG (or any other test framework) from the project would require removing 3 lines from project.xml... More to come in a wiki ;-) [*] - Testing specific tab in project properties is maybe already filed somewhere in bugzilla (I'd start searching in junit or j2se project components) - again more about this will be in a wiki