This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 234353 - Strange test results when running TestNG test suite for NB Module project
Summary: Strange test results when running TestNG test suite for NB Module project
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: TestNG (show other bugs)
Version: 7.4
Hardware: PC Windows 7
: P3 normal (vote)
Assignee: Theofanis Oikonomou
URL:
Keywords: API, API_REVIEW_FAST
Depends on:
Blocks:
 
Reported: 2013-08-13 22:03 UTC by František Mantlík
Modified: 2013-09-01 01:26 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Screenshot of the unexpected result (321.82 KB, image/png)
2013-08-13 22:03 UTC, František Mantlík
Details
proposed fix (8.86 KB, patch)
2013-08-23 14:26 UTC, Theofanis Oikonomou
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description František Mantlík 2013-08-13 22:03:34 UTC
Created attachment 138662 [details]
Screenshot of the unexpected result

When "Test file" is invoked from a TestSuite.xml file context menu of a NB Module project with TestNG testing infrastructure, strange test result is reported.

How to reproduce:
================

1. Create new Netbeans Module project with all settings on their default values.

2. Copy the whole contents of the package "teststorun" from the sample Jave SE application http://wiki.netbeans.org/wiki/images/c/c0/JUnitSampleProject_TS_68_JUnit.zip into Source Packages of the newly created project.

3. Right-click the "teststorun" package in the Projects menu and select Tools - Create tests. Select TestNG framework. Keep all checkboxes checked.

4. Invoke Test (Alt+F6) from the context menu of the project. Test runs and Test Results window opens. All 10 tests failed (which is expected result).

5. Invoke "Test file" context menu item from the TestSuite.xml file generated in the third step.

Expected result:
===============

The same test should run once more producing the same result, i.e. 10 failed tests should be reported in the Test Results window.

What happend:
============

TestNGException is reported in the Output window and the Test Results window reports: "28 tests passed, 10 tests failed". See attachment.

Notes:
=====

Tested on Windows 7 64 bit, JDK 1.7.0_25, NB7.4 Development build 201308112300.

The problem was discovered when following the http://wiki.netbeans.org/TS_72_UnitTesting - Test Suite 1.6 TestNG.
Comment 1 František Mantlík 2013-08-13 22:08:21 UTC
One more note:

Invoking "Debug test file" context menu item on TestSuite.xml produces no action and "Target "debug-test" does not exist in the project." message appears in the Output window.
Comment 2 Theofanis Oikonomou 2013-08-23 14:26:42 UTC
Created attachment 139140 [details]
proposed fix

The attached patch adds support for "Test File" and "Debug Test File" actions in the popup menu of a TestNG suite xml file. It also takes care of correctly utilizing the rerun action in the Test Results Window's toolbar. Finally some info about the extra options for test.includes and test.class properties is added in the apisupport.harness readme file.
Comment 3 Theofanis Oikonomou 2013-08-23 14:29:35 UTC
Jardo I do not think this needs an api review. If you agree I will integrate on Monday. Thank you
Comment 4 Theofanis Oikonomou 2013-08-26 08:36:50 UTC
Starting a review for the changes made in templates/common.xml script. 

The change is needed so that the "Test File" and "Debug Test File" actions for a TestNG suite xml file in nb projects can work. The solution is to use the already existing 'test.includes' and 'test.class' properties to carry the suite's name. These are then used by the script to determine which target to call. 

Please review the change. Thank you
Comment 5 Theofanis Oikonomou 2013-08-29 14:24:22 UTC
If there are no objections I will integrate tomorrow. Thank you
Comment 6 Theofanis Oikonomou 2013-08-30 10:13:35 UTC
no comments so far. I will integrate later today as this is needed for 7.4 release. Thank you
Comment 7 Theofanis Oikonomou 2013-08-30 13:21:08 UTC
Fixed: http://hg.netbeans.org/core-main/rev/3c279fcedfe8
Comment 8 Quality Engineering 2013-09-01 01:26:50 UTC
Integrated into 'main-silver', will be available in build *201309010001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/3c279fcedfe8
User: Theofanis Oikonomou <theofanis@netbeans.org>
Log: #234353 - Strange test results when running TestNG test suite for NB Module project