This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 271378 - Incorrect result bar for Suite (with @DataProvider)
Summary: Incorrect result bar for Suite (with @DataProvider)
Status: NEW
Alias: None
Product: java
Classification: Unclassified
Component: TestNG (show other bugs)
Version: 8.2
Hardware: PC Windows 10 x64
: P3 normal (vote)
Assignee: Theofanis Oikonomou
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-08-27 15:04 UTC by bytesArray
Modified: 2017-08-27 15:07 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
TestNg with dataprovider (4.36 KB, application/octet-stream)
2017-08-27 15:04 UTC, bytesArray
Details
first screen shot (67.38 KB, image/png)
2017-08-27 15:05 UTC, bytesArray
Details
second screenshot (66.55 KB, image/png)
2017-08-27 15:07 UTC, bytesArray
Details

Note You need to log in before you can comment on or make changes to this bug.
Description bytesArray 2017-08-27 15:04:36 UTC
Created attachment 165020 [details]
TestNg with dataprovider

Many things are not consistent in this verry little tryout of TestNG and it's @Dataprovider capabilities.

The test sources codes and 2 screenshots are attached.

the (stupid) test case is a digest oy my initial goal.
It builds a static dataset of pictures and their attributes,
and do some validations.

I noticed weird behaviours. (Green bar while the output status is that some tests FAILED ??)
Thought I was doing something wrong. 
Loose a couple of hours on this , eventually reaching the TestNG index.html in my /AppData/Local/Temp/.

So I ended up with the simple test attached, using an  Assert.assertTrue(currentBool);
the boolean coming from an getRandomBoolean() method.

As what is seems :
1st Defect :
-----------
 if the first test in the suite initiated via an dataprovider is OK, Whatever comes next the bar will be green ! 
Plus some other heuristics I didn't get.. @ some point the current netbeans' module code just don't take in account the fact that in a suite all tests must be OK before you can set the global status to PASSED. (and disrespect the TestNG's output)

2nd defect :
------------
Complete INCOHERENCE in the label :
"NO test passed, 1 test failed " ??? there was 9 tests, 4 passed.
the run after , thanks to the assertTrue based on  getRandomBoolean()'s value
Netbeans claims that "THE" test passed : there was not just a sole unique test. Do you mean "the SUITE" passed ?? (All this not caring about the fact that it failed in the 1st place)
Comment 1 bytesArray 2017-08-27 15:05:47 UTC
Created attachment 165021 [details]
first screen shot
Comment 2 bytesArray 2017-08-27 15:07:29 UTC
Created attachment 165022 [details]
second screenshot