Changeset: f06725cb6499
Author:    Theofanis Oikonomou <theofanis@netbeans.org>
Date:      2014-12-19 23:05
Message:

(In reply to netbeansuzr from comment #0)
> When running a Suite test from NetBeans groups all test methods under the
> Suite node.  It instead should use the following hierarchy:
> 
> Desired behaviour:
> 
> Suite
>    TestClass1
>       testClass1TestMethod1
>       testClass1TestMethod2
>    TestClass2
>       testClass2TestMethod1
>       testClass2TestMethod2
> 
> Current behaviour:
> 
> Suite
>    testClass1TestMethod1
>    testClass1TestMethod2
>    testClass2TestMethod1
>    testClass2TestMethod2

current behaviour was actually

Suite
    TestMethod1
    TestMethod2
    TestMethod1
    TestMethod2

> 
> The current output is confusing and not very useful.  It's unclear which
> tests belong with which test class.  I tried playing with making custom
> Suites and runners but I was not able to change this behaviour.
> 
> This becomes compounded even more when one suite includes another suite,
> which then includes test classes.  The output is the same as above with no
> categorization or grouping of tests.  This makes test suites pretty useless
> as it's hard to parse the output from running them.

which is indeed confusing especially when there are more test cases in a suite with the same test method names. I enhanced the behaviour so it looks like:

Suite
    testClass1TestMethod1
    testClass1TestMethod2
    testClass2TestMethod1
    testClass2TestMethod2

when running a suite file and like:

Testcase
    TestMethod1
    TestMethod2
    TestMethod1
    TestMethod2

when running a test file. Double-clicking on any leaf node will navigate to the respective test method. I hope now it is more usable. Thank you for reporting

Thank you very much.  That should do the trick.

Just two minor suggestions for improvement:
1) can you add an _ or - between the test class name and the test method name or a . or some sort of separator
2) can you sort the results in alphabetical order (this would work), or more precisely, sort them so that tests from the same class are grouped together.

It would also be nice if I could run all my unit tests for all my projects at the same time.  It's too much work every time to select each and every project and then hit run.  I think the output shows up in separate tabs in this case anyway and I don't want to have to look at a ton of tabs to find out if all the unit tests passed.

Right now I have some of my unit tests in a test only project just so that I can work around this issue, but this is not ideal as it's not truthful to what's going on.  The tests below with each project and I shouldn't have to create a test only project to work around this issue.

part 1) A reasonable solution might be to allow unit tests to be inherited from different projects just like source code and have all the unit tests compiled classes copied to the test projects build directories so they can be found by a suite in the test project.  This would allow running a single suite that finds all tests to find tests from different projects.

part 2) A step above this would be if project dependencies would also inherit libraries so that I wouldn't have to manually add the libraries to the test project that each project depends on.

I know some of this would be more work, but it would be a really useful feature and would make NetBeans so much better for testing.  Anything you could do to get closer to this feature over time and if you could do something now to make it workable (eg. implement part 1 now) would be much appreciated.

Integrated into 'main-silver', will be available in build *201412210001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/f06725cb6499
User: Theofanis Oikonomou <theofanis@netbeans.org>
Log: Task #249124 - The JUnit test output window doesn't display nested tests.  This can most commonly be seen when running Suite tests