This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I am working on a project with almost 400000 LOC, more thant 1000 unit tests and 100s of GUI tests. When I run a test suite, the result tree is displayed only when the full test suite is finished. The tree should be updated as soon a the result of a test is available. For example, if you have bugs like an infinite loop or a GUI test stuck due to a threading issue, you have to kill the unit test and you never know were is the problem. For the unit and integration tests in my project, it takes 10 minutes and for a GUI test it takes 16 minutes. When all the tests are failing since the first seconds, it is useful to know it immediately so you don't have to wait until the end. I can add that the filter to show failure doesn't work. Moreover, the history of previous tests is missing so when I have 5 tests on 1000 that fails, I have to write down these 5 tests to remember them before fixing and running each one individually. To summarise, I love NetBeans but I cannot use it in my company because the unit test support is too poor. Eclipse is far ahead in this area :(
Accepted. This is definitely doable but I do not have enough time to implement it is NB 6.5 (too many bugs to fix).
Removing the UI keyword, if any specific attention of UI/HIE folks is needed, please let me know.
Correct,no attention to the specific attention of UI/HIE folks is needed, the usability is more the point. Do you need that I write a complete use case to have a better idea for a version 7 of a cool unit test feature?
Probably a question for mpetras.
I think the use case for the requested enhancement is descriptive enough. But if you sum up all the weaknesses of the unit test support that prevent you from using NetBeans in your company, I will definitely appreciate it.
The main weaknesses of the current unit test support are: - unable to see the result of each test as soon as the test is executed so you can interrupt as soon as you see a critical error - unable to keep the history of previous run including the interrupted ones - unable to run only the failing tests from a previous test suite from the history (see issue 84523) - unable to run only one test method in a test case from the history (useful when you have one test for a core behaviour that fails and most of the tests depend on this core behaviour so you need to fix first the core behaviour) Sorry for this late answer, I was away for a while.
Something simple but very useful: a counter with the number of tests runs/total, Errors count and Failures count. When I see the failure counter going wild while the test is executing, I know that the test can be interrupted immediately. I tested the version 6.5RC2, it contains some very good improvements that are part of this request: the result is displayed even I interrupt the test suite, the 'show failure only' filter is working. I can rerun a method by right clicking but only when I create the test from a new project using JUnit4.
Integrated into 'main-golden', will be available in build *200812221122* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/9211feffa130 User: Michael Chernyshov <cms@netbeans.org> Log: #145850 [JUnit must display the result tree while executing a test suite]
The issue itself is fixed but the fix caused a regression: numbers of failed/passed/total tests are not calculated properly now. Need to synchronize test execution and results exposing threads. Will mark the request as "fixed" when synchronization is added (in progress currently).
Integrated into 'main-golden', will be available in build *200812260201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/d25dbf986339 User: Michael Chernyshov <cms@netbeans.org> Log: #145850 fix, part two (fixed regression in incorrect test numbers calculation)
Regressions fixed. Since other requests mentioned here are covered by another enhancement issues, closing this one as fixed.