This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Compilation errors are shown in results page but test cases which should run are not reported. Also if a test case crashes, it doesn't report that next tests didn't run.
I will consider this comment from jtulach: Btw. for the compilation failure problem: The simplest solution could be if each xtest run would have one artifitial test called compilation. Then there would be no need for changes in the beetle:8080/unittest just in the xtest way of producing output.
Comments per jskrivanek question: As it is stated there are 2 kind of problems. 1. Uncompiled tests 2. Critical problems during execution resulting in skipping certain amount of tests I don't know good solution for the first one except result logging infrastructure. In that case any compilation errors directly reflect in decreasing numbers of executed tests(if 2nd problem is solved) Second problem can be solved in Xtest infrastructure by adding all missed tests manually. Sequence: 1. Execute tests and get execution result XMLs 2. Use cfg-*.xml and test classes to build original set of tests (in other words set of tests that has to be executed) 3. Add all missed records into result XMLs with result "error" and message "Did Not Start" After step 3 result XMLs can be processed in usual ways.
Both problems fixed in trunk (XTest version 1.5.52). 1) Uncompiled tests /cvs/xtest/src/org/netbeans/xtest/NbExecutor.java,v <-- NbExecutor.java new revision: 1.29; previous revision: 1.28 If an unexpected critical error appears, it creates a new test bag results. In most case it is a compilation error but it can be something different. This test bag contains one suite with one test case. It satisfies that critical error appears in summary results. 2) Skipped and unfinished tests: /cvs/xtest/src/org/netbeans/xtest/testrunner/XMLReporter.java,v <-- XMLReporter.java new revision: 1.14; previous revision: 1.13 In method startTestSuite we create list of all test cases which should be executed. All test cases are initially signed as not started. Then in startTest right test case is found and it is signed as unfinished. Finally correct status is set in method endTest, addFail or addError. If a test case crashed somewhere in the middle its status remains "Did not start" or "Did not finished".
*** Issue 85689 has been marked as a duplicate of this issue. ***
I added tests: /cvs/xtest/test/unit/src/org/netbeans/xtest/test/UncompilableTest.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/CheckUncompilableTest.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/UnfinishedTest.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/CheckUnfinishedWithSetupTest74360.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/SpeedSuiteTest75001.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/CheckUnfinishedTest.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/CheckSpeedSuiteTest75001.java /cvs/xtest/test/unit/src/org/netbeans/xtest/test/UnfinishedWithSetupTest74360.java