Bug 171050

Note: these options are not available for PASSED nodes. For failed it works correctly.

I am attaching screenshot that compares the two situations. 

Created attachment 86767 [details]
Different actions in Results wndow

*** Issue 174296 has been marked as a duplicate of this issue. ***

Another inconsistency reported by user in issue 174269

This problem makes new features of Test Results window difficult to use, or is at least quite unpractical and confusing
-> raising to P2

Seems it is regression after a fix in the file
java.source.ant\src\org\netbeans\modules\java\source\ant\resources\junit-snippet.xml where functionality for generating
of the junit report in the xml form was removed.

Jesse,

Could you please, devote one's attention to this issue. 

Current implementation of the JUnit support provides generating of the JUnit results that can be based on either "brief"
or "xml" results, but if the "xml" results are not present then important functionality of the Test Result Window will
be lost. 

See the Issue 160777. Probably, we need review the fix and provide generating of the "xml" results in a temporary
directory so that it will be accessible for the JUnit support.

---
Victor

P.S. I think, the best solution in this area is to use the JUnit API [1] to run tests and obtain results instead of
parsing the output files, but it is not a way for the NB6.8 release.

[1] http://junit.org/apidocs/index.html?overview-summary.html

The temp file trick as used in junit-debug-snippet.xml could probably be used in junit-snippet.xml as well. Not clear to
me whether you can delete ${test.result.file} at the end of the target in this case. Probably a SIMPLEFIX.

If a client launches a junit target then it means that the client have an interest to the test results, and the results
must not be deleted during or immediately after executing of the target. The client is completely responsible in
removing requested results if there are no longer any needs in these results.
Nevertheless, functionality for removing results should be located in the same unit that creates these results.
Therefore, additionally to each target which launches test(s) a complementary target cleaning the test(s) results should
be implemented. The client must use such targets to clean all test results instead of singly trying to delete the results.
Note, client's strategies against when the test results will be deleted may be different.

Well in this case java.source.ant (the "client" I guess) is forced to produce XML output only for the benefit of the
junit module, this output must be in some temp dir (not inside the project directory), and there is no obvious hook for
it to clean up the result because it cannot know when the junit module has finished parsing this output.

If the junit module in fact parsed the XML file synchronously during the callback from AntLogger indicating that the
<junit> task has finished, then java.source.ant could simply <delete> the file immediately after <junit> in the same
target and all would be well. I just recall that this did not work in the case of junit-debug-snippet.xml; only the
junit module owner would be able to say why.

I guess, if we add back the <formatter type="xml"/> element to the <junit> task in the junit-snippet.xml file then it
will be enough for resolving this issue. I'll attach the issue-171050_junit-snippet.xml.patch
I've tested this fix against both project types: ant and maven. Seems all is OK, because in all cases a temporary XML
report has not been created in the project folders.

Jesse, please validate this solution, and apply the patch if you agree.

About deleting temporary XML reports... I think, adding of the <delete> task immediately after the <junit> task is not a
good idea, because in result we'll have a strange target that should be renamed to "do nothing", and intention of the
junit-snippet.xml will be unclear.

I guess, in all cases when a temporary XML report is created a client is the junit module. Hence, I've another solution.
The issue-171050_JUnitOutputReader.java.patch will be attached. 
Note, execution of the action "Test Alt+F6" at project node's context is a case when a client is a IDE user. In this
case, XML report will be created under the project folder, and it won't be deleted, because it is not a temporary report.   
If you agree then I'll fix it in this manner. 

Created attachment 89577 [details]
Please, review and apply this patch against the java.source.ant module

Created attachment 89578 [details]
Please, approve this fix

issue-171050_junit-snippet.xml.patch would seem to regress issue #160777: TEST-*.xml would be created in the project
directory (when using Ctrl-F6 with CoS enabled, say in an autoproject), in which case reportPath.startsWith(projectPath)
would be true so it would be left there.

As to issue-171050_JUnitOutputReader.java.patch, this seems unsafe. It is perfectly acceptable for a custom Ant script
to generate reports to some other directory, say a group build directory used by a set of related projects; this patch
would delete them without warning, possibly causing a <junitreport> later in the script to break.

Again I recommend that the style used in junit-debug-snippet.xml, that of creating a report in the temp dir, be copied
to junit-snippet.xml, as an immediate fix; and that we look for a way to safely delete the resulting file in both of
these cases. Ideally you could <delete> it right after <junit> in the script itself, but again this presumes that
parsing of the report is done synchronously by the junit module's AntLogger while receiving the task finished event for
<junit>; last I checked (when committing 918b569e5a3d) this did not happen and the JUnit module was confused by the
deleted report, perhaps because it runs parsing asynchronously.

BTW never use FileObject.getPath to get a File (or its path) from a FileObject. Use FileUtil.toFileObject.

OK. Two patches issue-171050_java.source.ant.patch and issue-171050_junit.patch provide the fix exactly in accord with
your suggestions.
The JUnit module provides synchronous parsing of the report, the Java Source to Ant Bindings module deletes an XML
report immediately after creating in both modes, i.e. in normal testing and in debugging of the tests.  

Could you please, apply the patches if you have no objections more, or give me know if I can singly complete this.

Created attachment 89764 [details]
The patch for the java.source.ant module

Created attachment 89765 [details]
The patch for the junit module

*** Issue 174173 has been marked as a duplicate of this issue. ***

junit part of patch looks fine, as well as junit-debug-snippet.xml.

For junit-snippet.xml, the patch can be considerably simplified. You can still use <junit> (no need to switch to
<java>); the todir attr of <test> can be set to ${java.io.tmpdir}.

OK. The issue-171050_java.source.ant_v2.patch uses the junit task.

Created attachment 89868 [details]
The patch for the java.source.ant module (version 2)

Looks good to me. If it works well in your manual tests, you should go ahead and commit (preferably as a single
changeset). Not sure if there is any applicable automated test infrastructure for this; would probably belong in the
junit module if so.

Fixed in the main trunk
http://hg.netbeans.org/main/rev/a9b198488c7d

Integrated into 'main-golden', will be available in build *200910221401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/a9b198488c7d
User: Victor G. Vasilyev <vvg@netbeans.org>
Log: #171050 - Inconsistent behavior of Test Result window

*** Issue 166682 has been marked as a duplicate of this issue. ***

*** Issue 162847 has been marked as a duplicate of this issue. ***

*** Issue 167429 has been marked as a duplicate of this issue. ***