Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 171050 - Inconsistent behavior of Test Result window
Inconsistent behavior of Test Result window
Status: RESOLVED FIXED
Product: java
Classification: Unclassified
Component: JUnit
6.x
All All
: P2 (vote)
: 6.x
Assigned To: Victor Vasilyev
junit-issues@java
67patch-candidate
:
: 162847 166682 174173 174296 (view as bug list)
Depends on: 160777
Blocks: 174173
  Show dependency treegraph
 
Reported: 2009-08-28 08:45 UTC by pribyl
Modified: 2009-10-30 12:09 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
:


Attachments
Different actions in Results wndow (33.09 KB, image/png)
2009-08-28 08:48 UTC, pribyl
Details
Please, review and apply this patch against the java.source.ant module (768 bytes, patch)
2009-10-16 00:29 UTC, Victor Vasilyev
Details | Diff
Please, approve this fix (1.61 KB, patch)
2009-10-16 00:30 UTC, Victor Vasilyev
Details | Diff
The patch for the java.source.ant module (5.03 KB, patch)
2009-10-20 14:05 UTC, Victor Vasilyev
Details | Diff
The patch for the junit module (890 bytes, patch)
2009-10-20 14:06 UTC, Victor Vasilyev
Details | Diff
The patch for the java.source.ant module (version 2) (3.62 KB, patch)
2009-10-21 14:42 UTC, Victor Vasilyev
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description pribyl 2009-08-28 08:45:04 UTC
Product Version: NetBeans IDE Dev (Build 200908270201)
Java: 1.6.0_15; Java HotSpot(TM) Client VM 14.1-b02
System: Windows XP version 5.1 running on x86; Cp1250; cs_CZ (nb)

Test Result window offers options like "Rerun" and "Debug" for all nodes (passed and failed) if the whole project is
tested (invoke Test on project's root node or press Alt+F6).
However if only a single file test (or test suite) is run, then those options are not available in the popup menu. It
offers only "Go to Source" that does not work.
 
I am sorry if the problem is already reported, it seems it exist for a quite a long time. But I was not able to find
such issue :-/ The tested project has Compile on Save ENABLED, so it is not the problem described in issue 161283
Comment 1 pribyl 2009-08-28 08:47:16 UTC
Note: these options are not available for PASSED nodes. For failed it works correctly.

I am attaching screenshot that compares the two situations. 
Comment 2 pribyl 2009-08-28 08:48:17 UTC
Created attachment 86767 [details]
Different actions in Results wndow
Comment 3 pribyl 2009-10-13 13:09:51 UTC
*** Issue 174296 has been marked as a duplicate of this issue. ***
Comment 4 pribyl 2009-10-13 13:14:29 UTC
Another inconsistency reported by user in issue 174269

This problem makes new features of Test Results window difficult to use, or is at least quite unpractical and confusing
-> raising to P2
Comment 5 Victor Vasilyev 2009-10-13 13:30:58 UTC
Seems it is regression after a fix in the file
java.source.ant\src\org\netbeans\modules\java\source\ant\resources\junit-snippet.xml where functionality for generating
of the junit report in the xml form was removed.
Comment 6 Victor Vasilyev 2009-10-13 14:43:52 UTC
Jesse,

Could you please, devote one's attention to this issue. 

Current implementation of the JUnit support provides generating of the JUnit results that can be based on either "brief"
or "xml" results, but if the "xml" results are not present then important functionality of the Test Result Window will
be lost. 

See the Issue 160777. Probably, we need review the fix and provide generating of the "xml" results in a temporary
directory so that it will be accessible for the JUnit support.

---
Victor

P.S. I think, the best solution in this area is to use the JUnit API [1] to run tests and obtain results instead of
parsing the output files, but it is not a way for the NB6.8 release.

[1] http://junit.org/apidocs/index.html?overview-summary.html
Comment 7 Jesse Glick 2009-10-13 17:38:26 UTC
The temp file trick as used in junit-debug-snippet.xml could probably be used in junit-snippet.xml as well. Not clear to
me whether you can delete ${test.result.file} at the end of the target in this case. Probably a SIMPLEFIX.
Comment 8 Victor Vasilyev 2009-10-14 11:04:39 UTC
If a client launches a junit target then it means that the client have an interest to the test results, and the results
must not be deleted during or immediately after executing of the target. The client is completely responsible in
removing requested results if there are no longer any needs in these results.
Nevertheless, functionality for removing results should be located in the same unit that creates these results.
Therefore, additionally to each target which launches test(s) a complementary target cleaning the test(s) results should
be implemented. The client must use such targets to clean all test results instead of singly trying to delete the results.
Note, client's strategies against when the test results will be deleted may be different.
Comment 9 Jesse Glick 2009-10-14 15:42:11 UTC
Well in this case java.source.ant (the "client" I guess) is forced to produce XML output only for the benefit of the
junit module, this output must be in some temp dir (not inside the project directory), and there is no obvious hook for
it to clean up the result because it cannot know when the junit module has finished parsing this output.

If the junit module in fact parsed the XML file synchronously during the callback from AntLogger indicating that the
<junit> task has finished, then java.source.ant could simply <delete> the file immediately after <junit> in the same
target and all would be well. I just recall that this did not work in the case of junit-debug-snippet.xml; only the
junit module owner would be able to say why.
Comment 10 Victor Vasilyev 2009-10-16 00:24:24 UTC
I guess, if we add back the <formatter type="xml"/> element to the <junit> task in the junit-snippet.xml file then it
will be enough for resolving this issue. I'll attach the issue-171050_junit-snippet.xml.patch
I've tested this fix against both project types: ant and maven. Seems all is OK, because in all cases a temporary XML
report has not been created in the project folders.

Jesse, please validate this solution, and apply the patch if you agree.

About deleting temporary XML reports... I think, adding of the <delete> task immediately after the <junit> task is not a
good idea, because in result we'll have a strange target that should be renamed to "do nothing", and intention of the
junit-snippet.xml will be unclear.

I guess, in all cases when a temporary XML report is created a client is the junit module. Hence, I've another solution.
The issue-171050_JUnitOutputReader.java.patch will be attached. 
Note, execution of the action "Test Alt+F6" at project node's context is a case when a client is a IDE user. In this
case, XML report will be created under the project folder, and it won't be deleted, because it is not a temporary report.   
If you agree then I'll fix it in this manner. 

Comment 11 Victor Vasilyev 2009-10-16 00:29:14 UTC
Created attachment 89577 [details]
Please, review and apply this patch against the java.source.ant module
Comment 12 Victor Vasilyev 2009-10-16 00:30:59 UTC
Created attachment 89578 [details]
Please, approve this fix
Comment 13 Jesse Glick 2009-10-16 17:29:14 UTC
issue-171050_junit-snippet.xml.patch would seem to regress issue #160777: TEST-*.xml would be created in the project
directory (when using Ctrl-F6 with CoS enabled, say in an autoproject), in which case reportPath.startsWith(projectPath)
would be true so it would be left there.

As to issue-171050_JUnitOutputReader.java.patch, this seems unsafe. It is perfectly acceptable for a custom Ant script
to generate reports to some other directory, say a group build directory used by a set of related projects; this patch
would delete them without warning, possibly causing a <junitreport> later in the script to break.

Again I recommend that the style used in junit-debug-snippet.xml, that of creating a report in the temp dir, be copied
to junit-snippet.xml, as an immediate fix; and that we look for a way to safely delete the resulting file in both of
these cases. Ideally you could <delete> it right after <junit> in the script itself, but again this presumes that
parsing of the report is done synchronously by the junit module's AntLogger while receiving the task finished event for
<junit>; last I checked (when committing 918b569e5a3d) this did not happen and the JUnit module was confused by the
deleted report, perhaps because it runs parsing asynchronously.

BTW never use FileObject.getPath to get a File (or its path) from a FileObject. Use FileUtil.toFileObject.
Comment 14 Victor Vasilyev 2009-10-20 14:03:39 UTC
OK. Two patches issue-171050_java.source.ant.patch and issue-171050_junit.patch provide the fix exactly in accord with
your suggestions.
The JUnit module provides synchronous parsing of the report, the Java Source to Ant Bindings module deletes an XML
report immediately after creating in both modes, i.e. in normal testing and in debugging of the tests.  

Could you please, apply the patches if you have no objections more, or give me know if I can singly complete this.
Comment 15 Victor Vasilyev 2009-10-20 14:05:28 UTC
Created attachment 89764 [details]
The patch for the java.source.ant module
Comment 16 Victor Vasilyev 2009-10-20 14:06:30 UTC
Created attachment 89765 [details]
The patch for the junit module
Comment 17 Victor Vasilyev 2009-10-20 15:53:03 UTC
*** Issue 174173 has been marked as a duplicate of this issue. ***
Comment 18 Jesse Glick 2009-10-20 17:59:15 UTC
junit part of patch looks fine, as well as junit-debug-snippet.xml.

For junit-snippet.xml, the patch can be considerably simplified. You can still use <junit> (no need to switch to
<java>); the todir attr of <test> can be set to ${java.io.tmpdir}.
Comment 19 Victor Vasilyev 2009-10-21 14:38:47 UTC
OK. The issue-171050_java.source.ant_v2.patch uses the junit task.
Comment 20 Victor Vasilyev 2009-10-21 14:42:33 UTC
Created attachment 89868 [details]
The patch for the java.source.ant module (version 2)
Comment 21 Jesse Glick 2009-10-21 15:46:25 UTC
Looks good to me. If it works well in your manual tests, you should go ahead and commit (preferably as a single
changeset). Not sure if there is any applicable automated test infrastructure for this; would probably belong in the
junit module if so.
Comment 22 Victor Vasilyev 2009-10-21 17:16:09 UTC
Fixed in the main trunk
http://hg.netbeans.org/main/rev/a9b198488c7d
Comment 23 Quality Engineering 2009-10-22 23:54:24 UTC
Integrated into 'main-golden', will be available in build *200910221401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/a9b198488c7d
User: Victor G. Vasilyev <vvg@netbeans.org>
Log: #171050 - Inconsistent behavior of Test Result window
Comment 24 Victor Vasilyev 2009-10-27 17:51:45 UTC
*** Issue 166682 has been marked as a duplicate of this issue. ***
Comment 25 Victor Vasilyev 2009-10-28 22:36:16 UTC
*** Issue 162847 has been marked as a duplicate of this issue. ***
Comment 26 Victor Vasilyev 2009-10-30 12:09:01 UTC
*** Issue 167429 has been marked as a duplicate of this issue. ***


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo