This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
If you encounter a wrong testcase while running a testcase, currently I can skip or fail the test. But the information that the testcase is invalid is lost. How about another state like "test case is wrong. Needs an update"? Or if you skip an entry you have to choose a reason "test case is wrong", "cannot execute testcase" ... from a dropdown?
Good idea, it may help to review and fix better the test case
Sorry validating too quickly. Event if green (passed) comment may be a cool think to have. (free text, dropdown, ...)
*** Bug 240313 has been marked as a duplicate of this bug. ***
Having such functionality is A but we also need the B i.e. what will happen to the gathered comments, who will process it etc.
True, I was only thinking about generating "comments" page to test run with all comments made to particular cases during testing so it would not affect specification as such. After that, someone would have to go through these comments (specification owner, anyone?) and resolve them
I was thinking more along the lines of a code table, but then it occurred to me that the only reasons I could think of were along the lines of: Unclear or terse instructions / Don't understand Steps are missing or incorrect for my Operating System Steps are missing or incorrect in general Other: [comment] Any of which should indicate that the owner of the test specification in question should review that particular test case with a fine eye.
Hi I was thinking on maybe complementary idea with #240258. 1) I imagine skipping a test A knowing it will take 30min and go to test B to have a maximum of coverture due to personnal schedule. 2) I skip the test A saying free text or a list as proposed by Lou. Deadline is not reached: 3a) I use #240258 to replay A, it complete it, it remove my now useless reason to skip A. Deadline is over: 3b) reason is kept on test A for final review.
I'd like to avoid free text option (it would allow accurate filtering in page with comments) , so here's a couple of possible predefined comments (thanks Lou and Eric): Unclear instructions / Don't understand Steps are missing or incorrect for my Operating System Steps are missing or incorrect in general Depends on previous case(s) which failed Missing sample file(s) required in test case Test case is obsolete No time to finish test case Any other ideas?
In debugger we had two almost identical test cases so something like "Duplicate test case" would be handy.
Thank you, I'll add it as well. I just pushed changes to repo (live in 15 minutes) which force user to provide reason for skipping case and save all comments to DB for further processing. TODO: page with comments + process to resolve them
Fixed, changes will be live in 15 minutes. Each test run has a link to comments page with list of comments added to test cases during testing. Anyone can mark the comment as "resolved" to indicate that issue with case has been resolved (since anyone can edit specification, no reason to limit resolve action to specification owners). User is also asked if he/she wants to resolve all similar comments as well (similar = same test case, same test suite and same comment). Comment can be added not only to skipped cases, but also for passed or failed (but it is required for skipped ones) There is no way to "reopen" resolved comment as I don't think it is necessary for this use case and I don't think creating "bugzilla light" in Synergy is a good idea.