This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 269912 - New TC centric view on Test Run
Summary: New TC centric view on Test Run
Status: STARTED
Alias: None
Product: qa
Classification: Unclassified
Component: Synergy (show other bugs)
Version: Dev
Hardware: PC Linux
: P1 normal (vote)
Assignee: Vladimir Riha
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-02-23 19:41 UTC by Jiri Kovalsky
Modified: 2017-05-14 08:42 UTC (History)
0 users

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Kovalsky 2017-02-23 19:41:00 UTC
What are in need to have another view on Synergy test runs which would compile a set of all test cases which were executed during a selected test run in one long document.

The view would only show the test cases which were really performed (both passed and failed) so skipped test cases will not be displayed and also those which do not match selected labels like "sanity", "acceptance" etc. will be ignored. The view would join all such test cases satisfying these criteria from all applicable test specifications into one long list and display them in this format:

ABCS 17.2.1 Sanity test run in Pre-Production
=============================================
Date: 2017-03-29 - 2017-03-29 (1 day)
Applied labels: sanity
Show test cases: passed [x] failed [x] skipped [ ]
[ ] Expand test cases

  1. Life Cycle Management test specification version 16.3.5

    1.1 Application Worklow test suite

      [+] Create Application test case | Jiří Prox | Internet Explorer on Windows 8.1 | Passed
      [+] Stage Application test case | Jiří Prox | Internet Explorer on Windows 8.1 | Passed
      [+] Publish Application test case | Jiří Prox | Internet Explorer on Windows 8.1 | Failed (#BUFP-10057)

    1.2 Multi User Workflow test suite

      [+] Add Multiple Users test case | Jiří Prox | Internet Explorer on Windows 8.1 | Passed

  2. Web Services test specification version 17.1.1

    2.1 Web Services Wizard test suite

      [+] Create HCM Business Object test case | Štěpán Žebra | Safari on Mac OS X 10.11 | Passed
      [+] Register Custom Web Service test case | Štěpán Žebra | Safari on Mac OS X 10.11 | Passed

etc.

Statistics:
===========
Pass rate: 100%, 14% with issues
Number of test cases: 423
Number of testers: 8
Total time: 25 hours and 29 minutes
Comment 1 Jiri Kovalsky 2017-02-23 19:42:58 UTC
Oh, and I forgot one important thing. This views would be either frozen or their static snapshots would be created upon freezing their test runs so that changing labels later will have zero impact on the view.
Comment 2 Vladimir Riha 2017-02-26 10:07:20 UTC
Jirko I think this is doable, I have a partially implemented it (not pushed yet). Right now I believe the backend part is done and part of client part (say the first easy 20%). I'll push the changes tomorrow so that backend will start collecting the required data and once I finish the UI, it can display also assignment finished after I pushed the backend changes.

The view will be limited though - it will only display data (no way to e.g. add/start/resume test assignment). I'm not yet sure how it will be linked to test run (maybe "Test Cases statistics") or something like that.
Comment 3 Jiri Kovalsky 2017-02-26 10:59:43 UTC
Thank you very much Lado for the good news! Let's link it as "Test Cases View" in the beginning. It's totally fine if it's just a read-only page displaying data with no controls except the checkboxes what should be visible.
Comment 4 Jiri Kovalsky 2017-03-20 16:58:28 UTC
I have looked at the deployed prototype and it looks amazing! Really great job Lado!

Just couple of comments:

1. Would it be possible to add the duration e.g. 6 days?
2. Statistic should be 100% in total i.e. Passed 95% (8% with issues), Failed 3%, Skipped 2%
3. Total time statistic is also missing
4. Could the "issues" link change to #BUFP-12345?

Once again wonderful work!
Comment 5 Vladimir Riha 2017-03-22 07:09:06 UTC
Thanks Jirko, I'll have a look at it probably on Friday. #1, #2 should be easily possible. #4 Could be possible, but the problem is there could be multiple issues and reported issues have a lot of forms - someone enters only number part, someone BUFP-NUMBER, someone BUFPNUMBER etc and also there could be multiple issues not only one... I'll do my best to improve it somehow albeit not sure how right now :)

Unfortunately #3 is not possible, this view uses different data than other test runs view and there is no information about duration. Could you please use this information from the "Statistics" view instead?
Comment 6 Vladimir Riha 2017-03-30 07:22:18 UTC
Jirko I've deployed the latest and greatest to our internal server, it should cover #1, #2 and partially #4. I'll look at #3 one more time during weekend to see what I can do about it (I'm slightly optimistic :))

Why #4 is partial:
 - if there are 2 or more issues, it will display the "issues" as until now
 - if there is only 1 issue, it will display exactly what the tester entered during his test assignment. So please use the format BUFP-XXXX when entering issues during testing. I can see that in 17.2.1 test run, there are some entered as BUFP-XXXX, but some only as XXXX

Thank you
Comment 7 Jiri Kovalsky 2017-05-14 08:42:47 UTC
Thank you Lado. The view looks really great now! As for the issues, can't you improve it this way?

12345 => BUFP-12345 (starts with number, prefix with BUFP-)
BUFP-12345 => BUFP-12345 (ideal case, do nothing)
BUFP12345 => BUFP-12345 (matches BUFP([0-9]*) regexp, insert dash between BUFP and $1)
MAX-54321 => MAX-54321 (anything else, leave as it is)