Bug 150613 - NetBeans doesn't report "Shoulda" tests in its output unless they fail
NetBeans doesn't report "Shoulda" tests in its output unless they fail
Status: VERIFIED FIXED
Product: ruby
Classification: Unclassified
Component: Testing
6.x
Macintosh Mac OS X
: P3 (vote)
: 6.x
Assigned To: Erno Mononen
issues@ruby
65fixes2-verified
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-10-18 16:45 UTC by bkocik
Modified: 2009-02-19 23:01 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description bkocik 2008-10-18 16:45:24 UTC
When using the "Shoulda" testing framework (http://thoughtbot.com/projects/shoulda), NetBeans will run your Shoulda
tests along with your traditional tests, but doesn't report on the Shoulda tests unless they fail.

To repro:

Create a project and install Shoulda (I'm doing the following with Ruby 1.8.7p72 and Rails 2.1.0):
$ rails nb_shoulda
$ cd nb_shoulda
$ script/generate controller main index
$ cd vendor/plugins
$ git clone git://github.com/thoughtbot/shoulda.git shoulda
$ cd ../..

Create a shoulda test in test/functional/main_controller.rb that looks like this:

  context "when index is called" do
    setup {get :index}
    should_respond_with :success
  end

Run the functional tests:
$ rake test:functionals

Note the test count in the output:

2 tests, 2 assertions, 0 failures, 0 errors

Now load that up as a project in NetBeans, and run the tests with NB's "Run/Debug Rake Task..." feature. Note the test
count in the output:

1 tests, 0 failures, 0 errors

NB is not counting the Shoulda test that passed. But it is running it, and it knows if it fails. You can verify by
changing the Shoulda test to expect, for example, HTTP 403 from the index action so that it will fail:

  context "when index is called" do
    setup {get :index}
    should_respond_with :forbidden
  end

Run the tests from within NB again, and note the count in the output. NetBeans is now reporting on the test, because it
failed:

2 tests, 1 failures, 0 errors
Comment 1 Erno Mononen 2008-10-18 22:03:42 UTC
Thanks for the report. Seems to be caused by that Shoulda uses a bit different format for notifying listeners 
registered for the TestCase::STARTED and FINISHED events. I have a fix ready, will push to the repo tomorrow or on 
Monday.
Comment 2 Erno Mononen 2008-10-20 09:16:29 UTC
Should be fixed now, changeset #1973dfdea1ca. There will be an automatic notification in this issue when the fix has 
been integrated. Please let use know if you find any other cases where Shoulda tests are not working, it is very much 
possible that there are such cases as we haven't done much (any) testing with the Shoulda framework. If the fix works 
well, I will try to have it included in the first patch for 6.5.
Comment 3 bkocik 2008-10-20 12:39:29 UTC
Outstanding! Thanks for the fast turnaround. I'll keep you posted if I find anything more.
Comment 4 Quality Engineering 2008-10-21 04:08:42 UTC
Integrated into 'main-golden', will be available in build *200810210201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/1973dfdea1ca
User: Erno Mononen <emononen@netbeans.org>
Log: #150613: NetBeans doesn't report "Shoulda" tests in its output unless they fail
Comment 5 joaovitor 2008-11-20 01:16:13 UTC
This issue is really fixed on nighly builds. Thanks.

I just don't understand why this didn't ship with NB6.5 final.
Why do you plan to release this fix only on NB7.0M1?
Comment 6 Erno Mononen 2008-11-20 08:14:36 UTC
> I just don't understand why this didn't ship with NB6.5 final.
> Why do you plan to release this fix only on NB7.0M1?

Unfortunately it was too late to fix this in 6.5, we were already in high resistance when the issue was first reported 
(only show stopper bugs can be fixed when in high resistance). If you can confirm that the issue is fixed and mark it 
as VERIFIED, we can include this in the next patch for 6.5.


Comment 7 bkocik 2008-11-26 11:59:57 UTC
Verified in nightly builds.
Comment 8 Erno Mononen 2008-11-26 12:26:13 UTC
Thanks for verifying! Adding as a candidate for the next patch. 
Comment 9 jinb 2009-01-07 19:31:16 UTC
fix backported into release65_fixes branch
http://hg.netbeans.org/release65_fixes/rev/cf4c925effad


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo