Bug 72080 - Would be nice to have a way to run one testXXX method
Would be nice to have a way to run one testXXX method
Status: RESOLVED FIXED
Product: java
Classification: Unclassified
Component: JUnit
5.x
All All
: P2 with 20 votes (vote)
: 7.0
Assigned To: Andrey Yamkovoy
junit-issues@java
JA_COMMUNITY
: PLAN
: 123790 153481 158091 158689 161725 (view as bug list)
Depends on: 142579 136580 137191 142763 153558 160194 188547
Blocks:
  Show dependency treegraph
 
Reported: 2006-01-31 19:11 UTC by Scott Violet
Modified: 2010-10-04 15:54 UTC (History)
16 users (show)

See Also:
Issue Type: ENHANCEMENT
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Scott Violet 2006-01-31 19:11:10 UTC
If I've done some refactoring that results in one testXXX method, often times
various other testYYY methods will fail. In this scenario I would like the
ability to run just the testXXX method, so as to avoid all the other output and
debug points. Would be nice if this were directly supported. Perhaps putting the
cursor in a method and saying run this method.
Comment 1 mhelff 2007-02-07 09:28:09 UTC
Both Eclipse and IDEA already have this feature, this is the feature i'm really
missing when working on unit tests
Comment 2 Andrei Badea 2007-12-06 11:05:26 UTC
Any chance this is addressed for 6.1?

Upping to P2 since the we lose against the competition for not having this feature.
Comment 3 tomwheeler 2008-04-17 22:19:31 UTC
*** Issue 123790 has been marked as a duplicate of this issue. ***
Comment 4 Torbjorn Norbye 2008-05-21 19:36:06 UTC
Note - the Ruby modules already have this functionality. We should make sure we have the same name for these actions
(and the same keybindings).

In Ruby they are called

Run Focused Test
Debug Focused Test

and are assigned to Ctrl-Alt-F6 and Ctrl-Alt-F5.
http://wiki.netbeans.org/KeymapProfileFor60#section-KeymapProfileFor60-RubyRubyOnRails
Comment 5 Marian Petras 2008-06-05 16:02:46 UTC
I made the first step of adding support for execution of a single test method - I made a patch for Ant 1.7.0, which
allows to specify one or more test methods of a test suite to be executed. It is also possible to specify that from all
selected tests, only those that failed the last time should be executed.

Mercurial changeset Id:
d466bd54f8ca
(http://hg.netbeans.org/main/rev/d466bd54f8ca)
Comment 6 Jesse Glick 2008-06-05 18:05:17 UTC
To the patch: please put property definitions in project.properties rather than build.xml. I would recommend a filename
of simply "72080.jar" (that it is a patch to Ant is clear from its location). Also it is a good idea to include the
actual patch (diff file) somewhere, e.g. "72080.diff".

To the key bindings: Ctrl-Alt-F<n> is a pretty dangerous binding. For example, on Linux this will normally switch to a
different console! As always, I recommend we be very conservative about using Alt in key bindings, and begin using more
multistroke bindings for less common actions.
Comment 7 Marian Petras 2008-06-05 18:29:23 UTC
Location of the property definition:
I think that the build file is the right location. The property does not just set some value (e.g. a target directory) -
it actually tells that something should be copied, which is not apparent from the rest of the build script. Actually, I
think that a properties file would be a bad location in this case. It should be clear from the build script what it does.

The name of the .jar file:
OK, I will change the value of the property to "ant/patches/72080.jar". I will not rename the .jar in junit/external.

The source code patch:
It is in the root directory of the .jar file. I saw this concept somewhere but I do not know how common or good it is.
Comment 8 Jesse Glick 2008-06-05 18:40:58 UTC
To the property definition: every other module in NetBeans which copies something using release.*, or in fact defines
any Ant property statically, puts this definition in project.properties. Most modules have a build.xml which is empty
other than the import and this is the desirable state. Be consistent with everyone else, please. Anyway common.xml and
projectized.xml do all sorts of steps automatically based on various property definitions (as well as project.xml); if
you want to know what your module's build does, it is unavoidable that you look at and understand these metadata files.

Keeping the diff in the JAR itself is OK too. (I only saw the Hg patch so did not notice this.)
Comment 9 Quality Engineering 2008-06-06 04:18:09 UTC
Integrated into 'main-golden', available in NB_Trunk_Production #240 build
Changeset: http://hg.netbeans.org/main/rev/d466bd54f8ca
User: mpetras@netbeans.org
Log: preparation for fix of issue #72080 (addition of ability to run a single test method):
added binary patch of Ant 1.7.0 that adds support for execution of a single test method and for execution of all failed test methods
Comment 10 Marian Petras 2008-06-06 13:25:09 UTC
From issue #136580, it seems that location of the property definition in project.properties is not just an alternative
but it is a necessity. Thanks for fixing it.
Comment 11 Jesse Glick 2008-06-06 21:08:08 UTC
Regarding property location: probably had to be initialized earlier than in projectized-common.files-init rather than after.
Comment 12 Quality Engineering 2008-06-07 16:11:22 UTC
Integrated into 'main-golden', available in NB_Trunk_Production #243 build
Changeset: http://hg.netbeans.org/main/rev/00fc8aa200ce
User: mpetras@netbeans.org
Log: binary patch for Ant renamed to "72080.jar" as suggested in the comment to issue #72080
Comment 13 Marian Petras 2008-07-10 12:43:01 UTC
Done on the side of the JUnit module and the ProjectAPI module (issue #137191).

New and modified files in the JUnit module:
    junit/manifest.mf
    junit/nbproject/project.xml
    junit/src/org/netbeans/modules/junit/output/Bundle.properties
    junit/src/org/netbeans/modules/junit/output/JUnitOutputReader.java
    junit/src/org/netbeans/modules/junit/output/OutputUtils.java
    junit/src/org/netbeans/modules/junit/output/Report.java
    junit/src/org/netbeans/modules/junit/output/TestMethodNode.java
    junit/src/org/netbeans/modules/junit/output/TestMethodNodeAction.java   (new)

The actual availability of support for execution of a single test method depends on whether the respective project
supports these actions. If the actions are supported, actions "Run Again" and/or "Debug" are available in the
context-menu of nodes representing individual test methods in the JUnit Test Results window.

At the moment, no project type supports these actions so the actions are in fact never available. I am going to enable
the actions in J2SE projects and NetBeans module projects in a day or two.
Comment 14 _ gsporar 2008-07-10 14:13:57 UTC
>I am going to enable the actions in J2SE projects and NetBeans module projects in a day or two.

Awesome!  :-)

Is there an IZ number for those changes?  I'd like to know once those changes get put into the trunk.

Comment 15 Quality Engineering 2008-07-10 15:46:43 UTC
Integrated into 'main-golden', available in NB_Trunk_Production #311 build
Changeset: http://hg.netbeans.org/main/rev/cc3ff20c3ae7
User: Marian Petras <mpetras@netbeans.org>
Log: added support for execution/debugging of a single test method (issue #72080)
Comment 16 Marian Petras 2008-07-10 17:08:22 UTC
Now that Ant in NetBeans has been updated to Ant 1.7.1, I also updated the binary patch of Ant.

Mercurial changeset Id:
0d9f2f6d877c
(http://hg.netbeans.org/main/rev/0d9f2f6d877c)
Comment 17 Quality Engineering 2008-07-17 04:31:51 UTC
Integrated into 'main-golden', available in NB_Trunk_Production #324 build
Changeset: http://hg.netbeans.org/main/rev/0d9f2f6d877c
User: Marian Petras <mpetras@netbeans.org>
Log: updated binary patch of Ant (see issue #72080) - now for Ant 1.7.1 (see issue #139445)
Comment 18 Jesse Glick 2008-08-04 19:36:20 UTC
How do you actually run the single test methods? The only way I have found in the GUI so far is "Run Again" in the
context menu of JUnit Test Results. Is there no way to run just one method from a test suite without first running the
entire test suite?
Comment 19 Torbjorn Norbye 2008-08-05 00:54:23 UTC
I don't know how this works today, but I know what I think it should do: through the editor context, and from the test
output.

1) Editor: You should be able to press a keybinding for "Run Focused Test", and this should run the test surrounding the
caret. 

2) From the test output: You should be able to right click on one or more tests in the test output and choose run to run
that test or set of tests.

We should also make sure this is consistent among the rest runners. I think we have (1) for Ruby, I'm not sure about (2)
- adding Erno to the CC.
Comment 20 Jesse Glick 2008-08-05 00:57:30 UTC
So Tor's #2 seems to exist today, but if #1 exists I cannot find it.
Comment 21 Erno Mononen 2008-08-05 08:08:01 UTC
> I think we have (1) for Ruby, I'm not sure about (2)

Right, we have (1). (2) we don't have currently, tracked in issue 136422. 

Comment 22 Marian Petras 2008-08-08 13:23:50 UTC
There is no other way, indeed. I am afraid it is too late for adding new menu items to pop-up menus.
Comment 23 Torbjorn Norbye 2008-08-08 16:06:00 UTC
Just FYI, we don't have an item in the editor context menu for this in Ruby either; it would be highly context dependent
(you wouldn't want "Run Focused Test" in the context menu for ALL Ruby files, only test files.)

The important thing is to have an action people can find in the keybindings dialog that will perform the test run since
it's key to workflow when working with unit tests. The keybindings are listed here:
http://wiki.netbeans.org/KeymapProfileFor60
(look for "focused" and you'll find it.)
Comment 24 Jesse Glick 2008-08-08 17:45:13 UTC
Note that on Linux Ctrl-Alt-Fn normally switches to a virtual console, so this is a poor choice for a key binding.
Comment 25 Peter Pis 2008-08-15 21:15:17 UTC
Not all issues that this issue depends on are fixed. Reopening.
Comment 26 Peter Pis 2008-11-20 11:51:18 UTC
*** Issue 153481 has been marked as a duplicate of this issue. ***
Comment 27 tomwheeler 2008-11-20 14:29:20 UTC
This was listed in the "New and Noteworthy" document as a feature available in 6.5 but it's not actually present in the
IDE.  

Since users were lead to believe it was available now, it would be nice if this could be fixed and included in a patch
to 6.5.
Comment 28 Jesse Glick 2008-11-20 19:56:53 UTC
Tom - read above comments. The feature *is* present if you know where to look. You can rerun a single test only after
running the containing suite, by right-clicking the result subnode in the JUnit Test Results window. Not available in
all project types, I think.
Comment 29 Marian Petras 2008-11-20 20:43:06 UTC
The implementation of support for execution of a single test method is composed of several parts:

1) A binary patch for Ant which extends the Ant's <junit> task such that an individual test method can be specified.
2) In the JUnit module, the UI support is added: If the project is probed for support of execution of a single test
method and if it the support is detected, the appropriate actions (Run Again, Debug) are made available in the context
menu in the test results window.
3) Each project type must be accomodated if these actions should be supported. This is because the tests are still
executed via Ant and each type of project uses a different build script. Note that there is a different build script
used if "Compile on Save" is enabled.

In NetBeans 6.5, the actions (Run Again, Debug) are only supported by the J2SE projects. In NetBeans 7.0, the support
for these actions should be added to project types. It should by mostly implemented by the respective project type
developer, with help from the JUnit developer. I will not develop the JUnit module any more for NB 7.0 (someone else
takes the development over) but I will be able to help with this.
Comment 30 Alexei Mokeev 2009-03-05 13:50:27 UTC
Assigning issues from Michael to Andrey.
Comment 31 tomwheeler 2009-03-06 21:47:51 UTC
Is this really going to be in 6.7M3 as the target milestone suggests?  
Comment 32 Alexei Mokeev 2009-03-12 14:52:05 UTC
*** Issue 158091 has been marked as a duplicate of this issue. ***
Comment 33 Alexei Mokeev 2009-03-12 14:53:07 UTC
*** Issue 158689 has been marked as a duplicate of this issue. ***
Comment 34 sgtgarcia 2009-03-12 15:07:27 UTC
As described in Issue 158689, Maven project is another problem.

Scenario in standard project:
- Run JUnit Test
- In JUnit Result view, right click on Test method
- A contextual menu appears (Run again, debug)
- Click on Run again, runs only the selected method

Scenario in Maven project:
- Run JUnit Test
- In JUnit Result view, right click on Test method
- Nothing appears
- Can't "Run again" a single Test method

Comment 35 Alexei Mokeev 2009-03-12 18:20:09 UTC
Fixed initial milestone setting. By no means this could be done for M3.
Comment 36 Andrey Yamkovoy 2009-04-02 13:05:39 UTC
*** Issue 161725 has been marked as a duplicate of this issue. ***
Comment 37 fommil 2009-04-11 00:28:55 UTC
While there is work going on here... I'd like there to be a menu entry (so I can bind a key combo to the action) which runs the test associated to the where the 
caret is currently located (e.g. caret is either in a method which has an obvious associated test, or in a test method)
Comment 38 Torbjorn Norbye 2009-04-11 00:50:53 UTC
See my May 21 comment regarding this; we have those actions (Run Focused Test, Debug Focused Test) in Ruby. Since the
test infrastructure is shared now (at least that's my impression) perhaps those actions are also available for junit
now? If not, they definitely should be.
Comment 39 fommil 2009-04-11 00:59:36 UTC
Not in 6.5.1 (I'm looking under the "Run" menu)... when was this integrated?
Comment 40 Torbjorn Norbye 2009-04-11 01:04:48 UTC
It's not in the menu; there are actions with default keybindings (and you should be able to find it in the keybindings
dialog (which has finally been rewritten in 6.7 and is pretty usable now.)  Note that I only know for a fact that this
is there for Ruby; I was speculating that it might also be available for Java too given that, -in 6.7-, they are sharing
a test infrastructure. It's probably not there in 6.5.1 - but if you try 6.7 you won't find it in the menu either
(unless Erno or somebody has added it?)
Comment 41 Jesse Glick 2010-02-05 18:45:36 UTC
(In reply to comment #9)
> added binary patch of Ant 1.7.0 that adds support for execution of a single
> test method

Note: this is tracked in the Ant system as

https://issues.apache.org/bugzilla/show_bug.cgi?id=34748

and the patch is being updated for 1.8.0 (bug #179463) in

https://svn.apache.org/repos/asf/ant/core/branches/run-single-test-method/

> and for execution of all failed test methods

This has been removed from the updated patch since the IDE does not seem to need it, and anyway 1.8.0 is said to include some such facility anyway (a failure recorder).
Comment 42 tomwheeler 2010-05-04 17:28:44 UTC
I'm sorry if I'm dense, but can I expect this to work in a NetBeans Platform project?  I am using NetBeans 6.9 beta, Ant 1.8.0 and JDK 1.6.0_20 and there is still no way to re-run a failed test in the IDE as far as I can tell.  If it is possible, please tell me how and I will document in the Wiki; if it's not possible, I will file an issue to request this.
Comment 43 tomwheeler 2010-06-02 22:25:00 UTC
This still does not work for NetBeans Platform projects as far as I can tell, even using NetBeans 6.9 RC2, Ant 1.8.0 and JDK 1.6.0_20.  

Should the Target Milestone be changed to indicate when this will be implemented?  I would imagine it's (unfortunately) not going to make it into the final 6.9 release.
Comment 44 Jesse Glick 2010-06-11 16:41:10 UTC
The Ant patch has been applied and should be included in the 1.8.2 release.
Comment 45 Andrey Yamkovoy 2010-07-07 14:24:53 UTC
I have implemented the actions "Run Focused Test" and "Debug Focused Test" for the java files available ONLY from the editor context menu. These actions will be enabled if the project's (which owns the file) ActionProvider supports such an actions. See issue 166252.

After that I don't see any issues on the JUnit side which are described in this enhancement. So I suggest to remove the issues 142579 and 142763 from the dependencies and mark this issue as resolved. Any objections?
Comment 46 Jesse Glick 2010-07-07 15:02:01 UTC
Please do not remove dependencies. There will be more work required before the complete feature is available from a user perspective in all applicable contexts, and the dependency tree is invaluable for finding related issues.
Comment 47 Jesse Glick 2010-07-08 17:45:11 UTC
I think this is fixed from the junit module side; the action appears and works for

1. j2seproject's with CoS enabled
2. Maven projects (CoS enabled for tests)
3. automatic projects

Other project types and scenarios can be handled as separate issues.
Comment 48 tomwheeler 2010-07-08 18:04:23 UTC
Can I expect it to work for a standard NetBeans Platform project (e.g. built using the Ant build harness)?
Comment 49 Jesse Glick 2010-07-08 20:49:34 UTC
Support in NBM projects is tracked separately (and in progress): #142763
Comment 50 Jesse Glick 2010-07-09 16:45:09 UTC
(In reply to comment #45)
> I have implemented the actions "Run Focused Test" and "Debug Focused Test" for
> the java files available ONLY from the editor context menu. These actions will
> be enabled if the project's (which owns the file) ActionProvider supports such
> an actions.

The question is whether these actions belong in the junit module. After all, SingleMethod is in projectapi, makes no mention of JUnit, and is available for any type of project - including non-Java projects, not to mention Java projects using e.g. TestNG. The usual style would be for the actions to be defined in projectui. The extra logic in the current actions is to verify that the selected method is in fact a JUnit test; this should really be left to the discretion of the project (in isActionEnabled), with a standard impl for JUnit 3.x/4.x present perhaps in java.api.common. The action name should likely also not contain the label "Test" (just referring to running or debugging the selected method) so it could be used also for e.g. running isolated functions in a scripting language (compare C-x C-e in Emacs Lisp mode).

However the action at least needs to identify the selected method name, which in its current implementation is done manually for Java source files only, which cannot be in projectui; this implementation would need to be somehow injected by content type. It might also be inefficient to run two Java source actions: one to identify the method name at caret, the second to verify that it is a test.

If these issues can be resolved, I think the implementation should be refactored out of the junit module. If done after this release, it would be mostly compatible; existing project types which unconditionally report enablement of COMMAND_*_SINGLE_METHOD on e.g. test/**Test.java would begin offering the action on inappropriate methods (e.g. setUp), which is not great but not a major problem.


BTW one possible bug in the identification of 3.x tests: if the class contains a public static Test suite() method, running an individual method is likely wrong (since this suite factory will be bypassed). I am not sure if the same applies to 4.x @RunWith.
Comment 51 David Konecny 2010-07-11 21:08:07 UTC
Is there an issue for web/ee projects? What needs to be done?
Comment 52 Jesse Glick 2010-07-12 14:17:31 UTC
(In reply to comment #51)
> Is there an issue for web/ee projects?

Might not be.

> What needs to be done?

Perhaps nothing; WebActionProvider already seems to support SingleMethod.COMMAND_*_SINGLE_METHOD. Not sure if, like j2seproject, these are supported only in CoS mode; if so, bug #142579 would be the right place to look for background.
Comment 53 David Konecny 2010-07-12 18:46:30 UTC
(In reply to comment #52)
> Perhaps nothing; WebActionProvider already seems to support
> SingleMethod.COMMAND_*_SINGLE_METHOD. Not sure if, like j2seproject, these are
> supported only in CoS mode; if so, bug #142579 would be the right place to look
> for background.

Thanks. EE and SE projects are using now the same base ActionProvider impl so most likely only CoS mode is supported.
Comment 54 Quality Engineering 2010-07-13 03:32:58 UTC
Integrated into 'main-golden', will be available in build *201007130001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/c13dd2500664
User: Jesse Glick <jglick@netbeans.org>
Log: #72080 refinement: action must be disabled in case the ActionProvider does not even support this command.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo