This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 227791 - Quick switch between CoS and full test mode
Summary: Quick switch between CoS and full test mode
Status: RESOLVED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Maven (show other bugs)
Version: 7.4
Hardware: All All
: P3 normal with 3 votes (vote)
Assignee: Tomas Stupka
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-03-21 20:13 UTC by Jesse Glick
Modified: 2018-04-16 15:25 UTC (History)
2 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse Glick 2013-03-21 20:13:14 UTC
According to how many comments deep into bug #217595 you read, CoS mode is on or off by default for various packaging types.

The problem I am running into these days is that for some projects with complex builds it may happen that most, though not all, of the tests run fine under CoS. For those that do, definitely you want to use it; for those that do not (e.g. because of source generation), you cannot. But having the IDE create or modify ${basedir}/nb-configuration.xml each time you switch from one set to the other is irritating; you have to remember to go back and delete (or ignore) this file later, and even making the switch requires opening the Properties dialog and navigating to the correct panel.

What I really want is a simple toggle button (in the Output Window, say) that lets me turn CoS on and off at whim, without making any changes to my project workspace. Normally I would leave it on, but if I run into a test failure that smells suspiciously like a CoS problem, all I need do is click off the button and press Ctrl-F6 again to fall back to running a complete Maven command. Sometimes I want to run the full Maven command even if the test is running OK in CoS mode, just because I want to confirm that it works in full Surefire, or because I want to copy the Maven command line to send to a friend or run on a Windows test box etc.

Alternately there could just be two actions: test with CoS, or test with Maven; and you could simply run whichever one you wish at the moment. Unfortunately that would consume extra keyboard shortcuts, especially when you add in combinations (debug, test single method, debug single method). It would probably be OK to have to use the main or context menu to invoke tests in one mode (leaving the other bound to the existing KB shortcuts) so long as Ctrl-F11 works to rerun the test, which certainly it would for Maven mode, and AFAICT currently also for CoS mode so long as this mode is enabled.

(All of this probably applies also to application execution, as opposed to tests, but I am rarely running main methods during development except in little test projects, so I have no strong opinion.)

Switching back and forth between the two modes would of course be safer and more efficient if (as requested in bug #217595 comment #9) the IDE would keep its generated class files in a separate location. This would simplify the tortured (and probably unreliable) logic by which the IDE currently tries to decide whether or not to synch to target/classes, and to clean up upon a mode switch; it could unconditionally synch generated classes to target/{,test-}classes-cos/. (Or simply run straight out of $cachedir/index/s*/java/14/classes/, if the *.sig files were renamed to *.class—they are no longer only signatures.)
Comment 1 Milos Kleint 2013-03-25 11:07:44 UTC
I'm currently in the process of rewriting CoS, getting rid of ant based javaRunner entirely. After it's done, the only failing point would be
1. wrongly generated classes by the IDE
2. missing stuff
3. not copied resources

all of which should be fixable by running a clean build I suppose. Will that make some problems go away (those caused by IDE's JavaRunner)? Or are some of the problems you encounter caused by 1-3 as well?
Comment 2 Jesse Glick 2013-03-25 22:46:46 UTC
I have never known JavaRunner per se to be a problem; it is 1-3 that I would care about (for some tests). CoS just cannot be expected to run every last correctly, and that is fine so long as there is an easy way to switch back to a plain Surefire invocation for those cases.

BTW there is no need to get rid of JavaRunner, merely to fix it to use ProcessBuilder to run Java directly. Nothing in its API forces it to use Ant, that was just the path of least resistance.
Comment 3 Jesse Glick 2013-04-11 15:00:45 UTC
Part of the problem with CoS is that the IDE’s parser is so often wrong. “Often” not relative to the size of the codebase—by that metric, it is usually correct—but in absolute terms: for a given largish project, chances are better than even that at least one of the classes used during a typical functional test run has been mishandled by the internal parser and so the CoS *.class is wrong: generally meaning that it has a bogus error in it, and often accompanied by endless “incorrect error badges” warnings in the log and classpath rescanning.

The solution seems obvious to me, and I am not sure why this has never been done right: always prefer target/classes/**/*.class from the Maven build to anything generated by the IDE, so long as that bytecode exists and is newer than the corresponding src/main/java/**/*.java. When bytecode in the real build directory is missing or clearly out of date, then and only then should the internal parser’s bytecode be swapped in.

This would not help you if the bogus errors were in the class you were actually editing. That sometimes happens, but is I think less common than errors in some unrelated source file you have not touched in months that is nonetheless loaded at some point by the test. For that common case, CoS as currently implemented is horrible; all you really want is for the IDE to run your tests using a standard classpath with a couple of freshly edited class files prepended.

If a class is deleted, you could just do nothing special (wait for the user to perform a clean build); or maybe run CoS with a temporary *.class with a static initializer throwing ClassNotFoundException. This of course only matters for code which probes for a class reflectively and then behaves differently if it is missing; irrelevant for regular statically-linked code.

A fix of issue #193183 could also be useful, though I think both are needed.
Comment 4 Jan Lahoda 2013-04-15 17:23:53 UTC
(In reply to comment #3)
> Part of the problem with CoS is that the IDE’s parser is so often wrong.
> “Often” not relative to the size of the codebase—by that metric, it is usually
> correct—but in absolute terms: for a given largish project, chances are better
> than even that at least one of the classes used during a typical functional
> test run has been mishandled by the internal parser and so the CoS *.class is
> wrong: generally meaning that it has a bogus error in it, and often accompanied
> by endless “incorrect error badges” warnings in the log and classpath
> rescanning.

I definitely would be happier if we fixed such problems, than if we tried to mask them (as there are many features that depend on the caches being right).

> The solution seems obvious to me, and I am not sure why this has never been
> done right: always prefer target/classes/**/*.class from the Maven build to
> anything generated by the IDE, so long as that bytecode exists and is newer
> than the corresponding src/main/java/**/*.java. When bytecode in the real build
> directory is missing or clearly out of date, then and only then should the
> internal parser’s bytecode be swapped in.

So it would be working as without <depend>? Aside for general "small" trouble (like annotation processor output handling, handling of classes whose binary names does not correspond to their source file name&location, broken timestamps in build/classes, etc.), I think this is clearly wrong: a project may compile fine before and after a change, but may fail if only the changed class is actually compiled, and that is generally wrong default behaviour. But, of course, there are limited usecases where this might be useful.

Even with <depend> (which severely limits the bug-tolerance advantages of this approach), I generally dislike the idea of mixing the javac-generated and IDE-generated classfiles. I am afraid that too many things could go wrong, and we would never be able to actually fix the problems - if we are unable to find reproducible usecases in the current much simpler situation, how can we hope to find them when we add another axis? (Aside for the general "small" trouble, I don't have a particular example of what could go wrong - but, after all, I don't have a particular (i.e. reproducible) example of what goes wrong in the current case, and yet we observe it sometime does go wrong.)

Regarding using the .sig files to directly run the application: the .sig files are not going to be renamed to .class, they were actually renamed from .class to .sig, so that we are able to very quickly detect from where we are loading the class. This is to prefer things from sources over things from the libraries, which is required for some projects.
Comment 5 Jesse Glick 2013-04-16 17:29:06 UTC
Seems like the perfect is the enemy of the good here. Given that the NB team has been trying to improve the Java parser for something like a decade, yet I have never seen any version of NB successfully parse any version of Jenkins sources (and have often seen it fail to parse its _own_ sources), I am not optimistic about a sudden breakthrough that would lead to 100% reliability. My suggestion in comment #3 was to make something which usually works (and as in comment #0 can be quickly bypassed), as opposed to the current CoS which (for some projects) usually does not work; for now the only choice is to disable CoS altogether.

FWIW, IntelliJ seems to do a better job running tests in the equivalent of CoS mode, though I ran into some other problems that I did not care to investigate more deeply.
Comment 6 Jesse Glick 2014-01-22 15:57:31 UTC
Any thoughts here? This issue is the main reason I keep IDEA installed.
Comment 7 Milos Kleint 2014-01-22 16:33:14 UTC
(In reply to Jesse Glick from comment #6)
> Any thoughts here? This issue is the main reason I keep IDEA installed.

I'm not getting feedback similar to yours from elsewhere. Is it because your projects are special in some way, too complex or something else? So I'm not sure it's actually a mainstream feature. I guess a third party plugin that modifies either global preferences or settings.xml file (and resides in global toolbar I guess, no way to plug into output bar) could be easy enough to write.

what I do get is that test execution is still "slow" for some when single test is executed.


Is your feedback still the same after 7.4 rewrite of CoS? better/worse? all executions are now done with maven, a subsequent "Run test" might not actually do what expect and a full project rebuild might be required then. With multiple project, we are at a full reactor build..
Comment 8 Jesse Glick 2014-01-31 01:52:39 UTC
> Is it because your projects are special in some way, too complex

Possibly.

> plugin that modifies either global preferences or settings.xml file

Perhaps, but see last para in comment #0.
Comment 9 peathal 2014-10-14 08:10:25 UTC
I have similar problems with CoS. A related and new issue with good description is here for submodules:

https://netbeans.org/bugzilla/show_bug.cgi?id=246176
Comment 10 Geertjan Wielenga 2014-10-14 08:20:25 UTC
Can you describe your similar problems, peathal? Exactly which kinds of applications (NetBeans Platform applications or Java SE Swing applications or Java EE applications or what?) are you creating? Can you provide step by step instructions so we can reproduce your problem?
Comment 11 Jesse Glick 2014-10-14 11:28:43 UTC
FWIW, in my case CoS _usually_ works, and when it does, great—it saves a lot of time rebuilding. It just fails to work (because of never-ending problems with the Java parser) often enough that when I want to get work done and not fight with my IDE, I need a button I can press to fall back to running from a full Maven build, without permanently modifying my project configuration.
Comment 12 peathal 2014-10-16 20:08:49 UTC
I've identified my CoS problems, see this bug report: https://netbeans.org/bugzilla/show_bug.cgi?id=247982

Jesse, with bugs in the Java parser -> couldn't you try to push people to fix those :) ? (like I'll now doing for my reported bugs)

> I need a button I can press to fall back to running from a full Maven build

When I had these problems I helped myself to create a custom goal "clean&install" clicked it, waited a bit and was fine ;)
Comment 13 Martin Balin 2016-07-07 08:38:48 UTC
This old bug may not be relevant anymore. If you can still reproduce it in 8.2 development builds please reopen this issue.

Thanks for your cooperation,
NetBeans IDE 8.2 Release Boss
Comment 14 Jesse Glick 2017-01-25 17:10:46 UTC
> with bugs in the Java parser -> couldn't you try to push people to fix those

I already vote for them, but they do not necessarily get fixed.
Comment 15 Jesse Glick 2017-01-25 17:13:41 UTC
BTW I have some related comments in #217595 & #242859.