Bug 207032

Here's a link to the original discussion thread in nbusers mailing list:
http://forums.netbeans.org/topic45782.html

Let me start from a broader view: while striving for total consistency is
definitely good, it is not the sole aspect. The other aspect is speed. It is
possible to get consistent results by removing CoS and forcing clean&build
before each run. But, that would obviously make the IDE unusable for anything
but toy projects. (I would even argue that for projects with annotation
processors, there is not real way to actually get consistent (correct) results
other than doing clean&build before any action - but that is a different
story.)

Without doing a clean build before every run (counting running tests as run,
too), there is a lot of ways to get balance between consistency and speed
appropriate for the given usecase, the ones that can be selected for J2SE
Projects are:
1. the Java indexing, when a signature of an element changes, finds all
dependent classes and re-indexes (recompiles) them. The main advantage is that
this forces recompiling of relatively small set of files (smaller than the
ant's depend task). The outcomes of this are used both for the editor features
and when the project is run with CoS enabled. There are several disadvantages,
too. Let me enumerate the most important ones (in my opinion): 1) the project
must be completely understandable by the Java infrastructure - custom tasks
that generate some data, etc. are generally causing troubles. 2) the run action
must wait for the indexing to finish, so that the results are correct (which
may not really be acceptable for some users). 3) (I admit that) there are bugs
in the code that computes what should be recompiled (on both sides: that it
recompiles less than necessary and more than necessary). Unfortunately, to fix
such a bug, it is necessary to understand what went wrong, and that is
typically impossible from the already wrong class file (although it is
sometimes possible to infer it from the ".err" file that contains compilation
errors produced for the given file). This is used for the J2SE project when the
"Project Properties/Build/Compiling/Compile on Save" option is checked.
2. use of the ant's depend task, which often recompiles (much) more than
necessary, and which does not handle the compile-time constants. But, as it is
much simpler, it works reliably inside its boundaries, and works with custom
additions to the build script. This is used for the J2SE project when the
"Project Properties/Build/Compiling/Track Java Dependencies" option is checked.
3. simply don't automatically recompile any files except those modified by the
user. If the user does change that is significant enough, the project must be
rebuilt manually. While this behavior may sound strange, I am sure there are
users with projects so big, that the other options would simply be too slow.
This is used when both the CoS and Track Java Dependencies options are
unchecked.

This particular bug is basically about option 2 and ant's <depend> inability to
deal with compile-time constants (although maven may have the same problem).
(This has nothing to do with javac, whatsoever. javac is a Java compiler and as
such, it is expected to adhere (as much as possible) to the Java language
specification, and the specification requires the compile time constants to be
inlined (see, e.g., JLS 13.4.9, which is a section on binary compatibility). So
javac does not have the option not to inline the compile time constants.) I see
two ways out of it, none without drawbacks:
-enhance the <depend> task inside NB so that it can handle the compile-time
constants. Could be done by having a (Java) indexer that would record
re-indexed files between project runs, and would clear the classfiles that
would correspond to them. The main disadvantage is that this depend task would
need to wait for the indexing to finish, which it did not need till now.
-force use of classfiles coming from the IDE's indexing, could be theoretically
done while still running the user's ant script. Would require valuable (and
precise) feedback from the users, so that we could fix the bugs mentioned for
option 1. Would still require waiting for the indexing to finish, obviously.

It is pretty difficult to say what is the correct component for this bug
(except that java/compiler isn't - this component is for the internal Java
parser/compiler, which cannot really affect anything in this regard). I will go
with ant (integration), as that is what integrates the <depend> task and it is
probably the place where an enhancement of the <depend> task would conceptually
belong.

BTW: I must say I am really flattered by this bug. A problem that is basically
ignored by most, although not all, projects out there is considered a P1 bug in
NetBeans. The NetBeans long-term quality must be very high.

Jan,
I'm not really familiar with all the other projects out there that are ignoring this problem.  I made it P1 because to me, it's pretty pernicious behavior.

I also have to admit to not fully understanding the reasoning in the 2nd way out of the ant "depends" problem: force use of Java Indexing.  You say that this would require the bugs in that area to be worked out.  But if those bugs were worked out, then why bother with the "depends" task solution at all?  Conversely, you make it sound like the java indexing solution will be difficult to make fool-proof without NB completely understanding the project structure.  But if that is so, then this 2nd way out of the ant "depends" problem will never work if it depends on this java indexing.

Thanks for having looked into this bug so quickly.

Certainly not P1. Most compile-time constants are not frequently, if ever changed; when they do, the workaround is merely to do a clean build (as you would in most Java build environments, including for example command-line Maven which basically works using strategy #3). Since there is no known practical way of improving dependency detection without sacrificing performance or robustness, I am closing this as designed.

(In reply to comment #2)
> This has nothing to do with javac

Not quite - javac will actually compile sources other than the ones you specified, if they can be found in the sourcepath and corresponding classfiles are missing, which can interact with things like <depend> in certain cases.

> the specification requires the compile time constants to be inlined

BTW I am considering advocating that this be changed in JDK 8, though it is a tricky subject for a few reasons (annotations, switch).

> ant (integration), as that is what integrates the <depend> task

Really it is java.j2seproject (and others) which choose to call <depend> or not.

(In reply to comment #4)
> Certainly not P1.

Then you certainly don't value program correctness much.  Hope you stay away from writing financial or medical software.

> Most compile-time constants are not frequently, if ever
> changed; when they do, the workaround is merely to do a clean build (as you
> would in most Java build environments, including for example command-line Maven
> which basically works using strategy #3). Since there is no known practical way
> of improving dependency detection without sacrificing performance or
> robustness, I am closing this as designed.

What difference does it make how often constants are changed or that there's a workaround - if the IDE user doesn't even know he needs a "workaround"??  If the user modifies 10+ files while fixing a bug or doing a refactoring and then hits "Run", I think most reasonable people would have the expectation that the IDE produced a completely "up-to-date" program - not just some approximation of one.

Netbeans is not just a build environment - it's an IDE which aims to hide/abstract-away build-related complexities.  Because you did that, you've invited non-professionals or even Java beginners into your "fold".  Clearly you can't expect those users (which, judging by the questions in nbusers these days, is the majority) to know when to hit "Run" and when to hit "Clean and Build".

If you're not going to change the behavior for the unchecked "CoS" case, at least make it clearer to your users (via adequate documentation) what the implication is.

I am using NetBeans IDE 6.8 (Build 200912041610). I am experiencing the identical behaviour reported by the original user and would like to provide some additional information to the discussion. I'll try my best not to rant, even though I've wasted hours of development time today trying to find a resolution and have had to sift through dozens of other resources before I landed on this bug report. Yes, I know that I just ranted about ranting :D

1. The issue in question does not depend on the type of code change, at least that is whay I'm seeing with my particular IDE configuration and version. Lots of the discussions regarding this issue (here and in other forums) have focused on changes in source code to STATIC variables and/or inner classes. Those discussions are ancillary because the problem exists irregardless of the type of code change. Therefore, suggestions that the problem is low impact because constants are not often changing in source code seem irrelevant to me.

2. I could be mistaken, but I'm pretty confident that this problem was not present in earlier versions of Netbeans, and this is what is most frustrating to me. I believe I started developing in version 6.5 at a time when I was just learning java and starting to use unit testing. At that time I wrote hundreds of tests, and I do not remember encountering this issue. At some point I moved to other projects where I had no control over Netbeans projects and was writing very few tests. It was then that I experienced the problem, and at that time I mistakenly assumed that it was because Compile on Save was turned off in this set of projects. I didn't have ownership of the projects so I could not turn COS on. Though the behaviour was annoying, I ignored it at the time and get accustomed to running Clean and Build every time I made a source code change and subsequently ran my Junit tests. Recently however, I began working in my earlier project set and started writing a signifcant number of tests, and because of the mounting frustration this behaviour presented, I wanted to find a resolution. Where I find this extrememtly frustrating is that I work in a fairly large codebase with 15 dependant Netbeans projects. I always use a test driven development mindset, so I'm always trying to write the test first before fully implementing new code changes. The test breaks, I go back to the source and make the changes until the test passes. I continue to repeat this cycle on the same source file. It is excruciating having to run a full Clean and Build on the project between each code change and subsequent test run. I tried recompiling just the single source file under test. The behavior is the same and you have to run Clean and Build.

3. This same behaviour is present in the Netbeans debugger. The following test code exhibits the faulty behaviour in both the Junit tests and the Netbeans debugger.

int x = 0;
if (x != 0) {

I am using NetBeans IDE 6.8 (Build 200912041610). I am experiencing the identical behaviour reported by the original user and would like to provide some additional information to the discussion. I'll try my best not to rant, even though I've wasted hours of development time today trying to find a resolution and have had to sift through dozens of other resources before I landed on this bug report. Yes, I know that I just ranted about ranting :D

1. The issue in question does not depend on the type of code change, at least that is what I'm seeing with my particular IDE configuration and version. Lots of the discussions regarding this issue (here and in other forums) have focused on changes in source code to STATIC variables and/or inner classes. Those discussions are ancillary because the problem exists irregardless of the type of code change. Therefore, suggestions that the problem is low impact because "constants are not often changing in source code" seem irrelevant to me.

2. I could be mistaken, but I'm pretty confident that this problem was not present in earlier versions of Netbeans, and this is what is most frustrating to me. I believe I started developing in version 6.5 at a time when I was just learning java and starting to use unit testing. At that time I wrote hundreds of tests, and I do not remember encountering this issue. At some point I moved to other projects where I had no control over Netbeans projects and was writing very few tests. It was then that I experienced the problem, and at that time I mistakenly assumed that it was present because Compile on Save was turned off in this set of projects. I didn't have ownership of the projects so I could not turn COS on. Though the behaviour was annoying, I ignored it at the time and got accustomed to running Clean and Build every time I made a source code change in order to run my Junit tests. Recently however, I began working in my earlier project set and started writing a signifcant number of tests. Because of the mounting frustration this behaviour presented, I wanted to find a resolution. Where I find this extrememtly frustrating is that I work in a fairly large codebase with 15 dependant Netbeans projects. I always use a test driven development mindset, so I'm always trying to write the test first before fully implementing new code changes. The test breaks, I go back to the source and make the changes until the test passes. I continue to repeat this cycle on the same source file. It is excruciating having to run a full Clean and Build on the project between each and every single code change and subsequent test run. I tried recompiling just the single source file under test as a workaround. The behavior is the same and you have to run Clean and Build.

3. This same behaviour is present in the Netbeans debugger. The following test code exhibits the faulty behaviour in both the Junit tests and the Netbeans debugger.

int x = 0;
if (x != 0) {
   throw new Exception("Condition failed");
}

A Junit test excecuting this code will not throw an exception, which is the correct behaviour. The source is changed to :

int x = 0;
if (x != 1) {
   throw new Exception("Condition failed");
}

If the project is not recompiled and the test is executed again, the test will not throw an exception. This contradicts what the user assumes would be the correct behaviour from the source code in view. I don't see how the argument that "everything looks right" can be applied.

If you use the same steps above and run the test in the debugger (I'm a heavy user of the debugger, especially when I'm troubleshooting unit tests) and step through every line of code, the debugger will behave identically. In the second code snippet, the evaluation of the source code in the debugger indicates to the user that "x must equal 1 even though you just assigned it a value of zero."

I find that disturbing.

Thank you for taking the time to evaulate the issue and reading this long post.

my apologies for posting my additional comments the first time before I had completed my post.

The followup poster didn't re-open the bug.  Since I don't know whether anyone looks at such followup comments once a bug has been marked as "RESOLVED" I'm re-opening it so it can be given a second thought.

In reply to comment #7.
I am not able to reproduce it a way you have described.
Don't you have track.file.changes in your project files?
Don't you use depend?

waiting for reporter response...

@Tomas Musil,
I also cannot reproduce the example given in comment #7.

I can, of course, still reproduce my original issue when CoS is not turned on.  But since nobody seems to agree that this is a big issue, I won't further comment.