For whatever reason after more use I'm seeing this more often -- and somehow even across projects!

I'll do some compiles and builds and then suddenly I start getting red error x's on various projects, packages, and
source files -- all related to usages of classes from within the project's output jar.

I'd argue for a P2 priority, but I don't want to delay NB 6's release over this.  I do, however, want to see this fixed
in the near term.

Oddly after restarting the IDE the project, package, and file error markers are still there, but if I open the files in
the editor there are no errors shown there.  Once I then open *another* file the file error marker goes away, etc.  Weird.

Correction -- open another file *after* the editor has finished scanning and annotating the file in question.

I'm elevating this to P2 as this is really quite obnoxious and is eroding developer confidence in NetBeans.  [I didn't
want NB 6.0's release held up by this, but now that 6.0 is out, this and performance are my top NB priorities overall.]

Its hard to say what happens without more info (e.g. reproducible test case). But output from IDE with the following
debugging options might give some pointers (it needs to be a trunk build, not NB6.0, and the output will be huge):
-J-Dorg.netbeans.modules.java.source.parsing.level=300 -J-Dorg.netbeans.modules.java.source.usages.RepositoryUpdater.level=0

I installed build 200712090000, loaded up a dozen or so of my projects, and reproduced the issue.  I then closed all but
1 project, shutdown the IDE, cleared the cache and logs, restarted to populate the cache, shutdown the IDE, and
restarted again.

At that point I just did a few F9 single-file compilations in my project.  The first few do not succeed in triggering
the issue, but the final one does.  Suddenly I have red x's all over the place -- specifically one for each use of a
ResourceBundle generated from one of our non-Java source files in the project.  This is with
-J-DCacheClassPath.keepJars=true set in addition to the logging directives requested.

In my case F9 triggers:
  1) an Ant <copy> of the existing project output jar to a temporary location
  2) a zip.exe update of the temporary copy
  3) an Ant <copy> (with overwrite="true") of the temporary copy over the existing project output jar.

I'd love to just do (2) by itself but this produces file locking issues (on Windows at least).

It would appear that there is some sort of a race in which the parser tries to look at the output jar during step (3)
above.  I'd be happy just knowing a way to tell the parser to wait until this step is done (though one would think I
shouldn't have to) or even a way to tell it about our non-Java sources (I've recently started hacking some simple NB
modules).  Right now, however, I'm kind of sunk.

I'll attach a zip of the logs shortly.

Created attachment 54097 [details]
Logs from bug reproduction with daily build

Removing keyword "INCOMPLETE" as this is about as complete as I think I can realistically make this.  It is not feasible
to reproduce one's entire free-form project environment, but I have reproduced the issue with all requested logging
enabled and described the situation as best I can.

P.S. I did try changing the Ant <copy> in step (3) to a <move> -- this just produced an error as Ant was unable to
remove the existing jar.

Note that it would be *really* nice if Java exposed an efficient jar/zip update API.  I have to use zip.exe as it is
many, many times faster than using <zip> or <jar> for large jars and only adding/updating a few entries therein.  I'm
updating these on compile.single rather than solely on build due to the size of the jar -- doing a full rebuild to
update a handful of classes is *not* an option.  [Every NetBeans user using the free-form projects in question would
flee to Eclipse immediately if I tried this.]

Is it possible that some file locking could/should just be added to BinaryAnalyser?

[I'm poking around in the source, but am pretty darn ignorant on NetBeans' internals.]

I reproduced the issue using <zip> to update the jar rather than zip.exe as well, so this is not specific to the
peculiarities of my Ant script.  Also, Java file locking won't help as this as all done within the same process (except
for the zip.exe usage itself).

It would seem best if NetBeans wouldn't try to re-analyze files noted in <built-to> elements during "build" and
"compile.single" actions at a minimum or if it noticed that these files continued to change thereafter, reanalyzed and
thus fixed up any mis-cues produced by reading the files while in flux.  I'm not certain from looking at the code what
NetBeans' normal strategy for "righting itself" after a miscued reading of a jar is.

I'm left wondering whether there isn't a way to "clue-in" NetBeans' parser about non-Java source files via a module. 
For instance, is it possible to write a small module that says for .xyz files I'll tell you about the Java class(es)
they embody?

I'm hopeful that there is such an approach as it seems like something that should be possible and having to use special
NetBeans command-line args (even if everything worked out right thereafter) is not really where we want to be...

If such a thing is possible I'd greatly appreciate some pointers.  The classes in question are always very simple, so
I'd hope this should be a simple thing...

As for what's behind this bug, I'm not sure why NetBeans looks right when the jar does not exist, is empty, or is being
rebuilt (or how to tell which) and then can't be bothered to look again when the jar is fully rebuilt.  I suppose I
could try placing some breakpoints from one NetBeans copy against the other...

I did some poking around in the debugger, but I'm pretty much lost in the wilderness here.  I notice enticing bits in
RepositoryUpdater like assureFilesWithErrorsRecompiled(), but I don't understand the intended operation of the algorithm
well enough to see what's going on.

Overall NetBeans seems to get lost when you update module jars underneath it (and are depending upon
-J-DCacheClassPath.keepJars=true) and never gets "unlost".

I've spent a bit more time in the debugger.  I'm still lost, but I'm becoming more and more convinced that this is a
systematic gap in the logic surrounding compile.single sorts of actions and -J-DCacheClassPath.keepJars=true -- rather
than some sort of a timing/race issue.

It would be great if some NetBeans developer could take a look at this as it is starting to drive me batty -- and I
can't seem to get enough traction to figure it out (in part because I haven't loaded sources for and built all of trunk
-- which is necessary for really useful NB debugging of this...)

I have a small test case (still dependent on our build framework/environment) that I've been playing with in a full
build from trunk and trying to debug from another NetBeans.

Could someone give me some pointers here?  For instance, where exactly should I be putting break points to determine
where/how/why the red error markers occur?  I've got breaks all over RepositoryUpdater, but that's not seeming to be the
issue...

From further debugging and experimentation it seems there is a timing element to this.

-J-DCacheClassPath.keepJars=true fully addresses up-front scanning/compiling/parsing sources in such situations.

The issue is that *sometimes* when a source file in such a situation needs to be recompiled/reparsed, e.g. when opening
it in the editor, the compiler does not get information about classes in the project output jars from the classpath
despite -J-DCacheClassPath.keepJars=true being used.  This does not seem to require that the output jar actually be
touched, but I'm not absolutely certain on this.

I've cranked up every logger that appears related in NetBeans to ALL and I still see the jar class files in question
noted in all the appropriate ProxyFileManager logs.  I've stepped through in the debugger and all would appear well --
until somehow the compiler does not get or discards this information, but I can't seem to determine where exactly to
look to determine what the compiler actually gets/uses.

Dusane, can you take a look at this one? Thanks.

Any progress on this?

Fix for issue #123869 ensures that each time a jar on classpath is updated, the sources are refreshed, which should
ensure that the error badges go away in your case.

Which versions is the fix for #123869 in?  I'd like to try them out.

Also I had thought of a means to try to reproduce this in a small free-form project, but not had chance to do so yet.

Essentially I think that apart from timing issues it should be quite possible to just have 2 classes:
  1) Class A which is available in A.class.hidden, i.e. via a class file but renamed to hide it from being recognized as
such.
  2) Class B which is defined via B.java and references class A, e.g. static final constants therein, A.class, etc.

The freeform project's build script would:
  1) For "build", copy A.class.hidden to a target build directory and rename it to A.class, compile B against it to the
same build directory, and then jar up the result.
  2) For compile.single, compile B against A and update the jar.  In my free-form projects I use native zip here.  [It
is horrible that Java has no way to efficiently update a zip file -- this is a performance killer for large jars.]

In my real case, of course, A.class is produced from another non-Java source file, which is why I think this case is
even of any broader interest.  NetBeans should be able to handle writing code against the result of non-Java source
files -- without having to use a separate directory tree for such sources (ours intentionally reside in the same
directory structure for easy reference and modularity).

It would appear that *something*, presumably the fix to issue #123869, pretty much made this issue disappear with the
latest 6.0 updates.

The issue is still quite alive and well in the 3/25 daily of NetBeans 6.1, though.  Has the fix to issue #123869 not
been merged to 6.1 yet?

I spoke too soon.  The issue was not as easily or quickly reproducible with the latest 6.0 updates, but it is clearly
still there :-(

*** Issue 131917 has been marked as a duplicate of this issue. ***

moving opened issues from TM <= 6.1 to TM=Dev

*** Issue 134292 has been marked as a duplicate of this issue. ***

Should be fixed in 6.5