Bug 152749

I'm pretty sure that performance team has arguments why is this useful and points you to documents what has been in 6.5
improved in this area..

The particulars of fixes to this mechanism are not as interesting as how the scanning makes it faster for the IDE to
operate over all.  I don't see the IDE operating faster when I now have to wait 5mins after rebuilding a library, for
the IDE to rescan all my projects and be happy that I didn't break any APIs.  I updated the library, rescan the library,
not all of my projects.  When I open a source file, scan it.  When I search a project, scan it.  When I search all
source files for some text, keep track of what you learn then.

I should only pay for the time it takes to search globally, when I issue for global searches.

Rescanning all of my projects after the library build is not something I need the IDE to do.  Instead, it is something
at a feature of the IDE wants to do, to "color" my project tab with information that I don't need.  So, it should do
that either completely in the background, without impact to the IDE editing, build and run mechanisms, or it should not
do it at all until I ask specifically for that information.

> ... I now have to wait 5mins after rebuilding a library, for the IDE to rescan all my projects ...

I think NB java team could answer, I believe it is possible to have such a setup that projects are not rescanned after 
rebuilding a library.

Note that for NB 7.0 we plan to do an experiment by providing a special plugin which when installed would disable 
automatic scanning and let the user invoke it explicitly if needed. It was planned mainly for turning off checking 
external changes after IDE restart (project reopen) but perhaps it could be extended also to some cases of changes on 
project classpath.

I was analyzing a heap dump when I decided to look in the source of a web project.  I clicked the + beside it, and it
said "Initializing project...".  It said that for all web projects.  I waited a minute and it seemed stuck.  I restarted
the IDE a few times, rebooted, and it still said that.  Just before I was ready to delete my .netbeans folder, I noticed
the status bar said it was scanning all of my projects!  I made a meal, ate it, and when I got back NetBeans was done
scanning and my projects became usable again. 

Added self to CC list

I just had another episode of this behavior.  It occurs to me that netbeans should really just be checking the date on
the library jars when the user invokes a build, and ignore .class files older than the libraries in the classpath to
facilitate a robust build.  Error badges are just not adding value to my experience given the limited amount of
information that they actually convey.  There is no list of errors or anything else accessible after all of that work. 
The fastest way to see the errors I have to deal with is to run the compiler and look at the error messages.  That
provides clickable links to the source files etc, which IS helpful in finding out what needs to be fixed.

But guess what, this is already how I work.  When I first start working each day, I sync the SCM to my machine, look for
library updates from my coworkers, build those libraries, and then any project I will work on, I select "Clean and
Build" on to make sure I can see exactly what the state of affairs are.  It's that tree of dependencies I know netbeans
developers are trying to assist me with.  But, the simple problem is that I can't make use of all the work that they are
trying to do, instantaneously, and the time it takes to do that work is just so counter productive compared to the tasks
that I actually need to focus on.

> Error badges are just not adding value to my experience given the limited 
> amount of information that they actually convey.  There is no list of 
> errors or anything else accessible after all of that work. 

If you keep the Tasks pane open it will show you a list of all errors, which file, line, etc. If you double click on one
of the tasks it will open the source file and put the cursor on the line with an error.

Okay, so that was not an obvious, to me, place to look.  Now I look there and see all kinds of things to click on.  I
click on some of them that describe unresolved external class references.  I click on various ones and I don't get any
margin markings for errors, and no orange/yellow warnings either.

For example, here is one of the nonsense messages:

Error	cannot find symbol symbol  : method print(java.lang.String) location: class System.out	Gsm_State.java	71
C:/cvs/java.net/javecho/src/gsm/encoder

It's telling me (indirectly, why isn't the same message here that is on the "resolve reference problems" dialog?) that
the project is referencing a non-existent JVM.  There are hundreds of these errors (and associated badges) when only one
is needed.

I had another case just now where I rebuilt one of my core libraries.  NB(version 6.1) went off to rescan/compile all of
the dependent projects.  I noticed another edit I needed to make on a library source file.  I started making that edit
and the editor noted a syntax error as I opened a try{}catch.  But, it did not note when the braces were correctly
matched.  I was able to finish my edit, start another build, and that finished as well.  It then kicked off another set
of scans/compiles, which I believe are now happening in parallel.  The library built without error, but my editor is
still full of red bars in the margin and red lines in the source because it is waiting for use of the compiler or
something to make progress.

For each of your projects, right click the project tree node, select properties.  Click the Libraries tree node, and at
the bottom of th escreen un-check the "Build projects on classpath" checkbox, and click OK.  Does this solve your problems?

Most of my large projects, where it seems a lot of the time is spent, are freeform and this option is not present on the
java sources classpath pane.

Jarda wanted to do some experiments with disabling startups scan and file events.

This sounds like a performance enhancement. If this requires something from java team please create sub issues and
assign only the subissues to us.

There is an experimental module on AU which attempts to turn off automatic scanning and adds File -> 'Refresh All' menu
item. See http://wiki.netbeans.org/ScanOnDemand for more details. 

One possible way to implement this is to be able to specify a list of excludes of source (or binary) roots. By exclude I
mean to be able to say that these roots should not be scanned and/or that these roots should not be watched for
modifications. This would allow the users that generate new source code during the build to generate the code into the
folders that would not trigger the otherwise enabled automatic scan. Of course there would have to be an UI that would
allow to index such roots on demant (e.g. action on the root "Index" or similar). Similarly it might be possible to
exclude a .jar file from indexing, again when someone e.g. generates new jars (with the same data) when doing build.

Rather than removing the initial scan and letting the IDE to work on staled data it seems to me better to speed up the indexing (compiling) itself.
1) Is it needed to do the full attribution? Yes-when compile on save is turned on, but enter + symbol index (like JDT) may be enough otherwise.  Attr is about 
20% of java initial scan.
2) There are also possible improvements in the lucene usage. I've some experiments which improve the performance of LuceneIndex by 30%. The lucene takes 
also about 20% of the whole java indexing. Also when doing the indexing using JSR166 on 4 cpu (2*2 cores) machine the performance improvement is even 
better, but probably not yet main stream.

I'm surprised to learn that Lucene is being used for the source index. Does this make any sense? Source code, unlike 
document text, is highly structured, and source code index lookups have quite different requirements from full text 
searches.

Hi Guys, 

I don't think the increasing the speed will resolve the issue. We need to make the indexing non blocking. The initial
index is not so much of the concern as it happens only once. It's all the features that are blocked while the
incremental scans are done that's the biggest killer. 

Thanks,
Nigel.

bump. I would really, really appreciate the option to completely disable background scanning of projects, and error indicators in my project, completely. I have other tools I use to determine errors, I don't need, nor do I want, visual indicators about errors while I'm writing code.

I'm not interested in scanning or compilation of my code. I just want Netbeans to act like a simple code highlighter. I find 'error badges' to be extremely distracting flashing error notifications, and I'd like the ability to turn them off, completely.

Literally Netbeans flashes red warnings anytime you type one single character, and won't take no for an answer.