As a background - we had a number of bugs related to JavaDataObject
not knowing its .class files or reporting it is out of date (when it
was compiled, actually). We did a minimal refresh in NB-3.3, and the
"global" refresh catched the corner cases, like files reached and
compiled through closure etc.

The refresh is not all-covering, actually, or should not be. It
refreshes a folder, and its children folders, which existed before
(not the new ones which appeared). I have no other clue about how to
prune the tree of potentially changed objects, sorry -- but maybe I
overlooked some trick, please advise.

IMHO not possible to prune refreshing too much further for 3.5.

But it could at least skip refreshing the whole SystemFS, couldn't it?
I've added a log to MFO.refresh and got answer "42" ;-)
All the SFS folders were refreshed, like:

MFO.refresh(DTDs/4_0)
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1071)
at o.o.fs.MultiFileObject.refresh(MultiFileObject.java:371)
at o.o.fs.AbstractFolder.refresh(AbstractFolder.java:389)
at o.o.fs.MultiFileObject.refresh(MultiFileObject.java:1178)
at o.o.fs.FileObject.refresh(FileObject.java:662)
at o.n.m.java.JExtCG.refresh(JExternalCompilerGroup.java:560)
at o.n.m.java.JExtCG.refresh(JExternalCompilerGroup.java:574)
at o.n.m.java.JExtCG.refresh(JExternalCompilerGroup.java:574)
at o.n.m.java.JExtCG.refreshFS(JExternalCompilerGroup.java:554)
at o.n.m.java.JExtCG.start(JExternalCompilerGroup.java:197)
at o.n.core.compiler.CEImpl$CT$GC.run(CompilationEngineImpl.java:280)

Nevertheless the SFS refresh take ~400-800ms (Why is SFS actually
marked as "Use in compiler?"), but the CVSFS, on which I had the
source, took 1300-3300ms ;-( while the refresh of the LocalFS
over the same data took ~500ms

Now, the strange thing is that you do explictely recursive refresh
forcing initialization of the FO's nobody is yet interrested in.
Is it neccessary?
Wouldn't fs.refresh(true) be sufficient?
(Why does a "refresh" of never used JavaCVS FS take 3s?)

I've tested the fs.refresh() approach and it works correctly
and is *way* faster. For some people (phrebejk with ~20 JavaCVS
FSes mounted), it makes a difference of ~97% (60s vs. 2s, he was
complaining very loudly...)
We should certainly fix this for 3.5

> We should certainly fix this for 3.5

agreed

Created attachment 9566 [details]
Proposed patch

Fixed in trunk: 
 
/cvs/java/src/org/netbeans/modules/java/JExternalCompilerGroup.java,v  
<--  JExternalCompilerGroup.java 
new revision: 1.47; previous revision: 1.46 
 
Comments: 
I excluded SFS from the refresh entirely; don't know why it has 
COMPILE capability, but will not dare to change that for 3.5 (and it will 
be hopefully trashed for the next version). 
 
The recursive refresh refreshes one level more: it refreshes pre-existing 
directories, which may not yet read their children (so it creates 
additional FileObjects). 
On the contrary, FileSystem.refresh() refreshes all live objects, so as I 
understand it, it will do equivalent of refresh() even on .properties, .gif 
and other files, which were ignored by the refresh code implemented in 
External Compiler (it avoided calling refresh() on them, if they were 
refreshed as a part of their parent refresh, file a bug against 
filesystems). 
Please attach a measurement for local filesystem, too. 
 
I do not consider this an appropriate fix for NetBeans 4.0, because the 
number of live FileObjects will become the union of all opened projects 
and the difference between number of operations done by 
HostFS.refresh() and refresh of only some files in some subtrees 
(targets of compilation) will be potentially great. 
 

For LocalFS, when I was testing it, the times were like 500ms vs 50ms
but it depends on many factors, like the number of live FOs.
The patch looks OK, I was trying about the same.


Without the fix, the JExtCG would spend quite some time even
on the filesystems which are collapsed in the explorer and otherwise
idle (as you said).

To the selective refresh: filesystems library is perfectly prepared
for this functionality so implementing it is just a question of
consensus and few lines of code.
Radek agreed it is a reasonable request so you can expect
something like FileObject.refresh(boolean expected, boolean recursive)

OK, great; for 3.* the difference b/w selective and full refresh of live 
FOs should not be so visible anyway, as all the FOs are used in the 
(single) opened project. 

Created attachment 9570 [details]
Binary patch for testing

Unzip the patch in the installation directory & play. 

The patch is OK.

The patch seems to be OK from QA point of view.

approved for 3.5

Merged into rel35: 
 
/cvs/java/src/org/netbeans/modules/java/JExternalCompilerGroup.java,v  
<--  JExternalCompilerGroup.java 
new revision: 1.46.2.1; previous revision: 1.46 
 

Cool!