Created attachment 77730 [details]
thread dump

Easily reproducible by unplugging a network cable and trying to expand a folder. The question is why is explorer +
FolderChildren blocking EQ while performing Filesystems operations? A wait cursor is correctly displayed but EQ ought
not be blocked - should just show the children when they arrive.

"Default RequestProcessor" daemon prio=1 tid=0x0a173988 nid=0x5583 runnable [0x78c36000..0x78c370b0]
....
        at org.openide.filesystems.AbstractFolder.getChildren(AbstractFolder.java:245)
        - locked <0x89d92220> (a org.openide.filesystems.AbstractFileObject)
        at org.openide.loaders.FolderChildren$1R.run(FolderChildren.java:150)
        at org.openide.loaders.FolderChildren.refreshChildren(FolderChildren.java:185)
        at org.openide.loaders.FolderChildren.addNotify(FolderChildren.java:279)
        at org.openide.nodes.Children.callAddNotify(Children.java:546)
        at org.openide.nodes.EntrySupport$Lazy.checkInit(EntrySupport.java:1024)
        at org.openide.nodes.EntrySupport$Lazy.getNodesCount(EntrySupport.java:1208)
        at org.openide.nodes.Children.getNodesCount(Children.java:480)
        at org.openide.loaders.FolderChildren.getNodesCount(FolderChildren.java:254)
        at org.openide.explorer.view.TreeView$6.run(TreeView.java:912)
        at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:573)
        at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1005)
"AWT-EventQueue-1" prio=1 tid=0x09d148a0 nid=0x5565 in Object.wait() [0x7acfe000..0x7acfeeb0]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x85fd0328> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:474)
        at org.openide.nodes.EntrySupport$Lazy.checkInit(EntrySupport.java:1063)
        - locked <0x85fd0328> (a java.lang.Object)
        at org.openide.nodes.EntrySupport$Lazy.getNodesCount(EntrySupport.java:1208)
        at org.openide.nodes.Children.getNodesCount(Children.java:480)
        at org.openide.explorer.view.VisualizerNode.getChildren(VisualizerNode.java:257)
        at org.openide.explorer.view.VisualizerNode.getChildren(VisualizerNode.java:247)
        at org.openide.explorer.view.VisualizerNode.getChildCount(VisualizerNode.java:324)
        at javax.swing.tree.DefaultTreeModel.getChildCount(DefaultTreeModel.java:168)
....

Passing to nodes for evaluation.

When FolderChildren were changed to use "lazy nodes" it started to compute keys synchronously in certain cases (like
addNotify()). We discussed it with jtulach and he consider current behavior as desirable and proposed solution that this
synchronous call for children would be replaced by getAttribute("getChildrenFast") allowing slow filesystems to return
empty array immediately and fire change later (see patch). Jesse, what do you think about this hack?

Created attachment 78726 [details]
patch

The patch looks wrong to me. The Filesystems layer is behaving correctly, as is the Datasystems layer. Returning an
empty children list from AbstractFileSystem.Children.list (which is what I think you are suggesting) would be incorrect
and probably cause a lot of problems: I do more with this FileSystem than simply display nodes for its data objects.

The problem is FolderChildren. It should not be computing keys synchronously when the filesystem is potentially slow,
e.g. when it is using a remote URL protocol. (I am not sure why it would compute keys synchronously when the filesystem
is "local" either - it could be NFS/CIFS/etc. - but that's not my business.)

Anyway the patch looks like it would cause an NPE for a non-AbstractFileSystem FileSystem impl which did not do anything
special.

Anyway it seems most of the time is spent creating the DataObject's for each FileObject (because of MIME resolution),
not getting the list of FileObject's per se. I thought the whole point of the lazy nodes effort was to make things like
this work incrementally.

Created attachment 78808 [details]
Corrected patch

Another option is to change FolderChildren to run refreshChildre() asynchronously all the time like FolderChildrenEager
and keep improvements like using FO as keys instead of DO.

Is the "corrected patch" intended to be on top of the previous patch, or a replacement for it? If the latter, where is
"getChildrenFast" coming from?

BTW is there some reason 'operation' is not an enum? Makes it hard to follow code when '10' is a special value...

It is replacement for previous patch. getAttribute("getChildrenFast") is meant as way how to ask slow FS for children
(if FS decide to support it). jtulach asked me to put here corrected patch because he got the feeling that you did not
understand the idea from previous patch. I agree operation codes should be replaced by more readable enum, but it is a
question for jtulach.

I don't think I understand the idea from the current patch either. What is there for the filesystem to "support"? I
extend AbstractFileSystem, and when asked for List.children(), I make an HTTP connection, parse the result, and return a
list of child names, as required by the contract. (See hudson/src/.../RemoteFileSystem.java.) AFAIK the filesystem is
behaving correctly already. I just want some way to indicate to the node tree that it should never call
FileObject.getChildren() in EQ - assuming that doing so is desirable for a FS based on java.io.File (*).

(*) Again, it is not obvious that calling FO.gC synch is even desirable for "local" filesystems, since these may in fact
be NFS mounts or similar and there may be significant network lag.

Implement AFS.Attr, in readAttribute check for getChildrenFast. Return new FO[0]. Start another thread that will do 
the HTTP connection and fire the childrenAdded change as soon as it is gets the result. In spite that the attribute 
has to be tested, documented in arch.xml, etc., this is far simpler and safe change than rewritting FolderChilden.

But both variants are possible and as (*) complete rewrite is even better. However I, as someone watching from 
distance, don't really want to push you in direction that you might be sorry for.

Integrated into 'main-golden', will be available in build *200904041400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/28e7b99fe2fc
User: Jesse Glick <jglick@netbeans.org>
Log: Using an enum for clarity. (Mentioned in #159628.)

That patch did not work for me; EQ was not blocked, but the folder never showed any children either. However an even
simpler variant did work: just use SHALLOW (not SHALLOW_IMMEDIATE) in addNotify for remote FSs.

A couple of other modules were also blocking EQ asking for folder children during popup menu construction, which I am
also fixing.

cdev #ef2e253813ab

Integrated into 'main-golden', will be available in build *200904071400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/ef2e253813ab
User: Jesse Glick <jglick@netbeans.org>
Log: Issue #159628: do not freeze EQ in folder expansion for a remote filesystem.

Verified

Product Version: NetBeans IDE Dev (Build 200904081400)
Java: 1.6.0_10; Java HotSpot(TM) Client VM 11.0-b15
System: Linux version 2.6.27-14-generic running on i386; UTF-8; en_US (nb)