Created attachment 3149 [details]
Full thread dump

I see also editor code in stack but lookup looks more suspicious =>
jtulach

dkaspar, can you reproduce the problem?  If the deadlock happens often
then please raise prio to P2, if it happens very often (sure steps to
reproduce it) then P1

IMO Lookup is (quite) innocent. I suspect that JavaDataObject
attempted to locate the indentation engine, while holding lock on
itself (createFromTemplate synchronizes *then* calls
handleCreateTemplate) - and the FolderRecognizer attempted to lock the
same (data) object.

Yes, I looked at it yesterday, to find out if it is editor issue.
The problem seems to be in these two threads

1.)
Thread-28:
at 
org.openide.loaders.MultiDataObject.handleCreateFromTemplate(MultiData
Object.java:523)
...
 synchronized (getSecondary ()) {
-getting lock
...


2.)
Folder recognizer:
at 
org.openide.loaders.MultiDataObject.markSecondaryEntriesRecognized(Mul
tiDataObject.java:209)
...
synchronized (secondary) {
-Deadlock
...

Good investigation Mato.

This is nearly unfixable. The problem is that MultiDataObject does
calls subclasses code while holding internal lock, it blocks on folder
processor and folder processor blocks on the MultiDataObject internal
lock. But I can imagine thousands different scenarios when the code
blocks.

Solutions:
1. either do not hold internal lock when calling subclasses methods =>
in this case we would not copy one entry (I guess)
2. improve the markAsRecognized => not sure whether it would help, but
might.
3. create easy fix in java data object

I vote for 3, fix the problem by overriding handleCreateFromTemplate
and initializing the indentation engine and after that calling to
super.handleCreateFromTemplate. If possible.

Then return the bug to loaders. We will write tests to identify the
possible deadlocks and try to enhance the behaviour somehow....

This bug is fully reproducible - raising priority.
How to reproduce
Installed ORBacus 4.x is needed.
Build CORBA.nbm
Start NB build #011026
Autoupdate CORBA.nbm
Create IDL from "CORBA | Simple" template in root of sampledir filesystem
Change IDL to this:
interface Simple {
  string op (in string aa);
};
Select Simple.idl in the Explorer
Do "Tools | CORBA Wizard..." main menu action.
Press Next.
Check Create Server, Create Client, Create Implementation checkboxes.
Press Next.
Select ORBacus 4.x ORB
Select IOR to standard output
Press Next three times
Press Finish
IDL is compiled, implementation is generated.
ERROR - Deadlock
See attachment for new Full Thread Dump

Created attachment 3166 [details]
new Full Thread Dump

Created attachment 3167 [details]
Correct Full Thread Dump - ignore previous attachment

Fixed in trunk.

/cvs/java/src/org/netbeans/modules/java/CodeGenerator.java,v  <-- 
CodeGenerator.java
new revision: delete; previous revision: 1.21
Checking in IndentFileEntry.java;
/cvs/java/src/org/netbeans/modules/java/IndentFileEntry.java,v  <-- 
IndentFileEntry.java
new revision: 1.6; previous revision: 1.5
Checking in JavaDataObject.java;
/cvs/java/src/org/netbeans/modules/java/JavaDataObject.java,v  <-- 
JavaDataObject.java

Reassigning back to openide and lowering the priority. The necessary
condition for the deadlock was broken in java module.

Yarda, up to you to decide what's next.  This is a general threading
issue.

Very likely we will not do anything with it for 3.3. A bit dangerous
to mangle with this code.


One possible fix would be to stop using locks and move the processing
into Folder Recognizer thread. That might be fine, because most of the
invocations are done by that thread anyway. The code would look like:

if (FolderList.isRecognizerThread ()) {
	runNow ();
} else {
	FolderList.postTask (runLater ());
}

But first of all someone should write a test that will reproduce the
deadlock and we will know what we want to fix.

The actual problem is fixed and I suggest not to fix the general issue
in current data system implementation, but concentrate on preventing
this in the new one.

Because the mark secondaryEntriesRecognized method is not in the new
design and all its tasks are handled by the infrastructure itself, I
believe that this kind of general problem cannot occur anymore.

I would like David to review the code in the new data systems and if
he agrees with this, then close this issue.

Taking into account Svata's fix and Jarda's claims I am closing this
as fixed for now.

Verified

Resolved for 3.4.x or earlier, no new info since then -> closing.