Created attachment 17827 [details]
3 thread dumps

It looks like there is several threads waiting on some Mutex but
nobody is inside any Mutex (and working or locked for some other reason).
So this seems to be a bug in Mutex implementation for me.

Not simple at all.
There are some threads inside some Mutexes and trying to obtain
others, but still it isn't clear what's wrong.
I'll attach commented snippet of the thread dump with all important
threads.
Generally, there are two Mutex instances involved, Children.MUTEX and
ProjectManager.mutex() (actually three, also Mutex.EVENT, but it has
no important role there).

In the attached partial dump, the first thread is writer in Ch.MUTEX
and tries also to obtain a read access on PM.mutex.

The second thread is just leaving write access of PM.mutex and running
postReadRequest() attached runnables. So the thread owns read access
on PM.mutex and also wants to get a write access on Ch.MUTEX (which is
blocked by 1st thread).

The third thread (AWT) just wants to enter as a writer to PM.mutex


My understanding of the problem is that our Mutex has no starvation
prevention policy and does no reordering.
The PM.mutex is only readlocked, so the first thread should be able to
proceed, but the third thread is queued before it with its write
request (please note that I haven't verified this behaviour of Mutex yet).

In that case, it may be possible to fix it in Mutex, but i'm still a
bit nervous about the first two threads and their locks.

Created attachment 17838 [details]
Simplified and commented thread dump.

I've managed to simulate the problem in a test (uff need some time to
single-threadize my brain again...).
Will continue later.

The fundamental problem is that Children.Keys should never have been
written to use a r/w lock at all. It doesn't make any sense - you
never "write" data to a C.K subclass, it simply listens for changes in
other data models and refreshes its display accordingly. It should be
running as a normal GUI component would, in the EQ with no funny locks.

ProjectsRootNode.ProjectChildren.stateChanges could of course refresh
its keys later. Might be best to simply call refreshKey in EQ (as
issue #35833's patch does for you automatically, in fact).

I've added test to Mutex that simulates the problem.
I'm blaming mutex for the problem although it would be simpler to fix
the problem elsewhere.

It seems like Mutex does not wake up other readers during
leaveX/attached readers processing. The third thread is not needed for
the deadlock.

Have a fix, Testing it now.

Fixed in Mutex:

openide/src/org/openide/util/Mutex.java,v1.62
openide/test/unit/src/org/openide/util/MutexTest.java,v1.11

v