Created attachment 41491 [details]
thread dump

old == 5.5,  5.0

The deadlock is caused by JsfJavaDataLoader using FileOwnerQuery to recognize
its primary file.

Increasing priority because this deadlock affects Mobility very frequently.

Created attachment 41537 [details]
mobile project from 5.5

this issue happens with "standard" NB distro when opening Mobility project. Test
project attached. This is stopper for M9

Visual web decided not to use FileOwnerQuery in JsfJavaDataLoader for NetBeans
5.5. I'm not sure why it comes back in NetBeans 6.

JsfJavaDataLoader only calls
org.netbeans.modules.visualweb.project.jsf.api.JsfProjectUtils.getJspForJava,
which in turn uses FileOwnerQuery.  I think this should be fixed in the
visualweb/project/jsf module rather than in the dataloader.

I try the attached project but cannot reproduce it. Is it always reproducible?

From the stack, jsfloader was called by threads "Folder recognizer" and
"Versioning", and enter Mutex.readAccess. Theoretically the whole IDE should not
hang because Opening a project should all under Mutex.readAccess unless "opening
a old mobility project" involved Mutex.writeAccess. If so, then the coding in
'opening old' is not corrent in mobility modules.

BTW, I did see one hang with the status line says:
    Compiling D:\J2MEUnitTestingProject\src

I think that is not right when opening a project.

Now jsfloader validates the data object iff it's under a web project with visual
web framework used. This should fix the issue.

Created attachment 41729 [details]
another thread dump

reopening, it happened again in following configuration

NetBeans IDE Dev (Build 070426)
1.6.0-rc; Java HotSpot(TM) Client VM 1.6.0-rc-b104
Windows XP version 5.1 running on x86
en_US (nb); Cp1252

Look into the new thread dump, there is NO visualweb modules involved. Actually
it was not visualweb issue neither though visualweb now makes its codes not
participate this race-condition. (mobility pack's below action triggers the
incorrect dataloader actions.)

The issue is in the mobility pack. As I said before, I see "Compiling
D:\J2MEUnitTestingProject\src" when opening this project. Recently Java Sources
module did change some architecture and make various codes easier go into
deadlock if not carefully. Looks like mobility pack's codes are doing
writeAccess actions while 'opening project' should under readAccess.

This is a different deadlock. The first one was caused by using FileOwnerQuery
from DataLoaders. 

This one is caused by direct propagation of project.properties change into all
listeners when project Mutex is locked in write mode.

I can fix this one in our CompiledSourceForBinaryQuery and J2MEProject.

I've found a dangerous pattern when a synchronized query method requres reading
from project properties.

The sample here is CompiledSourceForBinaryQuery that is first invoked from
RepositoryUpdater but could not proceed because at the same time
AntProjectHelper already holds write Mutex to the project.
When the AntProjectHelper is ready with writing it fires property changes which
lead into subsequent call of CompiledSourceForBinaryQuery still under write Mutex.
So the deadlock is here - two instances of the query - one inside synchronized
block and one holding write Mutex.

I've found similar dengerous patterns also in  
FileBuiltQueryImpl, JavadocForBinaryQueryImpl and FileEncodingQueryImpl.

All the patterns are now fixed in trunk, waiting for approval to integrate into M9.

Created attachment 41807 [details]
Diff of the suggested fixes

I see at least 2 thing which I don't like and it's not probably all
1) inconsistency in getRoots method of CompiledSourceForBinaryQuery and 
JavadocForBinaryQueryImpl
---
                FileObject[] fo = cache;
                if (fo == null) {
                    cache = new FileObject[0];
                    fo = createRoots();
---
                URL [] urls = cache;
                if (urls == null) {
                    urls = createRoots();

Why once is there  cache = new FileObject[0]; and in second case is missing 
cache=new URL[0] ?

2) Possible data integrity issue
   cache variable is changed in sychronized block, however URL [] urls = 
cache; or  FileObject[] fo = cache; are not synchronized
it should be something like sychronized(lock) { urls = cache };

thanks for the comments
ad #1 - I found it in Compiled query and missed in Javadoc query   - thanks
ad #2 - it is safe even without synchronisation, the code excepts all possibilities


As there are no objections I am merging the chages into M9

v