Bug 204427 - Deadlock while adding Spring MVC framework into Maven Web Application
Deadlock while adding Spring MVC framework into Maven Web Application
Status: VERIFIED FIXED
Product: javaee
Classification: Unclassified
Component: Spring
7.1
PC Windows 7
: P2 (vote)
: 7.1
Assigned To: Martin Fousek
issues@javaee
: 71_HR_FIX
: 204428 204840 205315 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-10-31 10:21 UTC by Martin Janicek
Modified: 2011-11-24 08:46 UTC (History)
10 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
Thread dump (42.54 KB, text/plain)
2011-10-31 12:49 UTC, Martin Janicek
Details
Thread dump - only important threads (5.82 KB, text/plain)
2011-11-01 13:59 UTC, Martin Janicek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Janicek 2011-10-31 10:21:29 UTC
Steps to reproduce:

1. Create Jave EE6 Maven Web Project
2. Go to Project Properties, frameworks tab
3. Add Spring Web MVC framework
4. Click OK --> application freeze

Probably related to changes made in web-main revision 1e746549ad1a.
Comment 1 David Konecny 2011-10-31 10:27:15 UTC
Can you attach a threaddump?
Comment 2 Martin Janicek 2011-10-31 12:49:46 UTC
Created attachment 112594 [details]
Thread dump
Comment 3 Martin Janicek 2011-10-31 13:09:27 UTC
Similar problem were in Ant projects long time ago (issue 68118).

I've found a way how to deal with this, but I'm not sure if the solution is correct. If I wrap applyChanges code to SwingUtilities.invokeLater(), then everything runs just fine. Similar code can be found for Ant projects.

Do you think that it's correct to do so even if the running code isn't AWT related only?
Comment 4 Martin Janicek 2011-10-31 15:31:35 UTC
*** Bug 204428 has been marked as a duplicate of this bug. ***
Comment 5 Martin Janicek 2011-11-01 13:56:17 UTC
Reassigning to Core team. CloneableEditorSupport code (concretely method openDocumentImpl()) leads to deadlock in some cases. I'm attaching thread dump of those two threads causing the deadlock. The situation why is it happening:

First, the thread with nid=0x1874 is started. It happens when Project properties panel is created and it gets a ProjectManager.mutex().writeAccess right after the creation so it has exclusive access (see CustomizerDialog class, line 294).

When some frameworks are added in the Frameworks customizer panel, there is a call for each concrete framework extenders. Those extenders need to create project files, modify configuration files and so on. Everything goes fine if there is no need for extenders to read some project data, BUT if extender needs to find project and read some information as in this case, then the application gets stucked.

In such a case second thread (nid=0x1aa8) is started inside of CloneableEditorSupport at the line 682 (the call was made from line 889). Right after creation of the second thread, the first thread is going to wait to the second one to finish it's operations (line 902). Here is the point when everything goes fine if there is no need to read some project data inside of framework extender. But if we need to do so, second thread is going to wait for ProjectManager.mutex().readAccess, but that LOCK is holden by first thread.

As a result the first thread has PM.mutex().writeAcces LOCK and it is waiting for the second one to finish, while the second thread is trying to get PM.mutex().readAccess to be able to finish.
Comment 6 Martin Janicek 2011-11-01 13:59:18 UTC
Created attachment 112669 [details]
Thread dump - only important threads
Comment 7 Martin Janicek 2011-11-08 15:15:27 UTC
*** Bug 204840 has been marked as a duplicate of this bug. ***
Comment 8 Martin Janicek 2011-11-11 11:51:29 UTC
Guys can you evaluate this?.. It has to be fixed for 7.1. Either on your side or if there is no time do so for 7.1 I can provide a workarround for now.
Comment 9 Miloslav Metelka 2011-11-11 16:19:46 UTC
The jsploader.BaseJspEditorSupport.loadFromStreamToKit() does some non-trivial processing that finally leads to the deadlock. Reassigning to web module for evaluation.
Comment 10 Marek Fukala 2011-11-14 16:05:24 UTC
fixed in web-main#c4e35a2cca9f

Martine, please verify the fix since it is your are now.
Comment 11 Martin Fousek 2011-11-15 10:51:15 UTC
Your fix looks really well but it will stuck later (createSyntax?). If I hacked that and removed the createSyntax we figured out that it's locking still somewhere else. It's going to be really complicated. :/
Comment 12 Quality Engineering 2011-11-15 15:41:20 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/c4e35a2cca9f
User: Marek Fukala <mfukala@netbeans.org>
Log: #204427 - Deadlock while adding Spring MVC framework into Maven Web Application
Comment 13 Marek Fukala 2011-11-15 16:31:13 UTC
the other deadlock in JspKit.createSyntaxSupport() waiting on project's lock fixed in web-main#b8d4d0280c8c
Comment 14 Marek Fukala 2011-11-15 16:36:11 UTC
now we have another one:

"Parsing & Indexing Loop (20111111-c15db42ed827)" daemon prio=1 tid=000000000217c400 nid=0xb58e6000 in Object.wait() [00000000b58e5000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at org.openide.util.Mutex$QueueCell.sleep(Mutex.java:1612)
	- locked <000000000f7ef3e0> (a org.openide.util.Mutex$QueueCell)
	at org.openide.util.Mutex.enterImpl(Mutex.java:727)
	at org.openide.util.Mutex.enter(Mutex.java:634)
	at org.openide.util.Mutex.readEnter(Mutex.java:617)
	at org.openide.util.Mutex.readAccess(Mutex.java:327)
	at org.netbeans.api.project.ProjectManager.findProject(ProjectManager.java:233)
	at org.netbeans.modules.projectapi.SimpleFileOwnerQueryImplementation.getOwner(SimpleFileOwnerQueryImplementation.java:131)
	at org.netbeans.api.project.FileOwnerQuery.getOwner(FileOwnerQuery.java:104)
	at org.netbeans.modules.csl.core.ProjectClassPathProvider.findClassPath(ProjectClassPathProvider.java:66)
	at org.netbeans.api.java.classpath.ClassPath.getClassPath(ClassPath.java:632)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.getClassPathEntry(RepositoryUpdater.java:1431)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.access$1900(RepositoryUpdater.java:149)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.scanFiles(RepositoryUpdater.java:2419)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$FileListWork.getDone(RepositoryUpdater.java:2719)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.doTheWork(RepositoryUpdater.java:2505)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task._run(RepositoryUpdater.java:4518)
	at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task.run(RepositoryUpdater.java:4455)
	at org.netbeans.modules.parsing.impl.TaskProcessor.callParserResultTask(TaskProcessor.java:618)
	at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:728)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:680)


versus

"Default RequestProcessor" daemon prio=1 tid=000000005200bc00 nid=0xb2a2d000 waiting on condition [00000000b2a2c000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <000000000a9ab858> (a java.util.concurrent.locks.ReentrantLock$FairSync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
	at java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:201)
	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
	at org.netbeans.modules.parsing.impl.TaskProcessor.acquireParserLock(TaskProcessor.java:508)
	at org.netbeans.modules.parsing.impl.Utilities.acquireParserLock(Utilities.java:76)
	at org.netbeans.modules.java.source.save.Reformatter$2.lock(Reformatter.java:330)
	at org.netbeans.modules.editor.indent.TaskHandler$MimeItem.lock(TaskHandler.java:541)
	at org.netbeans.modules.editor.indent.TaskHandler.lock(TaskHandler.java:277)
	at org.netbeans.modules.editor.indent.IndentImpl.reformatLock(IndentImpl.java:205)
	- locked <0000000006549d90> (a java.lang.Object)
	at org.netbeans.modules.editor.indent.api.Reformat.lock(Reformat.java:114)
	at org.netbeans.modules.editor.EditorModule.reformat(EditorModule.java:643)
	at org.netbeans.modules.editor.EditorModule.access$100(EditorModule.java:108)
	at org.netbeans.modules.editor.EditorModule$5.operationCreateFromTemplate(EditorModule.java:330)
	at org.openide.loaders.DataLoaderPool.fireOperationEvent(DataLoaderPool.java:240)
	at org.openide.loaders.DataObject.fireOperationEvent(DataObject.java:866)
	at org.openide.loaders.DataObject.createFromTemplate(DataObject.java:839)
	at org.openide.loaders.DataObject.createFromTemplate(DataObject.java:818)
	at org.netbeans.modules.spring.webmvc.SpringWebModuleExtender$CreateSpringConfig.createFromTemplate(SpringWebModuleExtender.java:418)
	at org.netbeans.modules.spring.webmvc.SpringWebModuleExtender$CreateSpringConfig.run(SpringWebModuleExtender.java:364)
	at org.openide.filesystems.EventControl.runAtomicAction(EventControl.java:127)
	at org.openide.filesystems.FileSystem.runAtomicAction(FileSystem.java:566)
	at org.netbeans.modules.spring.webmvc.SpringWebModuleExtender.extend(SpringWebModuleExtender.java:223)
	at org.netbeans.modules.maven.j2ee.customizer.CustomizerFrameworks.applyChanges(CustomizerFrameworks.java:129)
	at org.netbeans.modules.maven.j2ee.customizer.MavenCompositePanelProvider$2.actionPerformed(MavenCompositePanelProvider.java:155)
	at org.netbeans.modules.project.uiapi.CustomizerDialog$OptionListener.storePerformed(CustomizerDialog.java:338)
	at org.netbeans.modules.project.uiapi.CustomizerDialog$OptionListener.access$400(CustomizerDialog.java:242)
	at org.netbeans.modules.project.uiapi.CustomizerDialog$OptionListener$2$1.run(CustomizerDialog.java:300)
	at org.openide.util.Mutex.writeAccess(Mutex.java:397)
	at org.netbeans.modules.project.uiapi.CustomizerDialog$OptionListener$2.run(CustomizerDialog.java:294)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
Comment 15 Marek Fukala 2011-11-15 16:47:21 UTC
"Default RequestProcessor" thread under project's mutex write lock tries to acquire parser lock whereas respository updater does the opposite.

Likely the spring support should not create the templates under project's MUTEX write access.
Comment 16 Martin Fousek 2011-11-16 13:49:03 UTC
Understand. Spring is not the one who acquires the projects mutex write lock. It is acquired by CustomizerDialog.OptionListener#actionPerformed, so I'm not pretty sure how to workaround that problem. 

If I would omit calling project extending in the EDT... It could be done on the background but in such case the customizer's progress bar would disappear earlier than the whole Web framework support would be included into the project. Or I could perform only templates creation in the background but I would still need to wait for their creation and it would still lead to deadlock.

Tomas or Jesse, please could you advice the right way? Thanks...
Comment 17 Jesse Glick 2011-11-16 14:47:00 UTC
I do not see why EditorModule.reformat (syntactic level) should be waiting on the parser (semantic level), especially since this is being called from the rather low-level DataObject.createFromTemplate. A bug in editor IMO.
Comment 18 Quality Engineering 2011-11-16 15:46:27 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/b8d4d0280c8c
User: Marek Fukala <mfukala@netbeans.org>
Log: #204427 - fixed another deadlock
Comment 19 Tomas Zezula 2011-11-16 16:11:26 UTC
>I do not see why EditorModule.reformat (syntactic level) should be waiting on
>the parser (semantic level)
Parse is syntax, attribute is semantic level.

There are two problems:
1st) The SpringWebModuleExtender$CreateSpringConfig.createFromTemplate calls createFromTemplate (reformat) under project mutex.

2nd) The BaseJspEditorSupport.loadFromStreamToKit calls FileEncodingQuery (FileOwnerQuery) under editor lock

The CreateSpringConfig.createFromTemplate can be probably rescheduled (PM.mutex.pRA).
Comment 20 Jesse Glick 2011-11-16 16:37:07 UTC
(In reply to comment #19)
> 1st) The SpringWebModuleExtender$CreateSpringConfig.createFromTemplate calls
> createFromTemplate (reformat) under project mutex.

It does not call reformat. It just calls cFT, which is a simple and commonplace call that ought to be safe, and for which the return value is generally critical for subsequent code. If anything, it is EditorModule$5.operationCreateFromTemplate which ought to be invoke its body asynch - there is no contract saying that newly created files should be silently reformatted, and calling into complex APIs from a listener callback is always dangerous.
Comment 21 Martin Fousek 2011-11-16 17:51:04 UTC
> 2nd) The BaseJspEditorSupport.loadFromStreamToKit calls FileEncodingQuery
> (FileOwnerQuery) under editor lock

Suppose you are speaking about the original attached thread dumps. It's already solved by Marek and the new one is in comment #14.

> 
> The CreateSpringConfig.createFromTemplate can be probably rescheduled
> (PM.mutex.pRA).

Hmm probably not as a ideal solution. As I mentioned when I would reschedule that, the Saving properties notificator probably couldn't wait for these files creation (to prevent another deadlock). It looked to me that there isn't done nothing wrong (to call DO.cFT by saving project customizer) for now.
Comment 22 Tomas Zezula 2011-11-18 07:28:06 UTC
> It looked to me that there isn't done nothing wrong (to call DO.cFT by saving project customizer) for now.
Except calling 3rd party code under lock.
Comment 23 Martin Fousek 2011-11-21 12:53:36 UTC
Ok, after offline discussion with editor guys we come to following conclusion. The safest way, especially when the solution has to be transplanted into NB71, is really to run project extending in AWT EDT.

Fixed in web-main #f9f9e02059de.

BTW, just this last changeset should be enough for fixing that deadlock, so I will backport just this one. Jirko, could you verify that deadlock doesn't happen any more once #f9f9e02059de will be in the dev build?

Thanks you all for your comments...
Comment 24 Martin Fousek 2011-11-21 12:57:55 UTC
*** Bug 205315 has been marked as a duplicate of this bug. ***
Comment 25 Jesse Glick 2011-11-21 17:26:15 UTC
Regardless of what is done as a conservative fix for 7.1, I still think EditorModule.reformat was at fault here; the operationCreateFromTemplate handler should call it asynch.
Comment 26 Jiri Skrivanek 2011-11-22 09:25:45 UTC
Fix verified in custom dev build 20111122-9a8edc0fe2de.
Comment 27 Martin Fousek 2011-11-22 12:45:10 UTC
Please, could anybody (i.e. mjanicek as a owner of maven.j2ee) review the changeset web-main #f9f9e02059de to comply with HR process. Thanks.
Comment 28 Martin Janicek 2011-11-22 12:52:58 UTC
Yup, patch looks OK.
Comment 29 Quality Engineering 2011-11-22 15:50:59 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/f9f9e02059de
User: Martin Fousek <marfous@netbeans.org>
Log: #204427 - Deadlock while adding Spring MVC framework into Maven Web Application
Comment 30 Martin Fousek 2011-11-23 10:01:09 UTC
Transplanted as changeset: releases #d64decfd7680
Comment 31 Quality Engineering 2011-11-24 07:15:34 UTC
Integrated into 'releases'
Changeset: http://hg.netbeans.org/releases/rev/d64decfd7680
User: Martin Fousek <marfous@netbeans.org>
Log: #204427 - Deadlock while adding Spring MVC framework into Maven Web Application
(transplanted from f9f9e02059de6bc3163646785e5f42b26557cd15)
Comment 32 Jiri Skrivanek 2011-11-24 08:46:53 UTC
Verified in NetBeans IDE 7.1 RC1 (Build 201111232200).


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo