Bug 192354 - Random locks on OpenProjectList
Random locks on OpenProjectList
Status: NEW
Product: projects
Classification: Unclassified
Component: Generic Infrastructure
7.0
All All
: P2 with 2 votes (vote)
: TBD
Assigned To: Milos Kleint
issues@projects
EXCEPTIONS_REPORT perf-profileme 80pa...
: 7.3_WAIVER_APPROVED, 7.4_WAIVER_APPROVED, 8.0_WAIVER_APPROVED, PERFORMANCE, THREAD
Depends on: 239556
Blocks:
  Show dependency treegraph
 
Reported: 2010-11-23 17:15 UTC by azizur
Modified: 2014-03-05 07:46 UTC (History)
106 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
nps snapshot (9.86 KB, application/nps)
2010-11-23 17:15 UTC, azizur
Details

Note You need to log in before you can comment on or make changes to this bug.
Description azizur 2010-11-23 17:15:22 UTC
Build: NetBeans IDE 7.0 Beta (Build 201011152355)
VM: Java HotSpot(TM) Client VM, 17.1-b03, Java(TM) SE Runtime Environment, 1.6.0_22-b04
OS: Windows XP

User Comments:
GUEST: loading a project

GUEST: Start the first time



Maximum slowness yet reported was 51031 ms, average is 14223
Comment 1 azizur 2010-11-23 17:15:27 UTC
Created attachment 103255 [details]
nps snapshot
Comment 2 sc1 2011-03-04 10:48:13 UTC
That very oftne happens opüening files or changeing Project Properties.
In this case at least one of the two PCs processors is completely busy (50% load).
One cannot work with that ...
Comment 3 Jesse Glick 2011-03-07 23:48:32 UTC
No general fix seems forthcoming. Various things need to get a read lock on OPL - NewFile ctor, ProjectsRootNode.getHtmlDisplayName, InspectHierarchyAction.enable, etc. If something is holding a write lock for a long time, readers will wait. One example is AntProjectHelper.saveXml which by itself should be pretty fast, but the file events fired cause DataObjectPool.FSListener.fileChanged to call a long loop in getTargets for no clear reason. Another example is class loading for Xalan (see bug #179720). Can either avoid work done under a lock - hard to see how in general; call OPL from asynch threads (88b2b6dc1de9 should fix one of several cases); or rewrite OPL to use some kind of concurrent collections, and wait for race condition bugs.
Comment 4 Alexander Simon 2013-01-25 14:32:13 UTC
548 reports => P2
Comment 5 Milos Kleint 2013-01-25 14:47:25 UTC
as evaluated by jglick: If something is holding a write lock for a long time, readers will wait. No solution in sight. A lot of the reports are slow due to slow native IO. I have 20+ bugs like that with no solution pending. if File.exists() takes a long time, it's fairly little we can do, if it happens in AWT we can postpone it but sometimes the off-AWT threads just hold locks for a long time for that reason. Does that mean AWT should never aquire any locks? I don't see how that could work.
Comment 6 Egor Ushakov 2013-01-25 15:01:21 UTC
(In reply to comment #5)
> as evaluated by jglick: If something is holding a write lock for a long time,
> readers will wait. No solution in sight. A lot of the reports are slow due to
> slow native IO. I have 20+ bugs like that with no solution pending. if
> File.exists() takes a long time, it's fairly little we can do, if it happens in
> AWT we can postpone it but sometimes the off-AWT threads just hold locks for a
> long time for that reason. Does that mean AWT should never aquire any locks? I
> don't see how that could work.

There is significant effort to avoid IO in AWT, see bug 218138 and dependent bugs. Doing IO in AWT is a bad practice.
Comment 7 Milos Kleint 2013-01-25 15:20:24 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > as evaluated by jglick: If something is holding a write lock for a long time,
> > readers will wait. No solution in sight. A lot of the reports are slow due to
> > slow native IO. I have 20+ bugs like that with no solution pending. if
> > File.exists() takes a long time, it's fairly little we can do, if it happens in
> > AWT we can postpone it but sometimes the off-AWT threads just hold locks for a
> > long time for that reason. Does that mean AWT should never aquire any locks? I
> > don't see how that could work.
> 
> There is significant effort to avoid IO in AWT, see bug 218138 and dependent
> bugs. Doing IO in AWT is a bad practice.

IMHO that's unrealistic expectation (the amount of work will be tremendous, including never loading a class in AWT) and even if you succeed, you will still have a problem of AWT locking with other threads doing the work. In the end it might not even help the people that currently have problems with File related slowdowns as the general responsiveness of the IDE will remain slow in their case.  
The primary problem needing investigation is why sometimes File.lastModified(), File.exists(), opening jar etc takes tens of seconds. We (as far as I know) don't know that. Windows is to be blamed partly (as windows related reports are more common). But it can be anything from low memory for java,  low memory in general, other processes requesting IO, bug in java, our native fs listening etc. Or some other code in netbeans that is io intensive at times.
Comment 8 Milos Kleint 2013-01-25 15:33:58 UTC
with regard to not calling IO in awt in general:
With the current level or modularization and the code calling various API, it's fairly unclear for the guy writing the AWT piece what is being done by the underlaying layers. The behaviour also differs in dev builds versus stable builds when asserts are off. And of course changes in time. Bugs get fixed, new features added and what was safe is now calling File.lastModified. 
It could be solved by sort of indicative annotations maybe that would somehow bubble up the layers of code but still clumsy.
Comment 9 bht 2013-01-25 16:28:36 UTC
My reports if any in this context would be one of the slowest if not the slowest of all. I agree with Milos in that shifting the I/O to another thread would not necessarily improve responsiveness of the IDE. Possibly the contrary due to more complex processing.

In my case I see that I/O is directly translated into CPU load due to OS file caches. With a lot of I/O happening in a short time, of course that data must be processed. Memory usage goes up. Even if there is sufficient memory (no swapping) then the garbage collector kicks in. If in this scenario the elapse time of any I/O is measured (as seen in the log) then this is interpreted as slow I/O where in fact it is CPU starvation, and the I/O is as slow as anything else. I/O is probably more costly because there is much of it and it may involve JNA which is slower than JNI. I am not 100% sure about the swapping because I cannot really detect it. Perhaps there is even swapping involving disk cache?

I am hoping that some optimisation will be found to reduce I/O, memory consumption and CPU use. I find that the IDE slows down over time while memory consumption goes up slowly, but not as much as to explain the slowdown.
Comment 10 Egor Ushakov 2013-01-25 17:28:03 UTC
In our case File.lastModified(), File.exists() and other IO operations are slow because we're using projects located on NFS. Unfortunately it is a frequent situation in big SW companies and NB need to handle that somehow. Now during some operations UI just hangs every other second which looks bad.
Comment 11 Jesse Glick 2013-01-25 19:00:15 UTC
Note that some of the duplicates are probably quite distinctive and obviously bugs: EQ waiting for something known to be time-consuming, not just class loading etc. but real work like loading projects. The trouble is that the exception reporter just dumps hundreds of barely related slowness reports in one bucket.

As mentioned in comment #3, introducing some concurrency to OPL using finer-grained locks would probably fix some of these bugs, but it could also reintroduce lots of race conditions and deadlocks that used to be associated with OPL before it was simplified. Would probably recommend keeping simple threading semantics for OPL but ensuring that its callers are prepared for it to block sometimes.
Comment 12 Milos Kleint 2013-01-28 10:36:27 UTC
one of the heuristics that could release some pressure would be *not* calculating enablement of New File.. action based on number of opened projects. If it stays always enabled we can bypass quite a few hangs reported (unless there's a next action waiting in the queue right behind it) 
Just commenting it out appears to have no sideffects, clicking the action in tolbar just does nothing. if issue 186943 gets implemented it would actually show just a subset of file templates.
Comment 13 Marian Mirilovic 2013-01-30 17:32:28 UTC
As the list of reports is pretty long and groups many non-related problems, QE agrees to waive it for NetBeans 7.3 with promise that we will do our best to fix it in one of the following patches. Thanks for your understanding and patience.
Comment 14 Quality Engineering 2013-01-31 03:13:21 UTC
Integrated into 'main-golden', will be available in build *201301310001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/4381b7a9f57b
User: Milos Kleint <mkleint@netbeans.org>
Log: #192354 experimentally try disabling the enablement check on NewFile
Comment 15 Milos Kleint 2014-01-15 10:14:04 UTC
http://hg.netbeans.org/core-main/rev/e2bbf65912ea 
could have a significant influence on this one since 8.0 beta


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo