Bug 186744 - Adding library causes huge I/O load
Adding library causes huge I/O load
Status: RESOLVED FIXED
Product: java
Classification: Unclassified
Component: Source
6.x
Other Linux
: P2 (vote)
: 7.0
Assigned To: Tomas Zezula
issues@java
: PERFORMANCE, PLAN
Depends on: 191316
Blocks:
  Show dependency treegraph
 
Reported: 2010-05-26 14:12 UTC by Jaroslav Tulach
Modified: 2012-02-24 09:29 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
Result of self-profile and export to ... action, hopefully openable in NetBeans profiler (214.71 KB, application/x-gzip)
2010-05-26 15:15 UTC, Jaroslav Tulach
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jaroslav Tulach 2010-05-26 14:12:30 UTC
I have a project with bunch of libraries. When I add yet another one to the list of its libraries, the UI is completely frozen for few seconds and the I/O load is high. Stack trace shows:

"Parsing & Indexing Loop (100511-5b0ca7653b2c)" 
java.lang.Thread.State: RUNNABLE
java.io.UnixFileSystem.delete0(Native Method)
java.io.UnixFileSystem.delete(UnixFileSystem.java:266)
java.io.File.delete(File.java:921)
o.n.m.j.s.parsing.FileObjects.deleteRecursively(FileObjects.java:604)
o.n.m.j.s.indexing.JavaBinaryIndexer$1.run(JavaBinaryIndexer.java:81)
o.n.m.j.s.indexing.JavaBinaryIndexer$1.run(JavaBinaryIndexer.java:77)
o.n.m.j.s.usages.ClassIndexManager.prepareWriteLock(ClassIndexManager.java:112)
o.n.m.j.s.indexing.JavaBinaryIndexer.index(JavaBinaryIndexer.java:77)
o.n.m.parsing.spi.indexing.Indexable$MyAccessor$1.run(Indexable.java:160)
o.n.m.p.i.i.RepositoryUpdater.runIndexer(RU.java:237)
o.n.m.parsing.spi.indexing.Indexable$MyAccessor.index(Indexable.java:158)
o.n.m.p.i.i.RepositoryUpdater$Work.indexBinary(RU.java:1848)
o.n.m.p.i.i.RepositoryUpdater$AbstractRootsWork.scanBinary(RU.java:3064)
o.n.m.p.i.i.RepositoryUpdater$AbstractRootsWork.scanBinaries(RU.java:3040)
o.n.m.p.i.i.RepositoryUpdater$RootsWork.getDone(RU.java:2870)
o.n.m.p.i.i.RepositoryUpdater$Work.doTheWork(RU.java:2012)
o.n.m.p.i.i.RepositoryUpdater$Task._run(RU.java:3605)
o.n.m.p.i.i.RepositoryUpdater$Task.run(RU.java:3545)
o.n.m.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:577)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
java.util.concurrent.FutureTask.run(FutureTask.java:166)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
java.lang.Thread.run(Thread.java:636)

My interpretation of this is that there is really huge amount of sigtest files from libraries and the system deletes them all, before it starts new round of indexing. This seems to take really long time.
Comment 1 Jaroslav Tulach 2010-05-26 14:15:17 UTC
My suggestion is to reimplement the deleteRecursively method. Instead of doing the delete, it shall just rename the folder to some temporary name. Then it shall do the delete in a background thread, possibly using the idleIO(Runnable) provided by the masterfs. As a result the UI will not be blocked, the system will delay the removal of old files until later and the user will sooner see the reindexed values.
Comment 2 Jaroslav Tulach 2010-05-26 15:15:26 UTC
Created attachment 99499 [details]
Result of self-profile and export to ... action, hopefully openable in NetBeans profiler
Comment 3 Tomas Zezula 2010-11-03 15:42:19 UTC
The main problem is that deleteRecursively is called even when not needed. In your case it probably should not be called at all.
I've changed the impl to not to call it when nothing changed. Delete only changed files when something has changed. deleteRecursively is called only when huge amount of crcs has changed (rare case).
Anyway I will will an enhancement to benefit from slowIO when available as an API (http://netbeans.org/bugzilla/show_bug.cgi?id=191316).
Comment 4 Tomas Zezula 2010-11-03 15:52:07 UTC
Fixed as described above in jet-main 4f78de51f641
Comment 5 David Strupl 2012-02-24 09:29:35 UTC
Mentioned on http://wiki.netbeans.org/EditorPlan72 . Is there something to be done for 7.2 for this one?


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo