This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 186744

Summary: Adding library causes huge I/O load
Product: java Reporter: Jaroslav Tulach <jtulach>
Component: SourceAssignee: Tomas Zezula <tzezula>
Status: RESOLVED FIXED    
Severity: normal CC: dstrupl, issues
Priority: P2 Keywords: PERFORMANCE, PLAN
Version: 6.x   
Hardware: Other   
OS: Linux   
Issue Type: DEFECT Exception Reporter:
Bug Depends on: 191316    
Bug Blocks:    
Attachments: Result of self-profile and export to ... action, hopefully openable in NetBeans profiler

Description Jaroslav Tulach 2010-05-26 14:12:30 UTC
I have a project with bunch of libraries. When I add yet another one to the list of its libraries, the UI is completely frozen for few seconds and the I/O load is high. Stack trace shows:

"Parsing & Indexing Loop (100511-5b0ca7653b2c)" 
java.lang.Thread.State: RUNNABLE
java.io.UnixFileSystem.delete0(Native Method)
java.io.UnixFileSystem.delete(UnixFileSystem.java:266)
java.io.File.delete(File.java:921)
o.n.m.j.s.parsing.FileObjects.deleteRecursively(FileObjects.java:604)
o.n.m.j.s.indexing.JavaBinaryIndexer$1.run(JavaBinaryIndexer.java:81)
o.n.m.j.s.indexing.JavaBinaryIndexer$1.run(JavaBinaryIndexer.java:77)
o.n.m.j.s.usages.ClassIndexManager.prepareWriteLock(ClassIndexManager.java:112)
o.n.m.j.s.indexing.JavaBinaryIndexer.index(JavaBinaryIndexer.java:77)
o.n.m.parsing.spi.indexing.Indexable$MyAccessor$1.run(Indexable.java:160)
o.n.m.p.i.i.RepositoryUpdater.runIndexer(RU.java:237)
o.n.m.parsing.spi.indexing.Indexable$MyAccessor.index(Indexable.java:158)
o.n.m.p.i.i.RepositoryUpdater$Work.indexBinary(RU.java:1848)
o.n.m.p.i.i.RepositoryUpdater$AbstractRootsWork.scanBinary(RU.java:3064)
o.n.m.p.i.i.RepositoryUpdater$AbstractRootsWork.scanBinaries(RU.java:3040)
o.n.m.p.i.i.RepositoryUpdater$RootsWork.getDone(RU.java:2870)
o.n.m.p.i.i.RepositoryUpdater$Work.doTheWork(RU.java:2012)
o.n.m.p.i.i.RepositoryUpdater$Task._run(RU.java:3605)
o.n.m.p.i.i.RepositoryUpdater$Task.run(RU.java:3545)
o.n.m.parsing.impl.TaskProcessor$CompilationJob.run(TaskProcessor.java:577)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
java.util.concurrent.FutureTask.run(FutureTask.java:166)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
java.lang.Thread.run(Thread.java:636)

My interpretation of this is that there is really huge amount of sigtest files from libraries and the system deletes them all, before it starts new round of indexing. This seems to take really long time.
Comment 1 Jaroslav Tulach 2010-05-26 14:15:17 UTC
My suggestion is to reimplement the deleteRecursively method. Instead of doing the delete, it shall just rename the folder to some temporary name. Then it shall do the delete in a background thread, possibly using the idleIO(Runnable) provided by the masterfs. As a result the UI will not be blocked, the system will delay the removal of old files until later and the user will sooner see the reindexed values.
Comment 2 Jaroslav Tulach 2010-05-26 15:15:26 UTC
Created attachment 99499 [details]
Result of self-profile and export to ... action, hopefully openable in NetBeans profiler
Comment 3 Tomas Zezula 2010-11-03 15:42:19 UTC
The main problem is that deleteRecursively is called even when not needed. In your case it probably should not be called at all.
I've changed the impl to not to call it when nothing changed. Delete only changed files when something has changed. deleteRecursively is called only when huge amount of crcs has changed (rare case).
Anyway I will will an enhancement to benefit from slowIO when available as an API (http://netbeans.org/bugzilla/show_bug.cgi?id=191316).
Comment 4 Tomas Zezula 2010-11-03 15:52:07 UTC
Fixed as described above in jet-main 4f78de51f641
Comment 5 David Strupl 2012-02-24 09:29:35 UTC
Mentioned on http://wiki.netbeans.org/EditorPlan72 . Is there something to be done for 7.2 for this one?