This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 15378 - Incessant activity in folder recognizer during startup
Summary: Incessant activity in folder recognizer during startup
Status: CLOSED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Data Systems (show other bugs)
Version: 3.x
Hardware: PC Linux
: P1 blocker (vote)
Assignee: Jesse Glick
URL:
Keywords: PERFORMANCE
: 15381 (view as bug list)
Depends on:
Blocks:
 
Reported: 2001-09-11 21:47 UTC by Jesse Glick
Modified: 2008-12-22 20:33 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Some thread dumps (60.05 KB, text/plain)
2001-09-11 21:48 UTC, Jesse Glick
Details
Thread dumps while hanging with a new user dir (28.51 KB, text/plain)
2001-09-26 22:02 UTC, Roger Blumer
Details
Another dump while starting up with a new userdir (56.71 KB, text/plain)
2001-09-26 22:05 UTC, Roger Blumer
Details
Thread dump of hang during startup with new user dir. NOTE: previous attachment was hang with an EXISTING userdir. (49.33 KB, text/plain)
2001-09-26 22:14 UTC, Roger Blumer
Details
Possible patch (1.98 KB, patch)
2001-10-05 18:45 UTC, Jesse Glick
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse Glick 2001-09-11 21:47:22 UTC
[dev sep 10] IDE takes inordinately long to respond to pressing Finish
on last stage of Setup Wizard (Auto Update config)--I thought it was
deadlocked completely--and when it does start up, consumes 90% of
processor time for no apparent reason. Attached are a number of thread
dumps: one during startup when it looked frozen (no repaints etc.),
several more after startup (all looks normal). Any actions, such as
expanding a folder, take ten seconds or so, during which the folder
recognizer and request processor threads are very busy. After a few
minutes, things seem normal and unusual thread activity stops.
Comment 1 Jesse Glick 2001-09-11 21:48:33 UTC
Created attachment 2501 [details]
Some thread dumps
Comment 2 Jan Zajicek 2001-09-12 13:13:34 UTC
assigning
Comment 3 Jan Zajicek 2001-09-12 13:19:02 UTC
Vito, can you check this one, thanks. Maybe that this can be related
to the issue #15381.
Comment 4 Petr Nejedly 2001-09-12 15:36:08 UTC
*** Issue 15381 has been marked as a duplicate of this issue. ***
Comment 5 Petr Nejedly 2001-09-12 15:38:22 UTC
Try whether it happens also with clean userdir or only when
running against customized settings, see comments by #15381.
Comment 6 dmladek 2001-09-12 15:52:48 UTC
CC'ing myself.
And addig keyword:PERFORMANCE (hope that it is right:)
Comment 7 Jesse Glick 2001-09-13 23:02:06 UTC
In the cases where this has happened to me, I have run a complete
build including sanity-check, which uses nbbuild/testuserdir/ as
userdir. I then run ant tryme which again uses testuserdir/. Things
look OK at first, Setup Wizard appears. I select MDI mode as always,
hit Finish, then wizard appears to freeze for a long time and that is
when the heavy activity seems to start. Using other user directories
(such as my normal ~/nbdev/) recent builds seem OK.
Comment 8 Martin Entlicher 2001-09-17 10:38:50 UTC
The problem mentioned in issue #15381 (too many VCS Ignore List
Creation Threads - look into series of FullThreadDumps) has beed fixed
now in VcsFileSystem. It should speed up the folder recognizer
process.
Comment 9 Jesse Glick 2001-09-17 23:34:45 UTC
OK. I saw this problem with sources as of Sunday again--note that it
only happens when using an existing testuserdir/, when using a fresh
user directory all is well. So it must have something to do with
loading settings.
Comment 10 Vitezslav Stejskal 2001-09-18 09:21:46 UTC
Well, I am going to investigate it bit more, Yarda pointed me to 
similar problem caused by infinite recognition of files by different 
loaders--the root of the problem was in 
MultiFileLoader.checkConsistency (). Not sure if this is the case.
Comment 11 Roger Blumer 2001-09-21 00:10:23 UTC
I saw this yesterday on my Orion build, only with one new wrinkle:  in 
many cases the folder-recognizer activity NEVER stopped!!  
Since this is on a 2-cpu machine, it may be due to a race condition 
that is hit more often on a multi-cpu machine. One CPU was 
completely used the entire time the IDE was up (10 minutes for one 
case).

My Orion build was from 2001.09.19-18:51:09GMT.

On my machine(2-cpu, 800Mhz), I hit this continuous activity roughly 
75% of the time.  I then tried the same build on a 1-cpu 400 Mhz 
machine, and didn't get the problem in 3 tries (CPU usage went to 0 
once the IDE was up).

I have 2 long traces of several thread dumps during the CPU usage, if 
more thread dumps will help.
Comment 12 Petr Nejedly 2001-09-21 09:29:51 UTC
Was that 400MHz single CPU operated against the SAME userdir
or was it clean instalation? This problem doesn't affect
clean instalations.
Comment 13 Roger Blumer 2001-09-21 18:53:31 UTC
Good question.  The 3 startups on the 400Mhz single-cpu were 1 with a 
clean userdir, then 2 with that newly-created user-dir.

On the 800Mhz, 2 CPU machine, the 1st run was with an old userdir from 
a previous build, then 1 run with a clean user-dir, then the next 8 
runs were with that newly-created user-dir.

Do you think I'm seeing the same bug, or should I open a different 
one?  Or maybe something one of the EE modules does in its 
options/settings causes this bug to happen even with a clean user-dir?
Comment 14 Roger Blumer 2001-09-26 21:59:15 UTC
Increased priority to P1, since my latest build hung during 
firststart. So now I can't even start up the IDE (with an old OR new 
userdir).  I'll add a couple attachments with several stack traces 
during the hang/infinite loop.

My Orion build was started at "25 Sep 2001 23:52:06 GMT"
Again, this is on a 2-cpu machine.
Comment 15 Roger Blumer 2001-09-26 22:02:26 UTC
Created attachment 2718 [details]
Thread dumps while hanging with a new user dir
Comment 16 Roger Blumer 2001-09-26 22:05:01 UTC
Created attachment 2719 [details]
Another dump while starting up with a new userdir
Comment 17 Roger Blumer 2001-09-26 22:14:44 UTC
Created attachment 2720 [details]
Thread dump of hang during startup with new user dir.  NOTE: previous attachment was hang with an EXISTING userdir.
Comment 18 anovak 2001-09-27 15:20:58 UTC
I have just fixed a similar problem #15918
Can somebody try to reproduce this problem on tomorrow's (2001/09/28)
build?
Comment 19 Roger Blumer 2001-09-27 22:51:43 UTC
I tried this again with a build containing the FolderOption changes 
(build time 27 Sep 2001 17:00:14 GMT), and still had the same problem.

I had a little time to look into this, and added a trace to 
ModuleInstaller.loadOptionSection.  Twice the firstart hung with the 
splash screen saying "loading pieces of User Utilities".  The first 
time my trace put out
loading 
ManifestSection[org.netbeans.modules.corba.settings.CORBASupportSettin
gs]
loading 
ManifestSection[com.sun.forte4j.persistence.internal.ui.modules.settin
gs.TransparentPersistenceSettings]
getActiveSetting () -> null
getActiveSetting () -> null
loading ManifestSection[org.netbeans.modules.openfile.Settings]

And the 2nd time it stopped after the TransparentPersistenceSettings 
line.  So I removed the persistence-ui.jar from my modules directory, 
and then firststart finished correctly. This is an OK workaround for 
my current work.
I also don't get the continuous CPU usage after startup any more. 
Maybe TP has some settings with the same problem as the debugger 
settings (in 15381)?
Comment 20 Jesse Glick 2001-09-28 11:38:37 UTC
ModuleInstaller.loadOptionSection? Do you mean NbInstaller? Roger I
assume you are talking about dev builds (Orion) here...but e.g.
OpenFileSettings is for some time now not loaded from manifest but
from layer (3.3 settings change), so what exactly are you using??
Comment 21 Roger Blumer 2001-09-28 18:14:35 UTC
You're right, Jesse. It was NbInstaller.loadOptions.
Sorry for the typo.

I'm building the Orion/dev build, using the command:
 ffjbuild -clean -update -Dmoduleconfig=stable-with-apisupport ee
so I assume I'm getting whatever RE builds.

What is the issue about the OpenFile settings?
Could there be a problem with the build setup? 
Or do I have to delete my CVS directories & start with a new checkout 
because of some recent change?
(To make sure that trace wasn't due to an old userdir, I ran with a 
new userdir, and the line about OpenFile still comes out.)
Comment 22 Jesse Glick 2001-10-01 14:40:10 UTC
Roger--cvs update is fine, no need for a fresh checkout. The Orion
build scripts are apparently broken and you are actually running
slightly different code than exists in NetBeans. I will notify RE.
Comment 23 Jesse Glick 2001-10-05 18:44:15 UTC
OK, I think I have a fix for this with Vita's and Yarda's help. Was
able to reproduce the problem under *some circumstances* from the user
directory in #15381 (thanks Daniel for including this!). It is
probably only reproducible on certain machines (thankfully mine) and
under certain memory conditions (e.g. start another app and the
problem disappears).

Turns out that if conditions are exactly wrong, a cycle of six folders
develops (all in the system file system), and including the Services/
folder. (So if you delete the debugger project settings, this folder
behaves differently and the problem disappears. Nothing to do with
reading settings really.) Apparently folder #1 is asked for its
children, which makes it read its OpenIDE-Folder-Order attribute; a
FolderOrder object is created, the order computed, and something
finishes. Folder #2 does the same. Another two folders are also
processed, twice each (or something like this). By this time VM memory
runs out, the GC is run, and the cached FolderOrder's are discarded,
leaving you at the beginning of the loop again--because the "previous"
order is forgotten and it is assumed the children list has changed.

Fixable by keeping a strong reference from the folder FileObject to
its textual order, e.g. "earlier/middle/later". If the FolderOrder
object is GC'd, the next one for the same folder will start with the
correct order and no children change will be fired.

Have patch, will test some first.
Comment 24 Jesse Glick 2001-10-05 18:45:10 UTC
Created attachment 2881 [details]
Possible patch
Comment 25 Jesse Glick 2001-10-05 21:41:10 UTC
Fixed in FolderOrder.java 1.7. Note that I am only confident about
fixing #15381; the dumps reported here (by myself and Roger) appear to
be the same problem, but as I do not know how to reproduce these it is
impossible to be sure.
Comment 26 Roger Blumer 2001-10-05 21:56:18 UTC
I took the patch & added it to my 05-oct build & re-ran the firstart 
part of the build.  It ran correctly. I also brought up the IDE 
several times after firststart, and had no problems.
So this looks good.
I'll update my build with the official fix next week, and add more 
info only if the problem occurs again.
Comment 27 pzajac 2002-09-10 16:35:25 UTC
I seems be resolved. I'll mark it as verified.
Comment 28 Quality Engineering 2003-07-01 16:34:18 UTC
Resolved for 3.4.x or earlier, no new info since then -> closing.