This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 33474 - vcs file system mount action: IDE uses 98% of the CPU for at least 40 minutes+
Summary: vcs file system mount action: IDE uses 98% of the CPU for at least 40 minutes+
Status: VERIFIED FIXED
Alias: None
Product: obsolete
Classification: Unclassified
Component: vcscore (show other bugs)
Version: -S1S-
Hardware: All All
: P1 blocker (vote)
Assignee: Martin Entlicher
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-05-07 18:39 UTC by Peter Liu
Modified: 2003-07-08 15:32 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
The binary patch, that should improve the performance. To apply, put the jar into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder. (4.30 KB, application/octet-stream)
2003-05-07 19:02 UTC, Martin Entlicher
Details
The contextual diff of changes that fix this problem. (5.65 KB, patch)
2003-05-09 11:39 UTC, Martin Entlicher
Details | Diff
The new contextual diff of all changes. (7.74 KB, patch)
2003-05-09 13:37 UTC, Martin Entlicher
Details | Diff
The new binary patch. (4.56 KB, application/octet-stream)
2003-05-09 13:40 UTC, Martin Entlicher
Details
The new binary patch, that filters .nbintdb. Use with the latest build from May 11. Put into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder. (53.27 KB, application/octet-stream)
2003-05-12 14:51 UTC, Martin Entlicher
Details
The contextual diff. (4.89 KB, patch)
2003-05-12 16:39 UTC, Martin Entlicher
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Liu 2003-05-07 18:39:33 UTC
Description: 
RC2 build. Win 2k and Win XP (tested on 2 systems, jdk1_4_1_02b6 and beta 1.4.2)
now the the javacvs is gone, we need to switch to the generic CVS filesystem
mounting system.
As a plugin developer, we always mount the root of the app server s1as source
code (pretty big, but tha't not the issue). After mounting the source code, the
IDE starts some background activity (I presume to create the code completion
database for the cvs FS).
The IDE is barely usable for at least 40 minutes (maybe much more): it takes
between 70 and 98% of the CPU and the user can't do much...


xxx@xxxx 2003-05-06:
I've put the thread dumps into an attachment. BT+ has a problem with too long
texts in it's fields. This was a recommentation of the BT+ people after I filed
a servicedesk ticket.

Also it looks like the dumps were not completely included here, maybe Ludo has
the remainder somewhere?
-- Marco


Work Around: 
Do not use the CVS filesystem.
This defeats the whole value proposition of Nevada around VCS systems...
xxx@xxxx 2003-05-05
xxx@xxxx 2003-05-05

Suggested Fix: 
According to my tests it seems, that the root cause of the performance problems
are I/O operation when reading/storing the file attribute. The DefaultAttributes
are not efficient for storing of big objects.

Thus I've created a patch that works as described in the Evaluation:
The object integrity database is saved in a raw binary form (a serialized Java
object) in a file .nbintdb next to .nbattrs.

The patch Patch4858578.jar is attached in the Attachments. To apply, put the jar
into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder.

I've done some measurments of a test working directory with a bunch of modules
from cvs.netbeans.org:
The time of building of the parser database after the filesystem was mounted:
13  min  NetBeans 3.5 RC2, VCS Generic filesystem with CVS profile,
4   min  NetBeans 3.5 RC2 with the attached patch applied, VCS Generic FS,
2.5 min  NetBeans 3.5 RC2, Local Filesystem

The Generic VCS filesystem will always be slower, but it's evident, that the
patch improved the performance substantially. Also the IDE was subjectively more
responsive.

xxx@xxxx 2003-05-07

Please verify whether the attached patch helps. It needs to be decided whether
it should be included into NetBeans 3.5 / S1S 5.0 or not.
xxx@xxxx 2003-05-07

Will try today. Could we decrease the thread priority also?

xxx@xxxx 2003-05-07

Evaluation: 
xxx@xxxx 2003-05-06:
how does this work iff you simply mount this as a `normal' source directory
structure?  The CodeCompletion DB would have to updated as well?
-- Marco

It works fine if you do not use VCS FS...
(If it did not, we could not use Studio at all to develop the plugin;-)

The issue is really around the VCS FS: not usable for decent repository. You can
try it on you side with a Studio Module (from you CVS repository) and do some
measures.
xxx@xxxx 2003-05-06

On another machine: win2000, Dual CPU 600Mhz, the code completion on S1AS VCS
fileystem took one entire CPU for around 15 minutes to complete the database.
After that, the IDE was usable again.


xxx@xxxx 2003-05-06


I tried mounting the s1s 5 source tree using local and CVS filesystem on a
SunBlade 2000 machine. For local filesystem, to took ~5 1/2 minutes for the IDE
to finish creating the code completion DB. For VCS filesystem, it took ~12
minutes to finish.  From the thread dumps , it appears that the IDE is scanning
the entire source tree, parsing files and creating data objects . VCS filesystem
exacerbates the problem by doing additional things.

I will bugbridge this bug to Issuezilla tomorrow. Bugbridge is currently down.


xxx@xxxx 2003-05-06


It seems like this problem can be caused by the implementation of issue #31056.
This was not implemented in JavaCVS, only in VCS Generic.
However in the attached thread dumps I didn't find anything from VCS that can
cause some performance problems.
The first thread dump does not contain anything from VCS, the second contains
just reading of an integrity support attribute and the third writing of the
integrity support attribute.
If this is still causing problems, the whole feature would probably have to be
completely redesigned or removed.

xxx@xxxx 2003-05-07

One possible performance improvement that came to my mind is that we don't use
FileObject attributes for the Object Integriry database. Therefore it will not
be necessary to parse a long XML file. The database can be saved in a raw binary
form (a serialized Java object) in a file next to .nbattrs. It would have to be
measured whether it will lead to a substantial performance improvements.

xxx@xxxx 2003-05-07


A comment.
Comment 1 Martin Entlicher 2003-05-07 19:02:15 UTC
Created attachment 10247 [details]
The binary patch, that should improve the performance. To apply, put the jar into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder.
Comment 2 Martin Entlicher 2003-05-07 19:24:47 UTC
Copied the further comments:

We can not decrease the thread priority any more. It's already '1'.
Martin Entlicher 2003-05-07

I tried on my laptop (dell 650Mh, 512Mb):
without the patch, yeterday, I stopped the IDE after 45 minutes and it
was still constructing the database.
With the patch, the database is done in 18 minutes.
its size is 3.8Mb.

So this is MUCH better. Is is enough? I don't know;-) 18 minutes of
CPU is a lot...
xxx@xxxx 2003-05-07

You will never reach zero! The building of the parser database takes
some time, which is proportional to the filesystem size. This will
happen when you mount a local filesystem as well. My goal is not to
prolog that time too much. VCS does have to do some work due to the
implementation of issue #31056, but the CPU consumption should not be
bigger than is really necessary. Anyway the IDE should stay
responsive, which should be assured by the lowest possible thread
priority.

So someone would have to decide whether this is acceptable and whether
it's worth to integrate into FCS. My opinion is, that it should be
integrated, the fix is relatively safe and gives a substantial
performance improvement. It would need a throughout analysis to find
out whether an even better performance improvement is possible.

Martin Entlicher 2003-05-07
Comment 3 Martin Entlicher 2003-05-07 19:29:55 UTC
The bug is fixed in the main trunk after it was verified that the
attached patch really improves the performance:

/cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v 
<--  IntegritySupportMaintainer.java
new revision: 1.4; previous revision: 1.3
Comment 4 Peter Liu 2003-05-07 19:45:27 UTC
I just tried the patch and it did improve the performance for mount
CVS filesystem. However, the performance as a whole is still horrible.
This is not a VCS-only issue. The problem really has to do with the
fact that we are scanning the entire tree. I know it is probably too
late to make anymore changes. Shouldn't the code completion DB be
built on demand as the user access each directory?
Comment 5 Martin Entlicher 2003-05-09 11:39:38 UTC
Created attachment 10255 [details]
The contextual diff of changes that fix this problem.
Comment 6 Martin Entlicher 2003-05-09 13:32:41 UTC
I've found out, that in some cases the database file could be
corrupted (even when the IDE was not killed, but exited via Exit).
Thus the database needs to be written to a temp file first: .nbintdb~
and later renamed to .nbintdb
Also an adaptive schedule time was introduced. This improves the
performance a little bit more. This is fixed in the main trunk:

/cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v 
<--  IntegritySupportMaintainer.java
new revision: 1.5; previous revision: 1.4
Comment 7 Martin Entlicher 2003-05-09 13:37:49 UTC
Created attachment 10258 [details]
The new contextual diff of all changes.
Comment 8 Martin Entlicher 2003-05-09 13:40:29 UTC
Created attachment 10259 [details]
The new binary patch.
Comment 9 Martin Entlicher 2003-05-09 14:05:49 UTC
As was pointed out by the reviewer, when reading the database file we
should try to read the backup when the original does not exist (the
process could be killed before it renamed the .nbintdb~ to .nbintdb):

/cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v 
<--  IntegritySupportMaintainer.java
new revision: 1.6; previous revision: 1.5
Comment 10 Richard Gregor 2003-05-09 14:07:28 UTC
Code reviewed without objections.
Comment 11 Martin Entlicher 2003-05-09 14:32:27 UTC
The fix is merged into release35 branch upon a preliminary approval by
the release coordinator:

/cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v 
<--  IntegritySupportMaintainer.java
new revision: 1.1.2.4; previous revision: 1.1.2.3
Comment 12 _ ttran 2003-05-09 15:04:11 UTC
yes, I gave verbal approval to Martin.  thx for the fix
Comment 13 Jiri Kovalsky 2003-05-12 13:31:29 UTC
I have a comment here. The patch introduced new file .nbintdb that is
not being ignored by default. Can we avoid displaying this file ?
Therefore reopening to get this regression fixed. As for the fix of
the performance issue, it looks good so far. I am confirming that it
helped to reduce full CPU usage approximately to 20 - 30% for large
filesystems with 10000+ files. For filesystems with less than 5000
files it does not help nearly at all. Tested in Sun ONE Studio 5.0
build #030502 and Patch4858578.jar.
Comment 14 Martin Entlicher 2003-05-12 14:34:38 UTC
Nobody has found it earlier, because there's no periodic refresh :-(
The fix is simple - filter .nbintdb in VcsFileSystem.children().
Comment 15 Martin Entlicher 2003-05-12 14:51:06 UTC
Created attachment 10268 [details]
The new binary patch, that filters .nbintdb. Use with the latest build from May 11. Put into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder.
Comment 16 Martin Entlicher 2003-05-12 16:39:06 UTC
Created attachment 10273 [details]
The contextual diff.
Comment 17 Martin Entlicher 2003-05-12 16:43:58 UTC
The filtering is fixed in the main trunk:

/cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v 
<--  IntegritySupportMaintainer.java
new revision: 1.7; previous revision: 1.6
Checking in VcsFileSystem.java;
/cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v 
<--  VcsFileSystem.java
new revision: 1.226; previous revision: 1.225
Comment 18 Tomas Pavek 2003-05-12 16:48:37 UTC
The additional diff reviewed, looks OK.
Comment 19 Jiri Kovalsky 2003-05-12 16:53:52 UTC
Yes, the patch fixes the problem. No accessory file is visible.
Verified in RC1 build #200305112350 of NetBeans 3.5 with
Patch4858578_2.jar.
Comment 20 Martin Entlicher 2003-05-13 09:01:51 UTC
Integrated in release35 branch:

/cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v 
<--  IntegritySupportMaintainer.java
new revision: 1.1.2.5; previous revision: 1.1.2.4
Checking in VcsFileSystem.java;
/cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v 
<--  VcsFileSystem.java
new revision: 1.207.2.14; previous revision: 1.207.2.13
Comment 21 Jiri Kovalsky 2003-07-08 15:32:39 UTC
Verified in Sun ONE Studio 5.0 Standard Edition build #030528.