This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | vcs file system mount action: IDE uses 98% of the CPU for at least 40 minutes+ | ||
---|---|---|---|
Product: | obsolete | Reporter: | Peter Liu <petertliu> |
Component: | vcscore | Assignee: | Martin Entlicher <mentlicher> |
Status: | VERIFIED FIXED | ||
Severity: | blocker | ||
Priority: | P1 | ||
Version: | -S1S- | ||
Hardware: | All | ||
OS: | All | ||
Issue Type: | DEFECT | Exception Reporter: | |
Attachments: |
The binary patch, that should improve the performance. To apply, put the jar into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder.
The contextual diff of changes that fix this problem. The new contextual diff of all changes. The new binary patch. The new binary patch, that filters .nbintdb. Use with the latest build from May 11. Put into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder. The contextual diff. |
Description
Peter Liu
2003-05-07 18:39:33 UTC
Created attachment 10247 [details]
The binary patch, that should improve the performance. To apply, put the jar into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder.
Copied the further comments: We can not decrease the thread priority any more. It's already '1'. Martin Entlicher 2003-05-07 I tried on my laptop (dell 650Mh, 512Mb): without the patch, yeterday, I stopped the IDE after 45 minutes and it was still constructing the database. With the patch, the database is done in 18 minutes. its size is 3.8Mb. So this is MUCH better. Is is enough? I don't know;-) 18 minutes of CPU is a lot... xxx@xxxx 2003-05-07 You will never reach zero! The building of the parser database takes some time, which is proportional to the filesystem size. This will happen when you mount a local filesystem as well. My goal is not to prolog that time too much. VCS does have to do some work due to the implementation of issue #31056, but the CPU consumption should not be bigger than is really necessary. Anyway the IDE should stay responsive, which should be assured by the lowest possible thread priority. So someone would have to decide whether this is acceptable and whether it's worth to integrate into FCS. My opinion is, that it should be integrated, the fix is relatively safe and gives a substantial performance improvement. It would need a throughout analysis to find out whether an even better performance improvement is possible. Martin Entlicher 2003-05-07 The bug is fixed in the main trunk after it was verified that the attached patch really improves the performance: /cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v <-- IntegritySupportMaintainer.java new revision: 1.4; previous revision: 1.3 I just tried the patch and it did improve the performance for mount CVS filesystem. However, the performance as a whole is still horrible. This is not a VCS-only issue. The problem really has to do with the fact that we are scanning the entire tree. I know it is probably too late to make anymore changes. Shouldn't the code completion DB be built on demand as the user access each directory? Created attachment 10255 [details]
The contextual diff of changes that fix this problem.
I've found out, that in some cases the database file could be corrupted (even when the IDE was not killed, but exited via Exit). Thus the database needs to be written to a temp file first: .nbintdb~ and later renamed to .nbintdb Also an adaptive schedule time was introduced. This improves the performance a little bit more. This is fixed in the main trunk: /cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v <-- IntegritySupportMaintainer.java new revision: 1.5; previous revision: 1.4 Created attachment 10258 [details]
The new contextual diff of all changes.
Created attachment 10259 [details]
The new binary patch.
As was pointed out by the reviewer, when reading the database file we should try to read the backup when the original does not exist (the process could be killed before it renamed the .nbintdb~ to .nbintdb): /cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v <-- IntegritySupportMaintainer.java new revision: 1.6; previous revision: 1.5 Code reviewed without objections. The fix is merged into release35 branch upon a preliminary approval by the release coordinator: /cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v <-- IntegritySupportMaintainer.java new revision: 1.1.2.4; previous revision: 1.1.2.3 yes, I gave verbal approval to Martin. thx for the fix I have a comment here. The patch introduced new file .nbintdb that is not being ignored by default. Can we avoid displaying this file ? Therefore reopening to get this regression fixed. As for the fix of the performance issue, it looks good so far. I am confirming that it helped to reduce full CPU usage approximately to 20 - 30% for large filesystems with 10000+ files. For filesystems with less than 5000 files it does not help nearly at all. Tested in Sun ONE Studio 5.0 build #030502 and Patch4858578.jar. Nobody has found it earlier, because there's no periodic refresh :-( The fix is simple - filter .nbintdb in VcsFileSystem.children(). Created attachment 10268 [details]
The new binary patch, that filters .nbintdb. Use with the latest build from May 11. Put into <IDE-install>/modules/autoload/patches/org-netbeans-modules-vcscore/ folder.
Created attachment 10273 [details]
The contextual diff.
The filtering is fixed in the main trunk: /cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v <-- IntegritySupportMaintainer.java new revision: 1.7; previous revision: 1.6 Checking in VcsFileSystem.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v <-- VcsFileSystem.java new revision: 1.226; previous revision: 1.225 The additional diff reviewed, looks OK. Yes, the patch fixes the problem. No accessory file is visible. Verified in RC1 build #200305112350 of NetBeans 3.5 with Patch4858578_2.jar. Integrated in release35 branch: /cvs/vcscore/src/org/netbeans/modules/vcscore/objectintegrity/IntegritySupportMaintainer.java,v <-- IntegritySupportMaintainer.java new revision: 1.1.2.5; previous revision: 1.1.2.4 Checking in VcsFileSystem.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v <-- VcsFileSystem.java new revision: 1.207.2.14; previous revision: 1.207.2.13 Verified in Sun ONE Studio 5.0 Standard Edition build #030528. |