This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
When I mount a couple of Generic FS, I experience big slow-down of IDE (perceived) performance. It seems the VCS FS scale bad when doing substantial refreshes, overusing CPU etc. thus causing AWT starvation. Steps to reproduce: Checkout all nb cvs. Mount it as generic VCS FS (nb_all as root), mount also about 10 other generic VCS FS. Then try to work, open sources, edit them, compile etc. It is almost impossible to work with. Everything is too slow comparing to the normal use.
The problem probably is, that Java parser goes through the all packages recursively. This triggers some VCS processes to be executed (like creation of ignore lists for each folder, etc.) I will analyze what is exactly happening to see whether I can do anything about it.
Please try at least first schedule the tasks of VCS Status Updater with minimal priority. I tried it again -> mounted nb_all as vcs generic FS, expanded just the top and the IDE gets almost frozen. (also a couple of another cvs fs were mounted before but not so huge ones).
Also when I'm doing exit of IDE I experience this: The IDE GUI is closed but the actual exit takes over 5 or even more minutes. And all that even in case I unmounted the nb_all fs before the exit. It still makes wirtual refresh (VCSRefreshRequest), seems there were scheduled huge amount of those tasks in advance. Could it help to diagnose the problem?
I think, that it needs a more radical solution, than scheduling the tasks with minimal priority. Though lowering the priority could help. I'm almost sure, that these refreshes are triggerd by the parser, which scans all the code. I need to return just disk files and do not touch the cache (which starts the refresh if it's empty) in children(). It should work this way, but for some reason it seems, that it does work, that is not necessary. I'll do some investigation.
Yes the minimal priority shouldn't help much. I looked a bit at the code, and saw you are always creating new task when the request comes and schedule it immediatelly (0 ms) into request processor. I think you should scan the tasks which are already in the request processor, and when the new request for the same task comes, try to do following: 1) cancel the old one (it was not running yet) and if succesful schedule the new one 2) if the old one is already running then the new one schedule with some delay (e.g. 200 ms -> there could be possible to wait when the old one finishes and just the schedule it with some delay). This technique should prevent from overloading of the requestprocessor, what is the case I think with the huge FS's causing the scalability problem.
Could somebody explain, why has this changed to task? Isn't it clear defect?
Current policy is to mark all perfomance issues except memory leaks as tasks blocking top level performance features. Linked properly to issue 26581 and issue 28352.
OK, thanks.
postponing to milestone 2
Targeted to M3.
One performance improvement implemented in the main trunk: The ignore list is created when it is really needed. This will improve the performance when engines will scan the codebase (Java parser, compiler, etc.). Checking in vcscore/src/org/netbeans/modules/vcscore/caching/VcsCacheDir.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/caching/VcsCacheDir.java,v <-- VcsCacheDir.java new revision: 1.44; previous revision: 1.43 done Checking in vcscore/src/org/netbeans/modules/vcscore/caching/VcsFSCache.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/caching/VcsFSCache.java,v <-- VcsFSCache.java new revision: 1.36; previous revision: 1.35 done Checking in vcscore/src/org/netbeans/modules/vcscore/caching/VcsCache.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/caching/VcsCache.java,v <-- VcsCache.java new revision: 1.54; previous revision: 1.53 done Processing log script arguments... More commits to come... Checking in vcscore/src/org/netbeans/modules/vcscore/cache/FileSystemCache.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/cache/FileSystemCache.java,v <-- FileSystemCache.java new revision: 1.14; previous revision: 1.13 done Processing log script arguments... More commits to come... Checking in vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java; /cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v <-- VcsFileSystem.java new revision: 1.199; previous revision: 1.198 done
If I may add, if its related.. I would prefer no action upon mount of a file system. Often I mount maybe the 1.4.1 sources or something, and immediately the IDE starts parsing it seems. I would love the chance to tell it those files are for viewing only, before it starts all the hard work. Like the ability to set capabilities during the mount. If this is at all related..
I agree. The ability to set capabilities during the mount would be nice. But it will not be implemented. There's no time to do this into 3.5 and in 4.0 the approach will be completely different. You'll not need to mount the sources at all, you would be able to browse all disk from the Filesystem tab.
The second performance improvement integrated. The status change is fired lazily exactly as Peter suggested. Also more changes can be collected and fired as a one. /cvs/vcscore/src/org/netbeans/modules/vcscore/versioning/VersioningFileSystem.java,v <-- VersioningFileSystem.java new revision: 1.22; previous revision: 1.21 /cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v <-- VcsFileSystem.java new revision: 1.200; previous revision: 1.199 It seems, that these two changes lead to a considerably better performance when a large filesystem is mounted. So I'm resolving this as fixed.
I'm must make big thanks to Martin and Peter thet they've got rid of one of the most paintfull issue of VCS in general:-) I've tested it several times and the final and most drastick testing testing I performed on: NetBeans IDE Dev (Build 200308270100) Linux version 2.4.20-18.9 running on i386 Java 1.4.2; Java HotSpot(TM) Client VM 1.4.2-b28 with 3 mounted whole sources of netbeans: PRJ4-_PROTOTYPE, RELEASE35 as G-CVS, and TRUNK as JavaCVS (this one contains almost whole repository) and others c. 10 various CVS FSs and performing updates on them and editing files and compiling... and ide was still usable :-) (Of'course for some small exceptions;) but I think I could mark it as VERIFIED.