This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
BuildSys team has done a good job in eliminating the huge folder expansion regression, but there is still something strange in the code. Sometimes measurement shows me significantly longer time, while in other cases it shows good time. The problem manifests itself on linux (see the numbers below...). Here are results from testing on W2K (Dell Precision 220, PIII 800MHz, 512MB RAM, Windows 2000) and Linux (Dell Latitude C840, PIV 2.2GHz, 1024MB RAM, Red Hat 9). Each test was repeated several times. Expanding a folder containing 1000 very simple java files: W2K: 200403141900 - 5172ms, 5703ms, 5235ms 200403161900 - 5297ms, 5250ms, 5266ms Linux: 200403141900 - 4179ms, 3127ms, 3493ms 200403161900 - 9863ms, 4465ms, 17131ms, 4933ms, 15059ms, 4125ms
Tomasi, I've done measurements with today's trunk and expanding folder with 1000 javas is still problematic on linux. I've done 10 measurements on W2K and on Linux. Linux still shows a strange long time sometimes. W2K: 200405271200 (my build) - 5016ms, 5000ms, 5000ms, 4985ms, 5000ms, 5000ms, 5000ms, 5015ms, 5031ms, 5016ms Linux: 200405271200 (my build) - 3413ms, 9235ms, 4227ms, 4227ms, 4760ms, 4557ms, 4704ms, 9306ms, 9124ms, 4230ms This is hard to track in profiler because the CPU profiling makes the times much longer and you'd need many profiling snapshots to compare, find out which of them are the worse cases and compare with the better ones to find differences. But eventually this will perhaps be the only way to find the cause.
Tondo, can you remeasure it again. On my Linux workstation (Intel(R) Pentium(R) M processor 1600MHz, Linux 2.6.5-7.95)it works fine, the folder with 1000 files opens in 3s. Here are some screenshots of active threads in OptimizeIt.
Created attachment 16531 [details] Screenshot from OptimizeIt
Created attachment 16532 [details] Next screenshot from OptimizeIt
Please do a new measurement.
The new measurement. The same configuration, with jdk 1.4.2_04. W2K: 200407281800 - 6094ms, 7344ms, 7391ms, 5437ms, 7656ms, 7212ms Linux: 200407281800 - 2496ms, 2424ms, 2482ms, 2538ms, 2367ms, 2517ms
Passing back to Tomas for evaluation.
OK, it seems that the problems is on Windows.
Couple of comments. Other DataLoaders - we have slow ImageDataLoader (issue #47926) and also other loaders can perform their work. We may consider more explicit ordering of loader to move JavaDataLoader to the begining of the chain as it is primarily JavaIDE. Mainly we would like to avoid getMIMEType call during recognition that is done for most of XML DOs. Filesystems * MFS adds a layer to original local FS, * getPath() is slow (called often for example from getAttribute()) I tried to cache path for last folder. It is not very effective during startup but later the hit ratio improves to something like 1:15 but still not very good. JavaDataObject initialization. The most of time is spent in addCookieSet. At least I suggest to move it before propert change lsnr adding to reduce overhead when DO fire change of COOKIE_SET property.
And yet another comment: If there are many Java file the JavaNodes are created and schedule icon resolving that can be executed before painting of nodes or at least will cause heavy CPU utilization to complete icon badging. Some hot places here: JavaNode$IconResolver.run -> FileBuiltQuery.getStatus that spents a lot of time in adding linsteners JMManager.getResource followed by getPackageName in JavaNode.resolveIcons and hasMain in the same method
Radim already fixed the solvable Java related issues. Now the DataLoaders and FileSystems issues should be fixed. Reassigning to David
I am going to look to FileBuildQuery related issues.
If JavaDataLoader should move up in the chain of loaders then it is up to Java module to do that. The same answer is for JDO initialization. Passing to Radek to comment FS part.
Tonda, just for comparison what is the speed of opening folder with 1000 text files or 1000 html files?
We discussed all the mentioned filesystems comments lately and didn't find any resonable solution. Radim, file a task if you think that caching path is worth of implementing and I'm ready to fix it ASAP.
Results from automated tests [nb_dev](200409011800),[jdk1.5.0](b63) Hosts: RH Linux 9 JDS 2 Solaris 9 Win 2000 Win XP Expand folder with 50 java files 1st usage 701 751 904 562 635 2nd usage 391 390 348 292 289 Expand folder with 100 java files 1st usage 582 1111 624 463 479 2nd usage 390 387 361 290 297 Expand folder with 100 txt files 1st usage 564 517 536 364 416 2nd usage 415 406 335 339 295 Expand folder with 100 xml files 1st usage 1897 1859 2247 1859 1646 2nd usage 382 552 722 275 287 Explanation : - test of folder with 50 java files runs before test of folder with 100 java files, so in the time expansion is measured class loading (for action not for expansion) - java files expansion is measured with badging turned off (perf.dont.resolve.java.badges=true) Conclusion: I think folder with java files expansion meets UI criteria, but folder with 100 xml files expansion doesn't.
The only way how to improve the JavaDataObject is to change the datasystems API (allow to register lookup rather than the CookieSet). Java module can not do anything with this. Moving the java datatype up does not help much.
I have to disagree with David, JDO initialization is a issue of DataSystems and must be solved there.
Sure, DS is source of lot of problems. But I do not dare to change anything in DS for Beta2. Passing back to Tomas who said that there is one more thing he might improve. After that I suggest to close this issue as fixed. As Marian said it meets the UI criteria.
I've filled an enhancement for data systems: http://www.netbeans.org/issues/show_bug.cgi?id=48381 There are no more easy to fix solutions for this bug. We improved the hasMain and JavaDataObject performance. I've looked into FileBuiltQuery, but deferring adding of listeners in the StatusImpl from constructor to isBuilt will not help, since isBuilt is called when resolving icon badges. To make comparison to 3.6 the cross compilation (on NB 3.6) should be turned on.
As mentioned in issue #48381, I disagree with Tomas that "The only way how to improve the JavaDataObject is to change the datasystems API (allow to register lookup rather than the CookieSet). Java module cannot do anything with this." - JavaDataObject could override getCookie to manage its cookie handling quite explicitly, which could minimize initialization costs as well as reduce memory consumption per JDO.
OK, I will try this. It may help a bit.