This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 129620 - [PERFORMANCE] Refactoring is extremely slow
Summary: [PERFORMANCE] Refactoring is extremely slow
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 6.x
Hardware: All All
: P3 blocker (vote)
Assignee: Pavel Flaska
URL:
Keywords: PERFORMANCE, REGRESSION
: 131981 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-03-09 18:05 UTC by _ gtzabari
Modified: 2008-04-04 13:22 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Patch for a testing from Honza Lahoda (1.49 KB, text/plain)
2008-03-27 17:53 UTC, Pavel Flaska
Details

Note You need to log in before you can comment on or make changes to this bug.
Description _ gtzabari 2008-03-09 18:05:16 UTC
Netbeans doesn't take seem to advantage of multiple-core/cpu computers. For example, try a refactoring operation
involving moving a large number of classes from one package to another. On my quad-core machine the operation takes a
long while and while it runs I see about 25% usage per CPU core. There seems to be little or no hard-drive usage.

I suspect that there is major thread-contention on some shared data structure.

What I propose is adding a metric to nightly tests which would (somehow) show how well an operation forks across
multiple CPUs. Once the metric is in place module authors will have a better sense of how severe the problem is and we'd
go from there.
Comment 1 _ gtzabari 2008-03-23 07:26:00 UTC
dev build 200803200007

I take it back, I don't think this is a multi-core issue. It has been my experience that copying or moving classes is
extremely slow in Netbeans, especially if you have many classes in your project. My particular project has 129 Java
files but most of them are under 2 pages long.
Comment 2 Daniel Prusa 2008-03-25 12:03:12 UTC
Probably not a refactoring issue, copying files is slow even without refactoring.
Comment 3 _ gtzabari 2008-03-25 14:14:53 UTC
Issue 130958 and other memory leaks aggravate the problem but you're right that even simple copying is slow. I noticed
that while refactoring is taking place the status bar says:

"Saving file1"<pause>
"Saving file2"<pause>
... for all files that have to be modified

The pause between each file is anywhere from 500ms to 2 seconds depending on the type of refactoring operation and Java
heap size. Now consider what happens if you rename a class and 30 classes depend on it (I consider 30 to be relatively
small): you now have to wait 15 - 60 seconds for the operation to complete! And this is on my quad-core 2.4GHz cpu.

I would expect any refactoring to complete within 5 seconds or less. Maybe that's unrealistic but I get the feeling the
current implementation has some low-hanging fruit.
Comment 4 Daniel Prusa 2008-03-26 15:14:10 UTC
I have made some measurements and it seems that move class refactoring is really very slow.
To move 33 classes took over 4 minutes. All the classes resided in one package and there were only few their usages
outside the package.
Simple move of the classes is substantially faster, just couple of seconds.
Comment 5 Pavel Flaska 2008-03-27 12:48:08 UTC
There are several possible improvements:
i) There are too many api.java.ClassPath$RootsListener. Seems like a regression in comparison to 6.0. These listeners
call filesystems and there is spent most of the time when converting file to url.
ii) We can eliminate bunch of calls to ClassPath API and FileSystems directly in MoveClassTransformer. (Need to be
measured if this can help)


Comment 6 Pavel Flaska 2008-03-27 17:53:58 UTC
Created attachment 59234 [details]
Patch for a testing from Honza Lahoda
Comment 7 Pavel Flaska 2008-03-27 17:56:56 UTC
Reducing number of listeners (removing them if class path does not exist) helps a lot...

Without patch, moving 10 files in jEdit took 173.311 seconds on my workstation. When I applied patch, it took 10.005
seconds.
Comment 8 _ gtzabari 2008-03-27 18:01:48 UTC
That sounds like a great improvement but it still sounds like a lot to me. Is it unrealistic to expect moderate-size
refactoring to complete within 5 seconds? For starters, can we define what the operational complexity depends on? Does
it depend how many classes are being refactored? Does it depend on the total number of classes in the project? Does it
depend on some combination of the two?

From a purely end-user point of view I am expecting to refactor a "moderate sized project" (whatever that means) in
under 5 seconds on a modern PC. Another issue is that it seems to be faster to refactor a bunch of files at a time (an
entire package) than refactor each file individually and add up the total time.
Comment 9 Pavel Flaska 2008-03-27 18:10:29 UTC
I agree it is still slow and we should provide more improvements. But not sure what can be done right now for 6.1. And
bear in mind I even did not apply the patch, because we have to ensure this will not cause quality regressions, others
have to check the patch and test it.

Patch just fixes regression to 6.0.1 I think.
Comment 10 Jan Lahoda 2008-03-27 20:46:04 UTC
After some digging, it turned out that the root cause for retained RootsListener is in the Filesystems API, should be
fixed now:
http://hg.netbeans.org/main/rev/14878a081417
Pavel, could you please redo your measurements on a build with the above patch? Thanks. (Please disregard the patch - it
was only hiding away the real problem.)

Some things that I think we should investigate in the future:
-rewrite SourceUtils.getFile not to use the proxy ClassPath (may or may not help)
-if the files-to-be-refactored are still load into a Document, find out whether this is still necessary (there will be
problems with guarded blocks and Windows line endings, of course)
-if there are lots of calls to SourceUtils.getFile that are not necessary, removing them would probably help at least a bit.
Comment 11 Jan Becicka 2008-03-28 07:34:21 UTC
> http://hg.netbeans.org/main/rev/14878a081417
ooooo. Thanks for finding this! Big leak in Filesystems! This is the regression since 6.0

> if there are lots of calls to SourceUtils.getFile that are not necessary, removing them would probably help at least a
bit.

It looks like getFile() is not that cheap operation I thought. It should be placed properly only to those places, where
really needed.

Comment 12 Pavel Flaska 2008-03-28 11:14:30 UTC
> Pavel, could you please redo your measurements on a build with the above patch? Thanks. (Please disregard the patch - it
> was only hiding away the real problem.)

Yes, I will check the results immediately when I will be able to create a new build ;-(.
Comment 13 Pavel Flaska 2008-03-28 14:17:50 UTC
I've created the new build and tested. Works like a charm.
Comment 14 Jan Pokorsky 2008-04-04 13:21:59 UTC
*** Issue 131981 has been marked as a duplicate of this issue. ***