This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 147620 - Improve performance of Go To File by indexing
Summary: Improve performance of Go To File by indexing
Status: RESOLVED DUPLICATE of bug 122639
Alias: None
Product: utilities
Classification: Unclassified
Component: Jump To (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P1 blocker (vote)
Assignee: Vitezslav Stejskal
URL:
Keywords: PERFORMANCE
Depends on:
Blocks: 122639
  Show dependency tree
 
Reported: 2008-09-18 16:43 UTC by dafreels
Modified: 2009-04-03 12:17 UTC (History)
2 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Go To File Thread Dump (48.84 KB, text/plain)
2008-09-19 13:31 UTC, dafreels
Details

Note You need to log in before you can comment on or make changes to this bug.
Description dafreels 2008-09-18 16:43:16 UTC
[ BUILD # : 200809190201 ]
[ JDK VERSION : 1.6.0_04 ]

Using Go To File on a large project becomes unusable. this becomes an
issue when working with large web projects with many js, xml or
properties files.
Comment 1 dafreels 2008-09-18 16:53:04 UTC
Go To Type is also slow.
Comment 2 Peter Pis 2008-09-18 22:36:04 UTC
Could you please provide more details? Thread dump would be appreciated.

http://wiki.netbeans.org/GenerateThreadDump
Comment 3 dafreels 2008-09-19 13:31:20 UTC
Created attachment 70145 [details]
Go To File Thread Dump
Comment 4 Pavel Flaska 2008-09-22 09:33:19 UTC
Reporter's comment on mailing list:

I have noticed that the performance of the Go To Type functionality is hit or miss when it comes to performance. Sometimes I can get almost 
instantaneous results, other times I am having to wait many seconds (~5-20) before I get results. I feel this is unacceptable given that on the same 
machine Eclipse and Intellij are orders of magnitude faster. Intellij begins showing results almost immediately and Eclipse takes no longer than ~1-2 
seconds. So I did a little research and found the index cache where Netbeans stores class files and Lucene indexes. I have ~ 435 directories. Each 
directory has a Lucene index. I am not sure if the Netbeans code is pulling all those together at runtime or not, but I believe that it would be more 
efficient to combine these into a single index and allow Lucene to partition these as it sees fit.
Has any one else experienced this behavior? I had a few other developers in the office install Netbeans and they experienced similar behavior. I will be 
filing another issue but I doubt it can be fixed until 7.0. I think this issue should be taken very seriously since it presents a bad experience to developers 
who are ready to leave Eclipse for Netbeans.
David Freels
P.S. Go To File is unusable still. This is even after the performance improvements (they did help) made the last time I filed an issue.
Comment 5 Pavel Flaska 2008-09-23 07:28:14 UTC
Go To File is file search, there is no index for files. It means it is strongly related how fast is your disk, how OS caches the filesystems etc. That is also 
reason why it gives you different results. IntelliJ indexes the files when started, that is the reason why it is so fast.
Comment 6 Jiri Kovalsky 2008-09-23 14:55:41 UTC
David provided the thread-dump. Removing INCOMPLETE keyword.
Comment 7 Pavel Flaska 2008-09-23 17:18:38 UTC
Note:
For the Go To Type..., I have added additional logging to measure individual time consumed by providers:
http://hg.netbeans.org/main/rev/b630fab2f56a

Right now, you can switch it on with command line parameter:
-J-Dorg.netbeans.modules.jumpto.level=FINE.


Comment 8 Pavel Flaska 2008-09-23 17:24:44 UTC
As I described above, Go To File is strongly affected by disk, filesystem, operating system etc. If you use large source-base, it is obvious that going 
throughout the whole disk will take a lot of time. Only the way how to improve behavior dramatically is to index all files, which will take some time during the 
start-up. But it is another discussion. Closing as WONT fix, as currently there are no plans to index all files.
Comment 9 Tomas Zezula 2008-09-23 18:08:31 UTC
1) In my project group there are 47983 java files, go to type takes in average less than 3 sec, except of the first run when the files are not hot.
Intellij has faster go to type but as it loads the class index into memory it needs higher mx.
2) There was an issue in the Go to type causing Go to type not to cancel previous searches which slows it down, it was fixed in NB 6.5.
3)
>I am not sure if the Netbeans code is pulling all those together at runtime or not, but I believe that it would be more 
>efficient to combine these into a single index and allow Lucene to partition these as it sees fit.
Is a naive solution which unfortunately doesn't works at all, see the eclipse or intelliJ, there is the same partitioning of storages. Any modification,
especially cp changes will be very expensive. The overhead on federation is not very high as nb doesn't use the query API and uses
just the low level APIs and caches the index readers.
4) There is an RFE to improve go to file to use the index rather than the file search, scheduled for current + 1.

Comment 10 Tomas Zezula 2008-10-15 16:03:18 UTC
Unlike Go to type the Go to file doesn't benefit from an index. Either parsing API or some other module should provide file name index which can be used by 
this feature. This will dramatically decrease the time of search from minutes to seconds.
Comment 11 Jaroslav Tulach 2009-04-01 14:43:33 UTC
Tomáš just mentioned that he might look at the implementation one day.
Comment 12 Tomas Zezula 2009-04-01 14:47:37 UTC
I will do it this weekend.
Comment 13 Tomas Zezula 2009-04-03 09:59:29 UTC
Vita was faster and already implemented this.
Comment 14 Vitezslav Stejskal 2009-04-03 12:17:32 UTC

*** This issue has been marked as a duplicate of 122639 ***