I'm having issues which I find very annoying and prevents me from using the "Go to File" popup. I'm currently working on a Maven project with quite a lot of modules, but I only have 5 or 6 projects opened atm. There's also a lot of jar dependencies which could cause a lot of files to index.
I'm facing 2 problems:
- if I start typing a file name, the results area always show "No Files Found" and seem to do nothing unless I press the backspace key. At this moment it shows "Searching...". The strange thing is, if I start typing a file name and wait, it will eventually show up results even if I had no feedback about any actual search being in progress...
- the second problem is much more annoying, with both options checked (case sentitive and prefer current project), searching for a file name which will give a single result takes up to 25s!!
This is taking so long that I prefer searching the files by myself in the project structure :/
Netbeans 7.0 beta2 on Windows XP SP3, Intel Core2 duo @2.66GHz, 3.23Gb of RAM.
FYI, I have the same problem with netbeans 6.9.1 when I work on the same project.
Created attachment 106302 [details]
Profiler snapshot taken during go to file
The speed of indexed part was improved in the NB 7.0 by index caches, unfortunately part of it comes not from index and is searched on disk. If you attache the output of self profile? Thanks
It's already attached. :-) Thanks
Could it be related to my antivirus (McAfee) which is slowing down file scanning?
Unfortunately I don't have the necessary permissions to shut it down to make a test...
Finally I've got to the snapshot.
It's as I've described above the nearly whole time is spent in searching in non indexed parts of the project.
Here are the times:
Complete search: 76857ms
Search in indexed part: 2526ms
Search in non indexed part: 74331ms (The non indexed search is just going through the project folders ignoring the indexed folders (source roots) and listing them, the same as UNIX find command).
Yes, the Antivirus may have affect on it, but most of antivirus are activated by write not by read.
I am not Maven expert but it seems that the project has significant part which is not source and it's unindexed. Can you describe more the project structure? What is the biggest part of the project (excluding source roots - these are searched by parsing API in 2.5 s)?
Created attachment 106657 [details]
mvn dependency:tree extract
Indeed it is a maven project with quite a lot of dependencies (apache commons, ibatis, hibernate, spring etc.). I've attached a sample of dependency tree so that you can have an idea of the number of jars included. I cut the output because other modules have more or less the same dependencies.
For example our "core" project has about 160 dependencies (including transitive ones), 1600 classes (source code files) and several hundreds of JUnit test classes, XML files etc. The snapshot I provided initially was a search in this project.
I've tried using Intellij on the same project, and the go to file function finds instantly what I'm looking for. The problem is that NetBeans has a better maven integration IMHO, so I'd really like to continue using it without worrying about performance issues :)
What could I do to provide more information and help fixing this issue?
Any chance to see a performance improvement on this feature in NB7 final ?
Sorry I didn't got to it yet.
I will look at it later this week.
The dependency graph is not much helpful, also the number of sources is not helpful.
The snapshot shows that the indexed search is fast but significant time is spent in the non indexed part (non sources).
I've added a performance logging into jet-main b0eb35fa13ce.
When it becomes part of daily build can you try to run netbeans with -J-Dorg.netbeans.modules.jumpto.file.FileSearchAction.level=FINE to find out the times of indexed, providers and non indexed search.
Integrated into 'main-golden', will be available in build *201103160400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
User: Tomas Zezula <firstname.lastname@example.org>
Log: #195814 - performance logging
Created attachment 107039 [details]
Logs with o.n.m.jumpto.file.FileSearchAction.level=FINE
All right, I just downloaded the zip from hudson and did a few "go to file" with either indexed files (.java) or unindexed files (.xml or .jspx if I did understand correctly). The first result should be quite irrelevant because NB started scanning projects at the same time, but for the next searches there was no background task going on.
It seems that you are right, most of the time is spent searching in unindexed files (see attachment for full logs):
FINE [org.netbeans.modules.jumpto.file.FileSearchAction]: Worker for DealTermBusiness - started 3 391 ms.
FINE [org.netbeans.modules.jumpto.file.FileSearchAction]: Indexed Search: 0ms
FINE [org.netbeans.modules.jumpto.file.FileSearchAction]: Providers Search: 0ms
FINE [org.netbeans.modules.jumpto.file.FileSearchAction]: Unindexed search: 6 187ms
FINE [org.netbeans.modules.jumpto.file.FileSearchAction]: Worker for text DealTermBusiness finished after 9 578 ms.
FINE [org.netbeans.modules.jumpto.file.FileSearchAction]: Real search time 9578 ms.
I have one question though. How are compiled classes in JARs considered as? Are they indexed or unindexed resources?
The whole time is spent in the non indexed search (outside of sources).
The goto file does not search inside of jar files.
I cannot help much as the languages infrastructure does not know about non source folders.
That's quite interesting: I was wondering how to reduce the number of non-indexed files to scan, assuming that the search was done in opened projects. I closed my projects except one, which is quite small. The results were displayed instantly. I reopened the core project and I had no performance problem either. But when I opened the root project (which is only a pom project with multiple submodules), I get a huge search time again.
I only use this project to open submodules when needed, so I don't need it very often. I can get a satisfying workaround by opening only submodules I need...
The root project probably has lots of non indexed files.
I will try to experiment with it.
I don't know how the maven support works internally but I would guess that when I open the root project, NB also takes into consideration all the non-indexed files located in the submodules.
The root project has no direct non-indexed file, but the submodules do.
Yes, it seems so.
*** Bug 196960 has been marked as a duplicate of this bug. ***
Affects only non sources part of the project.
I will try to create a module which will pretend that the non sources part of the project is a "special" source which uses only file indexer.
Set target-milestone of open issue to TBD (was < 7.3, f.e. 6.8), so the issue doesn't get lost.
Set target-milestone of open issue to TBD (was < 7.3, f.e. 6.8), so the issue
doesn't get lost. This time the target milestone is really set.
I have this problem too, our project is a huge free form project and it takes too long to find any file. Any chance to have this addressed for next release?
The problem of the current GTF implementation is that it waits for the slowest provider before it displays the results. The slowest provider is the file crawling of all folders which are not under source root. Making the dialog to display partial results before the slowest provider completes will improve the search time for files under source roots.
>Any chance to have this addressed for next release?
Hard to say, there are no plans for next release yet.
Can you attach the nps snap shot.
Created attachment 131909 [details]
I hope it helps.
Can you add an option to 'Goto File' also in the Favourites tab. In this tab usually all the folder for database scripts are mounted, since they are not really (maven, ant) projects.
(In reply to comment #24)
> Can you add an option to 'Goto File' also in the Favourites tab. In this tab
> usually all the folder for database scripts are mounted, since they are not
> really (maven, ant) projects.
tomzi, can you file a new issue about it? I agree that this option could be helpful, but it is a different problem and should be treated as well. If you don't mind, I can do this for you, just let me know.
(In reply to comment #22)
> >Any chance to have this addressed for next release?
> Hard to say, there are no plans for next release yet.
Now that we have some plans for the next release, any chance to consider some improvements on this area? IMHO, it is a critical usability problem which could be considered a P2, but since it is just a matter of interpretation, you can disagree with me :)
Works without slowness for me in a medium-sized maven project.
Product Version: NetBeans IDE Dev (Build 201312190002)
Java: 1.7.0_45; Java HotSpot(TM) 64-Bit Server VM 24.45-b08
Runtime: Java(TM) SE Runtime Environment 1.7.0_45-b18
System: Windows 7 version 6.1 running on amd64; Cp1252; pt_BR (nb)
Okay, time for some real data. I tested it with the same hardware and project, but in Windows and Linux I had fairly different results. We have a project with 17.619 files where 10.509 are not indexed AFAIK. Our structure is something like that:
. other_files (10.509 files which are not mounted as source root in the project)
. src (default Maven structure with 6.982 files)
. (here more files like pom.xml etc)
Given this structure, AFAIK 'other_files' folder will not be index (as discussed before in this issue) and is probably the source of the issue because it is not mounted as a source root in the project.
I tested it in my Linux workstations searching for a single js file and it took something like 3 seconds, which is acceptable. But when we tested it in a Windows workstation it took more than 20 seconds, which now is not acceptable. I should highlight that the searched file is inside a folder which is mounted as source root in the project, so it probably is index.
Above is the configuration of the workstations.
Product Version: NetBeans IDE 8.0 RC1 (Build 201402242200)
Java: 1.7.0_51; Java HotSpot(TM) 64-Bit Server VM 24.51-b03
Runtime: Java(TM) SE Runtime Environment 1.7.0_51-b13
System: Windows 7 version 6.1 running on amd64; Cp1252; en_US (nb)
Product Version: NetBeans IDE Dev (Build 20140227-8aabb71e3f6d)
Updates: Updates available
Java: 1.7.0_40; Java HotSpot(TM) 64-Bit Server VM 24.0-b56
Runtime: Java(TM) SE Runtime Environment 1.7.0_40-b43
System: Linux version 3.4.63-2.44-desktop running on amd64; UTF-8; en_US (nb)
I hope this issue can have some attention soon.
20 votes ... at least P2
Could this somehow be related to https://netbeans.org/bugzilla/show_bug.cgi?id=241602 ?