This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 134558

Summary: Search FileObjects, not DataObjects
Product: utilities Reporter: Marian Petras <mpetras>
Component: SearchAssignee: Andrey Yamkovoy <kaktus>
Status: RESOLVED FIXED    
Severity: blocker CC: clairechung, nleck, pjiricka
Priority: P2 Keywords: API, PERFORMANCE
Version: 6.x   
Hardware: All   
OS: All   
Issue Type: DEFECT Exception Reporter:
Bug Depends on:    
Bug Blocks: 129557    
Attachments: Patch changing the Search mechanism to use FileObject's to search instead of DataObject's
Revised patch
Revised patch

Description Marian Petras 2008-05-06 06:14:59 UTC
The Search feature currently initializes a DataObject for each file it searches. This is time and memory consuming for
many DataObject types and it is not necessary to do so. It would be better to search just FileObjects and initialize
DataObjects only for FileObjects matching the search criteria.
Comment 1 nleck 2008-05-06 06:38:21 UTC
Hi Guys,

    We have ~5,000 source files. When we do a search on the source files
    in Solaris Netbeans 6.1 is taking > 10 minutes. The second time we
    do this search it takes a few seconds. The time is all in Netbeans
    as when we close open the IDE it's very slow again to do a search. A
    *find/grep *from the command line is fast.

    We also find opening a folder in the IDE is slow ( seconds)

    What is netbeans doing when we open a folder and is there a way to
    stop it ?

Thanks, 
Comment 2 clairechung 2008-08-27 12:34:22 UTC
I find this problem very troublesome too.  Netbeans is almost unusable for me  - On monday morning after our weekly 
reboot, the first thing I do is do a search over all of our files which can take about 45 mins.  If I do this, initial 
search, subsequent searches seem to be faster afterwards.
Still the 1 off 45mins is alot of wasted time & even after, the search speed is very very slow
Comment 3 Petr Jiricka 2008-08-27 13:43:57 UTC
Why is this an enhancement, not a defect? From the symptoms the users face it sounds like this should be a defect.
Comment 4 Marian Petras 2008-08-27 13:59:13 UTC
Changed the status to "defect".
Comment 5 nleck 2008-08-27 22:33:21 UTC
We aren't able to use the NB search at all as our ~5,000 source files are stored on a NFS mount ( mandatory, sorry) see
#144131 for more details. 

The command line below runs in seconds :- 

    find . -name "*.java" -exec grep "ABC" {} \; 

Also other IDEs search the sources quickly.
Comment 6 Andrey Yamkovoy 2008-12-23 12:06:38 UTC
Created attachment 75268 [details]
Patch changing the Search mechanism to use FileObject's to search instead of DataObject's
Comment 7 Andrey Yamkovoy 2008-12-23 12:12:22 UTC
Since the following API was changed reassign for API review:
- Added public interface SearchInfo.Files
- Utils class made public with one public static method
Comment 8 Jesse Glick 2009-01-05 17:19:42 UTC
BTW true Hg patches are generally preferred:
http://wiki.netbeans.org/HgHowTos#section-HgHowTos-DevelopAPIReviewPatchesUsingMQ


[JG01] Using FileObject (rather than File) has some overhead, too, though likely a lot less than DataObject adds. Have
you measured NB's time with FileObject (on a cold disk) and compared to command-line find/grep? What about when using File?


[JG02] SearchInfo.Files missing class Javadoc. Also be sure to add @since and apichanges.xml entry. Same for Utils.


[JG03] DataObject.getChildren automatically sorts files, usually by name. Probably you want to do the same manually when
working at the FileObject level. Otherwise search results will appear in unpredictable order.
Comment 9 Andrey Yamkovoy 2009-01-13 17:11:29 UTC
Created attachment 75772 [details]
Revised patch
Comment 10 Andrey Yamkovoy 2009-01-14 10:45:36 UTC
JG01: I am afraid we can't use File instead of FileObject mainly because we need to know charset to read file correctly.
As for comparing performance with command-line find tools I have not measured it.

JG02: Done in the revised patch.

JG03: I plan to implement sorting result functionality on the result model side (issue 119818). It allows us to sort all
search results together instead of only for each folder level. Also by this way we can implement different sorting
functions and add UI to allow user to use it. So I think it will be overhead to do sorting on the FileObject level.
Comment 11 Jaroslav Tulach 2009-01-17 14:35:17 UTC
Y01 I do not see a single line of test in the patch. Please cover the API with a testcase. 

Especially the CompoundIterators semantics deserve some stiffening with a test, imho.
Comment 12 Andrey Yamkovoy 2009-01-19 16:15:06 UTC
Created attachment 75996 [details]
Revised patch
Comment 13 Andrey Yamkovoy 2009-01-19 16:18:01 UTC
In the revised patch 2 tests for the new API were added and others were fixed.
Comment 14 Jaroslav Tulach 2009-01-20 10:09:23 UTC
Y02 Add private constructor to Utils class.

Otherwise OK, I guess.
Comment 15 Andrey Yamkovoy 2009-01-20 16:39:30 UTC
Y02: Will do before pushing the changes into repository.
Comment 16 Andrey Yamkovoy 2009-01-23 12:19:58 UTC
I will integrate the last patch plus changes for Y02 in 24h.
Comment 17 Andrey Yamkovoy 2009-01-23 15:09:55 UTC
Fixed in the trunk.
Comment 18 Quality Engineering 2009-01-24 18:51:27 UTC
Integrated into 'main-golden', will be available in build *200901241401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/4396d95a883f
User: Andrey Yamkovoy <kaktus@netbeans.org>
Log: Fix for #134558 - Search FileObjects, not DataObjects