Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 168817 - slow initial indexing of C++ project
slow initial indexing of C++ project
Status: VERIFIED FIXED
Product: editor
Classification: Unclassified
Component: Parsing & Indexing
6.x
Sun All
: P2 (vote)
: 6.x
Assigned To: Vitezslav Stejskal
issues@editor
: PERFORMANCE
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-07-20 14:40 UTC by Vladimir Voskresensky
Modified: 2009-07-24 11:12 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
:


Attachments
the idea is to get mime-type info somewhere else, not by expensive creating of snapshot (1.50 KB, patch)
2009-07-20 14:48 UTC, Vladimir Voskresensky
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Voskresensky 2009-07-20 14:40:03 UTC
Hi, Vita,

I have checked the current performance of indexing C++ project and it is still very slow for initial scan.

I will investigate the reason, if you'd like
Comment 1 Vladimir Voskresensky 2009-07-20 14:47:09 UTC
Well, it's again the very expensive creation of useless snapshots.
I will attach the patch which speeds up initial indexing 3x times
Comment 2 Vladimir Voskresensky 2009-07-20 14:48:04 UTC
Created attachment 84934 [details]
the idea is to get mime-type info somewhere else, not by expensive creating of snapshot
Comment 3 Vladimir Voskresensky 2009-07-20 14:49:06 UTC
with the proposed change time is improved from 
Complete indexing of 3 source roots took: 149192 ms
to
Complete indexing of 3 source roots took: 43312 ms
Comment 4 Jan Jancura 2009-07-21 08:59:54 UTC
Hi Vladimir.

It looks like this issue is already fixed in trunk. Can you check it? Or do you want to fix it in some other branch?
Comment 5 Vladimir Voskresensky 2009-07-21 09:27:01 UTC
Hi, Jan.
Vita have changed files crawling and closed issue 166340 with the request to open new one if we find the problem again.
I have checked the influence of fixes of issue 166340 and found that initial scanning phase is still very very slow (as
you can see 150 sec :-( ).
I have checked it in trunk version of cnd-main repository => I have profiled the problematic scanning and found that
fixing  issue 166340 introduced creation of non-used snapshot (very expensive) for the only reason of detecting
mime-type => I have changed it in the proposed patch.

Btw, I have checked your trunk as well
http://hg.netbeans.org/jet-main/file/tip/parsing.api/src/org/netbeans/modules/parsing/impl/indexing/RepositoryUpdater.java
and I see that the issue exists (see line 1303).
So, I have checked it :-)
Comment 6 Jan Jancura 2009-07-21 13:48:09 UTC
Yes, but last Vita's change fo RepositoryUpdater is opposite to your patch:
http://hg.netbeans.org/jet-main/diff/f22eac907102/parsing.api/src/org/netbeans/modules/parsing/impl/indexing/RepositoryUpdater.java

So, I am confused. I will reassign this issue to Vita. He should be back soon.
Comment 7 Vladimir Voskresensky 2009-07-21 14:07:58 UTC
Oh, I see. That explains, why before Vita's optimization I have seen 3x speed up and after the fix initial scan became
slow again. So, as result the improvement was only with "up-to-date" check and nothing was changed in initial scan phase
Comment 8 Quality Engineering 2009-07-23 07:53:11 UTC
Integrated into 'main-golden', will be available in build *200907230201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/59600af3ff86
User: Vladimir Voskresensky <vv159170@netbeans.org>
Log: fix for IZ#168817: slow indexing of C++ project
- do not create snapshots to obtain mime-type information only
Comment 9 Vitezslav Stejskal 2009-07-23 09:33:55 UTC
Unfortunately, Vladimir's http://hg.netbeans.org/main-golden/rev/59600af3ff86 is not correct. The mime type has to be
taken from the snapshot otherwise indexing is not going to work in embedded scenarios. I'll have a closer look and try
to come up with a correct solution that would improve situation for cnd. In the meantime I'm going to backout 59600af3ff86.
Comment 10 Vitezslav Stejskal 2009-07-23 10:24:15 UTC
Oh, I've now seen http://hg.netbeans.org/main-golden/rev/b4073f9088e6, which reverts 59600af3ff86. Thanks Vladimir
Comment 11 Vladimir Voskresensky 2009-07-23 12:56:34 UTC
Vita, thanks for the evaluation. I expected that it might be incorrect in general case...
Can we enhance ResultIterator with getMimeType to prevent creating of snapshost only for getting mime type?
Comment 12 Vitezslav Stejskal 2009-07-23 19:10:15 UTC
This should fix the problem - http://hg.netbeans.org/jet-main/rev/c633b15a60d9 - even though I was not able to see 3x
improvement reported by Vladimir. I generally seem to be getting much higher numbers than Vladimir. Vladimir, could you
please remeasure this again and either confirm the fix or reopen this issue? Thanks
Comment 13 Vladimir Voskresensky 2009-07-23 21:50:14 UTC
Hello, Vita,
Ok. I will check tomorrow (and profile further if needed). 
Btw, do you know that you can turn off parsing part of C++ project (and leave only scanning phase) in project's context
menu Code Assistance->C/C++ Code Assistance 
Comment 14 Quality Engineering 2009-07-24 05:42:35 UTC
Integrated into 'main-golden', will be available in build *200907240201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/c633b15a60d9
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #168817: do not attempt to parse snapshots of files that have no EmbeddingIndexer
Comment 15 Vitezslav Stejskal 2009-07-24 08:34:49 UTC
"Btw, do you know that you can turn off parsing part of C++ project" - Neat, thanks. Now I know :-).
Comment 16 Vladimir Voskresensky 2009-07-24 11:12:19 UTC
verified
Complete indexing of 3 source roots took: 43223 ms
I see the speedup, thanks!


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo