fixed in web-main#f5fcfd15a1c6 by implementing some FileObject->ClasspathInfo->JavaSource second level cache in the ELTypeUtilities. If the ClasspathInfo changes (resp. the underlying Classpath changes) the affected entries are cleared.

Martine, please verify the performance improvement and Tome, if you have some time please review the code, thanks.

Of course, I will try it out. I also asked tmysik for verifying - he was able to check it with bigger real project. Thank you Marek for fixing that so quickly.

Integrated into 'main-golden', will be available in build *201105280401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/f5fcfd15a1c6
User: Marek Fukala <mfukala@netbeans.org>
Log: #198881 - Bad performance in ELTypeUtilities

Well, not sure whether I can verify this issue, likely not; I have a build with this changeset but the code completion for _all_ beans (see initial description of #198783 for more information) takes:
~10 seconds for first invocation
~6 seconds for the next invocations

Unfortunately, user experience is not so good because 6 seconds for code completion is still quite a lot. Are we interested in e.g. profiler snapshot? Or is there anything else I can do? Or is it a use case (code completion for _all_ beans) that we don't want to "support"?

Thanks guys.

Product Version: NetBeans IDE Dev (Build 20110527-8c744797283f)
Java: 1.6.0_24; Java HotSpot(TM) 64-Bit Server VM 19.1-b02
System: Linux version 2.6.38-8-generic running on amd64; UTF-8; cs_CZ (nb)

I am afraid this will not work.
1st) The ClassPathInfo should not be used as a key, it's mutable object.
2nd) There is no need to add the listener onCPInfo when CPInfo changes the Source is still valid and
refreshed.

After an offline discussion with Tomas Zezula we agreed the fix works (at least the JavaSource is cached), just the ClasspathInfo listener is not necessary (doesn't work due to the CPI.hashCode() change). Once the CPI changes a new one along with new JavaSource will be created, the old references to the JavaSource should be cleared since the map is a WeakHashMap.

Right.
The listener should be removed as it's useless.
The fix works if there is no classpath change. If there is a cp change the caching does not work. But it's not mem leak (at least I hope :-)) because it's WeakHashMap.
As the EL module should be completely rewritten (it validates the java.source lock model) I agree that there is no need to solve the classpath change case.
Anyway with the fix the situation should be much better than without the fix.

the listener has been removed in web-main#71f435b51024

(In reply to comment #4)
> completion is still quite a lot. Are we interested in e.g. profiler snapshot?
Sure, please attach it. I wonder how come the fix hasn't influenced the performance...

(In reply to comment #9)
> Sure, please attach it. I wonder how come the fix hasn't influenced the
> performance...

OK, so reopening...

Thanks.

Created attachment 108596 [details]
CC - first invocation

Created attachment 108597 [details]
CC - second invocation

I'm going to rewrite the web.el module so the features like code completion, refactoring etc. doesn't invoke many UserTask-s during the code execution but instead they will run their code inside a single UserTask. This should(hopefully) eliminate most of the performance problems.

Currently it looks like caching the JavaSource didn't help much, there might be possibly a problem with the many JavaSource.runUserActionTask() run during the code completion run.

BTW I found some JS.runUserActionTask() in the EDT from withind ELJavaCompletionItem.getLs/RsHtml() which is definitively wrong and will be addressed along with the mentioned rewrite.

Integrated into 'main-golden', will be available in build *201105310954* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/7d26a616655e
User: Marek Fukala <mfukala@netbeans.org>
Log: #198881 - adding forgotten classpathinfo listener removal.

changeset:   194862:d4135ef76ff1
summary:     #198881 - redesigning the java infrastructure access within the web.el module. Now the code fulfills the general contract - lock the java infrastructure first, then access the java model.

I'd like to ask tmysik to verify the performance in his testing project.

Thanks Marku!

Integrated into 'main-golden', will be available in build *201106010401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/7d26a616655e
User: Marek Fukala <mfukala@netbeans.org>
Log: #198881 - adding forgotten classpathinfo listener removal.

There's a still a performance bottleneck in org.netbeans.modules.web.jsf.editor.el.WebBeansELVariableResolver.getWebBeans() which is called very often (at least once per one EL in a page). It triggers the MetadataModel.runReadAction() and then inside the internal java source task it resolves the cached ElementHandle-s to Element-s which is very slow. I need to do implement some context aware caching at the client side.

Just for curiosity, error checking of a page with 250 expressions takes about 6 seconds, mostly spent in the mentioned ElementHandle to Element conversion. The MM.runREadAction() is also called 250x :-(.

This problem also significantly influences the performance of the EL occurrences finder.

(In reply to comment #15)
> I'd like to ask tmysik to verify the performance in his testing project.

Just tried and the results are really great! First CC takes less than 1 second, next CC is nearly instant.

Greate job, thanks a lot.

changeset:   194994:5f560fb41770
summary:     #198881 - adding a compilation context caching of the ELVariableResolvers results.

...now everything is even more faster. Especially the hints and occurrences finder in a file with many ELs are instant compared to the previous seconds.

Integrated into 'main-golden', will be available in build *201106021001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/7d26a616655e
User: Marek Fukala <mfukala@netbeans.org>
Log: #198881 - adding forgotten classpathinfo listener removal.

changeset:   200027:54f390c7884e
branch:      release701
user:        Marek Fukala <mfukala@netbeans.org>
date:        Fri May 27 14:21:27 2011 +0200
summary:     #198881 - Bad performance in ELTypeUtilities

changeset:   200028:0aa7ac184fa8
branch:      release701
user:        Marek Fukala <mfukala@netbeans.org>
date:        Mon May 30 10:58:58 2011 +0200
summary:     #198881 - adding forgotten classpathinfo listener removal.

changeset:   200029:6020d073c6bc
branch:      release701
user:        Marek Fukala <mfukala@netbeans.org>
date:        Mon May 30 14:45:12 2011 +0200
summary:     ##198881 - removing needless ClasspathInfo listener

changeset:   200030:f42ebcd7f2c3
branch:      release701
user:        Marek Fukala <mfukala@netbeans.org>
date:        Wed Jun 01 14:59:24 2011 +0200
summary:     #198881 - redesigning the java infrastructure access within the web.el module. Now the code fulfills the general contract - lock the java infrastructure first, then access the java model.

changeset:   200031:067d1035a17a
branch:      release701
user:        Marek Fukala <mfukala@netbeans.org>
date:        Thu Jun 02 10:22:23 2011 +0200
summary:     #198881 - adding a compilation context caching of the ELVariableResolvers results.

Integrated into 'main-golden', will be available in build *201106031000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/5f560fb41770
User: Marek Fukala <mfukala@netbeans.org>
Log: #198881 - adding a compilation context caching of the ELVariableResolvers results.