Bug 221331 - "Action Items" tab makes C/C++ IDE unusable (PropertyDefinition.CACHE 1.5Gb RAM)
"Action Items" tab makes C/C++ IDE unusable (PropertyDefinition.CACHE 1.5Gb RAM)
Status: RESOLVED FIXED
Product: web
Classification: Unclassified
Component: CSS Editor
7.3
PC Solaris
: P1 (vote)
: 7.3
Assigned To: Marek Fukala
Vladimir Riha
: PERFORMANCE
: 221772 221936 222173 (view as bug list)
Depends on:
Blocks: 184292
  Show dependency treegraph
 
Reported: 2012-11-02 10:28 UTC by soldatov
Modified: 2012-11-21 15:51 UTC (History)
9 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
Beggining of indexing (525.14 KB, application/octet-stream)
2012-11-02 13:38 UTC, Alexander Simon
Details
just before IDE hung. IDE have eaten 2G memory (102.49 KB, application/octet-stream)
2012-11-02 13:40 UTC, Alexander Simon
Details
NB project for firefox (237.58 KB, application/x-bzip)
2012-11-06 16:36 UTC, Vladimir Voskresensky
Details
Classes with most instances (46.73 KB, image/png)
2012-11-08 15:14 UTC, Tomas Zezula
Details
GC root path (74.71 KB, image/png)
2012-11-08 15:16 UTC, Tomas Zezula
Details

Note You need to log in before you can comment on or make changes to this bug.
Description soldatov 2012-11-02 10:28:19 UTC
Scenario:
- Switch on C/C++|Other|Support indexing of non C/C++/Fortran files in projects (slow down parsing)
- Open mozilla-central project (Firefox's sources)
- Wait 10 minutes
- Push Window|Action Items
===>

main problems:
infinite "Backgroung scaning of projects" progress bar
IDE freezes very often
Comment 1 Alexander Simon 2012-11-02 12:27:07 UTC
It seems indexing infrastructure unusable for mozilla project.
Comment 2 Alexander Simon 2012-11-02 12:29:46 UTC
see also bug #221342 and bug #221337
Comment 3 Alexander Simon 2012-11-02 12:32:44 UTC
P2 because:
- IDE was freezed more then 30 minutes
- IDE eat 2G memory
Comment 4 Tomas Zezula 2012-11-02 13:05:58 UTC
Please attach IDE self profile, according to the linked issue the CSS & HTML parsing is done which has not much common with general p.a. There is already an issue about expensive HTML validation https://netbeans.org/bugzilla/show_bug.cgi?id=206025.
Comment 5 Alexander Simon 2012-11-02 13:38:30 UTC
Created attachment 127011 [details]
Beggining of indexing
Comment 6 Alexander Simon 2012-11-02 13:40:52 UTC
Created attachment 127013 [details]
just before IDE hung. IDE have eaten 2G memory
Comment 7 Tomas Zezula 2012-11-02 13:42:17 UTC
Thanks!
Comment 8 Tomas Zezula 2012-11-02 13:47:13 UTC
CSS parsing.
If it's possible can you attach a longer snapshot, for example 2 or 5 mins.
In the snapshot there is only CSS parsing but not HTML validator which I expect to be there as well.
Thanks!
Comment 9 Alexander Simon 2012-11-02 13:56:20 UTC
(In reply to comment #8)
> CSS parsing.
> If it's possible can you attach a longer snapshot, for example 2 or 5 mins.
> In the snapshot there is only CSS parsing but not HTML validator which I expect
> to be there as well.
> Thanks!
Unfortunately I cant because IDE died (not responsible).
Comment 10 Tomas Zezula 2012-11-02 13:58:09 UTC
>Wait 10 minutes
Can you try to to start sampling a bit earlier?
After 5 mins?
Comment 11 Alexander Simon 2012-11-02 14:11:47 UTC
(In reply to comment #10)
> >Wait 10 minutes
> Can you try to to start sampling a bit earlier?
> After 5 mins?
I give IDE 2G heap.
IDE start indexing.
I start profiling.
When IDE eat 1900M memory I take first snapshot.
After that IDE becames partly unresponsible, but I managed to take short second snapshot before IDE died.
Comment 12 Tomas Zezula 2012-11-02 14:15:43 UTC
Sorry, my fault I've overlooked the first snapshot.
That's exactly what I wanted.
Thanks!
Comment 13 Tomas Zezula 2012-11-02 15:04:19 UTC
Looking at the first snapshot the most time is spent in indexing javascript.
Seems a memory leak in the javascript as it takes 1900M.
I will clone mozilla central and generate heap dump to find out more.
Comment 14 Petr Pisl 2012-11-06 16:12:21 UTC
Tomas thanks for the help. I don't have the Mozilla central and I'm going to stop in your office tomorrow.
Comment 15 Vladimir Voskresensky 2012-11-06 16:35:30 UTC
#mkdir firefox
#cd firefox
#export CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot
#cvs checkout mozilla/client.mk
#cd mozilla 

create .mozconfig file with the following content:
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox
ac_add_options --enable-extensions=default,webservices
ac_add_options --enable-application=browser
ac_add_options --enable-debug

#make -f client.mk checkout 

Then extract inside top firefox folder (on the same level with mozilla folder) attached nb project
Comment 16 Vladimir Voskresensky 2012-11-06 16:36:37 UTC
Created attachment 127239 [details]
NB project for firefox
Comment 17 Vladimir Voskresensky 2012-11-06 16:45:47 UTC
To turn on TODO support (which activates js parsing) you should Tools->Options->C/C++->Other and set 
"Support indexing of non C/C++/Fortran files in projects (slow down parsing)"
Comment 18 Tomas Zezula 2012-11-06 17:07:03 UTC
Thanks Vladimir. I've already downloaded the mozilla-central project from Alex.
I will work on it tomorrow.
Comment 19 Tomas Zezula 2012-11-08 15:08:24 UTC
There is a large memory leak in the CSS support which holds 900MB of memory when IDE is on 1.5GB of memory, when IDE reaches 2GB limits it consumes 1.5GB.
Most of the classes are:
char[], StringBuilder, FixedTextGrammarElement which in fact are single reference chain: char[]<-StringBuilder<-FixedTextGrammarElement .
These instances are held by PropertyDefinition.CACHE which is referenced by various subtypes of the CssModule registered in global lookup and hold by it.
The CACHE itself is a WeakHashMap<FileObject,GroupGrammarElement>. The WHM conditions are satisfied,  the map s used and value does not refer to key. However the map grows very fast and the FIleObjects are still alive so the values are not freed.

If you are interested in heap dump it's available.
I will attach some visual VM screens.
Comment 20 Tomas Zezula 2012-11-08 15:14:39 UTC
Created attachment 127403 [details]
Classes with most instances
Comment 21 Tomas Zezula 2012-11-08 15:16:56 UTC
Created attachment 127404 [details]
GC root path
Comment 22 Marek Fukala 2012-11-08 15:32:11 UTC
The caching has been changed recently due to some architecture changes, but the main problem here is IMO not the cache but the size/number of the held objects. It should definitively not be such big. I'll take a more detailed look...
Comment 23 Marek Fukala 2012-11-08 15:33:37 UTC
marian, why you changed this to P1?
Comment 24 Tomas Zezula 2012-11-08 15:37:34 UTC
The memory leak happens for me even when task list (Action Items) is closed. The CSS TLIndexer (TaskList indexers) is called even when the Task list is closed. I personally believe that it's wrong the task list indexers should not be called when the task list is closed and the indexers should be refreshed when the task list is opened. But this will require new API in task list. Probably something we should look at as it affects all project types.
Comment 25 Marek Fukala 2012-11-09 08:37:50 UTC
fixed in web-main#552597ead66c

I had to restrict the possibilities of CssEditorModule in terms of what properties it can inject so a reasonable caching is possible.
Comment 26 Quality Engineering 2012-11-10 03:27:12 UTC
Integrated into 'main-golden', will be available in build *201211100001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/552597ead66c
User: Marek Fukala <mfukala@netbeans.org>
Log: #221331 - "Action Items" tab makes C/C++ IDE unusable (PropertyDefinition.CACHE 1.5Gb RAM)
Comment 27 Marek Fukala 2012-11-12 10:39:12 UTC
*** Bug 221936 has been marked as a duplicate of this bug. ***
Comment 28 Petr Cyhelsky 2012-11-14 10:43:51 UTC
*** Bug 221772 has been marked as a duplicate of this bug. ***
Comment 29 Petah 2012-11-14 23:20:23 UTC
I don't know if this is related, but even the dev build is still having this problem:

http://netbeans.org/bugzilla/show_bug.cgi?id=221840
Comment 30 Tomas Hurka 2012-11-15 09:18:27 UTC
*** Bug 221772 has been marked as a duplicate of this bug. ***
Comment 31 Petr Cyhelsky 2012-11-15 09:51:23 UTC
*** Bug 221772 has been marked as a duplicate of this bug. ***
Comment 32 Tomas Hurka 2012-11-21 15:51:56 UTC
*** Bug 222173 has been marked as a duplicate of this bug. ***


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo