This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 143080 - java.lang.OutOfMemoryError: Java heap space during indexing of PHP include path
Summary: java.lang.OutOfMemoryError: Java heap space during indexing of PHP include path
Status: VERIFIED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Text (show other bugs)
Version: 6.x
Hardware: All All
: P1 blocker (vote)
Assignee: Jaroslav Tulach
URL:
Keywords: PERFORMANCE, REGRESSION
: 143027 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-08-06 16:16 UTC by Tomas Mysik
Modified: 2008-12-22 13:33 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Screenshot of the references from heapdump (149.68 KB, image/png)
2008-08-08 09:02 UTC, Marek Fukala
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tomas Mysik 2008-08-06 16:16:57 UTC
I added my PHP include path (/usr/share/php5 and /usr/share/php folders) to the IDE global PHP include path - and 
after long time this exception occurs.

SEVERE [global]
Root: /usr/share/php5 File: Tags.php Bootpath: ClassPath[Entry[file:/usr/share/php5/]] Classpath: 
ClassPath[Entry[file:/usr/share/php5/]] Sourcepath: ClassPath[Entry[file:/usr/share/php5/]]
Caused by: java.lang.OutOfMemoryError: Java heap space
        at org.netbeans.lib.lexer.LAState$LargeState.<init>(LAState.java:400)
        at org.netbeans.lib.lexer.LAState$LargeState.upgrade(LAState.java:412)
        at org.netbeans.lib.lexer.LAState.add(LAState.java:114)
        at org.netbeans.lib.lexer.inc.IncTokenList.tokenOrEmbeddingImpl(IncTokenList.java:191)
        at org.netbeans.lib.lexer.inc.IncTokenList.tokenOrEmbedding(IncTokenList.java:180)
        at org.netbeans.api.lexer.TokenSequence.moveNext(TokenSequence.java:454)
        at org.netbeans.modules.javascript.editing.embedding.JsModel.extractJavaScriptFromPHP(JsModel.java:255)
        at org.netbeans.modules.javascript.editing.embedding.JsModel.getJsCode(JsModel.java:154)
        at org.netbeans.modules.javascript.editing.embedding.JsTranslatedSource.getSource(JsTranslatedSource.java:69)
        at org.netbeans.napi.gsfret.source.ParserTaskImpl.parse(ParserTaskImpl.java:139)
        at org.netbeans.modules.gsfret.source.usages.RepositoryUpdater.batchCompile(RepositoryUpdater.java:2040)
        at 
org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker.updateFolder(RepositoryUpdater.java:1405)
        at 
org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker.scanRoots(RepositoryUpdater.java:1130)
        at org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker.access$1900
(RepositoryUpdater.java:652)
        at org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker$1.run(RepositoryUpdater.java:790)
        at org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker$1.run(RepositoryUpdater.java:680)
        at org.netbeans.modules.gsfret.source.usages.ClassIndexManager.writeLock(ClassIndexManager.java:123)
        at org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker.run(RepositoryUpdater.java:677)
        at org.netbeans.modules.gsfret.source.usages.RepositoryUpdater$CompileWorker.run(RepositoryUpdater.java:652)
        at org.netbeans.napi.gsfret.source.Source$CompilationJob.run(Source.java:1300)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
        at java.util.concurrent.FutureTask.run(FutureTask.java:123)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)

Product Version: NetBeans IDE Dev (Build 080805)
Java: 1.5.0_16; Java HotSpot(TM) Client VM 1.5.0_16-b02
System: Linux version 2.6.26-gentoo running on i386; UTF-8; cs_CZ (nb)
Comment 1 Tomas Mysik 2008-08-06 16:19:58 UTC
Additional information:

/usr/share/php5 - 42M, 4660 files
/usr/share/php  - 21M, 1933 files
Comment 2 Tomas Mysik 2008-08-06 17:04:37 UTC
During indexing, IDE is completely unusable (100% CPU) and after 9 minutes (!) exception occurs.
(May or may not be related to the issue #132312.)
Comment 3 Petr Pisl 2008-08-06 18:03:17 UTC
This must to be a regression from this Monday, Tuesday or Wednesday. 
Comment 4 Marek Fukala 2008-08-07 12:29:40 UTC
I'll look at it ASAP
Comment 5 Marek Fukala 2008-08-07 15:04:03 UTC
The indexing slowness may be caused by the garbage collector trying to free some memory. There is an issue #143027
filled against javascript support complaining about memory leaks after initial indexing in web project with some
javascripts.

OTOH I am not sure whether the issue is regression from this week as ppisl claims. 

I'll try to analyze the heapdumps attached to the other issue. 
Comment 6 Petr Pisl 2008-08-07 16:06:01 UTC
It must to be regression from this week, because I measured the indexing of php files on Sunday. The indexing of my
testing project took 66 seconds. Now after 5 minutes I obtain OutOfMemory. 
Comment 7 Petr Pisl 2008-08-07 16:07:51 UTC
Just to be sure, I have tried it in this build

Product Version: NetBeans IDE Dev (Build 200808040201)
Java: 1.6.0_10-beta; Java HotSpot(TM) Client VM 11.0-b12
System: Linux version 2.6.24-19-generic running on i386; UTF-8; en_US (nb)
Userdir: /home/petr/.netbeans/de

and it works. It really takes "only" 66 seconds, no exception. 
Comment 8 Petr Pisl 2008-08-07 16:14:09 UTC
The most important thing is, whether this regression is in the beta as well or not. If it's in beta, then has to be
fixed. I will try it.
Comment 9 Petr Pisl 2008-08-07 16:18:53 UTC
I have just tested 

Product Version: NetBeans IDE 6.5 Beta (Build 200808070301)
Java: 1.6.0_10-beta; Java HotSpot(TM) Client VM 11.0-b12
System: Linux version 2.6.24-19-generic running on i386; UTF-8; en_US (nb)
Userdir: /space/tmp/userdir/567

and this build is OK. It doesn't contain the regression. It can not beta stopper.
Comment 10 Marek Fukala 2008-08-08 08:55:29 UTC
From looking into the heapdump I guess this is a regression from jtulach's changeset 94375:fef78628ae8e from Aug 4th -
he introduced a new static field 'charsets' into DataEditorSupport which  seems to indirectly hold all DataObject
instancies which used DataEditorSupport to load the content into editor.

Jardo, can you please take a look at this issue? It is 100% reproducible if you follow the steps above. I cannot attach
the heapdump here so I'll at least make a screenshot of the part I belive is the source of the leak.

Let me know if we (or gsf) does something wrong if you belive your change is ok.

Thanks
Comment 11 Marek Fukala 2008-08-08 09:02:15 UTC
Created attachment 66890 [details]
Screenshot of the references from heapdump
Comment 12 Jaroslav Tulach 2008-08-08 10:07:30 UTC
The objects are supposed to be held only during save/load. The described behaviour would be quite surprising.
Comment 13 Marek Fukala 2008-08-08 10:13:58 UTC
So maybe I am wrong, not completely sure. Will you check it then?
Comment 14 Jaroslav Tulach 2008-08-08 12:13:40 UTC
I guess the get in following diff in http://hg.netbeans.org/main/rev/fef78628ae8e shall be remove:

         } finally {
-            charsetForSaveAndLoad = null;
-            charsetForObject = null;
+            charsets.get(tmpObj);
         }
Comment 15 Jaroslav Tulach 2008-08-08 12:18:02 UTC
changeset:   95246:574db98aeaff
tag:         tip
user:        Jaroslav Tulach <jtulach@netbeans.org>
date:        Fri Aug 08 13:17:11 2008 +0200
summary:     #143080: Use remove for cleanup not get
Comment 16 Jaroslav Tulach 2008-08-08 12:19:00 UTC
*** Issue 143027 has been marked as a duplicate of this issue. ***
Comment 17 Quality Engineering 2008-08-09 03:41:30 UTC
Integrated into 'main-golden', available in build *200808090201* on http://bits.netbeans.org/dev/nightly/
Changeset: http://hg.netbeans.org/main/rev/574db98aeaff
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #143080: Use remove for cleanup not get
Comment 18 Petr Pisl 2008-08-11 09:49:11 UTC
I can confirm that in todays builds, the problem is not reproducible anymore.
Comment 19 Tomas Mysik 2008-08-11 10:05:14 UTC
Verified.