Bug 177889 - OutOfMemoryError: Java heap space
OutOfMemoryError: Java heap space
Status: RESOLVED WORKSFORME
Product: versioncontrol
Classification: Unclassified
Component: Subversion
6.x
All All
: P3 (vote)
: 6.x
Assigned To: Ondrej Vrabec
issues@versioncontrol
EXCEPTIONS_REPORT
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-11-30 22:52 UTC by nehalem
Modified: 2010-05-03 09:52 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
stacktrace (3.24 KB, text/plain)
2009-11-30 22:52 UTC, nehalem
Details

Note You need to log in before you can comment on or make changes to this bug.
Description nehalem 2009-11-30 22:52:09 UTC
Build: NetBeans IDE 6.8 RC1 (Build 200911252200)
VM: Java HotSpot(TM) 64-Bit Server VM, 14.3-b01-100, Java(TM) SE Runtime Environment, 1.6.0_17-b04-237-10M3013
OS: Mac OS X, 10.6.2, x86_64

User Comments:
GUEST: Project was scanned

GUEST: Starting up Netbeans

GUEST: Still geting this error with default install of latest Daily build. Was not doing anything but waiting for the "projects scan" to complete. I notice that this error possibly relates to the failure to copy a complete set of a php project's app and web files to the local web Document folder as specified in the php properties dialog box.

nehalem: I have tried to open and have the new netbeans scan my work project many times.  Each time it runs out of memory.

GUEST: Just sitting there, with 'Scanning Projects' running.

GUEST: this error occoured while "scanning projects" just after starting netbeans.

The project is a php project.



Stacktrace: 
java.lang.OutOfMemoryError: Java heap space
        at org.netbeans.lib.lexer.LexerInputOperation.createDefaultTokenInstance(LexerInputOperation.java:303)
        at org.netbeans.lib.lexer.LexerInputOperation.createToken(LexerInputOperation.java:281)
        at org.netbeans.spi.lexer.TokenFactory.createToken(TokenFactory.java:102)
        at org.netbeans.spi.lexer.TokenFactory.createToken(TokenFactory.java:86)
        at org.netbeans.lib.html.lexer.HTMLLexer.token(HTMLLexer.java:1096)
        at org.netbeans.lib.html.lexer.HTMLLexer.nextToken(HTMLLexer.java:443)
Comment 1 nehalem 2009-11-30 22:52:12 UTC
Created attachment 91891 [details]
stacktrace
Comment 2 Vitezslav Stejskal 2009-12-01 10:46:57 UTC
nahlem, could you please send us the heapdump generated by the IDE? Either attach it here or send it to vstejskal@netbeans.org. Thanks
Comment 3 nehalem 2009-12-02 10:11:58 UTC
(In reply to comment #2)
> nahlem, could you please send us the heapdump generated by the IDE? Either
> attach it here or send it to vstejskal@netbeans.org. Thanks

How do I do this?  I should mention this is a PHP project as well.  Seems the PHP scanner probably has some issues.
Comment 4 Vitezslav Stejskal 2009-12-03 06:30:35 UTC
Go to http://netbeans.org/projects/editor/downloads and upload the file there and tell us what file it is. You will have to be logged in. If the file contains data that you don't want to be publicaly visible, I'll move the file to an internal storage here at Sun. Thanks
Comment 5 nehalem 2009-12-04 14:22:41 UTC
Actually, what I don't understand is where do I get that heapdump that it generates?

Also, I can confirm the same behaivor on RC2.  In addition all of us who have tried it at my workplace are having this issue.
Comment 6 nehalem 2009-12-04 14:23:48 UTC
OH, wait, on RC2 I didn't have the problem, but a co-worker did.  Strange.  I'll keep playing with it.
Comment 7 Jiri Prox 2009-12-05 02:04:23 UTC
Here is guide how to generate heapdump:
http://wiki.netbeans.org/FaqNetBeansAndOOME
Comment 8 nehalem 2009-12-07 08:51:28 UTC
Ok, I've tried to set my netbeans to automatically create the heap dump when it happens.  With RC2 it's happening less, but in visual VM I just did, it came dangerously close.

I can upload that one if you would like (even though it doesn't create the exception).
Comment 9 Vitezslav Stejskal 2009-12-07 09:46:32 UTC
(In reply to comment #8)
> Ok, I've tried to set my netbeans to automatically create the heap dump when it
> happens.  With RC2 it's happening less, but in visual VM I just did, it came
> dangerously close.
> 
> I can upload that one if you would like (even though it doesn't create the
> exception).

Yes, please. Thanks. It's better than nothing. But if you manage to salvage a dump from the real OOME please upload it as well.
Comment 10 nehalem 2009-12-07 11:12:47 UTC
Okay, I'll upload.  Please remove quickly as it is sensitive.  It's also about four hundred mb.  

Finally.  I am having a hard time staying with RC2.  The IDE is always comsuming 30 to 100% of my cpu.  Maybe you could see if something points to that too.  This is after all scanning and no tasks are being performed in the bottom right status bar.
Comment 11 nehalem 2009-12-07 11:38:34 UTC
I don't see an upload option on that downloads page.
Comment 12 Vitezslav Stejskal 2009-12-08 01:29:14 UTC
(In reply to comment #11)
> I don't see an upload option on that downloads page.

I'm afraid that you will have to join the project (editor.netbeans.org) as an observer.
Comment 13 nehalem 2009-12-11 08:52:33 UTC
Well this is still happening on the release for us.  I will give me heap but can't figure out how to join that group.  Is there no easier way to send this stuff to you guys.
Comment 14 Vitezslav Stejskal 2009-12-13 00:15:00 UTC
I think in 6.9dev builds there is, see issue #173477. However, I'm not sure if the same can be done from 6.8. Jindra might know more and/or suggest a workaround for 6.8 users like yourself. Jindra, could you please help? Thanks

Re. joining the editor project - It's a bit unusual, but all you have to do is to click 'Bookmark This Project' from the project's page (eg. editor.netbeans.org). This will add the project to your 'My Page' and also grant you the observer role. Very intuitive, eh? :-)

http://netbeans.org/projects/help/pages/GettingInvolvedInProjects
Comment 15 Jindrich Sedek 2009-12-14 01:37:40 UTC
Issue #173477 is fixed only in trunk, I didn't do it for 6.8 because I'm not sure how long it might take to upload heapdump files and it needs to be well tested before releasing it.
Comment 16 nehalem 2010-02-16 00:27:55 UTC
Ok, finally added a heap dump.  It's called "Memory exception heap dump " on that page.

I hope it helps.  Netbeans 6.8 has really been problematic for us.  With the latest updates it's not only constantly giving me a heap overflow, but also constantly running the CPU.  Again, this seems pretty common with my team.  I've finally reverted to 6.7.1 since it's simply unusable.  I'll do anything I can to help, hope this bug isn't buried too low now.
Comment 18 Vitezslav Stejskal 2010-02-17 03:03:02 UTC
The biggest objects on the heap seem to be from subversion module:

org.netbeans.modules.subversion.DiskMapTurboProvider#1   ... 30,373,838
org.netbeans.modules.subversion.DiskMapTurboProvider$1#1 ... 30,371,376
java.util.concurrent.ConcurrentHashMap#23                ... 30,371,352
java.util.concurrent.ConcurrentHashMap$Segment[]#23      ... 30,371,256
Comment 19 Ondrej Vrabec 2010-02-17 06:53:20 UTC
1) It seems you have too much entries in subversion cache. Could you check if you have your home dir versioned? Is there by any chance .svn folder?
2) Please delete the whole ~/.netbeans/6.8/var/cache/svncache folder and run IDE with -J-Dorg.netbeans.modules.subversion.FileStatusCache.recursiveScan=false - does this help?
3) Start IDE with -J-Dorg.netbeans.modules.subversion.level=500 and attach the message log after few minutes of IDE's work (~/.netbeans/6.8/var/log/message.log)
Comment 20 nehalem 2010-02-20 20:33:58 UTC
Ok, follows your instructions:
1) Ok, yes there was a single .svn folder in my home directory.  NO IDEA WHY.  I deleted it (no subfolder had a .svn folder).  I did delete my cache.
2) I ran it with this both of the startup arguments you provided (I think, I added it to the netbeans.conf).  I started up netbeans and I got the same memory exception.
3) I attached my message.log in the upload directory.  Please take a look.
Comment 21 Ondrej Vrabec 2010-02-21 00:10:46 UTC
I looked at the message log and there seems to be no problem with subversion now, no excessive scanning, running svn status commands only inside '/Applications/MAMP/htdocs' which is svn versioned (that's versioned intentionaly, i presume?).
Could you then do the same steps as the last time (delete the svn cache, setting the switches) and upload:
1) generated heap dump
2) created ~/.netbeans/6.8/var/cache/svncache
3) the message log again

There will probably be some new interesting information

BTW, you upload the message.log directly to 'http://netbeans.org/projects/editor/downloads'. Please upload these files to the folder representing the corresponding issue, so we have all the related files together in one folder.
Comment 22 nehalem 2010-04-06 15:40:55 UTC
Okay, I had to revert for a while to deal with a deadline.  I did download m1 of 6.9 and can confirm that this problem still happens with that.  I will get you what you requested very soon.
Comment 23 Ondrej Vrabec 2010-04-06 17:32:39 UTC
(In reply to comment #22)
> Okay, I had to revert for a while to deal with a deadline.  I did download m1
> of 6.9 and can confirm that this problem still happens with that.  I will get
> you what you requested very soon.

We've recently made some performance improvements in subversion support, could you please test and provide requested info with the latest dev build (http://bits.netbeans.org/download/trunk/nightly/latest/)?
Comment 24 nehalem 2010-04-06 19:59:02 UTC
(In reply to comment #23)
> We've recently made some performance improvements in subversion support, could
> you please test and provide requested info with the latest dev build
> (http://bits.netbeans.org/download/trunk/nightly/latest/)?


Still happened.  Want me to provide info from the dev build or 6.8?
Comment 25 Ondrej Vrabec 2010-04-06 20:37:30 UTC
> Still happened.  Want me to provide info from the dev build or 6.8?
dev build
Comment 26 nehalem 2010-04-23 21:54:49 UTC
Hi, I have uploaded the requested dumps (archived).  I put in a single file in a folder called "Bug 177889".  Please remove after getting in case there is something private contained within.  Please let me know if that provides the data needed.  

BTW, not sure if it makes a difference but I have a new machine.  I guess the out of memory issue (or heap amount available) is not tied to RAM since my new machine has a lot more (8gb).

Thanks!
Comment 27 Ondrej Vrabec 2010-05-03 09:52:29 UTC
Hi, thanks for the files, I looked at your heap dump and message log. 
The heap dump does not indicate any excessive memory consumption (used memory is about 70MB), subversion does not use as much memory as it was indicated by the first heap dump you provided (probably because you deleted the subversion cache on disk).

I would say the error you're getting is caused by a memory usage peek while your files are being scanned, because the message log contains:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
at java.util.ArrayList.add(ArrayList.java:351)
at org.netbeans.lib.lexer.BatchTokenList.tokenOrEmbeddingImpl(BatchTokenList.java:179)
at org.netbeans.lib.lexer.BatchTokenList.tokenOrEmbedding(BatchTokenList.java:170)
at org.netbeans.api.lexer.TokenSequence.moveNext(TokenSequence.java:454)
at org.netbeans.modules.javascript.editing.embedding.JsEmbeddingProvider.extractJavaScriptFromHtml(JsEmbeddingProvider.java:428)
at org.netbeans.modules.javascript.editing.embedding.JsEmbeddingProvider.access$700(JsEmbeddingProvider.java:68)
at org.netbeans.modules.javascript.editing.embedding.JsEmbeddingProvider$HtmlTranslator.translate(JsEmbeddingProvider.java:415)
at org.netbeans.modules.javascript.editing.embedding.JsEmbeddingProvider.getEmbeddings(JsEmbeddingProvider.java:76)

It seems that your problem is caused not by subversion any more but rather by javascript parser (the OOME in javascript is tracked in bug #184677).
Try to increase allowed memory size (run IDE with -J-Xms1024m -J-Xmx1024m switches, it could help). If you still get the error even with the switches, report it with the exception reporter in IDE.

I'm closing this one, it seems no longer valid. Please report the error again with the exception reporter, it will be assigned correctly to javascript parsing.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo