Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 222158 - OutOfMemoryError: Java heap space
OutOfMemoryError: Java heap space
Status: RESOLVED WONTFIX
Product: platform
Classification: Unclassified
Component: JDK Problems
7.3
All All
: P3 (vote)
: 7.3
Assigned To: Stanislav Aubrecht
issues@platform
http://bugs.sun.com/bugdatabase/view_...
EXCEPTIONS_REPORT, jdk_bug_8003887
:
: 222087 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-11-14 23:38 UTC by Petah
Modified: 2013-01-10 09:50 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
stacktrace (1.07 KB, text/plain)
2012-11-14 23:38 UTC, Petah
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Petah 2012-11-14 23:38:06 UTC
This bug was originally marked as duplicate of bug 185063, that is already resolved. This bug is still valid, so this seems to be another bug, but it might be related.

Build: NetBeans IDE Dev (Build nbms-and-javadoc-9600-on-20121114)
VM: Java HotSpot(TM) Client VM, 20.0-b11, Java(TM) SE Runtime Environment, 1.6.0_25-b06
OS: Windows 7

Stacktrace: 
java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2786)
   at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
   at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
   at java.io.PrintStream.write(PrintStream.java:430)
   at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:161)
   at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)
Comment 1 Petah 2012-11-14 23:38:08 UTC
Created attachment 127819 [details]
stacktrace
Comment 2 Petah 2012-11-15 09:37:48 UTC
Exactly the same symptoms as occurred for me in 

http://netbeans.org/bugzilla/show_bug.cgi?id=221772

NB scans, chews up all the memory with no scanning progress, then starts to GC thrash, then hits an OutOfMemoryError.
Comment 3 Martin Entlicher 2012-11-18 11:23:05 UTC
Please provide a heap dump.
I do not think, that this is caused by some memory problem in uihandler, output to a GZIPOutputStream should not require an excessive amount of memory. I think that something else consumes memory and GZIPOutputStream simply did not have enough space.

Please provide a heap dump http://wiki.netbeans.org/FaqMemoryDump
Comment 4 Martin Entlicher 2012-11-18 11:52:04 UTC
I'm sorry, the heap dump is there http://statistics.netbeans.org/exceptions/exception.do?id=629229 I didn't notice it.
It's truncated, but it looks like the problem is in PHP, several large char arrays are held by PHP.
Comment 6 Tomas Mysik 2012-11-19 06:06:34 UTC
Ondro, I guess that these arrays are in the editor - can you evaluate please? If not, please reassign back to me.

Thanks a lot.
Comment 7 Ondrej Brejla 2012-11-19 09:45:47 UTC
Sorry, but I'm not able to ungzip any of heapdumps :( I can download them, but it's somehow corrupted. Nevertheless, if there is some issue with big char arrays in PHP parser, I made some improvements, but it _always_ will be erroneous to parse a really big files and I can't do anything with that :/
Comment 8 Tomas Hurka 2012-11-19 09:51:28 UTC
It looks like there is a loop in  F:\xmod . See messages.log attached to exception report.
Comment 9 Ondrej Brejla 2012-11-19 10:04:24 UTC
As Tomas wrote, it seems that there is a loop in mounted F:\xmod. It's really not a good idea to do that. Maybe tmysik will add something ;)
Comment 10 Petah 2012-11-19 19:21:53 UTC
Do you mean a symlink loop?

In that case yes there probably is.

f:\xmod\* is a bunch of composer modules. Some modules depend on other modules.

So for example when developing f:\xmod\xmod-cms the vendor directory would have a link f:\xmod\xmod-cms\vendor\xmod\common -> f:\xmod\xmod-common

Should NB just resolve each path to its absolute canonical path before scanning? Not only to avoid symlink loops, but also as it should be faster.

Also on a side note, does NB hash a file before it scans it? No point scanning an identical file twice.
Comment 11 Petah 2012-11-19 19:24:11 UTC
Also "but it _always_ will be erroneous to parse a really big files"

That fair enough if NB can't parse big files, but could you make NB tell the users  if a file is too big? Or make a setting with like "max file size to scan"?
Comment 12 Tomas Mysik 2012-11-20 09:39:51 UTC
The cause here seems to be a loop on your FS (see comment #8) - I doubt that we can do anything with it. In any case, reassigning to parsing - could you please answer questions from comments #10 and #11?

Thanks.
Comment 13 Petah 2012-11-20 09:57:26 UTC
You want me to reply to my own questions? Or are you talking to some one else?

And like I said in comment 10, NB should avoid symlink loops by looking up the absolute canonical path (i.e. in PHP you would `realpath($file)` the file first)
Comment 14 Tomas Mysik 2012-11-20 10:30:13 UTC
(In reply to comment #13)
> You want me to reply to my own questions?

No :)

> Or are you talking to some one else?

Yes, to the owner of the parsing component (I reassigned this issue there).
Comment 15 Tomas Zezula 2012-11-20 13:41:13 UTC
In fact the parsing.api resolves the path to the canonical ones to detect cycles and prevents them.
It seems that java.io.File.getCanonicalFile() does not work on Windows. The F: is a network drive exported from UNIX or NT junctions are used to simulate symlinks? 
Cycle in FS on Windows is not a good idea.
Comment 16 Petah 2012-11-20 19:50:23 UTC
F:\ is just a standard NTFS partition on a normal hard drive.

The symlinks are made using `mklink /D target source`

I have never had any other problem with other software and these symlinks. 

I ran a quick test in Java

F:\> mkdir link-target
F:\> mklink /D link f:\link-target
symbolic link created for link <<===>> f:\link-target 

public static void main(String[] args) throws IOException {
    File file = new File("f:\\link");
    System.out.println(file);
    System.out.println(file.getCanonicalFile());
    System.out.println(file.getCanonicalFile().getCanonicalFile());
}

f:\link
F:\link
F:\link

So it does seem like it might be a Java problem.
Comment 17 Petah 2012-11-20 19:57:44 UTC
Looking it up I don't see an easy way around it with out using java.nio in Java 1.7

http://docs.oracle.com/javase/tutorial/essential/io/links.html
http://stackoverflow.com/questions/813710/java-1-6-determine-symbolic-links
Comment 18 Tomas Zezula 2012-11-21 10:00:02 UTC
>So it does seem like it might be a Java problem.
Yes, this is what I've written in comment #15 and why I've assigned it to JDK Problems category.
I have an idea (rather workaround) but can be tried.
As you correctly pointed out we cannot link to JDK 7 as NB has to run at least on 2 newest JDKs,
however we we can use reflection to invoke it when IDE is running on JDK 7.

As I don't have Windows I need you to test if Files.isSymbolicLink() works fine for mklink and also measure at least a bit a performance of the IO call. Done by simple test below.

Here is the code:


import java.io.File;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;

class Test {
    public static void main(String[] args) {
        final File f = new File ("f:\\link");
        long st = System.nanoTime();
        boolean res = false;
        for (int i=0; i<100_000; i++){
            Path p = f.toPath();
            res = Files.isSymbolicLink(p);
        }
        long et = System.nanoTime();
        System.out.println(String.format(
            "file: %s, is symlink: %b, time of 100,000 invocations: %d ms",
            f,
            res,
            (et-st)/1000000));
    }
}
Comment 19 Stanislav Aubrecht 2012-11-22 10:18:21 UTC
(In reply to comment #18)

> As I don't have Windows I need you to test if Files.isSymbolicLink() works fine
> for mklink and also measure at least a bit a performance of the IO call. Done
> by simple test below.

I ran that test using:

java version "1.7.0_10-ea"
Java(TM) SE Runtime Environment (build 1.7.0_10-ea-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)

The result is:

file: d:\link, is symlink: true, time of 100,000 invocations: 6500 ms
Comment 20 Stanislav Aubrecht 2012-11-22 13:22:28 UTC
JDK team suggests this workaround: file.toPath().toRealPath()
Comment 21 Petah 2012-11-23 09:42:18 UTC
Why is this set to wont fix?
Comment 22 Stanislav Aubrecht 2012-11-23 09:51:43 UTC
(In reply to comment #21)
> Why is this set to wont fix?

Because it is a bug in JDK implementation.
Comment 23 Petah 2012-11-23 10:00:57 UTC
So I am not allowed to use Netbeans?
Comment 24 Stanislav Aubrecht 2012-11-23 10:13:26 UTC
(In reply to comment #23)
> So I am not allowed to use Netbeans?

Either wait till the bug is fixed in new JDK release or don't use symbolic links.
You can also open a new issue against Parsing and Indexing module to try some workaround.
Comment 25 Tomas Zezula 2013-01-10 09:50:45 UTC
*** Bug 222087 has been marked as a duplicate of this bug. ***


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo