This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 197988 - OOME when opening Maven project
Summary: OOME when opening Maven project
Status: RESOLVED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Maven (show other bugs)
Version: 7.0
Hardware: PC Linux
: P3 normal (vote)
Assignee: Jesse Glick
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-22 08:01 UTC by _ rkubacki
Modified: 2011-05-04 15:25 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
IDE log (329.36 KB, text/x-log)
2011-04-29 08:56 UTC, _ rkubacki
Details

Note You need to log in before you can comment on or make changes to this bug.
Description _ rkubacki 2011-04-22 08:01:56 UTC
NB7.0

Full track record is in http://statistics.netbeans.org/analytics/exception.do?id=505252. See 'Seq: 1912' there and you will see this entry:

<record>
  <date>2011-04-22T09:13:15</date>
  <millis>1303456395063</millis>
  <sequence>1912</sequence>
  <level>SEVERE</level>
  <thread>24</thread>
  <exception>
    <message>java.lang.OutOfMemoryError: Java heap space</message>
    <frame>
      <class>org.apache.maven.index.updater.IndexDataReader</class>
      <method>readUTF</method>
      <line>154</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.IndexDataReader</class>
      <method>readField</method>
      <line>144</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.IndexDataReader</class>
      <method>readDocument</method>
      <line>119</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.IndexDataReader</class>
      <method>readIndex</method>
      <line>86</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.DefaultIndexUpdater</class>
      <method>unpackIndexData</method>
      <line>508</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.DefaultIndexUpdater</class>
      <method>loadIndexDirectory</method>
      <line>196</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.DefaultIndexUpdater</class>
      <method>access$300</method>
      <line>75</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.DefaultIndexUpdater$LuceneIndexAdaptor</class>
      <method>setIndexFile</method>
      <line>641</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.DefaultIndexUpdater</class>
      <method>fetchAndUpdateIndex</method>
      <line>860</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.apache.maven.index.updater.DefaultIndexUpdater</class>
      <method>fetchAndUpdateIndex</method>
      <line>156</line>
      <file>${netBeansDir}modules/ext/maven/indexer-core-4.0.0.MINDEXER-12+MINDEXER-13.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl</class>
      <method>indexLoadedRepo</method>
      <line>526</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven-indexer.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl</class>
      <method>access$700</method>
      <line>165</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven-indexer.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl$2</class>
      <method>run</method>
      <line>614</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven-indexer.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl$2</class>
      <method>run</method>
      <line>605</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven-indexer.jar</file>
    </frame>
    <frame>
      <class>org.openide.util.Mutex</class>
      <method>writeAccess</method>
      <line>436</line>
      <file>${netbeans.home}lib/org-openide-util.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl</class>
      <method>indexRepo</method>
      <line>605</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven-indexer.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.indexer.api.RepositoryIndexer</class>
      <method>indexRepo</method>
      <line>62</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven-indexer.jar</file>
    </frame>
    <frame>
      <class>org.netbeans.modules.maven.ProjectOpenedHookImpl$1</class>
      <method>run</method>
      <line>185</line>
      <file>${netBeansDir}modules/org-netbeans-modules-maven.jar</file>
    </frame>
    <frame>
      <class>org.openide.util.RequestProcessor$Task</class>
      <method>run</method>
      <line>1424</line>
      <file>${netbeans.home}lib/org-openide-util.jar</file>
    </frame>
    <frame>
      <class>org.openide.util.RequestProcessor$Processor</class>
      <method>run</method>
      <line>1968</line>
      <file>${netbeans.home}lib/org-openide-util.jar</file>
    </frame>
  </exception>
</record>

Overall it is weird because log shows -Xmx1024m and the heap that was taken has ~200M only and it is Java heap space OOME (someone's allocating too big array?).
Comment 1 Jesse Glick 2011-04-22 15:29:04 UTC
IndexDataReader.readUTF (from the Maven Indexer Core library) does indeed allocate a couple of arrays and a String, though these would typically be tiny (identifiers and the like). It is possible that when fed malformed input it could be tricked into allocating a huge array, since it first reads a 32-bit length field (unlike DataInputStream.readUTF which uses a 16-bit length). It is also conceivable that a valid index contained an 800 Mb indexable field value, though I think you would have noticed a >800 Mb index being downloaded! Either way, the code could probably be made more robust by catching OOME during array allocation and rethrowing as an IOE with diagnostics.

The line number is a bit unclear to me because inferring the location from the SVN tag plus a patch I applied gives "c = bytearr[count] & 0xff;" which cannot throw OOME, whereas javap seems to say "int utflen = in.readInt();" which also should not - however the line right after that is "byte[] bytearr = new byte[utflen];" which would be the plausible failure point (maybe the JVM misreports OOME locations by fencepost error).

Do you have any idea how to reproduce this? In particular, which repository index was being read?
Comment 2 _ rkubacki 2011-04-25 16:16:07 UTC
I've added  -J-Dorg.netbeans.modules.maven.level=FINE now but manual re-indexing of Maven index passed this time. I have no idea what exactly was the problem. If you have some tips let me know and I can try them.
Comment 3 _ rkubacki 2011-04-29 08:54:48 UTC
It happened again today.

FINE [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: No mirror for http://bits.netbeans.org/maven2/.index/
FINE [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Loading Context :netbeans
FINE [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: No mirror for http://www.jasperforge.org/maven2/.index/
FINE [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: Loading Context :jaspersoft
SEVERE [org.openide.util.RequestProcessor]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.maven.index.updater.IndexDataReader.readUTF(IndexDataReader.java:154)
Comment 4 _ rkubacki 2011-04-29 08:56:36 UTC
Created attachment 108014 [details]
IDE log

complete IDE log file from last incident
Comment 5 Jesse Glick 2011-05-04 15:19:33 UTC
Reproducible on <http://www.jasperforge.org/maven2/>. Using a dev build I get just a warning for that URL:

INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl]: could not index jaspersoft
java.io.IOException: Provided input contains unexpected data (0x01 expected as 1st byte)!
	at org.apache.maven.index.updater.IndexDataReader.readHeader(IndexDataReader.java:113)
	at org.apache.maven.index.updater.IndexDataReader.readIndex(IndexDataReader.java:75)
	at org.apache.maven.index.updater.DefaultIndexUpdater.unpackIndexData(DefaultIndexUpdater.java:508)
	at org.apache.maven.index.updater.DefaultIndexUpdater.loadIndexDirectory(DefaultIndexUpdater.java:196)
	at org.apache.maven.index.updater.DefaultIndexUpdater.access$300(DefaultIndexUpdater.java:75)
	at org.apache.maven.index.updater.DefaultIndexUpdater$LuceneIndexAdaptor.setIndexFile(DefaultIndexUpdater.java:641)
	at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:860)
	at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:156)
[catch] at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.indexLoadedRepo(NexusRepositoryIndexerImpl.java:519)

That is thanks to <http://jira.codehaus.org/browse/MINDEXER-20> which makes the indexer fail immediately when given some kinds of garbage. In this case it seems that jasperforge.org is serving a 404-type HTML page for the Nexus index URLs... but with a 200 status code! Definitely a bug to be filed for that site.

Since the repo apparently cannot be indexed, you should not add it to Maven Repository Browser.
Comment 6 Jesse Glick 2011-05-04 15:25:43 UTC
(In reply to comment #1)
> the code could probably be made more robust

I filed http://jira.codehaus.org/browse/MINDEXER-28 for tracking. Probably will not do anything more about it, since there is a workaround (do not add this repo), and MINDEXER-20 is already in for 7.0.1.