This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 157533 - OOME when need to index larger amount of sources
Summary: OOME when need to index larger amount of sources
Status: RESOLVED FIXED
Alias: None
Product: ruby
Classification: Unclassified
Component: Editing (show other bugs)
Version: 6.x
Hardware: All All
: P1 blocker (vote)
Assignee: Erno Mononen
URL:
Keywords:
: 154870 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-01-28 09:59 UTC by Martin Krauskopf
Modified: 2009-03-17 08:33 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Krauskopf 2009-01-28 09:59:48 UTC 
To reproduce, it's enough to create a new Ruby project with default platform (just click through the New Project
wizard). Then watch the memory consumption during indexing. It stills grows, eats about few hundreds of MBs. After
indexing is done those hundreds of MBs are released. Problem is when user has a lot of core libraries (often the case in
Ruby) the OOME is thrown.
Comment 1 Martin Krauskopf 2009-01-28 10:01:44 UTC 
PS: seems it started to happen few days (5+/-) ago.
Comment 2 Vitezslav Stejskal 2009-02-04 09:56:37 UTC 
*** Issue 154870 has been marked as a duplicate of this issue. ***
Comment 3 Vitezslav Stejskal 2009-02-25 13:18:52 UTC 
The problem was in RubyIndexer and related code. It was opening editor documents when indexing files, the reason for
that was that it needed access to the lexer tokens for the indexed file. This can be now achieved by lexing snapshots.
Opening editor documents is very expensive, because they are designed to support full-featured editing and so will
initialize tons of stuff that is just not needed for indexing (eg. coloring, hints, etc)

With the following fix it's possible to index all ruby platform/gems/etc bundled stuff within 128m memory and in
reasonable time (~ 10sec). I would recommend all the other CSL based languages to review what their indexers do and use
Snapshot.getTokenHierarchy() whenever they need lexer tokens and don't have a document. The current approach was to
forcibly open the document, get its token hierarchy and then carry on. But this approach is flawed and leads to
performance problems.

http://hg.netbeans.org/jet-parsing-api/rev/02b15aca7832
Comment 4 Vitezslav Stejskal 2009-02-25 13:19:35 UTC 
The fix is in jet-parsing-api.
Comment 5 Quality Engineering 2009-03-17 08:33:16 UTC 
Integrated into 'main-golden', will be available in build *200903170201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/02b15aca7832
User: Vita Stejskal <vstejskal@netbeans.org>
Log: #157533: preferring Snapshot over Document when indexing (eg. it significantly reduces the amount of memory needed for indexing and increases speed)