Bug 123872 - Reduce memory consumption for huge projects
Reduce memory consumption for huge projects
Status: RESOLVED FIXED
Product: cnd
Classification: Unclassified
Component: Code Model
6.x
All All
: P2 (vote)
: 7.3
Assigned To: Vladimir Voskresensky
issues@cnd
: PERFORMANCE
: 120036 139352 141303 186765 (view as bug list)
Depends on: 89648 120042 120168 122313 135585 197297 215226
Blocks: 108501
  Show dependency treegraph
 
Reported: 2007-12-12 11:34 UTC by Alexander Simon
Modified: 2013-01-15 11:26 UTC (History)
2 users (show)

See Also:
Issue Type: TASK
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Simon 2007-12-12 11:34:21 UTC
Target is opening OpenSolaris priject in 1 Gb Xmx memory
Comment 1 Alexander Simon 2007-12-12 14:18:01 UTC
fixing:
-use one instance of equal strings at project reading. It save 1% of memory.

CVS log:
Checking in ConfigurationXMLCodec.java;
/shared/data/ccvs/repository/cnd/makeproject/src/org/netbeans/modules/cnd/makeproject/configurations/ConfigurationXMLCodec.java,v
 <--  ConfigurationXMLCodec.java
new revision: 1.14; previous revision: 1.13
done
Comment 2 Alexander Simon 2007-12-12 15:38:07 UTC
fixing:
- make lazy cookies (CppEditorSupport, BinaryExecSupport) in CndDataObject. It save 3,6% of memory.

CVS log:

Checking in highlight/src/org/netbeans/modules/cnd/highlight/error/HighlightProvider.java;
/shared/data/ccvs/repository/cnd/highlight/src/org/netbeans/modules/cnd/highlight/error/HighlightProvider.java,v  <-- 
HighlightProvider.java
new revision: 1.10; previous revision: 1.9
done
Checking in highlight/nbproject/project.xml;
/shared/data/ccvs/repository/cnd/highlight/nbproject/project.xml,v  <--  project.xml
new revision: 1.19; previous revision: 1.18
done
Checking in core/src/org/netbeans/modules/cnd/loaders/HDataObject.java;
/shared/data/ccvs/repository/cnd/core/src/org/netbeans/modules/cnd/loaders/HDataObject.java,v  <--  HDataObject.java
new revision: 1.6; previous revision: 1.5
done
Checking in core/src/org/netbeans/modules/cnd/loaders/CndDataObject.java;
/shared/data/ccvs/repository/cnd/core/src/org/netbeans/modules/cnd/loaders/CndDataObject.java,v  <--  CndDataObject.java
new revision: 1.8; previous revision: 1.7
done
Comment 3 Alexander Simon 2007-12-13 07:52:34 UTC
fixing:
- make lazy ItemComfiguration initialization. It saves 3,4% of memory.

CVS log:
Checking in ItemConfiguration.java;
/shared/data/ccvs/repository/cnd/makeproject/src/org/netbeans/modules/cnd/makeproject/api/configurations/ItemConfiguration.java,v
 <--  ItemConfiguration.java
new revision: 1.12; previous revision: 1.11
done
Comment 4 Alexander Simon 2007-12-14 11:44:24 UTC
fixing:
-own unique name key, cache compound names in renderer, cache canonical names. It save 8% of memory.

CVS log:
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/AstUtil.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/AstUtil.java,v  <--  AstUtil.java
new revision: 1.12; previous revision: 1.11
done
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/DeclarationContainer.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/DeclarationContainer.java,v 
<--  DeclarationContainer.java
new revision: 1.10; previous revision: 1.9
done
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/FileContainer.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/FileContainer.java,v  <-- 
FileContainer.java
new revision: 1.18; previous revision: 1.17
done
RCS file: /shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/FixedString.java,v
done
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/FixedString.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/FixedString.java,v  <-- 
FixedString.java
initial revision: 1.1
done
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/AstRenderer.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/csm/core/AstRenderer.java,v  <-- 
AstRenderer.java
new revision: 1.27; previous revision: 1.26
done
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/repository/OffsetableDeclarationKey.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/repository/OffsetableDeclarationKey.java,v
 <--  OffsetableDeclarationKey.java
new revision: 1.5; previous revision: 1.4
done
Checking in repository/src/org/netbeans/modules/cnd/repository/sfs/SimpleFileIndex.java;
/shared/data/ccvs/repository/cnd/repository/src/org/netbeans/modules/cnd/repository/sfs/SimpleFileIndex.java,v  <-- 
SimpleFileIndex.java
new revision: 1.7; previous revision: 1.6
done
Checking in modelimpl/src/org/netbeans/modules/cnd/modelimpl/uid/UIDObjectFactory.java;
/shared/data/ccvs/repository/cnd/modelimpl/src/org/netbeans/modules/cnd/modelimpl/uid/UIDObjectFactory.java,v  <-- 
UIDObjectFactory.java
new revision: 1.10; previous revision: 1.9
done
Comment 5 Alexander Simon 2008-03-13 18:23:33 UTC
In NB 6.1 memory was reduced on 16%.
Next memory reducing can be achieved by:
- use char sequence in ANTLR & APT
- keep part of repository keys in memory
- use number for repository keys
Comment 6 Alexander Simon 2008-06-27 10:47:44 UTC
Save 2 bytes on each object:
http://hg.netbeans.org/main/rev/9c6d965606b7
Comment 7 Vladimir Kvashin 2008-07-23 11:15:45 UTC
*** Issue 141303 has been marked as a duplicate of this issue. ***
Comment 8 Vladimir Kvashin 2008-07-23 11:16:16 UTC
*** Issue 120036 has been marked as a duplicate of this issue. ***
Comment 9 Vladimir Kvashin 2008-07-23 11:34:38 UTC
*** Issue 139352 has been marked as a duplicate of this issue. ***
Comment 10 Vladimir Kvashin 2008-07-23 11:34:42 UTC
Comments from another issue by philbyjohn@netbeans.org
that is made a dup pf this one

OS: Fedora8 with Kernel 2.6.25.6-27.fc8
System: AMD Turion on Toshiba Satellite A215
RAM: 1 GB

Problem Also noted on 
OS: Windows 2000 Professional
System: 2.8GHz IBM Think Centre 8434
RAM: 777,200KB

I tried to setup a project giving it a Makefile from the latest Linux kernel I had downloaded. But NetBeans6.1 just
hangs. NetBeans takes up huge memory and it parses the project every time you open it. NetBeans hangs while showing
"Parsing LinuxKernel ..." at 11% on status bar and the Memory shows 335.2/337.4MB.


Indeed I was able to resolve the hanging issue on both Linux Fedora8 and Windows2000 with the -J-Xmx700m and
-J-XX:PermSize=96m set in 
etc/netbeans.conf. But the IDE is non responsive to mouse clicks and is very sluggish.

Compared to an agile editor such as Source Insight, NetBeans is way off when it comes to speed. It took exactly 47
minutes for the parsing to 
complete on Linux and 32 minutes on a Windows system. Whereas Source Insight does the job in 8 minutes flat on Linux
(using wine) and 5 
minutes on Windows.


Outline of the steps that I had followed.
1)Go to http://www.kernel.org/
2)Download the latest Linux Kernel (as I type 2.6.26) by clicking on the "F" link which stands for "full source".
3)Extract and create a new project by specifying the "Makefile" in the directory "linux-2.6.26"

Also, I am Using NetBeans IDE 6.1, JDK Update 5 all contained in the free installation DVD available on request.


VK: I'd like to note that the project is large: it has 6,608,795 lines in 19,800 files (both header and source)
VK: With -Xmx1563m I got it parsed within 7 minutes (on multy-CPU machine, but with one parsing thread). NB memory
indicator shows 870/1200 Mb after parsing is finished.

Comment 11 Vladimir Voskresensky 2012-07-31 12:23:35 UTC
Goal:
Nb 7.2 needs 1.7Gb of memory to parse Mozilla sources. 
7.3 should use less than 1Gb
Comment 12 Vladimir Voskresensky 2012-07-31 12:33:28 UTC
*** Bug 186765 has been marked as a duplicate of this bug. ***
Comment 13 Vladimir Voskresensky 2012-09-09 18:29:55 UTC
parse Mozilla sources - 700mb
parse Linux sources - 1470mb


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo