Bug 62435

Summary: OutOffMemoryError from WritableXMLFileSystem
Product: apisupport Reporter: Martin Krauskopf <mkrauskopf>
Component: ProjectAssignee: Jesse Glick <jglick>
Status: CLOSED FIXED QA Contact: issues <issues.netbeans.org>
Priority: P1 CC: issues, mkleint, rmatous
Version: 5.xKeywords: PERFORMANCE
Target Milestone: 5.x   
Hardware: PC   
OS: Linux   
Whiteboard:
Issue Type: DEFECT Exception Report:
Attachments: helper? patch

Description Martin Krauskopf 2005-08-17 00:12:22 UTC
Just try to run DataModelTest(real case), which I'm goint to commit just now, or
apply attached patch of CreatedModifiedFilesTest (less real case).

The patch also contained adjusted CreatedModifiedFilesFactory where you can put
a breakpoint which will be reached before OutOffMemoryError happened (but
probably another counter on your machine - for me it is 44). The when you run
DataModelTest it should reach the breakpoint. I reached in this manner
WritableXMLFileSystem:1058. But probably not important and is different in
particular cases on different configurations. And the best is probably to play
with WritableXMLFileSystemTest - here I gave up :)

Maybe problem with FileSystems, don't know....
Comment 1 Martin Krauskopf 2005-08-17 00:15:13 UTC
Created attachment 23966 [details]
helper? patch
Comment 2 Jesse Glick 2005-08-17 11:10:04 UTC
I was running DataModelTest from the new action template wizard before with no
problems - is this the same unit test or have you made significant changes?
Comment 3 Martin Krauskopf 2005-08-17 11:30:39 UTC
No significat changes. The test in the CVS doesn't work for me - OOME. To get to
to the same problem you may try to increase loop counter in the patch to e.g. 300.
Radku or Milosi could you try to run the DataModelTest (from CVS) and tell us
the result - or better 'ant clean test' in apisupport/project - you _should_ get
OOME. Thanks.
Comment 4 Jesse Glick 2005-08-17 11:39:27 UTC
I haven't tried it today, but at least when I checked in the big CMF patch, I
had run all the apisupport tests several times over, using both JDK 1.4 and 1.6,
and never saw any OOME.

That's not to say that the infrastructure is memory-efficient - I haven't
checked yet. AbstractFileSystem should not have a lot of overhead, since Radek
has worked on this in the past, but I don't know about TAX. What I know of the
TAX code would indicate that it's not especially efficient for really large
documents - probably a bit worse than non-lazy-node-hack Xerces DOM - but we're
not talking about really large documents here. If you're getting an OOME on
anything under many thousands of files in size, something must be seriously
amiss. I can try to take a look in INSANE or a profiler.
Comment 5 Martin Krauskopf 2005-08-17 11:45:53 UTC
When you checked in a big CMF patch everything works for me as well - because
there weren't second half of the patch for DataModel. You have to check out new
DataModelTest and run it. It contains two methods. When I comment out one of
them (no matter which one) it runs ok. But if I leave the test as it is (with
both method "enabled") it result in OOME.
It is less than one hundred files I think.
Comment 6 Martin Krauskopf 2005-08-17 11:47:15 UTC
> of the patch for DataModel

sorry, should be "of the test for DataModel"
Comment 7 Jesse Glick 2005-08-17 16:55:15 UTC
Initial analysis: 1000 files added creates:

  17878 javax.swing.text.GapContent$MarkData
  17873 javax.swing.text.GapContent$StickyPosition
  17427 javax.swing.text.AbstractDocument$LeafElement
  16071 javax.swing.text.GapContent$UndoPosRef
   6890 java.lang.String
   6566 [C
   2293 [Ljava.lang.Object;
   2052 [Ljavax.swing.text.Element;
   1966 java.util.Hashtable$Entry
   1663 java.util.Vector
   1471 java.util.HashMap$Entry
   1030 javax.swing.text.AbstractDocument$ElementEdit
   1030 javax.swing.text.AbstractDocument$DefaultDocumentEvent
    515 javax.swing.text.GapContent$RemoveUndo
    515 javax.swing.text.GapContent$InsertUndo
...

Clearly the Document is not too happy with all the changes. The question is why
it has so many positions in it...
Comment 8 Jesse Glick 2005-08-17 17:50:15 UTC
Seems that every tree change modifies the document - adding to the undo stack.
TAX has a deferred fire mode, but this just seems to fire all the same events
later, which is pretty useless. Will look for options, but may need to do
something drastic...
Comment 9 Jesse Glick 2005-08-17 19:37:48 UTC
Doing a lot better by just throwing out all of the xml/core infrastructure
(XMLDataObject and its apparatus) and managing load/save/modify/reload cycle
manually, with no javax.swing.text.Document.

Not as nice if the user wishes to edit the layer as text - with the basic impl,
he will need to save changes to disk to see them structurally, and structural
changes will modify the file on disk so it better not be edited in memory - but
probably OK for now. A more sophisticated impl could check to see if the file
was already modified (by the user), and if so, read from/write to the Document
rather than disk.
Comment 10 Jesse Glick 2005-08-17 22:29:53 UTC
Should be far better now. Memory consumption for simple files looks to be around
a few Kb per file; add more for attributes, long names, etc.

committed   * Up-To-Date  1.16       
apisupport/project/nbproject/project.properties
committed   * Up-To-Date  1.30        apisupport/project/nbproject/project.xml
committed   * Up-To-Date  1.5        
apisupport/project/src/org/netbeans/modules/apisupport/project/layers/LayerNode.java
committed   * Up-To-Date  1.4        
apisupport/project/src/org/netbeans/modules/apisupport/project/layers/LayerUtils.java
committed   * Up-To-Date  1.5        
apisupport/project/src/org/netbeans/modules/apisupport/project/layers/WritableXMLFileSystem.java
committed   * Up-To-Date  1.2        
apisupport/project/test/unit/src/org/netbeans/modules/apisupport/project/layers/LayerTestBase.java
committed   * Up-To-Date  1.5        
apisupport/project/test/unit/src/org/netbeans/modules/apisupport/project/layers/WritableXMLFileSystemTest.java
committed   * Up-To-Date  1.151       ide/golden/deps.txt
Comment 11 Martin Krauskopf 2005-08-18 09:19:36 UTC
Thanks for the fast fix. Verified
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo