This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 22789 - XML Module cannot handle large XML files
Summary: XML Module cannot handle large XML files
Status: RESOLVED FIXED
Alias: None
Product: xml
Classification: Unclassified
Component: TAX/Lib (show other bugs)
Version: -S1S-
Hardware: PC Windows 3.1/NT
: P2 blocker (vote)
Assignee: issues@xml
URL:
Keywords: PERFORMANCE
: 22163 22206 (view as bug list)
Depends on: 21525
Blocks: 22152 22163 22206 27998
  Show dependency tree
 
Reported: 2002-04-25 13:47 UTC by tkellerer
Modified: 2008-02-17 14:56 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description tkellerer 2002-04-25 13:47:56 UTC
When trying to open a large XML file (in my case
12MB) the parser stops with an "Internal parser
error". I guess it's due to memory restrictions
(the VM for NetBeans is limited to 180MB).

It would nice if it was possible to open files
like that.
Comment 1 _ lkramolis 2002-04-25 14:23:09 UTC
Thanks for quick response. If it is possible could you attached zipped
the XML file? We could test it on real document -- our test documents
can differ by element deep level, number of attributes, etc. Thanks.
Comment 2 tkellerer 2002-04-25 14:52:24 UTC
I'm sorry but the file is catalog file with detailed information about
products and prices of one of our customers, and I'm not allowed to
pass that on to third parties...
Comment 3 _ lkramolis 2002-04-25 18:37:34 UTC
OK, I will try docbook files mentioned in issue #22152.
Comment 4 _ lkramolis 2002-05-07 10:13:30 UTC
It is general problem when you want to load/parse big document into
object structure and keep it in memmory.

It needs too big changes in tree structure what is long-term work. I
suggest to mark it is FFJ 4.0 waiver.
Comment 5 _ lkramolis 2002-05-09 07:52:16 UTC
FFJ 4.0 waiver approved.
Comment 6 _ pkuzel 2002-06-03 12:03:23 UTC
Thomas could you please attach the exception? Memory restrictions
should produce OutOfMemoryError instead of the "Internal parser
error".

Anyway large documents often produces OutOfMemoryError. It can be
solved by utilizing MDR with its disk swapping feature.
Comment 7 _ pkuzel 2002-06-03 14:58:26 UTC
*** Issue 22206 has been marked as a duplicate of this issue. ***
Comment 8 _ pkuzel 2002-06-03 15:26:05 UTC
*** Issue 22163 has been marked as a duplicate of this issue. ***
Comment 9 Marek Grummich 2002-07-19 16:07:17 UTC
Target milestone was changed from not determined to TBD
Comment 10 Jesse Glick 2002-07-25 17:36:35 UTC
If this is known to happen for some files, probably a candidate for
the release notes.
Comment 11 John Jullion-ceccarelli 2002-07-25 18:20:54 UTC
It's mentioned directly in the docs (Editing an XML 
Document). Should it go into the release notes as well?
Comment 12 Jaroslav Tulach 2002-07-26 07:58:30 UTC
This bug is reported in version <= 3.4dev and still not fixed. Due to that it
forbids the release candidate for 3.4 to be promoted. Are you aware of that and
are you intensively working on the fix? If not, you should consider some
corrective action.
Comment 13 _ lkramolis 2002-07-26 08:48:53 UTC
XML model (TAX) is not designed for large document -- then I could say
it is as designed behaviour, however I do not agree with it.

John mentioned that it is described in XML documentation and maybe it
should be mentioned in release notes too.

Other work on TAX was for 3.4 postponed, large files tree editing is
not our strategic feature. One important was fixed: referenced TAX
instances are correctly weak - not used references can be released by GC.
Comment 14 _ pkuzel 2002-07-26 10:36:57 UTC
We estimated that proper fix implementation requires
3 developer/weeks (rewrote parser and introduce positions).

Based on this estime the strategy commite marked XML performance
as non strategic issue for Sun's contribution to NetBeans.

From Sun's point of view it was classified as nice to have enhancement.
From user's point of view it is a bug. 
I do not know how to express this state using IZ.
Comment 15 _ lkramolis 2002-07-29 17:27:44 UTC
3.4_WAIVER: architecture changes are necessary (no simple fix is possible).
Comment 16 Patrick Keegan 2002-07-29 22:14:58 UTC
proposed release note (combined with note for 22152): 

Large and complex XML files open very slowly in the tree
editor and might cause OutOfMemory errors.
Workaround: Open such files with the Edit command to edit them in the 
text editor instead of the tree editor.
Comment 17 iformanek 2002-07-30 16:49:20 UTC
I agree with waiver for 3.4
Comment 18 Patrick Keegan 2002-07-30 17:37:47 UTC
I agree with 3.4 waiver; leaving my name on CC for relnote purposes
Comment 19 Jesse Glick 2002-07-31 05:49:10 UTC
Could this just be marked dupe of issue #22152, or vice-versa?
Comment 20 _ lkramolis 2002-07-31 09:20:46 UTC
I agree with it. If nobody objects I will mark this as duplicate of
issue 22152 (which already contains discussion about possible solution).
Comment 21 Martin Schovanek 2003-01-06 13:03:05 UTC
Version was changed on 'S1S 4.2'.
Comment 22 _ pkuzel 2003-02-14 11:31:27 UTC
S1S5_WAV request justification:

It is general problem when you want to load/parse big document into
object structure and keep it in memmory.

It needs too big changes in tree structure what is long-term work. I
suggest to postpone it until common.


Estimated fix:

Not decided, depends on MDR or equalent availability
Comment 23 Jan Chalupa 2003-02-17 18:03:59 UTC
Reopening. Please don't use the RESOLVED LATER status flag. There is
no common agreement on its meaning. In reality, the issue is not
RESOLVED, is it?
Comment 24 _ pkuzel 2003-02-17 18:36:16 UTC
LATER means that I cannot fix it under current contraints.
Transitively this issue is known contraint/limitation of current
implementation.

LATER is optimistics alternative to WONTFIX.
Comment 25 _ pkuzel 2003-03-03 17:54:14 UTC
I'm working on fix for Generate DTD action. Removing TAX dependency in
this action implementation will further minimize the problem occurency
probability.
Comment 26 _ pkuzel 2003-03-04 17:17:21 UTC
<--  GuiUtil.java
new revision: 1.4.2.1; previous revision: 1.4
<--  Bundle.properties
new revision: 1.18.4.1; previous revision: 1.18
<--  GenerateDTDSupport.java
new revision: 1.14.2.1; previous revision: 1.14

Generate DTD action made more lightweight. Now it creates tiny memory
structure containing only statistical data instead of huge model of
whole source XML document.
Comment 27 _ pkuzel 2003-03-04 17:32:27 UTC
Marking as FIXED in reduced set of client modules (excluding Visual
XML editor). I scanned the reduced set and found only usage of TAX DTD
models. I have never seen DTD bigger that 1Mb.

TAX XML models are now utilized only in Visual XML editor and I filled
issue #31656 to address it separately.
Comment 28 Martin Schovanek 2003-03-13 13:53:33 UTC
VERIFIED
Comment 29 Patrick Keegan 2003-04-02 14:01:22 UTC
removing RELNOTE keyword
Comment 30 mprpatel 2008-02-17 11:16:21 UTC
what kind of forum is this? not giving any kind of solution for our problem???? we are working on the same problem 
since a week and still not getting the problem.. we get the error of "outofmemory"... the first worst thing about 
netbeans is that it takes eons to startup its modules and then works too slow giving errors of out of memory... we 
have a 2GB RAM with 360GB hard disk and still the same problem... do we have to buy a super computer and make netbeans 
work on it?? we are plannin to change the IDE if the same problem persists... netbeans is really the worst IDE... 
sorry, but before changing the IDE would like to have a solution to memory problem if you can suggest
Comment 31 Jesse Glick 2008-02-17 14:56:08 UTC
Was spuriously reopened.