This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 87151 - Performance: re-opening HL7 schema file is slow
Summary: Performance: re-opening HL7 schema file is slow
Status: VERIFIED FIXED
Alias: None
Product: xml
Classification: Unclassified
Component: Schema Tools (show other bugs)
Version: 5.x
Hardware: All All
: P3 blocker (vote)
Assignee: Samaresh Panda
URL:
Keywords: PERFORMANCE
: 97193 (view as bug list)
Depends on:
Blocks: 89268
  Show dependency tree
 
Reported: 2006-10-13 23:47 UTC by Samaresh Panda
Modified: 2008-02-15 22:22 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
HL7 zip file (190.81 KB, application/x-compressed)
2006-10-13 23:48 UTC, Samaresh Panda
Details
Apache/Resolver performance degradation for hl7 impacts ABE (12.07 KB, application/octet-stream)
2006-11-21 01:38 UTC, Ayub Khan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Samaresh Panda 2006-10-13 23:47:21 UTC
Unzip the attahced zip file.

Open fields.xsd file and when you click on DV IDE hangs. CPU consumption is 100%
for quite some time. After a long time, DV appears blank and you see a NPE as
follows:

java.lang.NullPointerException
   at
org.netbeans.modules.xml.axi.impl.AXIModelImpl.fromSameSchemaModel(AXIModelImpl.java:103)
   at
org.netbeans.modules.xml.axi.impl.AXIModelBuilder.getAXIComponent(AXIModelBuilder.java:258)
   at
org.netbeans.modules.xml.axi.impl.AXIModelBuilder.visit(AXIModelBuilder.java:237)
   at
org.netbeans.modules.xml.schema.model.impl.ComplexExtensionImpl.accept(ComplexExtensionImpl.java:69)
   at
org.netbeans.modules.xml.schema.model.visitor.DeepSchemaVisitor.visitChildren(DeepSchemaVisitor.java:245)
   at
org.netbeans.modules.xml.axi.impl.AXIModelBuilder.visit(AXIModelBuilder.java:214)
   at
org.netbeans.modules.xml.schema.model.impl.ComplexContentImpl.accept(ComplexContentImpl.java:85)
   at
org.netbeans.modules.xml.axi.impl.AXIModelBuilder.populateChildren(AXIModelBuilder.java:71)
   at
org.netbeans.modules.xml.axi.AXIComponent.populateChildren(AXIComponent.java:463)
Comment 1 Samaresh Panda 2006-10-13 23:48:23 UTC
Created attachment 35201 [details]
HL7 zip file
Comment 2 Samaresh Panda 2006-10-13 23:59:34 UTC
Fix for the NPE has been integrated. Assgning to Girish and also lowering the
priority, since this is not a common case.

Here is the diff:
http://xml.netbeans.org/source/browse/xml/axi/src/org/netbeans/modules/xml/axi/impl/AXIModelBuilder.java?r1=1.1.2.23.6.1&r2=1.1.2.23.6.2
Comment 3 Samaresh Panda 2006-10-14 04:15:19 UTC
Even though, the schema validation passes, while building the axi model, some
schema components are null.

In this case, the base for a complex extension is null. I'll have to take care
of this situation. Ideally it shouldn't happen.

In any case I'm going to integrate a fix, but the problem remains, that is DV
shows up after the fix, but IDE is still unresponsive and CPU consumption is 100%.
Comment 4 Ayub Khan 2006-11-21 01:38:56 UTC
Created attachment 36153 [details]
Apache/Resolver performance degradation for hl7 impacts ABE
Comment 5 Ayub Khan 2006-11-21 02:08:41 UTC
Fixed XDM/NamedNodeMapImpl for the perf degradation during init of attributes.

/cvs/xml/xdm/src/org/netbeans/modules/xml/xdm/nodes/NodeImpl.java
new revision: 1.2.2.19.6.2; previous revision: 1.2.2.19.6.1

/cvs/xml/xdm/src/org/netbeans/modules/xml/xdm/nodes/NamedNodeMapImpl.java
new revision: 1.1.2.3.12.1; previous revision: 1.1.2.3

With above fix, Switch to Design view now takes 90 sec with 256/512MB jvm memory
setting.

Further perf bottleneck is found in Apache/Resolver. See attached (issue87151.txt)

(Note: with jvm setting of 128MB as in NB55 default setting, the switch to
Design view would run forever, and is again found to be a Apache/Resolver problem)
Comment 6 Samaresh Panda 2006-12-05 00:40:28 UTC
Significant improvement. Now, switching to DV takes about 30sec, however the job
is not done yet.

For now...
AXIModelUpdater.java: 1.1.2.36.6.2;
AXIModelBuilder.java: 1.1.2.24.6.1;
Util.java: 1.1.2.65.6.3;
ElementImpl.java: 1.1.2.46.6.2;
AXIModelImpl.java: 1.1.2.42.6.1;
Comment 7 Samaresh Panda 2006-12-05 23:35:49 UTC
Much better.
AXIModelImpl.java: 1.1.2.42.6.2;
Comment 8 Samaresh Panda 2006-12-06 21:39:30 UTC
We these fixes, we have brought down the creation of model from 105sec to 2.x
sec. With this change, switching to DV is quite fast (<20sec), but not quite
acceptable.
Comment 9 Samaresh Panda 2006-12-19 00:56:38 UTC
Some more improvement in the model side. This time we're caching AXIDocument's
children.

AXIComponent.java: new revision: 1.1.2.80.6.3;
ModelAccessImpl.java: new revision: 1.1.2.23.6.4;
AXIModelBuilder.java: new revision: 1.1.2.24.6.3;
AXIDocumentImpl.java: new revision: 1.1.2.3.24.2;
ElementImpl.java: new revision: 1.1.2.46.6.4;
AXIModelImpl.java: new revision: 1.1.2.42.6.6;

Comment 10 Samaresh Panda 2006-12-19 01:02:03 UTC
One more fix in UI
Checking in ContainerPanel.java;
/cvs/xml/schema/abe/src/org/netbeans/modules/xml/schema/abe/ContainerPanel.java,v
 <--  ContainerPanel.java
new revision: 1.1.2.37.6.2; previous revision: 1.1.2.37.6.1
Comment 11 Ayub Khan 2007-01-08 22:52:48 UTC
Fix XDM related perf for this issue.

1. Use jdk Collections.sort in DiffFinder.
2. Use attribute list instead of getAttributes wherever possible.

Files changed:
/cvs/xml/xdm/src/org/netbeans/modules/xml/xdm/nodes/NodeImpl.java
new revision: 1.2.2.19.6.3; previous revision: 1.2.2.19.6.2

/cvs/xml/xdm/src/org/netbeans/modules/xml/xdm/nodes/Element.java
new revision: 1.2.2.13.6.3; previous revision: 1.2.2.13.6.2

/cvs/xml/xdm/src/org/netbeans/modules/xml/xdm/diff/DiffFinder.java
new revision: 1.2.2.22.6.1; previous revision: 1.2.2.22

Comment 12 Nam Nguyen 2007-01-09 00:14:18 UTC
The changes look fine.
Comment 13 _ rkubacki 2007-01-09 10:32:34 UTC
Why is such a long computation realized in event dispatch thread? Then it is
obvious that the application cannot update its UI while data processing runs.
Comment 14 Samaresh Panda 2007-02-02 22:28:02 UTC
After a lot of discussion, here is what we came up with:
 1. collapse top level elements header if the no. of global elements
    are more than 50. They get shown when user expands.
 2. If rendering of these GEs take more than 3sec, we should show
    hourglass, status progress bar - along with a message.

Integrating the fix for #1. We already show the hourglass.
Checking in InstanceDesignerPanel.java;
/cvs/xml/schema/abe/src/org/netbeans/modules/xml/schema/abe/InstanceDesignerPanel.java,v
 <--  InstanceDesignerPanel.java
new revision: 1.1.2.43.6.2; previous revision: 1.1.2.43.6.1

For the remaining items in #2, I have filed a separate issue. See
http://www.netbeans.org/issues/show_bug.cgi?id=94158.
Comment 15 tonybeckham 2007-02-12 21:33:01 UTC
There is no longer any exceptions thrown and switching views is possible in just
a few seconds.  With previos release, it was not even possible to change views
and just to open the large schema took almost 10 seconds.  Now the schema opens
instantly and switching to design view is instantaneous and expanding nodes in
design view takes around 7 seconds for the very large schema.  Marking issue as
VERIFIED.
Comment 16 tonybeckham 2007-03-02 00:28:38 UTC
Reopening-

Using Milestone7b, Opening HL7 takes longer than it did with Coke.
Comment 17 Samaresh Panda 2007-03-02 00:41:53 UTC
Ayub is working on this.
Comment 18 Samaresh Panda 2007-03-05 17:01:44 UTC
Opening HL7 schema file takes more than 10sec now, hence this was reopened.
Comment 19 Samaresh Panda 2007-03-05 17:04:20 UTC
*** Issue 97193 has been marked as a duplicate of this issue. ***
Comment 20 tonybeckham 2007-03-06 17:23:21 UTC
I was not aware of the fact that the first opening is faster.  With clean
environment and build 070306_4 the time to first completely open HL7 takes
around 7 sec.
Comment 21 Samaresh Panda 2007-03-06 23:58:47 UTC
Opening HL7 for the first time takes about 5-7 secs and the numbers have not
changed as per our PBS criteria.

However, reopening the same file takes a long time (10-12sec) and I think we
could get some numbers there. We have a fix, and the impact is on all models. I
would prefer fixing that in 6.0 and not on the freeze date for 5.5.1.

Moving this bug to 6.0, also I'm changing the summary.
Comment 22 Samaresh Panda 2007-03-30 23:09:51 UTC
Opening larger schemas in the schema editor is pretty fast. However reopening
the same schema takes longer. Downgrading to P3 with a new summary.
Comment 23 Samaresh Panda 2007-09-20 23:38:13 UTC
I think, reopening is much better now. Please verify.
Comment 24 tonybeckham 2007-10-16 00:31:00 UTC
Product Version: NetBeans IDE Dev (Build 200710151100)
Java: 1.6.0_03; Java HotSpot(TM) Client VM 1.6.0_03-b05
System: Linux version 2.6.20-16-generic running on i386; UTF-8; en_US (nb)

With 6.0 Beta2 to reopen fields.xsd it took about 12 sec.  This seams acceptable to me as a time to reopen a large
schema.  Before this is verified, I want to get a benchmark for an older build.  Will any build before Sept. 20 be
sufficient?  Maybe M9 or something?
Comment 25 tonybeckham 2007-10-16 01:13:07 UTC
Initial opening is much faster now; M9 takes about 7 sec for first opening and with Beta2 this is only a couple of
seconds.  

For reopening (what this bug is about), M9 was faster @ around 4 sec. whereas Beta2 is around 12 sec.

I did these performance tests with just my watch so the results are unofficial.  The system I used was:
Linux 2.6.20-16-generic with java 1.6.0_03, 2GB RAM, AMD Opteron(tm) Processor 148 @ 1000 MHz.
Comment 26 tonybeckham 2008-02-15 22:22:49 UTC
Fix Verified:
Performance has significantly improved since moving fold to lexar.  Now even reopening fields.xsd takes only 2 sec.

Product Version: NetBeans IDE Dev (Build 200802151203)
Java: 1.5.0_13; Java HotSpot(TM) Client VM 1.5.0_13-119
System: Mac OS X version 10.5.2 running on i386; MacRoman; en_US (nb)