Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 100520 - Transformation on OTA in salami slice pattern takes a long time
Transformation on OTA in salami slice pattern takes a long time
Status: RESOLVED WONTFIX
Product: xml
Classification: Unclassified
Component: Schema Tools
5.x
All All
: P3 (vote)
: 6.x
Assigned To: Samaresh Panda
issues@xml
: PERFORMANCE
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2007-04-10 20:25 UTC by Mikhail Matveev
Modified: 2008-06-23 23:03 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mikhail Matveev 2007-04-10 20:25:53 UTC
1. Create project.
2. Open OTA_TravelItinerary schema.
3. Apply Salami Slice Design Pattern.
Result: IDE stops responding (I have been waiting for 2 hours before kill it).
Comment 1 Mikhail Matveev 2007-04-11 17:55:26 UTC
Note: I used memory options -J-Xms32m -J-Xmx256m
Comment 2 Ayub Khan 2007-04-18 05:21:26 UTC
If I use memory options -J-Xms32m -J-Xmx512m with the same scenario, then
transformaton happens within 5 min. I think there is regression (in
performance), it used to take only 60sec for transforming OTA to Salami Slice
3-4 months before.
Comment 3 tonybeckham 2007-04-18 19:27:16 UTC
System info:
NetBeans IDE Dev (Build 070418)
1.5.0_07; Java HotSpot(TM) Client VM 1.5.0_07-87
Mac OS X version 10.4.9 running on i386
en_US (nb); MacRoman

Options -J-Xms32m -J-Xmx512m (just using 512 here because thats what ayubskhan
did to have it work.  This is a whole other issue since it has been discussed
that 512 is way too high of a setting and negatively impacts the entire IDE) 
Nevertheless,  when I apply the design pattern the dialogbox with the progress
bar is shown for about 5-7 minutes then goes away as if the process finished. 
However, my processor remains pinned and since I started the IDE from command
line, I can see that there is continual operations going on.  I watched the
output for about 20 minutes then decided it was enough and killed it.  I think
we have a big problem here that needs attention.
Comment 4 Mikhail Matveev 2007-04-18 19:57:26 UTC
I got same results as Tony on WinXP, JDK 1.6.
Comment 5 Samaresh Panda 2007-04-18 21:48:45 UTC
Guys, I just installed a 04/17 build. I changed the heap settings to 256mb,
which is what EntPack recommends. The transformation took me about 7mins. BTW,
it also works with 128mb, but takes a lot more time.

I agree with Tony for the fact that processor was 100% occupied. The traces are
very much same as the one mentioned earlier, see
http://www.netbeans.org/issues/show_bug.cgi?id=87024.

I know this is a regression from Gavotte and Coke. Earlier, the same used to
take about a min. I'm changing the summary and priority accordingly.
Comment 6 Mikhail Matveev 2007-04-19 18:41:09 UTC
Checked with the last build. Yes, it works now as Sam described. Agree with
summary and priority change.
Also I got some irregular exceptions when worked with transformed schema, will
try to reproduce it consistently.
Comment 7 Mikhail Matveev 2007-06-21 14:26:20 UTC
I have checked with M10 on WinXP, Java 1.6_01.
It stops responding after applying Salami Slice, both with defaults command line options and with -J-Xms32m -J-Xmx512m.

I raise priority to P1 again.
Comment 8 Samaresh Panda 2007-08-15 20:04:14 UTC
We have unit tests to measure transformation performance, however the unit tests run on smaller schemas. The unit tests
fail if we try to run them against larger schemas with OOM. Also transformation in general requires more memory.

Here is a summary of my work on this bug so far:
- Of course the first thing you do is, profile it. I was using M10, have filed a bug (issue: 108557) using profiler.
Somehow I've managed to profile using a smaller schema.
- The profiler results are not very helpful. They all point to calls in xdm, which are IMO already very optimal.
- Then I thought I'll profile on OTA schema. It takes more than 24 hours to profile and the end result is, profiler
comes back with a message something like "more data than it can handle". Everything has its limit and I do not know if
its a bug.

I'm in touch with Ayub (original author of transformation) on this. He agrees that not much can be done on xdm front. So
perhaps another option is to revisit the way we transform today and I don't think we have enough time for that in 6.0.

Do note that schema model, axi models are based on xdm which is based on editor's syntax support. The general consensus
is that, we rather move to the new lexer apis if we really want improvements in memory consumption (98405) and performance.

As a workaround, perhaps we could warn the user before (s)he applies the pattern. I just verified, it takes about 7minutes.
Comment 9 Srividhya Narayanan 2007-08-16 20:07:27 UTC
updating the bug summary to reflect that this happens on a single pattern.
Comment 10 Samaresh Panda 2007-08-16 20:20:15 UTC
Forgot to mention, the "Salami Slice" pattern is an extreme corner case. Here is the text from the wizard:

"In the Salami Slice design, all elements are global. There is no nesting of element declarations and element
declarations can be reused throughout the schema. All elements must be defined within the global namespace."

I'm not an expert in authoring schema, however with my limited knowledge, how often do you create schemas with only
global elements? I should have made this clear in the beginning, my mistake.
Comment 11 Samaresh Panda 2007-08-18 01:25:29 UTC
Since Misha is on vacation, I would like Tony to verify the same and have him say whether or not it should be a P1.

IMO, it's not a P1 because of the following reasons:
1. We have released this feature in NB5.5 and NB5.5.1 EntPack with similar issues and no complains from any outside
users as yet.
2. For long running tasks such as this, there is a progress bar that gives feedback to the user.
3. Salami Slice is a very uncommon pattern.

Tony, let is know what you think about this?
Comment 12 Samaresh Panda 2007-08-18 01:34:27 UTC
The progress dialog is modal and it doesn't let user to do anything until it is finished. I'm wondering if this is what
he's referring to as "IDE stops responding"?
Comment 13 tonybeckham 2007-08-18 05:44:45 UTC
I will take time and consider downgrading later. What I can say now from applying the design pattern on OTA schema is that:
1. It does take a long time (like 10 minutes on a well equipped machine)
2. The progress bar is only there for the first 2/3 or so of the time the process takes.  After the progress bar goes
away there is still some time before the IDE will respond to anything.
Comment 14 Samaresh Panda 2007-08-21 19:57:47 UTC
As per agreement with iteam, downgrading this to P2.
Comment 15 Samaresh Panda 2008-01-21 17:38:58 UTC
As per iteam discussion, added a warning message for salami slice pattern on any large files more than 1/2 mb.
/cvs/xml/schema/abe/src/org/netbeans/modules/xml/schema/abe/wizard/SchemaTransformPatternSelectionUI.java,v  <-- 
SchemaTransformPatternSelectionUI.java
new revision: 1.7; previous revision: 1.6
/cvs/xml/schema/abe/src/org/netbeans/modules/xml/schema/abe/wizard/SchemaTransformPatternSelection.java,v  <-- 
SchemaTransformPatternSelection.java
new revision: 1.5; previous revision: 1.4
/cvs/xml/schema/abe/src/org/netbeans/modules/xml/schema/abe/wizard/Bundle.properties,v  <--  Bundle.properties
new revision: 1.7; previous revision: 1.6
/cvs/xml/schema/abe/src/org/netbeans/modules/xml/schema/abe/wizard/SchemaTransformPatternSelectionUI.form,v  <-- 
SchemaTransformPatternSelectionUI.form
new revision: 1.4; previous revision: 1.3
Comment 16 Samaresh Panda 2008-06-23 23:03:31 UTC
A new warning warns user that this action will take several seconds. I think we should consider working on this only if
there are external users want this really badly.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo