This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 196669

Summary: Dragging and dropping very large HTML table into PHP Web Page makes IDE unresponsive and after renders strangely (see attached)
Product: editor Reporter: maghiel <maghiel>
Component: Parsing & IndexingAssignee: Tomas Zezula <tzezula>
Status: VERIFIED FIXED    
Severity: normal CC: mfukala, mmirilovic, Selpi, sks3286, vriha
Priority: P4 Keywords: PERFORMANCE
Version: 7.0   
Hardware: PC   
OS: Windows 7   
Issue Type: DEFECT Exception Reporter:
Attachments: DnD large html table result
A snapshot taken during shifting approx. 45000 html lines.
A snapshot taken during shifting approx. 45000 html lines with disabled filling of stacktraces
Screenshot after sometime (tried to maximize and restore window first, so controls disapeared)
another screenshot

Description maghiel 2011-03-14 12:16:54 UTC
[ BUILD # : 201103120000 ]
[ JDK VERSION : 1.6.23 ]

Dragging and dropping very large HTML table into PHP Web Page makes IDE
unresponsive for many seconds (a minute?) and after renders strangely.
Please see attached screenshot.

Test case:
# {{testcase|DnD big Table HTML component to HTML code.}}
## DnD Table HTML component to the editor.
## In the "Insert Table" dialog, specify large values for rows and columns
(like 1000), change values of other fields and press OK.
#* {{result|EXPECTED RESULT: After reasonable time (few seconds), HTML tags
added to DnD position. They correspond to the visual representation of Table
HTML component in the "Insert Table" dialog (with made changes).}}
'''*FAIL*'''
Comment 1 maghiel 2011-03-14 12:18:01 UTC
Created attachment 106976 [details]
DnD large html table result
Comment 2 sks3286 2011-03-23 03:57:03 UTC
I have similar experience. In fact, on my WinXP system, the CPU utilisation goes up to a full 100% after which I am forced to "End task tree" for the IDE since the NB IDE stops responding to any input.
Comment 3 Sc0tTyXL 2011-03-25 10:51:55 UTC
I experienced the same problem during my RC1 certification test, so this bug is confirmed.
Comment 4 Tomas Mysik 2011-06-03 10:23:21 UTC
Batch reassigning.
Comment 5 Tomas Mysik 2011-06-08 10:15:42 UTC
Editor area.
Comment 6 vladimircs 2011-07-03 00:29:29 UTC
The error also appears in Ubuntu Linux, bug confirmed while performing the test on RC 7.0.1
Comment 7 Ondrej Brejla 2011-09-15 09:32:15 UTC
Reproducible in 7.1dev. Same for .html files so reassigning to html (it's its pallete).
Comment 8 Marek Fukala 2011-10-29 06:53:50 UTC
1) I cannot reproduce the painting problem.

2) If I insert 200x200 table into an html file most of the CPU time is spent in 

   a. very little time in html formatting ... getTagChildren() method - fixed in web-main#3f365fc85630

   b. document.insert/removeUpdate() -> o.n.m.parsing.impl.event.resetState() which schedules a RP task (for reparse likely) but this involves creation of RP$SlowItem.fillInStactrace() which is terribly slow.

the #b case is more nicely reproducible if you then select the whole file and invoke shift line action -> most of the time spent in document updates (isolated line shifts) causing the RP$SlowItem creation

Since the issue is easily reproducible I'm not attaching any thread snapshots here.

I'm not sure what the right component should be - likely parsing.api since its support schedules that many RP tasks.
Comment 9 Quality Engineering 2011-10-30 15:12:50 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/3f365fc85630
User: Marek Fukala <mfukala@netbeans.org>
Log: #196669 - HtmlIndenter caching of tag's children
Comment 10 Tomas Zezula 2011-11-01 12:07:50 UTC
In reply to comment #8.
>I'm not sure what the right component should be - likely parsing.api since its support schedules that many RP tasks.
Parsing api does not schedule RP tasks, it just reschedules single one. Please reevaluate.
Comment 11 Marek Fukala 2011-11-01 12:19:51 UTC
(In reply to comment #10)
> In reply to comment #8.
> >I'm not sure what the right component should be - likely parsing.api since its support schedules that many RP tasks.
> Parsing api does not schedule RP tasks, it just reschedules single one. Please
> reevaluate.

I'm sorry I wanted to write ...schedules that many RP *items*. Please re-read my comment above:

   b. document.insert/removeUpdate() -> o.n.m.parsing.impl.event.resetState()
which schedules a RP task (for reparse likely) but this involves creation of
RP$SlowItem.fillInStactrace() which is terribly slow.

RP$Task.schedule() creates the RP$SlowItem.
Comment 12 Marek Fukala 2011-11-01 12:22:06 UTC
> ...schedules that many RP *items*.
To be really accurate: 

Parsing API schedules one task many times which involves creation of RP$SlowItem on each RP$Task.schedule(...).
Comment 13 Tomas Zezula 2011-11-01 12:27:11 UTC
The evaluation is a  bit "nonsense".

>Thee are document.insert/removeUpdate fired by document
> b. document.insert/removeUpdate() -> o.n.m.parsing.impl.event.resetState()
>which schedules a RP task (for reparse likely) but this involves creation of
>RP$SlowItem.fillInStactrace() which is terribly slow.
>RP$Task.schedule() creates the RP$SlowItem.

The reason why the schedule is done is to collapse the events, so the category is wrong.
Better is to attach the snapshot.
Comment 14 Jan Lahoda 2011-11-01 12:39:44 UTC
(In reply to comment #13)
> The evaluation is a  bit "nonsense".
> 
> >Thee are document.insert/removeUpdate fired by document
> > b. document.insert/removeUpdate() -> o.n.m.parsing.impl.event.resetState()
> >which schedules a RP task (for reparse likely) but this involves creation of
> >RP$SlowItem.fillInStactrace() which is terribly slow.
> >RP$Task.schedule() creates the RP$SlowItem.
> 
> The reason why the schedule is done is to collapse the events, so the category
> is wrong.
> Better is to attach the snapshot.

Snapshot would definitely be better, but from the description it seemed that the problem Marek sees is that Task.schedule is creating stack traces and is being called a lot. I do not think that the real problem is that Task.schedule is being called many times - its the duty and point of the RP to handle that. The problem is the stack trace creation, which is (say) an unfortunate decision in the platform. I would propose to simply disable the stack trace creation, by changing:
private static final RequestProcessor RP = new RequestProcessor ("parsing-event-collector",1);
=>
private static final RequestProcessor RP = new RequestProcessor ("parsing-event-collector",1, false, false);

Not sure if that helps in the original case (the stacktrace creation is enabled only when assertions are enabled, and assertions were almost surely disabled in 7.0RC1).
Comment 15 Tomas Zezula 2011-11-01 12:46:04 UTC
I've already this this change.
But I doubt it's real slowness, it's rather safe point.


The following code:
long s = System.currentTimeMillis();
        for (int i=0; i<100000; i++) {
            StackTraceElement[] st = Thread.currentThread().getStackTrace();
        }
        long e = System.currentTimeMillis();
        System.out.println(e-s);



takes 100000 stack traces and it takes 356ms.
Comment 16 Marek Fukala 2011-11-01 13:08:01 UTC
>(In reply to comment #13)
>> The evaluation is a  bit "nonsense".
>I'm not sure what the right component should be - likely parsing.api since its
>support schedules that many RP tasks.

Why nonsense? The issue should possibly be filed against platform (RequestProcessor) rather than parsing.api. I understand the need for the events aggregation, I was just wondering if you could possibly do it in a RP without the filling stacktraces. 

I may attach the snapshot but since as I've already said it is so easy to reproduce anyone can make own in a minute.
Comment 17 Tomas Zezula 2011-11-01 13:10:44 UTC
The category is wrong and probably it's not problem at all, see comment #15.
This is what I've complained about.
Comment 18 Marek Fukala 2011-11-01 13:23:29 UTC
Created attachment 112660 [details]
A snapshot taken during shifting approx. 45000 html lines.

~16 seconds (of ~28) is spent in the RP$SlowItem creation
Comment 19 Marek Fukala 2011-11-01 13:34:50 UTC
Created attachment 112662 [details]
A snapshot taken during shifting approx. 45000 html lines with disabled filling of stacktraces
Comment 20 Marek Fukala 2011-11-01 13:38:32 UTC
>The category is wrong and probably it's not problem at all, see comment #15.
>This is what I've complained about.

The first snapshot contains just the part where the RP$SlowItem-s are created. 

The second one spans over the whole "shift" operation where I've disabled the filling in stacktraces. 

The whole line shift operation is approximately 2 times faster (~ 14seconds instead of the ~30 with filling in stacktraces enabled).
Comment 21 Tomas Zezula 2011-11-02 09:29:56 UTC
Thanks Marku
Comment 22 Tomas Zezula 2011-11-02 13:40:29 UTC
Changed SI to FI in jet-main 19512d556e0d.
But from real time measurement of shifting 3856 lines of HTML file only 811ms is spent in RP.T.schedule (SlowItem), and the SI is only in dev builds.
Comment 23 Quality Engineering 2011-11-04 15:03:43 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/19512d556e0d
User: Tomas Zezula <tzezula@netbeans.org>
Log: #196669:Dragging and dropping very large HTML table into PHP Web Page makes IDE unresponsive
Comment 24 Vladimir Riha 2011-12-01 16:25:12 UTC
verified

Product Version: NetBeans IDE 7.1 RC2 (Build 201111302200)
Java: 1.7.0_02; Java HotSpot(TM) Client VM 22.0-b10
System: Linux version 3.0.0-13-generic-pae running on i386; UTF-8; en_US (nb)
Comment 25 mrpc 2012-04-14 15:32:06 UTC
Created attachment 118272 [details]
Screenshot after sometime (tried to maximize and restore window first, so controls disapeared)

I've got it while I was doing a sanity test for 7.1.2:

   Product Version: NetBeans IDE 7.1.2 (Build 201204101705)
   Java: 1.7.0_04; Java HotSpot(TM) 64-Bit Server VM 23.0-b21
   System: Mac OS X version 10.7.3 running on x86_64; US-ASCII; el_GR (nb)

Took too much time to finish (netbeans was frozen), messed up the rendering and consumed tons of RAM.

Closed netbeans, reopened and tried to make it happen again, this time running the profiler to submit a snapshot, but it never became usable again to get the file. I tried to quit netbeans, but it didn't terminate, so I forced quit. 

The table I tried to add had 1000 rows / 200 columns.
Comment 26 Vladimir Riha 2012-04-14 16:06:25 UTC
I usually try sizes like 500x200 which works fine (takes about 7s but that is understandable). I'm not sure how much is generating such a huge table in static HTML a "real-world" scenario (even the 500x200 generates over 100k lines!)... Can you generate the 500x200 table?
Comment 27 mrpc 2012-04-14 16:43:51 UTC
Created attachment 118274 [details]
another screenshot

500x200 Works (it takes some time)
But 1000x200 works on:
   Product Version: NetBeans IDE 7.1 (Build 201112051121)
   Java: 1.6.0_31; Java HotSpot(TM) 64-Bit Server VM 20.6-b01-415
(after upgrade to 7.1.2). 

It becomes very slow - in the limits of unusable, but at least it works. So there is a small difference in performance, even if it's not realistic. Check screenshot. I think it's an issue of what Java SDK I use (when I upgraded 7.1  to 7.1.2 it was still using the old Java SDK).

I don't even know if it matters as an issue, as it is unrealistic.
Comment 28 Vladimir Riha 2012-04-14 17:38:04 UTC
I agree, not very realistic case. I would change it to P4 if you wouldn't mind. Also you've changed "Assigned To" to yourself. If it was by mistake can you change it back please?
Comment 29 mrpc 2012-04-14 18:45:13 UTC
I don't even know how I did that :p Perhaps it happened because I reopened?
Of course I agree for P4, or even closing it, but I had to report it :)
(In reply to comment #28)
> I agree, not very realistic case. I would change it to P4 if you wouldn't mind.
> Also you've changed "Assigned To" to yourself. If it was by mistake can you
> change it back please?
Comment 30 Marek Fukala 2012-04-16 09:55:45 UTC
BTW I've recently added a warning to the create table dialog if one tries to create a table with 10000 cels or bigger.
Comment 31 Vladimir Riha 2012-04-16 11:05:01 UTC
Thanks, I noticed the red msg but didn't know when exactly it appears
Comment 32 Marek Fukala 2012-04-16 11:13:22 UTC
so are you saying you found it not unintuitive or confusing? The message should appear once you modify the rows and columns input fields so rows multiplied by columns is bigger than 100*100.
Comment 33 Vladimir Riha 2012-04-17 11:10:00 UTC
(In reply to comment #32)
> so are you saying you found it not unintuitive or confusing? The message should
> appear once you modify the rows and columns input fields so rows multiplied by
> columns is bigger than 100*100.

No, it's clear. Maybe a small modification: You can generate table that has 0 rows and more than 100000 columns and in that case <thead> (and in future maybe also tfoot) is still generated but the warning is not shown . So solution would be either alter the sum to be able to handle 0 rows or not to generate thead if number of rows is 0. In my opinion first solution would be better, some may use it to statically generate header/footer and use some code (js, php, template specific...) between to generate the real content.
Comment 34 Marek Fukala 2012-04-18 08:50:00 UTC
vlado, change the code so it handles the situation you mention. To me it seems a P6, so I consider the issue as fixed.
Comment 35 Vladimir Riha 2012-04-18 14:36:06 UTC
(In reply to comment #34)
> vlado, change the code so it handles the situation you mention. To me it seems
> a P6, so I consider the issue as fixed.

OK :) Thanks

main #35bdafe70c9b
Comment 36 Quality Engineering 2012-04-19 10:01:42 UTC
Integrated into 'main-golden', will be available in build *201204190400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/35bdafe70c9b
User: Vladimir Riha <vriha@netbeans.org>
Log: #196669 - warning for cornes case with 0 rows
Comment 37 Marek Fukala 2012-04-19 13:15:54 UTC
I do thank you!
Comment 38 Marek Fukala 2012-04-20 11:31:45 UTC
*** Bug 205369 has been marked as a duplicate of this bug. ***