This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 194527

Summary: [REGRESSION] High number of positions when editing java and C++ files
Product: editor Reporter: Vladimir Voskresensky <vv159170>
Component: Painting & PrintingAssignee: Miloslav Metelka <mmetelka>
Status: VERIFIED FIXED    
Severity: normal CC: issues, juhrik, mmirilovic
Priority: P1 Keywords: PERFORMANCE, REGRESSION
Version: 7.0   
Hardware: All   
OS: All   
Issue Type: DEFECT Exception Reporter:
Bug Depends on:    
Bug Blocks: 194570    

Description Vladimir Voskresensky 2011-01-20 14:56:18 UTC
MarkVector performance was regressed in trunk 10 times.
It affects any modification actions.
I.e. using as test file:
trunk/cnd.modelimpl/src/org/netbeans/modules/cnd/modelimpl/parser/generated/CPPParser.java
In 6.9.1 reformat (to emulate big number of modifications) was blocked in [1] within 3 minutes, now it is blocked for 20 minutes. [2]

Note: Before press format you should wait till finished worker "Highlights-Coalescing"

Btw "Highlights-Coalescing" works long time in trunk as well (3 minutes) staying in MarkVector.createMark method [2]

[1] 6.9.1 stack
"org.netbeans.api.progress.ProgressUtils" daemon prio=3 tid=0x0afdd400 nid=0x28 runnable [0x4f5ff000]
   java.lang.Thread.State: RUNNABLE
        at org.netbeans.modules.editor.lib.impl.MarkVector.moveOffsetGap(MarkVector.java:703)
        at org.netbeans.modules.editor.lib.impl.MarkVector.update(MarkVector.java:286)
        - locked <0x637e0ce0> (a org.netbeans.modules.editor.lib.impl.MarkVector)
        at org.netbeans.editor.DocumentContent$Edit.undoOrRedo(DocumentContent.java:449)
        at org.netbeans.editor.DocumentContent$Edit.<init>(DocumentContent.java:404)
        at org.netbeans.editor.DocumentContent.remove(DocumentContent.java:119)
        at org.netbeans.editor.BaseDocument.remove(BaseDocument.java:929)
        at org.netbeans.modules.java.source.save.Reformatter.reformatImpl(Reformatter.java:316)
        at org.netbeans.modules.java.source.save.Reformatter.reformat(Reformatter.java:139)
        at org.netbeans.modules.editor.indent.TaskHandler$MimeItem.runTask(TaskHandler.java:550)
        at org.netbeans.modules.editor.indent.TaskHandler.runTasks(TaskHandler.java:317)
        at org.netbeans.modules.editor.indent.IndentImpl.reformat(IndentImpl.java:320)
        at org.netbeans.modules.editor.indent.FormatterImpl.reformat(FormatterImpl.java:190)
        at org.netbeans.editor.ActionFactory$FormatAction$1$1.run(ActionFactory.java:1683)
        at org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:344)
        at org.netbeans.editor.ActionFactory$FormatAction$1.run(ActionFactory.java:1651)
        at org.netbeans.modules.progress.ui.RunOffEDTImpl$1.run(RunOffEDTImpl.java:160)
        at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1418)
        at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1957)

[2] format action in trunk itself 
"org.netbeans.api.progress.ProgressUtils" daemon prio=3 tid=0x09463800 nid=0x3c runnable [0x50fff000]
   java.lang.Thread.State: RUNNABLE
        at org.netbeans.modules.editor.lib.impl.MarkVector.moveOffsetGap(MarkVector.java:679)
        at org.netbeans.modules.editor.lib.impl.MarkVector.update(MarkVector.java:286)
        - locked <0x63745ef8> (a org.netbeans.modules.editor.lib.impl.MarkVector)
        at org.netbeans.editor.DocumentContent$Edit.undoOrRedo(DocumentContent.java:449)
        at org.netbeans.editor.DocumentContent$Edit.<init>(DocumentContent.java:389)
        at org.netbeans.editor.DocumentContent.insertString(DocumentContent.java:112)
        at org.netbeans.editor.BaseDocument.insertString(BaseDocument.java:771)
        at org.netbeans.modules.java.source.save.Reformatter.reformatImpl(Reformatter.java:317)
        at org.netbeans.modules.java.source.save.Reformatter.reformat(Reformatter.java:140)
        at org.netbeans.modules.editor.indent.TaskHandler$MimeItem.runTask(TaskHandler.java:548)
        at org.netbeans.modules.editor.indent.TaskHandler.runTasks(TaskHandler.java:316)
        at org.netbeans.modules.editor.indent.IndentImpl.reformat(IndentImpl.java:325)
        at org.netbeans.modules.editor.indent.api.Reformat.reformat(Reformat.java:154)
        at org.netbeans.editor.ActionFactory$FormatAction$1$1.run(ActionFactory.java:1705)
        at org.netbeans.editor.GuardedDocument.runAtomicAsUser(GuardedDocument.java:344)
        at org.netbeans.editor.ActionFactory$FormatAction$1.run(ActionFactory.java:1661)
        at org.netbeans.modules.progress.ui.RunOffEDTImpl$1.run(RunOffEDTImpl.java:160)
        at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1424)
        at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1968)

[3] in trunk wait for annotations displaying before pressing format action
"Highlights-Coalescing" daemon prio=3 tid=0x08b61c00 nid=0x1a runnable [0x52bff000]
   java.lang.Thread.State: RUNNABLE
        at org.netbeans.modules.editor.lib.impl.MarkVector.createMark(MarkVector.java:127)
        at org.netbeans.editor.DocumentContent.createPosition(DocumentContent.java:125)
        at javax.swing.text.AbstractDocument.createPosition(AbstractDocument.java:816)
        - locked <0x61a92c60> (a org.netbeans.modules.editor.NbEditorDocument)
        at org.netbeans.modules.editor.lib2.view.ViewBuilder.createNextView(ViewBuilder.java:459)
        at org.netbeans.modules.editor.lib2.view.ViewBuilder.createViews(ViewBuilder.java:335)
        at org.netbeans.modules.editor.lib2.view.ViewUpdates.buildViews(ViewUpdates.java:146)
        at org.netbeans.modules.editor.lib2.view.ViewUpdates.checkRebuild(ViewUpdates.java:516)
        at org.netbeans.modules.editor.lib2.view.ViewUpdates$FactoriesListener.viewFactoryChanged(ViewUpdates.java:542)
        at org.netbeans.modules.editor.lib2.view.EditorViewFactory.fireEvent(EditorViewFactory.java:224)
        at org.netbeans.modules.editor.lib2.view.HighlightsViewFactory$1.run(HighlightsViewFactory.java:119)
        at org.netbeans.editor.BaseDocument.render(BaseDocument.java:1419)
        at org.netbeans.modules.editor.lib2.view.HighlightsViewFactory$1.run(HighlightsViewFactory.java:125)
        at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1424)
        at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1968)
Comment 1 Miloslav Metelka 2011-01-21 14:18:18 UTC
MarkVector was not changed for ages (last change except license updates was 2007) so the only explanation is that there's unusually high number of marks to be handled (likely due to a high number of document position for that particular document).
Comment 2 Vladimir Voskresensky 2011-01-21 14:41:52 UTC
(In reply to comment #1)
> MarkVector was not changed for ages (last change except license updates was
> 2007) 
you are right, we faced it's slowness for several releases :-)

> so the only explanation is that there's unusually high number of marks to
> be handled (likely due to a high number of document position for that
> particular document).
so, it's worth to investigate the issue (why slow component is used 10 times more causing 10 times slower performance as a result). May be you can find a way to improve/replace this slow element?
Comment 3 Miloslav Metelka 2011-01-26 09:52:37 UTC
Hi Vladimir,
 I have a brand new DocumentContent ready that will feature Position instances sharing but I first need to get rid of the o.n.editor.Mark support so I will put it in after-7.0 release.
 Anyway I still believe that for some reason you have a high number of position objects (MarkVector.java:703 corresponds to mark array iterating) and so that the MarkVector itself is not the culprit so I mark the issue as invalid. If I'm wrong please reopen. I've added a dump of the mark count into BaseDocument.toStringDetail() for a quick check of how many marks are present at the moment.
Comment 4 Miloslav Metelka 2011-01-26 09:58:57 UTC
http://hg.netbeans.org/jet-main/rev/e0d2f0fe39b7
Comment 5 Vladimir Voskresensky 2011-01-26 19:20:39 UTC
after discussion and some investigations we've decided to reopen it:
-------
Vladimir, you're right - the position count is high for java too, in the meantime I've used the added markcount info and for 80-line java file it's 1104 marks which is pretty high. I'm now searching what's holding the positions.
--------
Comment 6 Quality Engineering 2011-01-27 08:27:46 UTC
Integrated into 'main-golden', will be available in build *201101270001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/e0d2f0fe39b7
User: Miloslav Metelka <mmetelka@netbeans.org>
Log: #194527 - MarkVector is 700% slower comparing to 6.9.1 - Info about mark count added to BaseDocument.toStringDetail().
Comment 7 Vladimir Voskresensky 2011-01-27 09:12:42 UTC
Mila, may be there are too much objects attached to the same position? 
Then you can use new version of WeakSet which has method putIfAbsent to share same object between all clients
Comment 8 Miloslav Metelka 2011-01-31 15:12:30 UTC
I have added position creation stack tracing and the number of positions is roughly 2.7x number of lines:
1 position per line is for a line element start position.
1 position per line is for a paragraph view's start. With the new document content the same position instance would be shared in this case.
0.7 position per-line corresponds to other structures like folds, annotations etc.

Anyway during the reformat the view hierarchy updates are turned off. The view hierarchy is rebuilt after the reformat gets finished.

http://hg.netbeans.org/jet-main/rev/5ca6146a126b
http://hg.netbeans.org/jet-main/rev/64e6f65c2206

I'll now debug the individual inserts/removals done by the reformatter and the individual document listeners.
Comment 9 Quality Engineering 2011-02-01 05:56:47 UTC
Integrated into 'main-golden', will be available in build *201102010000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/5ca6146a126b
User: Miloslav Metelka <mmetelka@netbeans.org>
Log: #194527 - High number of positions when editing java and C++ files - added extra debugging; fixed missing read-lock in AnnotationView.
Comment 10 Miloslav Metelka 2011-03-22 11:33:41 UTC
Oops, at first look I have overlooked a missing break of the mark iteration loop in MarkVector.moveOffsetGap() but it obviously degrades performance, sorry for that.
I have double-checked that there's no similar error across the sources but it should be fine since all other usages come from OffsetGapList where the moveOffsetGap() does not split below/above gap iteration so it's fine.
Vladimir, thanks for finding this.

http://hg.netbeans.org/jet-main/rev/f5a1a3394a44
Comment 11 Vladimir Voskresensky 2011-03-22 11:40:43 UTC
let's nominate for 7.0, right?
Comment 12 Miloslav Metelka 2011-03-22 16:12:39 UTC
Ok, Vladimir, please verify that the fix works for you. Thanks.
Comment 13 Vladimir Voskresensky 2011-03-22 21:35:09 UTC
8 minutes without fix on very fast machine...
will check speed when 
http://hg.netbeans.org/jet-main/rev/f5a1a3394a44
is propagated into cnd-main
Comment 14 Vladimir Voskresensky 2011-03-23 07:23:04 UTC
Mila, works great!
Now time is 90 secs on the same machine.

Great work!
Thanks,
Vladimir.
Comment 15 Jaromir Uhrik 2011-03-23 08:28:58 UTC
Based on the verification above I agree with integration to nb70.
Comment 16 Quality Engineering 2011-03-23 09:59:14 UTC
Integrated into 'main-golden', will be available in build *201103230400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/f5a1a3394a44
User: Miloslav Metelka <mmetelka@netbeans.org>
Log: #194527 - High number of positions when editing java and C++ files - performance improvement of marks iteration loop.
Comment 17 Miloslav Metelka 2011-03-23 22:23:58 UTC
Integrated into release70:
f5a1a3394a44 transplanted to e61447073489
Comment 18 Jaromir Uhrik 2011-03-31 13:41:13 UTC
Vladimir, could you please verify also in the latest 70 build? 
Thanks.
Comment 19 Vladimir Voskresensky 2011-03-31 16:02:09 UTC
verified in 7.0