This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 28297 - [2004-05-26] Rewrite output window
Summary: [2004-05-26] Rewrite output window
Status: RESOLVED FIXED
Alias: None
Product: cnd
Classification: Unclassified
Component: Terminalemulator (show other bugs)
Version: 3.x
Hardware: PC Windows ME/2000
: P2 blocker (vote)
Assignee: _ tboudreau
URL:
Keywords: PERFORMANCE, UMBRELLA
Depends on: 36735 21724 27652
Blocks: 20522 24824 30536 30564 30587 33384 33605 34908 35131 35336 35667 38478 39918 41306 41924 42011
  Show dependency tree
 
Reported: 2002-10-26 21:13 UTC by _ pkuzel
Modified: 2008-12-23 08:40 UTC (History)
2 users (show)

See Also:
Issue Type: TASK
Exception Reporter:


Attachments
example (1.71 KB, text/plain)
2002-10-31 09:44 UTC, akemr
Details
Profiler data (7.13 KB, text/plain)
2002-11-07 22:32 UTC, _ pkuzel
Details
An extremely simplified OutputTabTerm (7.71 KB, text/plain)
2003-08-13 16:30 UTC, _ tboudreau
Details
Simple test writing from three threads (1.00 KB, text/plain)
2003-08-13 16:31 UTC, _ tboudreau
Details

Note You need to log in before you can comment on or make changes to this bug.
Description _ pkuzel 2002-10-26 21:13:41 UTC
Measured at PIII@600 it takes about 1 sec to 
write a long line (>500chars - debugger 
commanline) while starting debugging session.

It's nearly 8% of debugger session startup 
time.
Comment 1 akemr 2002-10-31 09:43:18 UTC
This is not OW issue IMO - I'll attach simple example, printing long
line takes about 10 - 150 ms. Could you try it?
Comment 2 akemr 2002-10-31 09:44:41 UTC
Created attachment 7820 [details]
example
Comment 3 _ pkuzel 2002-11-01 09:09:54 UTC
I made similar simple benchmark with the same result. I'll analyze
profiling snapshot further.
Comment 4 _ pkuzel 2002-11-07 22:32:14 UTC
Created attachment 7890 [details]
Profiler data
Comment 5 Marian Mirilovic 2002-12-06 17:14:18 UTC
reassigne to Tim, new owner of terminal emulator
Comment 6 _ tboudreau 2003-01-22 09:23:04 UTC
Ivan mentions that term generates a couple monospaced fonts on init.
I wonder if this could be triggering the same problem the Welcome
Screen did, forcing initialization of a bunch of system font
stuff.

Ales, when you got this 8% figure, what was the state of the
IDE?  Was this the first output sent to the OW in the session?
Comment 7 _ tboudreau 2003-07-01 17:15:31 UTC
Could I get some more data here?  If I am reading the 
attached numbers correctly, it looks like 2/3 of that
time is actually in the winsys.

There are two suspicious run() calls that suggest that
perhaps the threading/synchronization usage could be 
improved.

Was this test done on a cold-start (i.e. there was *no*
output window present until the debugger started)?

There is some stuff that looks a little peculiar, like the
RepaintTimer code - I don't know why the standard 
Swing repaint(nnn milliseconds) wasn't good enough.
Maybe some kind of thread-safety thing?

Comment 8 _ tboudreau 2003-07-09 07:14:02 UTC
FWIW, the more I read the code for OutputTabTerm, the more I am
convinced it should be written from scratch.  Reasons:

 - It tries to do way, way too much in one class, such as
    - Manage not one, but two term components (see known bugs
      with the invisible splitter getting focus)
    - Is responsible for pattern recognition/hyperlinking in
      weird ways - surely the thing that requests an output
      window be created knows better what should be matched and
      what should be done with it.  What is a method called
      "parseException" doing here?
    - Strange uses of threading for repainting, well, I'm trying
      to figure out what some of the other stuff is for

Better separation of concerns would be a big help to make this
class something more maintainable, or at least comprehensible.

Note the comment beginning:
// We use terminalemulator.ActiveTerm's ActiveRegions to simulate
// hyperlinks, but not in the obvious way.

Given that we own Term and we own OutputTabTerm, couldn't we do
something better than hacking around our own code?
Comment 9 _ tboudreau 2003-08-13 16:28:22 UTC
I was curious what was possible here, so I replaced 
OutputTabTerm with an intentionally naive implementation,
which:

 - Just uses a JTextArea, no Term (though presumably it
   should work equally well)
 - Overrides all of the methods of OutputWriter/PrintWriter
   that use synchronization to not use synchronization
 - Simply appends text to using JTextArea.append()
 - Used no synchronization or threading control except
   for the initial instantiation of the component

Then I wrote a test which starts three threads that 
simultaneously write to the output window in a loop.

The result:
No problems whatsoever.  No mixing of text simultaneously
added by different threads, or anything else.

Now, there probably is some synchronization code in the
Document object that text is being appended to, or something
such.  The point is, it's possible to do what OutputTabTerm
does very, very simply and have it work just fine.

I'll attach the example & test.
Comment 10 _ tboudreau 2003-08-13 16:30:39 UTC
Created attachment 11310 [details]
An extremely simplified OutputTabTerm
Comment 11 _ tboudreau 2003-08-13 16:31:23 UTC
Created attachment 11311 [details]
Simple test writing from three threads
Comment 12 _ tboudreau 2003-08-13 16:32:11 UTC
BTW, the simplified OTT & test were done on a multiprocessor machine.
Comment 13 _ tboudreau 2003-08-24 01:39:49 UTC
Changing topic from "Improve output window performance" to
rewrite OutputTabTerm.  Reasons enumerated above.
Comment 14 Jesse Glick 2003-09-18 06:11:40 UTC
Tim - umbrella issues *depend on*, not *block*, the subissues.
Comment 15 ivan 2003-12-06 03:59:40 UTC
Some history and suggestions on how to divide-and-conquer.
------------- part i ----------
If Term receives one character it calls repaint().
If you feed it one character at a time you get an expensive
repaint() per character. How do you solve this? You buffer.
When do you flush the buffer? When it's' full.
What about a half-full buffer? You need a timer to flush it.
That's what repaintTimer() is about (in OW).
But in principle we should be able to reduce repaints by using
	repaint(nnn milliseconds)
I'm about to commit Term where a timer of 20 msec is used and I've
had good experiences with it in standalong uses of Term.
I tried turning timerMode off but results were inconclusive.
------------ part ii -----------
Term needs to have chars added to it on the AWT EQ.
I used to use invokeAndWait() as a way of controlling input rate,
but that caused deadlocks. So we switched to invokeLater()
but that ended up swamping the EQ thread. 
So someone went and added. 'runnables' and syncObject.wait().
This is a scheme where a "job" gets put on the EQ and
grows bigger until it gets dispatched as opposed to each batch
of character inputs putting an event on the EQ.
I believe that the real reason for the swamping was that most stuff
in NB happens in RequestProcessors which are low priority, while
the io copier thread (dunno it's name) is at a regular 5/6 priority
and swamps everything else. So we might be abel to do away with 
the 'runnables' and ther attendant waits simply by altering the
priority of the copier thread to 1.

Comment 16 _ tboudreau 2004-03-01 13:42:57 UTC
FWIW, some preliminary work is in core/output on branch output_cleanup

Nothing exciting or even very functional yet, but some usable basics like cleanly splitting 
the GUI portion side of the universe from the InputOutput implementation, etc - this was 
causing problems in the new winsys, where the InputOutput impl is designed to be 
accessed from any thread and implemented on a winsys component where the winsys 
wants it to be accessed only from the EQ.

Probably the big open question is the best way to handle threading issues; what I've done 
thus far is to intentionally make it as thread-unsafe as possible (reimplement all 
synchronized methods of writer/printwriter to be unsynchronized); seems better to get it 
to the point where there are problems, and then handle the threading issues cleanly in one 
place.

What's there does work with multiple threads - it keeps a list of CharBuffers and allocates 
a new one for each call to println().  That's fine for long strings but won't fly for single char 
operations.  Periodically (on paint() and if close() or flush() is called), the GUI polls and 
siphons off the new data if any.

More interesting maybe is the possibility of polling to advise the Term that something may 
have changed, and the ability for the Term to do two things over a buffer:

 - A tail(linecount) function that will fetch a specified number of lines at the bottom of the 
buffer, so any buffer contents that have accrued since the last paint that don't actually 
need to be displayed don't need to be touched

 - A linecount() function over a buffer, where the Term is scrolled not to the bottom, and 
you just need a valid linecount to adjust the scroll model.  Keeping a running count of new 
\n's should be pretty cheap, but there may be corner cases that mean it won't work.

Prerequisite for either of these is a cleaner pipe between the Term and the output window, 
where instead of pushing lines into its internal buffer, you hand it a buffer and advise it 
when an interesting change may have occured, so it can poll and possibly redisplay.

Anyway, all this is just in the research phase right now.
Comment 17 _ tboudreau 2004-03-28 23:41:46 UTC
Created a branch output_rewrite.  Base tag is BLD200403281800
Comment 18 _ tboudreau 2004-05-26 15:10:36 UTC
Reassigning this issue back to myself.
Comment 19 _ tboudreau 2004-05-26 15:14:13 UTC
And closing.  core/output2 is now the default output window for NetBeans.  

TBD:  What to do with the existing issues on core/term and core/output.  The entire tree 
of open issues can be considered fixed with respect to NetBeans releases, but are not 
fixed in core/term or core/output, which are no longer used in NetBeans.
Comment 20 Quality Engineering 2008-12-23 08:40:39 UTC
moving terminal emulator issues to terminalemulator component.
To see the correct version and target milestone of this issue look at Issue
Activity table.