This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 180013 - remote gdb hangs at startup & "unrecognized line:" messages.
Summary: remote gdb hangs at startup & "unrecognized line:" messages.
Status: STARTED
Alias: None
Product: third-party
Classification: Unclassified
Component: DBX-Gui (show other bugs)
Version: -S1S-
Hardware: Sun Solaris
: P4 normal (vote)
Assignee: ivan
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-29 18:35 UTC by ivan
Modified: 2010-06-10 09:16 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ivan 2010-01-29 18:35:28 UTC
When running gdb remotely it's startup sometimes hangs with the
console only showing:

~"GNU gdb 6.6\n"
~"Copyright (C) 2006 Free Software Foundation, Inc.\n"
~"GDB is free software, covered by the GNU General Public License, and you are\n"
~"welcome to change it and/or distribute copies of it under certain conditions.\n"
~"Type \"show copying\" to see the conditions.\n"

If it doesn't hang then every command issued work but the console also
shows (in blue):
    unrecognized line: <echo of the entered command>
Comment 1 ivan 2010-01-29 18:40:52 UTC
Both of these are happening because remote gdb execution (via spawn)
uses full pty's. The initial terminal height may be too low so
gdbs readline "more mode" kicks in and asks:

"---Type <return> to continue, or q <return> to quit---

while MIPRoxy is waiting for a prompt and will not answer this question.
Deadlock ensues.

The "unrecognized line" messages come from MIProxy.processLine()
the default case. What's happening here is that pty's have echoing
enabled by default and it's this echoed line which doesn't get
processed correctly. (Although this is harmless).

I dunno how soon the switch to dlight.nativeexecution will happen
but gdb mode needs to not atrophy until then.

The fix is to enhance the spawn protocol to allow the creation
of raw pty's.
Comment 2 ivan 2010-02-03 20:46:58 UTC
details:   http://lessing.sfbay.sun.com/hg/toolshg/rev/f708d9d903f2

Quick workaround. Force initial Term size to be 24.
Comment 3 ivan 2010-02-19 16:30:45 UTC
Fixes in t-common:
        - spawn.gp enhanced to pass a bool 'rawpty' flag to start(),
          start_cmd() and new_pty() glue messages.
        - spawn_svc now honors these flags.
          In the case of Solaris we just don't push ptem/ldterm/ttcompat
          In the case of Linux we have to use cfmakeraw() because pty's
          are cooked by default.
        - Realized that base.so's UnixPty also ignores raw mode on linux
          so added cfmakeraw() there as well.

        Compatibility: if an older client talks to this spawn_svc or a
        a client using these changes talks to an older spawn_svc
        pty rawness behaviour will be as before ... i.e. as if raw
        flag never existed.

        Turns out "not pushing ldterm etc on solaris" is not exactly
        the same as "using cfmakeraw() on lunix". cfmakeraw() still
        causes isatty() in the child process to return true so
        gdb still goes into more more. We do get rid of the echoes though.

        The proper fix would be to not use pty's at all, just pipes
        or fifos but I'm short of time and we'll be switching to
        CND's remote development soon anyway.

Fixes in dbxgui
http://lessing.sfbay.sun.com/hg/toolshg/rev/86c0f92e73c7
http://lessing.sfbay.sun.com/hg/toolshg/rev/8d4a01c7050e
    This requires a corresponding fix from t-common.
    shelf.zip has been updated.

    The spawn protocol now takes a 'rawpty' flag for start(), start_cmd() and
    new_pty() so now ExecutorUnix.startEngine() passes this flag which
    it gets from TermComponent.

    ExecutorUnix.startShellCmdRemoteViaGlue() sets 'rawpty' to true
    by default. This pathway is mainly used for executing the
    'ps' command during remote attaches.

    In the case of ExecutorUnix.startIO() we hardcode 'rawpty' to false
    because we always want pty's for pio.

    Undid the previous workaround of setting cells.height = 24

    When we create a remote console for gdb, in GdbIOPack.setup(),
    we now pass TermComponent.RAW_PTY.

    In order for isRaw() to work with a generic TermComponent factored
    flag stuff from TermComponentUnix into TermComponent and added
    predicates for testing the flags: isRaw(), isActive() and isPacketMode().

    All of these changes still make sense even if the ultimate implementation
    in spawn & base still has to go to using pipes or somesuch.