Bug 270963

Created attachment 164625 [details]
gdb log (compressed)

Product Version: NetBeans IDE Dev (Build 201706250001)
Java: 1.8.0_121; Java HotSpot(TM) 64-Bit Server VM 25.121-b13
Runtime: Java(TM) SE Runtime Environment 1.8.0_121-b13
System: Linux version 4.11.3-200.fc25.x86_64 running on amd64; UTF-8; en_US (nb)
User directory: /home/toddb/.netbeans/dev
Cache directory: /home/toddb/.cache/netbeans/dev

Have you looked at the GDB log? 

What I can see is there a tons of the messages:
*running,thread-id="all"
(gdb) 
~"\nProgram"
~" received signal SIGINT, Interrupt.\n"
~"0x00007ffff6bedaf0 in __read_nocancel () from /lib64/libc.so.6\n"
*stopped,reason="signal-received",signal-name="SIGINT",signal-meaning="Interrupt",frame={addr="0x00007ffff6bedaf0",func="__read_nocancel",args=[],from="/lib64/libc.so.6"},thread-id="1",stopped-threads="all",core="7"
59171-stack-list-frames --thread 1


Do you use gdb server?(https://sourceware.org/bugzilla/show_bug.cgi?id=18315) Do you have local or remote case?
Do you read stdin in your program?

If you do not need to caught SIGINT you can as a workaround try to mark this signal as ignored:
1) Start GDB session
2) Open project properties-> Debugger and select Ignored in Signals dialog. 

Let me know if it helps. And please provide more details. 

I do not think it has something to do with mouse outside IDE. At least GDB log says it has problems with SIGINT as I pointed above

> Have you looked at the GDB log? 
Yes.

> What I can see is there a tons of the messages:
Yes, its as if sigint is being generated from somewhere else. Its not from me. I'm just sitting there, watching the fireworks as the debugger is being driven by events from somewhere else.

> Do you use gdb server?
No

> Do you have local or remote case?
Local

> Do you read stdin in your program?
Yes, but not when this occurred. The program is doing some very long computations, after which it will read stdin for commands. And eventually it got
to that point, would pause briefly on the read of stdin, and then continue.

> select Ignored in Signals dialog.
I could do that. However, I believe that gdb uses sigint to stop a running process. I don't know if that will be a problem; e.g. unable to stop a running process.

Does NB use sigint to interrupt gdb? If so, could it be that NB is incorrectly generating sigint?

(In reply to tbrunhoff from comment #4)
> > Have you looked at the GDB log? 
> Yes.
> 
> > What I can see is there a tons of the messages:
> Yes, its as if sigint is being generated from somewhere else. Its not from
> me. I'm just sitting there, watching the fireworks as the debugger is being
> driven by events from somewhere else.
> 
> > Do you use gdb server?
> No
> 
> > Do you have local or remote case?
> Local
> 
> > Do you read stdin in your program?
> Yes, but not when this occurred. The program is doing some very long
> computations, after which it will read stdin for commands. And eventually it
> got
> to that point, would pause briefly on the read of stdin, and then continue.
> 
> > select Ignored in Signals dialog.
> I could do that. However, I believe that gdb uses sigint to stop a running
> process. I don't know if that will be a problem; e.g. unable to stop a
> running process.

Did it help?
> 
> Does NB use sigint to interrupt gdb? If so, could it be that NB is
> incorrectly generating sigint?

No, we do not use signals for gdb on Linux. We use MI interface and just send the command when you push Pause button for example. 

I need some time to investigate the problem.  But please give me more details:
have you changed the GDB version? updated libc recentely?

> Did it help?
I'm sorry, I'm working on a different system today.

> have you changed the GDB version? updated libc recentely?
Sort of.  One system is running Fedora 21 and another is running Fedora 25, but both systems have seen this issue.

> have you changed the GDB version? updated libc recentely?
Both of my systems are now running a vanilla Fedora 25. Failures (where gdb 'continues' the program without user input and any future breakpoints are hit and continued, also without user input.

GNU gdb (GDB) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".

Product Version: NetBeans IDE Dev (Build 201706250001)
Java: 1.8.0_121; Java HotSpot(TM) 64-Bit Server VM 25.121-b13
Runtime: Java(TM) SE Runtime Environment 1.8.0_121-b13
System: Linux version 4.11.5-200.fc25.x86_64 running on amd64; UTF-8; en_US (nb)
User directory: /home/toddb/.netbeans/dev
Cache directory: /home/toddb/.cache/netbeans/dev

Today I'm working on a complex executable with 15 threads, and heavy use of mutexes and semaphores.

One scenario occurs when I step through a mutex, and I lose control instantly. Also, I cannot stop the debug session without exiting the ide.

Stepping may be one of the more important components to repeating this.  Generally I find my self avoiding stepping, and simply setting breakpoints and using continue.

Another scenario is when I hit continue with several breakpoints set and the debugger continues without stopping at any breakpoint. And of course the gdb log is gone by then. Generally, continue and breakpoints work fine.  I cannot predict when this will happen.

I don't know if this is relevant, but frequently I have 2 or 3 projects open which share the same code. And even if I am careful to open source files within the project that is being debugged, the breakpoints end up listed with one of the other projects. Also note that breakpoints associated with a project that gets closed remain.

At this point, this may be the best I can do without more guidance from you on how to find this problem.

> Can you close Variables/Watches/Debugging windows and tell me if it helped.
It does not help.

*** Bug 271010 has been marked as a duplicate of this bug. ***

I think, I've got some scenario...
Enabled Breakpoint is not hit very frequently if I have visible pinned watch in editor.
Enabled Breakpoint is not hit not that frequently if I have visible Variables/Watches view and no pinned watches at all.

I think it might be the consequence of the fix done for Bug #269968 "debugger continues from unexpected breakpoint place after tooltip evaluation".

It seems that now breakpoints are enabled too late, so when there are number of user-set breakpoints for debugger it takes time to enable them all back.
So, when continue is invoked it by-passes not yet re-activated breakpoint.

Todd, could you, please, open Window->Debugging->Watches and in it's context menu enable "Show pinned watches". 
I had situations when there was forgotten pinned watch which caused the issue.
Because pinned watch tries to update it's value as well and for that all breakpoints are disabled

Maria, For your record:
Recently I've got very strange behavior:
Session stopped on breakpoint 
(BitcodeWriter.Emit(uint32_t Val, unsigned NumBits)
and I switched to another activity.
Then come back to NB and pressed continue.
After hit in the same method, opened Variables (only 4 variables are displayed).

Then pressed continue and saw how IDE opens different places where I have breakpoints and blinks in Variables view and continues itself to some other breakpoint, blinks again, and so on and so on.

This auto-continue stopped only when I closed Variables view.

Created attachment 164812 [details]
log when IDE auto-continues from one breakpoint to another

In the attached log there is:
- some info where you can try find problems why auto-continue 
- then I disabled all breakpoints in file /home/vv159170/devarea/LLVM39/llvm/include/llvm/Bitcode/BitstreamWriter.h
- then enabled one at line 107
- then re-run debug session
- then took log file and attached here

Hope it helps

One more observation:
- when I select some expression in editor to see it's value 
i.e. 'Out.size()' in BitstreamWriter.h:
size_t GetBufferOffset() const { return Out.size(); }
tooltip doesn't appear after 10 secs, so I decide to continue
and after that nothing happens (I have unchecked mode "Finish debuggins session When program exits")
it looks like program finishes, because breakpoints were disabled for evaluation of expression when I pressed continue.

(In reply to tbrunhoff from comment #9)
> > Can you close Variables/Watches/Debugging windows and tell me if it helped.
> It does not help.

Todd, I am om vacation now and will return to work next Monday. Will do my best to fix debugger bugs.

==Maria

(In reply to Vladimir Voskresensky from comment #16)
> One more observation:
> - when I select some expression in editor to see it's value 
> i.e. 'Out.size()' in BitstreamWriter.h:
> size_t GetBufferOffset() const { return Out.size(); }
> tooltip doesn't appear after 10 secs, so I decide to continue
> and after that nothing happens (I have unchecked mode "Finish debuggins
> session When program exits")
> it looks like program finishes, because breakpoints were disabled for
> evaluation of expression when I pressed continue.
Gues about the reason:
When some expression is chosen to be shown in tooltip and gdb can not evaluate it => we never enable breakpoints back: the extract from the log:
915-break-disable
916-gdb-set unwindonsignal on
915^done
(gdb) 
916^done
(gdb) 
917-data-evaluate-expression "FDI->second->DeclIDs.size()"
917^error,msg="Cannot evaluate function -- may be inlined"
(gdb) 
918-break-disable
919-gdb-set unwindonsignal on
918^done
(gdb) 
919^done
(gdb) 
920-data-evaluate-expression "FDI->second->DeclIDs.size()"
920^error,msg="Cannot evaluate function -- may be inlined"
(gdb) 

And there is no paired -break-enable

Created attachment 164815 [details]
patch_v1

Please try this patch

(In reply to Vladimir Voskresensky from comment #18)
> (In reply to Vladimir Voskresensky from comment #16)
> > One more observation:
> > - when I select some expression in editor to see it's value 
> > i.e. 'Out.size()' in BitstreamWriter.h:
> > size_t GetBufferOffset() const { return Out.size(); }
> > tooltip doesn't appear after 10 secs, so I decide to continue
> > and after that nothing happens (I have unchecked mode "Finish debuggins
> > session When program exits")
> > it looks like program finishes, because breakpoints were disabled for
> > evaluation of expression when I pressed continue.
> Gues about the reason:
> When some expression is chosen to be shown in tooltip and gdb can not
> evaluate it => we never enable breakpoints back: the extract from the log:
> 915-break-disable
> 916-gdb-set unwindonsignal on
> 915^done
> (gdb) 
> 916^done
> (gdb) 
> 917-data-evaluate-expression "FDI->second->DeclIDs.size()"
> 917^error,msg="Cannot evaluate function -- may be inlined"
> (gdb) 
> 918-break-disable
> 919-gdb-set unwindonsignal on
> 918^done
> (gdb) 
> 919^done
> (gdb) 
> 920-data-evaluate-expression "FDI->second->DeclIDs.size()"
> 920^error,msg="Cannot evaluate function -- may be inlined"
> (gdb) 
> 
> And there is no paired -break-enable

please try patch attached

> open Window->Debugging->Watches and in it's context menu enable "Show pinned watches".
Today, the debugging sessions started doing the same thing: continuing without user input.  Your description of breakpoints being set too late certainly matches the symptoms I see.

So I marked watches as "show pinned watches". That did not change anything that I could see in the ide. However, assuming that it would have caused the evaluation with breakpoints disabled, I tried more debugging sessions, with (a) all watches disabled, then (b) watch window closed, then (c) variables window closed. Each step seemed to lessen the number of "continue without user input".

On the other hand, eventually the "continue without user input" ceased altogether over several debug runs. So I added opened the variables and watches windows, added some watches, and now I am not running into any issues. I suppose that is possible since this is clearly a timing issue.

One thing that might be a partial workaround, is for the ide to be aware of whether watches are function calls, because variables (in c/c++) can always be evaluated in gdb without executing code. I admit this could get complicated when evaluating an expression that looks like a class member, but where the usage invokes an overload. Just an idea.

And I can get back to the issue by adding more breakpoints. That would make sense if enabling each breakpoint takes some amount of time. On the other hand, cutting back to one breakpoint I still run into the issue.

So perhaps it has nothing to do with the number of breakpoints. Very frustrating.

Ah.... So closing all evaluation windows (variables, watches, stack) appears to address the race between continuing and re-enabling the breakpoints.

I've been fighting with the ide and debugging for the last 15 minutes where all breakpoints have been ignored, even just one. Finally, when I closed all the evaluation windows, it stops on the breakpoints.

I got to the point where 5 breakpoints and variables/watches/stack windows all closed, the breakpoints worked.  If I added one more breakpoint, it would reliably continue without user input.

I've downgraded to netbeans-trunk-nightly-201702170002-cpp-linux-x64.sh, which fell before the fix described in Bug #269968 was released on 2017-03-07. That seems to work.

If your new proposed patch doesn't fix this, please consider backing out the older one from Bug #269968.

Please enable all breakpoints before debug  if any (need to do it once when you will get this fix)

changeset:   304371:eb8703fcad6f
tag:         tip
user:        Maria Dalmatova <mromashova@netbeans.org>
date:        Tue Jul 25 09:27:00 2017 +0300
summary:     patch for bz#270963 - gdb continues automatically and bz#270912 - Netbeans disables all breakpoints when starting to run in Debug mode

(In reply to Maria Tishkova from comment #25)
> Please enable all breakpoints before debug  if any (need to do it once when
> you will get this fix)
> 
> changeset:   304371:eb8703fcad6f
> tag:         tip
> user:        Maria Dalmatova <mromashova@netbeans.org>
> date:        Tue Jul 25 09:27:00 2017 +0300
> summary:     patch for bz#270963 - gdb continues automatically and bz#270912
> - Netbeans disables all breakpoints when starting to run in Debug mode

Please, remove unused method with one runnable

I'm testing today's build with the fix. We need to provide patch for broken 8.2

Integrated into 'main-silver', will be available in build *201707260001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/eb8703fcad6f
User: Maria Dalmatova <mromashova@netbeans.org>
Log: patch for bz#270963 - gdb continues automatically and bz#270912 - Netbeans disables all breakpoints when starting to run in Debug mode

Please comment if it works for you

I haven't intensive debugging after that, but may be Todd can comment?

Btw, can we add time-stamp to gdb log entries?
Because it looks like we send some commands which are very expensive for gdb to execute. With timing we can see when command was send and when reply was received.
May be we need to search for alternative MI-commands or shortcuts (like disable-bps/evaluate/enable-bps).

Also it is not clear when tooltip evaluation is finished and failed. Can we show gdb error message in tooltip when it failed?
I know that until it is not finished (with any result OK/Failed). I can not do anything else in IDE with debugger.

> comment if it works for you
I did some work Thursday and Friday, and had no issues. Looks like I have some debugging ahead of me today.

> can we add time-stamp to gdb log entries?
I like that. This should make it more obvious when stepping takes a long time.

When I have CallStack Window and Watches window open 
then IDE after "continue" jumps from breakpoint to breakpoint again.
Watches has 1 entry for printing P->getPassName() (helpful for PMTopLevelManager::schedulePass method)

Created attachment 164911 [details]
gdb log with jumps

(In reply to Vladimir Voskresensky from comment #33)
> When I have CallStack Window and Watches window open 
> then IDE after "continue" jumps from breakpoint to breakpoint again.
> Watches has 1 entry for printing P->getPassName() (helpful for
> PMTopLevelManager::schedulePass method)

What do you mean by "jump"?

(In reply to Maria Tishkova from comment #35)
> (In reply to Vladimir Voskresensky from comment #33)
> > When I have CallStack Window and Watches window open 
> > then IDE after "continue" jumps from breakpoint to breakpoint again.
> > Watches has 1 entry for printing P->getPassName() (helpful for
> > PMTopLevelManager::schedulePass method)
> 
> What do you mean by "jump"?
Probably the same behaviour as https://netbeans.org/bugzilla/show_bug.cgi?id=270963#c13 but with minimal skips, so:
- IDE opens editor at really the next expected breakpoint
- but self-continue (jumps) to another document and stops in this case on the next breakpoint.

changeset:   304439:65beb5b68407
tag:         tip
user:        Maria Dalmatova <mromashova@netbeans.org>
date:        Fri Aug 11 12:29:08 2017 +0300
summary:    fix for bz#270963 - gdb continues automatically and bz#270070 - significant slowdown when debugging (was: editing certain files)
1) fixed problem with jumping when breakpoint hitted
2) minimized enabling/disabling breakpoints for data expression evaluation that speed up stepping
3) show tooltip with OUT_OF_SCOPT value if selected expression is incorrect

I will mark it as resolved, please test the fix and re-open if still re-producible

Integrated into 'main-silver', will be available in build *201708120001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/65beb5b68407
User: Maria Dalmatova <mromashova@netbeans.org>
Log: fix for bz#270963 - gdb continues automatically and bz#270070 - significant slowdown when debugging (was: editing certain files)
1) fixed problem with jumping  when breakpoint hitted
2) minimized enabling/disabling breakpoints for data expression evaluation that speed up stepping
3) show tooltip with OUT_OF_SCOPT value if selected expression is incorrect