This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 212950 - Race condition in Lexer and infinite loop in SyntaxHighlighting
Summary: Race condition in Lexer and infinite loop in SyntaxHighlighting
Status: VERIFIED WORKSFORME
Alias: None
Product: editor
Classification: Unclassified
Component: Painting & Printing (show other bugs)
Version: 7.2
Hardware: PC Windows XP
: P3 normal (vote)
Assignee: Miloslav Metelka
URL:
Keywords: TEST, THREAD
Depends on:
Blocks:
 
Reported: 2012-05-24 12:06 UTC by Jiri Skrivanek
Modified: 2013-09-03 07:20 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Thread dump. (26.29 KB, text/plain)
2012-05-24 12:06 UTC, Jiri Skrivanek
Details
Corrupted token list (shows negative offsets of some already existing tokens IHC==Identity Hash Code of the tokens is the same) (19.31 KB, text/plain)
2012-07-02 12:06 UTC, Miloslav Metelka
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Skrivanek 2012-05-24 12:06:51 UTC
Created attachment 119813 [details]
Thread dump.

Deadlock between HintsController worker and org.netbeans.editor.CustomFoldManager appeared in automated tests while testing php editor.

Product Version         = NetBeans IDE Dev (Build 201205240002) (#fbd4c2288712)
Operating System        = Windows XP version 5.1 running on x86
Java; VM; Vendor        = 1.7.0_02; Java HotSpot(TM) Client VM 22.0-b10; Oracle Corporation
Runtime                 = Java(TM) SE Runtime Environment 1.7.0_02-b13
Comment 1 Jan Lahoda 2012-05-28 05:35:32 UTC
Not sure there is an actual deadlock: the AWT thread is runnable (and presumably also running), and both the "HintsController worker" and "org.netbeans.editor.CustomFoldManager" are waiting for it to finish. Maybe an infinite (or very long) loop?
Comment 2 Jan Lahoda 2012-05-28 05:36:37 UTC
BTW, the profiler snapshot might be useful in this case (as opposed to actual deadlocks, where thread dumps are typically more useful), to see the "dynamic" behavior.
Comment 3 Miloslav Metelka 2012-05-28 14:21:50 UTC
It also seems to me more like an infinite loop, I've done a change in SyntaxHighlighting so it might be affected although the present tests pass ok.
Is it reproducible?
Comment 4 Jiri Skrivanek 2012-05-31 13:40:49 UTC
Yes, it seems like infinite loop. Processor is at 99%. I can reproduce it on remote hudson slave but I can't get any profiler results from it.
Comment 5 Miloslav Metelka 2012-06-22 13:32:02 UTC
I went through the SyntaxHighlighting class code but I did not find any problematic case.
 Since this does happen in a real IDE use marking as P3 until Jirka provides a reproducible test case.
Comment 6 Jiri Skrivanek 2012-06-25 10:42:26 UTC
It can be reproduced with this hudson job:
http://hudson4qe.cz.oracle.com/job/_Temp-php.editor/
Comment 7 Miloslav Metelka 2012-06-29 12:33:00 UTC
It's an infinite loop. Output of the test shows:

[junit] SyntaxHighlighting.HSImpl <111,112>
    [junit] SH.moveTheSequence(): Highlight: <111,112> attrs=AttrSet[0,1]@12558708 S_NEXT_TOKEN, seq#=1
    [junit] SyntaxHighlighting.HSImpl <111,112>
    [junit] SH.moveTheSequence(): Highlight: <111,112> attrs=AttrSet[0,1]@12558708 S_NEXT_TOKEN, seq#=1
    [junit] SyntaxHighlighting.HSImpl <10,118>
    [junit] SH.moveTheSequence(): Highlight: <10,-1073741707> attrs=AttrSet[0,1]@11306677 S_NEXT_TOKEN, seq#=1
    [junit] SH.moveTheSequence(): Highlight: <-1073741707,-1073741707> attrs=AttrSet[0,1]@11306677 S_NEXT_TOKEN, seq#=1
    [junit] SH.moveTheSequence(): Highlight: <-1073741707,-1073741707> attrs=AttrSet[0,1]@11306677 S_NEXT_TOKEN, seq#=1
    [junit] SH.moveTheSequence(): Highlight: <-1073741707,-1073741707> attrs=AttrSet[0,1]@11306677 S_NEXT_TOKEN, seq#=1
    [junit] SH.moveTheSequence(): Highlight: <-1073741707,-1073741707> attrs=AttrSet[0,1]@11306677 S_NEXT_TOKEN, seq#=1
etc.

I will do some more research why this happens and possibly make SyntaxHighlighting more robust for this case.
Comment 8 Miloslav Metelka 2012-07-02 12:03:27 UTC
I have added an extra debug and there seems to be a racing condition that corrupts lexer tokens and because of that the SyntaxHighlighting then gets into an infinite loop.
Comment 9 Miloslav Metelka 2012-07-02 12:06:14 UTC
Created attachment 121650 [details]
Corrupted token list (shows negative offsets of some already existing tokens IHC==Identity Hash Code of the tokens is the same)
Comment 10 Miloslav Metelka 2012-07-02 13:09:21 UTC
There were several non-readlocked accesses mostly from typing hooks which I've fixed. I'll check whether it has any effect on this issue.
http://hg.netbeans.org/jet-main/rev/ed973b1fad76
Comment 11 Miloslav Metelka 2012-07-03 08:58:17 UTC
It looks that it helped. The test failed but this time it's likely due to too much logging output from the test causing it to time out. So I've disabled the logging and put an official jet-main build there and restarted the test.
Comment 12 Quality Engineering 2012-07-03 11:10:48 UTC
Integrated into 'main-golden', will be available in build *201207030834* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/ed973b1fad76
User: Miloslav Metelka <mmetelka@netbeans.org>
Log: #212950 - Race condition in Lexer and infinite loop in SyntaxHighlighting - fixed non read-locked lexer accesses.
Comment 13 Miloslav Metelka 2012-07-04 12:48:29 UTC
This fix makes the SyntaxColoring more robust against invalid offset values:
http://hg.netbeans.org/jet-main/rev/46e32441600f
I will re-run the test to test that.
Comment 14 Quality Engineering 2012-07-08 02:53:34 UTC
Integrated into 'main-golden', will be available in build *201207080002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/46e32441600f
User: Miloslav Metelka <mmetelka@netbeans.org>
Log: #212950 - Race condition in Lexer and infinite loop in SyntaxHighlighting - be more robust against invalid offsets from lexer.
Comment 15 Miloslav Metelka 2012-07-11 06:51:00 UTC
The test now does not timeout so marking as fixed.
Comment 16 Jiri Skrivanek 2012-07-18 11:35:05 UTC
The test now sometimes passes but usually it fails.
Comment 17 Jiri Skrivanek 2013-09-03 07:20:06 UTC
It is no more reproducible in build 20130902001, JDK7u40.