This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 116939 - java.util.ConcurrentModificationException: Caller uses token sequence which is no longer valid. Underlying token hierarchy has been modified: modCount=1 != upToDateModCount=3
Summary: java.util.ConcurrentModificationException: Caller uses token sequence which i...
Status: VERIFIED FIXED
Alias: None
Product: javaee
Classification: Unclassified
Component: Code (show other bugs)
Version: 6.x
Hardware: All All
: P3 blocker (vote)
Assignee: Marek Fukala
URL: http://statistics.netbeans.org/except...
Keywords:
: 119462 119755 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-09-27 13:22 UTC by Jiri Skrivanek
Modified: 2007-10-30 11:12 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter: 2767

Attachments
stacktrace (5.71 KB, text/plain)
2007-09-27 13:22 UTC, Jiri Skrivanek 		Details
Different exception from EmbeddingUpdater appearing while running automated tests. (3.76 KB, text/plain)
2007-09-27 13:27 UTC, Jiri Skrivanek 		Details
Stack trace from EmbeddingUpdater. (3.47 KB, text/plain)
2007-10-23 11:30 UTC, Jiri Skrivanek 		Details
Stack trace from SyntaxHighlighting. (7.76 KB, text/plain)
2007-10-23 11:31 UTC, Jiri Skrivanek 		Details
Stack trace from SyntaxParser. (1.99 KB, text/plain)
2007-10-23 11:32 UTC, Jiri Skrivanek 		Details
Stacktrace of the THC.rebuild() call (1.08 KB, text/plain)
2007-10-24 09:05 UTC, Miloslav Metelka 		Details
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Skrivanek 2007-09-27 13:22:41 UTC 
Build: NetBeans IDE Dev (Build 200709270433)
VM: Java HotSpot(TM) Client VM, 1.6.0_03-b02
OS: Windows XP, 5.1, x86

User Comments: 
To reproduce manually:

- add glassfish server in Services view
- open project nb_all\web\jspdebug\test\qa-functional\data\MainTestApplication
- open index.jsp and you get the exception
Comment 1 Jiri Skrivanek 2007-09-27 13:22:46 UTC 
Created attachment 49673 [details]
stacktrace
Comment 2 Jiri Skrivanek 2007-09-27 13:27:02 UTC 
Created attachment 49674 [details]
Different exception from EmbeddingUpdater appearing while running automated tests.
Comment 3 Miloslav Metelka 2007-10-22 10:10:41 UTC 
I have improved synchronization - the TokenSequence's constructor takes IncTokenList.modCount() which was not
synchronized. I have fixed that.
Checking in EmbeddedTokenList.java;
/cvs/lexer/src/org/netbeans/lib/lexer/EmbeddedTokenList.java,v  <--  EmbeddedTokenList.java
new revision: 1.19; previous revision: 1.18
done
Checking in inc/IncTokenList.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/IncTokenList.java,v  <--  IncTokenList.java
new revision: 1.15; previous revision: 1.14
done
I'm wondering whether this shouldn't be integrated into beta2 as well. The fix is simple and the extra synchronization
should not cause any deadlocks or other problems. QE please decide.
Comment 4 Jiri Skrivanek 2007-10-23 11:25:17 UTC 
I cannot reproduce it manually but it still appears in automated tests:

cd web/jspdebug/test
ant -Dj2ee.appserver.path=<path_to_glassfish>

I will attach 3 different stack traces which I got.
Comment 5 Jiri Skrivanek 2007-10-23 11:30:40 UTC 
Created attachment 51480 [details]
Stack trace from EmbeddingUpdater.
Comment 6 Jiri Skrivanek 2007-10-23 11:31:17 UTC 
Created attachment 51481 [details]
Stack trace from SyntaxHighlighting.
Comment 7 Jiri Skrivanek 2007-10-23 11:32:50 UTC 
Created attachment 51482 [details]
Stack trace from SyntaxParser.
Comment 8 Marek Fukala 2007-10-23 11:59:24 UTC 
fixed the ISE from SyntaxParser

Checking in SyntaxParser.java;
/cvs/html/editor/lib/src/org/netbeans/editor/ext/html/parser/SyntaxParser.java,v  <--  SyntaxParser.java
new revision: 1.15; previous revision: 1.14
done
Comment 9 Miloslav Metelka 2007-10-24 09:03:22 UTC 
After hours of unsuccessful searching I've finally realized that in this particular case the TokenHierarchy's modCount
is not incremented by document modifications but by TokenHierarchyControl.rebuild() being called by
org.netbeans.modules.web.core.syntax.JSPKit$LexerColoringListener.recolor(JSPKit.java:154)

Rebuilding TH increments the hierarchy's modCount to invalidate all tokenSequences since all the tokens are being thrown
away and recreated. To ensure that existing TSes don't fail it's necessary to call THC.rebuild() under document's
write-lock. Reassigning to Marek.

I've just made a one-line commit in lexer too since the rebuild() actually incremented TH's modCount twice so I've
removed the one extra increment.
/cvs/lexer/src/org/netbeans/lib/lexer/TokenHierarchyOperation.java,v  <--  TokenHierarchyOperation.java
new revision: 1.27; previous revision: 1.26
Comment 10 Miloslav Metelka 2007-10-24 09:05:29 UTC 
Created attachment 51550 [details]
Stacktrace of the THC.rebuild() call
Comment 11 Miloslav Metelka 2007-10-24 09:14:23 UTC 
*** Issue 119462 has been marked as a duplicate of this issue. ***
Comment 12 Marek Fukala 2007-10-24 09:32:47 UTC 
fixed. Milo, please verify the fix.

BTW, I would really appreciate if we had non-blocking NbEditorDocument.isRead/WriteLocked() so we can put some
reasonable tests in the code supposing not/being run under the lock.

Checking in JSPKit.java;
/cvs/web/jspsyntax/src/org/netbeans/modules/web/core/syntax/JSPKit.java,v  <--  JSPKit.java
new revision: 1.52; previous revision: 1.51
done
Comment 13 Miloslav Metelka 2007-10-24 10:01:11 UTC 
Thanks, Marek, I'll check the fix.
There is editor/util/.../DocumentUtilities.isRead/WriteLocked(doc) already but see the javadoc first how they operate
(we are limited by AbstractDocument's final read/writeLock() methods).
Comment 14 Marek Fukala 2007-10-24 10:13:43 UTC 
Thanks. As for the isR/WLocked() - it was ment for you(lexer) mainly - if there was such check in the infr., we could
find such issues faster probably.
Comment 15 Miloslav Metelka 2007-10-24 11:06:54 UTC 
Yes, I plan to do similar thing as part of the Lexer API cleanup that I'm just going to do. A check that could be turned
on as part of the logging and would ensure that there are appropriate locks.
Comment 16 Jiri Skrivanek 2007-10-26 10:21:44 UTC 
Verified.
Comment 17 Miloslav Metelka 2007-10-30 11:12:18 UTC 
*** Issue 119755 has been marked as a duplicate of this issue. ***