This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 125313 - GroovyLexer.nextToken() swallows last doublequote in input
Summary: GroovyLexer.nextToken() swallows last doublequote in input
Status: VERIFIED FIXED
Alias: None
Product: groovy
Classification: Unclassified
Component: Editor (show other bugs)
Version: 6.x
Hardware: All All
: P1 blocker (vote)
Assignee: schmidtm
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-01-15 17:43 UTC by schmidtm
Modified: 2008-10-30 14:25 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description schmidtm 2008-01-15 17:43:55 UTC
Hi,

The GroovyLexer swallows the very last doublequote (") from the input stream. This means the last Token is one char
shorter than it should be.

These are the tests which expose this error (in GroovyLexerBatchTest.java):

testGstringLexing4
testMultipleStringConstants
testMultipleStringConstantsClosed
testMultipleStringConstantsDosTerminated
testLengthTest2
testLengthTest5
testQuotes

One sidenode: If one changes in GroovyLexer.java in (as of this writing) Line 115 : lexerInput.readLength()
into lexerInput.readLengthEOF()


   109      public Token<GroovyTokenId> nextToken() {
   110          try {
   111              groovyjarjarantlr.Token antlrToken = scanner.nextToken();
   112              if (antlrToken != null) {
   113                  int intId = antlrToken.getType();
   114
   115                  int len = lexerInput.readLengthEOF() - myCharBuffer.getExtraCharCount();
   116                  if ( antlrToken.getText() != null ) {
   117                      len = Math.max( len, antlrToken.getText().length() );

all the tests succeed but then i get a reproducible exception in the editor:

WARNING [org.netbeans.modules.editor.lib2.highlighting.CompoundHighlightsContainer]: The layer failed to supply highligh
ts: org.netbeans.modules.editor.lib2.highlighting.SyntaxHighlighting@111cdd6
java.lang.IllegalStateException: Lexer org.netbeans.modules.groovy.editor.lexer.GroovyLexer@116a2ac returned null token
but lexerInput.readLength()=1 - these characters need to be tokenized. Fix the lexer.
        at org.netbeans.lib.lexer.LexerUtilsConstants.checkLexerInputFinished(LexerUtilsConstants.java:116)
        at org.netbeans.lib.lexer.LexerInputOperation.nextToken(LexerInputOperation.java:276)
        at org.netbeans.lib.lexer.inc.IncTokenList.tokenOrEmbeddingContainerImpl(IncTokenList.java:197)
        at org.netbeans.lib.lexer.inc.IncTokenList.tokenOrEmbeddingContainer(IncTokenList.java:192)
        at org.netbeans.api.lexer.TokenSequence.moveNext(TokenSequence.java:405)
        at org.netbeans.modules.editor.lib2.highlighting.SyntaxHighlighting$HSImpl.moveTheSequence(SyntaxHighlighting.ja
va:541)
        at org.netbeans.modules.editor.lib2.highlighting.SyntaxHighlighting$HSImpl.moveNext(SyntaxHighlighting.java:285)

        at org.netbeans.spi.editor.highlighting.support.OffsetsBag.addAllHighlightsImpl(OffsetsBag.java:574)
Comment 1 martin_adamek 2008-01-16 07:10:57 UTC
See issue 124912. Problem was fixed there I guess.
Comment 2 Lukas Jungmann 2008-09-29 12:01:24 UTC
v.