This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 93644

Summary:	Fix incremental lexing for embedded sections
Product:	editor	Reporter:	Miloslav Metelka <mmetelka>
Component:	Lexer	Assignee:	issues@editor <issues>
Status:	RESOLVED FIXED
Severity:	blocker	CC:	mfukala, pbuzek, pjiricka, vstejskal
Priority:	P1
Version:	6.x
Hardware:	PC
OS:	Linux
Issue Type:	DEFECT	Exception Reporter:
Bug Depends on:
Bug Blocks:	99229, 99946
Attachments:	html file

Description Miloslav Metelka 2007-01-31 09:10:06 UTC

Currently the incrementality is only used for top-level language of a mutable
input. If a top-level token gets relexed its possible embedded section token
list gets lost. Instead the embedded token list needs to be fixed by incremental
algorithm.

Comment 1 Miloslav Metelka 2007-03-22 09:41:59 UTC

I should have the fix ready during today hopefully before noon.

Comment 2 Miloslav Metelka 2007-03-23 14:28:22 UTC

Fixed in trunk:
Checking in src/org/netbeans/api/lexer/TokenChange.java;
/cvs/lexer/src/org/netbeans/api/lexer/TokenChange.java,v  <--  TokenChange.java
new revision: 1.7; previous revision: 1.6
done
Checking in src/org/netbeans/api/lexer/TokenHierarchyEvent.java;
/cvs/lexer/src/org/netbeans/api/lexer/TokenHierarchyEvent.java,v  <-- 
TokenHierarchyEvent.java
new revision: 1.7; previous revision: 1.6
done
Checking in src/org/netbeans/lib/lexer/inc/TokenListUpdater.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenListUpdater.java,v  <-- 
TokenListUpdater.java
new revision: 1.7; previous revision: 1.6
done
Checking in src/org/netbeans/lib/lexer/inc/TokenListChange.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenListChange.java,v  <-- 
TokenListChange.java
new revision: 1.5; previous revision: 1.4
done
Checking in src/org/netbeans/lib/lexer/inc/IncTokenList.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/IncTokenList.java,v  <-- 
IncTokenList.java
new revision: 1.7; previous revision: 1.6
done
Checking in src/org/netbeans/lib/lexer/inc/TokenChangeInfo.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenChangeInfo.java,v  <-- 
TokenChangeInfo.java
new revision: 1.4; previous revision: 1.3
done
Checking in src/org/netbeans/lib/lexer/inc/TokenHierarchyEventInfo.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenHierarchyEventInfo.java,v  <-- 
TokenHierarchyEventInfo.java
new revision: 1.3; previous revision: 1.2
done
Checking in api/apichanges.xml;
/cvs/lexer/api/apichanges.xml,v  <--  apichanges.xml
new revision: 1.15; previous revision: 1.14
done
Checking in nbproject/project.properties;
/cvs/lexer/nbproject/project.properties,v  <--  project.properties
new revision: 1.9; previous revision: 1.8
done
Checking in src/org/netbeans/lib/lexer/TokenHierarchyOperation.java;
/cvs/lexer/src/org/netbeans/lib/lexer/TokenHierarchyOperation.java,v  <-- 
TokenHierarchyOperation.java
new revision: 1.8; previous revision: 1.7
done
Checking in src/org/netbeans/lib/lexer/EmbeddingContainer.java;
/cvs/lexer/src/org/netbeans/lib/lexer/EmbeddingContainer.java,v  <-- 
EmbeddingContainer.java
new revision: 1.5; previous revision: 1.4
done
Checking in src/org/netbeans/lib/lexer/EmbeddedTokenList.java;
/cvs/lexer/src/org/netbeans/lib/lexer/EmbeddedTokenList.java,v  <-- 
EmbeddedTokenList.java
new revision: 1.4; previous revision: 1.3
done
Checking in test/unit/src/org/netbeans/lib/lexer/test/LexerTestUtilities.java;
/cvs/lexer/test/unit/src/org/netbeans/lib/lexer/test/LexerTestUtilities.java,v 
<--  LexerTestUtilities.java
new revision: 1.10; previous revision: 1.9
done
RCS file:
/cvs/lexer/test/unit/src/org/netbeans/lib/lexer/inc/EmbeddingUpdateTest.java,v
done
Checking in test/unit/src/org/netbeans/lib/lexer/inc/EmbeddingUpdateTest.java;
/cvs/lexer/test/unit/src/org/netbeans/lib/lexer/inc/EmbeddingUpdateTest.java,v 
<--  EmbeddingUpdateTest.java
initial revision: 1.1

Comment 3 Lukasz Grela 2007-04-03 13:46:24 UTC

I still can reproduce this bug, it not occur always, it seems that it depends on
content of html file. I attached a html file to reproduce this bug, if you
remove <form> tag with content from attached file the bug will not occur
anymore. I also increase priority as it is very annoying bug.

Comment 4 Lukasz Grela 2007-04-03 13:47:12 UTC

Created attachment 40323 [details]
html file

Comment 5 Vitezslav Stejskal 2007-04-05 00:18:13 UTC

There have been some problems reported as issue #99229 and issue #99946 that
seem to be caused by incremental lexing of embedded tokens.

Comment 6 Miloslav Metelka 2007-04-05 14:13:42 UTC

There was an error in the token list updater that caused more than one token to
be relexed which prevented optimized updating of the embedded sections. It is
now fixed.
The token list modifications can now be logged by
-J-Dorg.netbeans.lib.lexer.TokenHierarchyOperation.level=FINE
including embedded sections changes (property mentioned in arch doc as well).

Checking in src/org/netbeans/lib/lexer/inc/TokenListUpdater.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenListUpdater.java,v  <-- 
TokenListUpdater.java
new revision: 1.9; previous revision: 1.8
done
Checking in src/org/netbeans/lib/lexer/inc/TokenHierarchyEventInfo.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenHierarchyEventInfo.java,v  <-- 
TokenHierarchyEventInfo.java
new revision: 1.4; previous revision: 1.3
done
Checking in src/org/netbeans/lib/lexer/inc/TokenListChange.java;
/cvs/lexer/src/org/netbeans/lib/lexer/inc/TokenListChange.java,v  <-- 
TokenListChange.java
new revision: 1.6; previous revision: 1.5
done
Checking in src/org/netbeans/lib/lexer/token/AbstractToken.java;
/cvs/lexer/src/org/netbeans/lib/lexer/token/AbstractToken.java,v  <-- 
AbstractToken.java
new revision: 1.6; previous revision: 1.5
done
Checking in arch.xml;
/cvs/lexer/arch.xml,v  <--  arch.xml
new revision: 1.10; previous revision: 1.9
done
Checking in src/org/netbeans/lib/lexer/LexerUtilsConstants.java;
/cvs/lexer/src/org/netbeans/lib/lexer/LexerUtilsConstants.java,v  <-- 
LexerUtilsConstants.java
new revision: 1.9; previous revision: 1.8
done
Checking in src/org/netbeans/lib/lexer/EmbeddingContainer.java;
/cvs/lexer/src/org/netbeans/lib/lexer/EmbeddingContainer.java,v  <-- 
EmbeddingContainer.java
new revision: 1.6; previous revision: 1.5
done
Checking in src/org/netbeans/lib/lexer/TokenHierarchyOperation.java;
/cvs/lexer/src/org/netbeans/lib/lexer/TokenHierarchyOperation.java,v  <-- 
TokenHierarchyOperation.java
new revision: 1.9; previous revision: 1.8
done
Checking in src/org/netbeans/lib/lexer/EmbeddedTokenList.java;
/cvs/lexer/src/org/netbeans/lib/lexer/EmbeddedTokenList.java,v  <-- 
EmbeddedTokenList.java
new revision: 1.5; previous revision: 1.4