This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 253863 - [newcodemodel] broken code model in file if file has been excluded from project
Summary: [newcodemodel] broken code model in file if file has been excluded from project
Status: VERIFIED FIXED
Alias: None
Product: cnd
Classification: Unclassified
Component: Code Model (show other bugs)
Version: 8.1
Hardware: All All
: P1 normal (vote)
Assignee: Vladimir Voskresensky
URL:
Keywords: 81_HR_FIX
Depends on:
Blocks:
 
Reported: 2015-07-27 13:55 UTC by soldatov
Modified: 2015-12-22 02:28 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
proposed patch for deep reparsing utils (9.55 KB, patch)
2015-10-01 11:52 UTC, Alexander Simon
Details | Diff
Refactoring #1 that is needed for fixing this (not a fix yet) (19.05 KB, patch)
2015-10-05 14:32 UTC, Vladimir Kvashin
Details | Diff
Refactoring #2 that is needed for fixing this (not a fix yet) (25.55 KB, patch)
2015-10-05 14:34 UTC, Vladimir Kvashin
Details | Diff
Test (10.87 KB, patch)
2015-10-05 15:09 UTC, Vladimir Kvashin
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description soldatov 2015-07-27 13:55:14 UTC
Scenario:
- Create Welcome project
- Open welcome.cc file
- Add #include "welcome.h" line
- Save file
- Delete or comment #include "welcome.h" line
- Open welcome.h file
- Modify and Save file
==> empty Navigator and broken code model in this file
Comment 1 Vladimir Kvashin 2015-09-25 22:51:51 UTC
Can not reproduce this on bits built today (96abf49f3ad4). Does this repeat?  Please reopen if it does.
Comment 2 soldatov 2015-09-28 08:24:56 UTC
Easy to reproduce in NetBeans IDE Dev (Build 201509280002)
Comment 3 Vladimir Voskresensky 2015-09-28 08:28:17 UTC
then this is regression
Comment 4 DimaZh 2015-09-30 14:53:24 UTC
A show-stopper
Comment 5 Alexander Simon 2015-10-01 11:52:46 UTC
Created attachment 156475 [details]
proposed patch for deep reparsing utils
Comment 6 Jiri Kovalsky 2015-10-01 15:05:58 UTC
Setting TM accordingly.
Comment 7 Vladimir Voskresensky 2015-10-04 20:53:51 UTC
Btw, I think due to this issue I've got exceptions[1] in console when was doing:
1) Create Welcome sample
2) open welcome.cc and copy #include <iostream> line
3) open welcome.h and paste #include <iostream>, Save all
4) switch to welcome.cc, replace included file to be #include <stream>, Save
5) Go to welcome.h and hyperlink on #include <iostream>
=> exceptions

[1] Exceptions
INFO [cnd.logger]: ClankFileInfoImpl{convertedToAPT=false; hasTokenStream=false, current=file=3:/usr/include/c++/4.8/string,
currentInclude=ClankInclusionDirective{
[1581-1598] annotation=null,
resolvedPath=ResPath{/usr/include/c++/4.8/string in /usr/include/c++/4.8},
spelling=string,
}}
 vs. 
PARSED FileImpl @-1645020165:1006450926 /usr/include/c++/4.8/iostream prj:667971514KeyBasedUID on ProjectKey /usr/include/c++/4.8 NOT_BEING_PARSED
java.lang.Exception: ClankFileInfoImpl{convertedToAPT=false; hasTokenStream=false, current=file=3:/usr/include/c++/4.8/string,
currentInclude=ClankInclusionDirective{
[1581-1598] annotation=null,
resolvedPath=ResPath{/usr/include/c++/4.8/string in /usr/include/c++/4.8},
spelling=string,
}}
 vs. 
PARSED FileImpl @-1645020165:1006450926 /usr/include/c++/4.8/iostream prj:667971514KeyBasedUID on ProjectKey /usr/include/c++/4.8 NOT_BEING_PARSED
[catch] at org.netbeans.modules.cnd.utils.CndUtils.assertTrueInConsole(CndUtils.java:218)
	at org.netbeans.modules.cnd.utils.CndUtils.assertPathsEqualInConsole(CndUtils.java:277)
	at org.netbeans.modules.cnd.modelimpl.parser.clank.ClankTokenStreamProducer$FileTokenStreamCallback.onEnter(ClankTokenStreamProducer.java:411)
	at org.netbeans.modules.cnd.apt.impl.support.clank.ClankPPCallback.onEnter(ClankPPCallback.java:339)
	at org.clang.tools.services.support.FileInfoCallback.FileEnterImpl(FileInfoCallback.java:777)
	at org.clang.tools.services.support.FileInfoCallback.FileChanged(FileInfoCallback.java:334)
	at org.clang.lex.PPCallbacks.FileChanged(PPCallbacks.java:97)
	at org.clang.lex.Preprocessor.EnterSourceFileWithLexer(Preprocessor.java:7166)
	at org.clang.lex.Preprocessor.EnterSourceFile(Preprocessor.java:2130)
	at org.clang.lex.Preprocessor.HandleIncludeDirective(Preprocessor.java:8053)
	at org.clang.lex.Preprocessor.HandleIncludeDirective(Preprocessor.java:7719)
	at org.clang.lex.Preprocessor.HandleIncludeDirective(Preprocessor.java:7688)
	at org.clang.lex.Preprocessor.HandleDirective(Preprocessor.java:4279)
	at org.clang.lex.Lexer.LexTokenInternal(Lexer.java:2800)
	at org.clang.lex.Lexer.Lex(Lexer.java:378)
	at org.clang.lex.Preprocessor.Lex(Preprocessor.java:2372)
	at org.clang.tools.services.impl.PreprocessorSupport.ConsumePreprocessedInput(PreprocessorSupport.java:411)
	at org.clang.tools.services.impl.PreprocessorSupport.dumpPreprocessedEntry(PreprocessorSupport.java:299)
	at org.clang.tools.services.impl.PreprocessorSupport.preprocessEntries(PreprocessorSupport.java:228)
	at org.clang.tools.services.impl.PreprocessorSupport.preprocess(PreprocessorSupport.java:170)
	at org.clang.tools.services.ClankPreprocessorServices.preprocess(ClankPreprocessorServices.java:67)
	at org.netbeans.modules.cnd.apt.impl.support.clank.ClankDriverImpl.preprocessImpl(ClankDriverImpl.java:157)
	at org.netbeans.modules.cnd.apt.support.ClankDriver.preprocess(ClankDriver.java:104)
	at org.netbeans.modules.cnd.modelimpl.parser.clank.ClankTokenStreamProducer.getTokenStream(ClankTokenStreamProducer.java:161)
	at org.netbeans.modules.cnd.modelimpl.csm.core.FileImpl.createAndCacheFullTokenStream(FileImpl.java:1204)
	at org.netbeans.modules.cnd.modelimpl.csm.core.FileImpl.getTokenStream(FileImpl.java:1273)
	at org.netbeans.modules.cnd.modelimpl.impl.services.MacroExpansionDocProviderImpl.getFileTokenSequence(MacroExpansionDocProviderImpl.java:853)
	at org.netbeans.modules.cnd.modelimpl.impl.services.MacroExpansionDocProviderImpl.expand(MacroExpansionDocProviderImpl.java:510)
	at org.netbeans.modules.cnd.modelimpl.impl.services.MacroExpansionDocProviderImpl.getMacroTable(MacroExpansionDocProviderImpl.java:1537)
	at org.netbeans.modules.cnd.modelimpl.impl.services.MacroExpansionDocProviderImpl.expand(MacroExpansionDocProviderImpl.java:331)
	at org.netbeans.modules.cnd.api.model.services.CsmMacroExpansion.expand(CsmMacroExpansion.java:161)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmExpandedTokenProcessor.create(CsmExpandedTokenProcessor.java:101)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmExpandedTokenProcessor.create(CsmExpandedTokenProcessor.java:93)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmCompletionQuery.processTokensInFile(CsmCompletionQuery.java:726)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmCompletionQuery.queryImpl(CsmCompletionQuery.java:566)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmCompletionQuery.query(CsmCompletionQuery.java:530)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmCompletionQuery.query(CsmCompletionQuery.java:523)
	at org.netbeans.modules.cnd.completion.csm.CompletionUtilities.findItemsReferencedAtCaretPos(CompletionUtilities.java:163)
	at org.netbeans.modules.cnd.completion.impl.xref.ReferencesSupport.findDeclaration(ReferencesSupport.java:491)
	at org.netbeans.modules.cnd.completion.impl.xref.ReferencesSupport.findDeclaration(ReferencesSupport.java:428)
	at org.netbeans.modules.cnd.completion.impl.xref.ReferencesSupport.findReferencedObject(ReferencesSupport.java:227)
	at org.netbeans.modules.cnd.completion.impl.xref.ReferenceImpl.getReferencedObjectImpl(ReferenceImpl.java:134)
	at org.netbeans.modules.cnd.completion.impl.xref.ReferenceImpl.getReferencedObject(ReferenceImpl.java:96)
	at org.netbeans.modules.cnd.highlight.semantic.ModelUtils$FieldReferenceCollector.visit(ModelUtils.java:146)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter$1.visit(SemanticHighlighter.java:227)
	at org.netbeans.modules.cnd.completion.impl.xref.FileReferencesImpl._accept(FileReferencesImpl.java:157)
	at org.netbeans.modules.cnd.completion.impl.xref.FileReferencesImpl.accept(FileReferencesImpl.java:109)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter.updateImpl(SemanticHighlighter.java:219)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter.access$500(SemanticHighlighter.java:91)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter$TaskContext.run(SemanticHighlighter.java:433)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
Comment 8 Vladimir Voskresensky 2015-10-04 20:57:46 UTC
(In reply to Vladimir Voskresensky from comment #7)
> 4) switch to welcome.cc, replace included file to be #include <stream>, Save
misprint, should be:
4) switch to welcome.cc, replace included file to be #include <string>, Save
Comment 9 Vladimir Voskresensky 2015-10-04 21:06:22 UTC
trace from other place
[catch] at org.netbeans.modules.cnd.utils.CndUtils.assertTrueInConsole(CndUtils.java:218)
	at org.netbeans.modules.cnd.utils.CndUtils.assertPathsEqualInConsole(CndUtils.java:277)
	at org.netbeans.modules.cnd.modelimpl.parser.clank.ClankTokenStreamProducer$FileTokenStreamCallback.onExit(ClankTokenStreamProducer.java:468)
	at org.netbeans.modules.cnd.apt.impl.support.clank.ClankPPCallback.onExit(ClankPPCallback.java:389)
	at org.clang.tools.services.support.FileInfoCallback.FileExitImpl(FileInfoCallback.java:812)
	at org.clang.tools.services.support.FileInfoCallback.FileChanged(FileInfoCallback.java:340)
	at org.clang.lex.Preprocessor.HandleEndOfFile(Preprocessor.java:4014)
	at org.clang.lex.Lexer.LexEndOfFile(Lexer.java:4025)
	at org.clang.lex.Lexer.LexTokenInternal(Lexer.java:1901)
	at org.clang.lex.Lexer.Lex(Lexer.java:378)
	at org.clang.lex.Preprocessor.Lex(Preprocessor.java:2372)
	at org.clang.tools.services.impl.PreprocessorSupport.ConsumePreprocessedInput(PreprocessorSupport.java:411)
	at org.clang.tools.services.impl.PreprocessorSupport.dumpPreprocessedEntry(PreprocessorSupport.java:299)
	at org.clang.tools.services.impl.PreprocessorSupport.preprocessEntries(PreprocessorSupport.java:228)
	at org.clang.tools.services.impl.PreprocessorSupport.preprocess(PreprocessorSupport.java:170)
	at org.clang.tools.services.ClankPreprocessorServices.preprocess(ClankPreprocessorServices.java:67)
	at org.netbeans.modules.cnd.apt.impl.support.clank.ClankDriverImpl.preprocessImpl(ClankDriverImpl.java:157)
	at org.netbeans.modules.cnd.apt.support.ClankDriver.preprocess(ClankDriver.java:104)
	at org.netbeans.modules.cnd.modelimpl.parser.clank.ClankTokenStreamProducer.getMacroUsages(ClankTokenStreamProducer.java:197)
	at org.netbeans.modules.cnd.modelimpl.parser.clank.ClankTokenStreamProducer.getMacroUsages(ClankTokenStreamProducer.java:111)
	at org.netbeans.modules.cnd.modelimpl.parser.clank.ClankFileInfoQuerySupport.getMacroUsages(ClankFileInfoQuerySupport.java:90)
	at org.netbeans.modules.cnd.modelimpl.impl.services.FileInfoQueryImpl.getMacroUsages(FileInfoQueryImpl.java:331)
	at org.netbeans.modules.cnd.highlight.semantic.ModelUtils.getMacroBlocks(ModelUtils.java:120)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticEntitiesProvider$MacrosCodeProvider.getBlocks(SemanticEntitiesProvider.java:354)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter.updateImpl(SemanticHighlighter.java:205)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter.access$500(SemanticHighlighter.java:91)
	at org.netbeans.modules.cnd.highlight.semantic.SemanticHighlighter$TaskContext.run(SemanticHighlighter.java:433)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
Comment 10 Vladimir Kvashin 2015-10-05 14:32:52 UTC
Created attachment 156518 [details]
Refactoring #1 that is needed for fixing this (not a fix yet)
Comment 11 Vladimir Kvashin 2015-10-05 14:34:31 UTC
Created attachment 156520 [details]
Refactoring #2 that is needed for fixing this (not a fix yet)

I had to attach 2 change sets since I have to temporarily to switch to anther issue (issue 255678) and I'm afraid I can loose these changes.
Comment 12 Vladimir Kvashin 2015-10-05 15:09:28 UTC
Created attachment 156521 [details]
Test
Comment 13 Quality Engineering 2015-10-09 01:22:30 UTC
Integrated into 'main-silver', will be available in build *201510090002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/90b1c537c2f0
User: Vladimir Kvashin <vkvashin@netbeans.org>
Log: Refactroing while fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project
Comment 14 Quality Engineering 2015-10-13 01:33:14 UTC
Integrated into 'main-silver', will be available in build *201510130002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/ed5af31c0cda
User: Vladimir Voskresensky <vv159170@netbeans.org>
Log: fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project (binaries)
- don't keep stat misses in NB mode
Comment 15 Quality Engineering 2015-10-18 01:28:15 UTC
Integrated into 'main-silver', will be available in build *201510180002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/5c28952a2547
User: Vladimir Voskresensky <vv159170@netbeans.org>
Log: fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project
- clean up before semantic changes
Comment 16 Quality Engineering 2015-10-22 01:27:34 UTC
Integrated into 'main-silver', will be available in build *201510220002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/06554bbbb09d
User: Vladimir Voskresensky <vv159170@netbeans.org>
Log: fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project
- remove methods from ClankFileInfo which belongs to PreprocessorOutput
Comment 17 Vladimir Voskresensky 2015-10-22 07:37:19 UTC
I integrated the latest fixes into cnd-main.


Please, verify.
Comment 18 Antonin Nebuzelsky 2015-10-22 14:32:05 UTC
This needs to be backported to release81 today and marked as 81_HR_FIX.
Comment 19 ilia 2015-10-22 15:22:08 UTC
Transplanted to releases~release81:
  hg.netbeans.org/releases/rev/3fdede9a7d8e - [extra fix #253863 - [newcodemodel] broken code model in file if file has been excluded from project (binaries)]
  hg.netbeans.org/releases/rev/0b5e77a36efc - [fixed #253863 - [newcodemodel] broken code model in file if file has been excluded from project (i18n)]
  hg.netbeans.org/releases/rev/221e01164cf8 - [fixed #253863 - [newcodemodel] broken code model in file if file has been excluded from project (binaries)]
  hg.netbeans.org/releases/rev/87a793552629 - [fixed #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/f1859e00bea5 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/e865baa89db1 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/f69d9adb8d87 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/648041d13432 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/3444b363d9e0 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/8a615728e7bd - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/1d80bdd72396 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/fa2847fe01f2 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/fb10e0b0bde1 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/d74f29bde0d6 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project (i18n)]
  hg.netbeans.org/releases/rev/305ffe260406 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/994d7f369e6e - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/b3f586fcb718 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/1d253b1ef950 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/42bdc494f0b3 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/0222aa5f0e72 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/69cf2b31251b - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/939b4c41598b - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/46eb0d1e6b8d - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project (binaries)]
  hg.netbeans.org/releases/rev/d74a8edb9479 - [fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
  hg.netbeans.org/releases/rev/d1fa937e7756 - [refactoring: corrected IncludeInfo method names  (fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project)]
  hg.netbeans.org/releases/rev/8529ffbb52be - [Refactoring: token stream parameters are for clank producer only! (fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from   project)]
  hg.netbeans.org/releases/rev/2eacf1d02378 - [Refactroing while fixing #253863 - [newcodemodel] broken code model in file if file has been excluded from project]
Comment 20 Quality Engineering 2015-10-23 01:25:28 UTC
Integrated into 'main-silver', will be available in build *201510230002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/3267a07a6082
User: Vladimir Voskresensky <vv159170@netbeans.org>
Log: fixed #253863 - [newcodemodel] broken code model in file if file has been excluded from project
- finished recovery of corrupted include chains
Comment 21 soldatov 2015-10-26 13:11:58 UTC
Verified in NetBeans IDE 8.1 (Build 201510222201)
Comment 22 Quality Engineering 2015-12-22 02:28:06 UTC
Integrated into 'main-silver', will be available in build *201512220002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/e6a42bc95634
User: Alexander Simon <alexvsimon@netbeans.org>
Log: additional fix to commit (do not generate unneeded strings):
fixing #253863 - [newcodemodel] broken code model in file if file has been
excluded from project
- for requests (except parsing) use InterestedFileImplPreprocessorCallback
which is responsible for correct restoring include chains (transplanted from
0eacd81849eb04ed2b1af83534d02ad4d1d735fd)
(transplanted from b0d5933f393625e7ede2db459c2aa862e54acafc)