This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 271076 - Slow parse of remote ccfe 12.6
Summary: Slow parse of remote ccfe 12.6
Status: STARTED
Alias: None
Product: cnd
Classification: Unclassified
Component: Remote (show other bugs)
Version: 8.2
Hardware: All All
: P3 normal (vote)
Assignee: Vladimir Kvashin
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-11 18:03 UTC by Vladimir Kvashin
Modified: 2017-07-22 02:32 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
proposed patch (5.84 KB, patch)
2017-07-20 10:31 UTC, Alexander Simon
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Kvashin 2017-07-11 18:03:09 UTC
Alexander S complained that remote ccfe (with sources and artifacts in SCA, accessed via full remote from SPb) was parsing more than 24 hours.

That's probably because of Alexanders setup specific (for example as I know his network home is mounted on SPb server even when working from SCA). In my case the same project was parsed within half an hour or so (which is probably ok with 200ms ping).

However I see some strangeness that can definitely slow down things.

Strangeness #1:
 
with my -J-Dnativeexecution.support.logger.level=0 -J-Dremote.support.logger.level=0 options I see lots of messages like 

"Refreshing /net/dv104/export/projects/ide/inaccuracy_projects/studio12.6
/lang/acomp/src/code.h took 416 ms"

Debugger shows the stack
"org.openide.text Document Processing"
	at org.netbeans.modules.remote.impl.fs.RemotePlainFile.refreshImpl(RemotePlainFile.java:439)
	at org.netbeans.modules.remote.impl.fs.RemotePlainFile.refreshImpl(RemotePlainFile.java:377)
	at org.netbeans.modules.remote.impl.fs.RemoteDirectory.ensureChildSync(RemoteDirectory.java:1780)
	at org.netbeans.modules.remote.impl.fs.RemotePlainFile.getInputStream(RemotePlainFile.java:277)
	at org.netbeans.modules.remote.impl.fs.RemoteFileObject.getInputStream(RemoteFileObject.java:398)
	at org.netbeans.modules.cnd.source.CppEditorSupport.loadFromStreamToKit(CppEditorSupport.java:207)
	at org.openide.text.DocumentOpenClose$DocumentLoad.atomicLockedRun(DocumentOpenClose.java:764)
	at org.openide.text.DocumentOpenClose$DocumentLoad.run(DocumentOpenClose.java:615)
	at org.netbeans.editor.GuardedDocument.runAtomic(GuardedDocument.java:321)
	at org.openide.text.NbDocument.runAtomic(NbDocument.java:436)
	at org.openide.text.DocumentOpenClose$DocumentLoad.run(DocumentOpenClose.java:655)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)

which in turn is caused by 
"Fix registration KeyBasedUID on FileKey (/home/vkvashin/NetBeansProjects/acomp_1/N/, /net/dv104/export/projects/ide/inaccuracy_projects/studio12.6/lang/acomp/src/amsg.h)"
	at org.openide.text.DocumentOpenClose$DocumentLoad.<init>(DocumentOpenClose.java:595)
	at org.openide.text.DocumentOpenClose.initLoadTaskLA(DocumentOpenClose.java:425)
	at org.openide.text.DocumentOpenClose.open(DocumentOpenClose.java:228)
	at org.openide.text.CloneableEditorSupport.openDocument(CloneableEditorSupport.java:574)
	at org.openide.text.DataEditorSupport.openDocument(DataEditorSupport.java:589)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmCompletionQuery.performQueryTask(CsmCompletionQuery.java:376)
	at org.netbeans.modules.cnd.completion.cplusplus.ext.CsmCompletionQuery.queryObjects(CsmCompletionQuery.java:307)
	at org.netbeans.modules.cnd.completion.cplusplus.CsmExpressionResolverImpl.resolveObjects(CsmExpressionResolverImpl.java:64)
	at org.netbeans.modules.cnd.api.model.services.CsmExpressionResolver$Default.resolveObjects(CsmExpressionResolver.java:436)
	at org.netbeans.modules.cnd.api.model.services.CsmExpressionResolver.resolveObjects(CsmExpressionResolver.java:103)
	at org.netbeans.modules.cnd.api.model.services.CsmExpressionResolver.resolveObjects(CsmExpressionResolver.java:90)
	at org.netbeans.modules.cnd.modelimpl.csm.core.AstRenderer.findGlobalHard(AstRenderer.java:690)
	at org.netbeans.modules.cnd.modelimpl.csm.core.AstRenderer.isVariableOrFunctionName(AstRenderer.java:561)
	at org.netbeans.modules.cnd.modelimpl.csm.core.AstRenderer.isRefToVariableOrFunction(AstRenderer.java:535)
	at org.netbeans.modules.cnd.modelimpl.csm.core.AstRenderer.isFuncLikeVariable(AstRenderer.java:435)
	at org.netbeans.modules.cnd.modelimpl.csm.FunctionImplEx.fixFakeRegistration(FunctionImplEx.java:372)
	at org.netbeans.modules.cnd.modelimpl.csm.core.FileImpl.fixFakeFunctionRegistrations(FileImpl.java:2046)
	at org.netbeans.modules.cnd.modelimpl.csm.core.FileImpl.fixFakeRegistrations(FileImpl.java:2017)
	at org.netbeans.modules.cnd.modelimpl.csm.core.FileImpl.onProjectParseFinished(FileImpl.java:934)
	at org.netbeans.modules.cnd.modelimpl.csm.core.FakeRegistrationWorker$FixRegistrationRunnable.run(FakeRegistrationWorker.java:146)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)

It isn't correct to refresh a file each time it is being read.
Comment 1 Alexander Simon 2017-07-20 10:22:41 UTC
I'm not sure that stacks reflects my problem.
I see a problem with updating project view after parsing each file.
Typical hot spot is a finding project item by file object.
Stack example:
"Code Model Parser 3: Parsing /header.h" #107 daemon prio=1 os_prio=64 tid=0x00000000064d7800 nid=0x71 runnable [0xffff80ffb94fc000]
   java.lang.Thread.State: RUNNABLE
    at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
    at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)
    at java.io.File.isDirectory(File.java:849)
    at java.io.File.toURI(File.java:732)
    at org.openide.util.BaseUtilities.toURI(BaseUtilities.java:1645)
    at org.openide.filesystems.FileUtil.normalizeFileOnUnixAlike(FileUtil.java:1697)
    at org.openide.filesystems.FileUtil.normalizeFileImpl(FileUtil.java:1684)
    at org.openide.filesystems.FileUtil.normalizePath(FileUtil.java:1619)
    at org.netbeans.modules.remote.support.LocalFileSystemProvider.getFileObject(LocalFileSystemProvider.java:137)
    at org.netbeans.modules.remote.spi.FileSystemProvider.getFileObject(FileSystemProvider.java:221)
    at org.netbeans.modules.cnd.api.remote.RemoteFileUtil.getFileObject(RemoteFileUtil.java:92)
    at org.netbeans.modules.cnd.makeproject.api.configurations.Item.getFileObjectImpl(Item.java:436)
    at org.netbeans.modules.cnd.makeproject.api.configurations.Item.getFileObject(Item.java:404)
    at org.netbeans.modules.cnd.makeproject.ui.ViewItemNode.stateChanged(ViewItemNode.java:438)
    at org.netbeans.modules.cnd.model.services.CsmCodeAssistanceProvider.fireChanges(CsmCodeAssistanceProvider.java:336)
    at org.netbeans.modules.cnd.model.services.CsmCodeAssistanceProvider.fileParsingFinished(CsmCodeAssistanceProvider.java:314)
    at org.netbeans.modules.cnd.modelimpl.csm.core.ProgressSupport.fireFileParsingFinished(ProgressSupport.java:127)
    at org.netbeans.modules.cnd.modelimpl.csm.core.ParserQueue.onFileParsingFinished(ParserQueue.java:906)
    at org.netbeans.modules.cnd.modelimpl.csm.core.ParserThread._run(ParserThread.java:164)
    at org.netbeans.modules.cnd.modelimpl.csm.core.ParserThread.run(ParserThread.java:74)
    at org.netbeans.modules.cnd.modelimpl.csm.core.ParserThreadManager$Wrapper.run(ParserThreadManager.java:92)
    at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
    at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
    at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
    at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
Comment 2 Alexander Simon 2017-07-20 10:24:42 UTC
Probably Vladimir cannot reproduce a bug because he did not expand project tree while parsing. I'd recommend perform select main file in project action.
Comment 3 Alexander Simon 2017-07-20 10:31:09 UTC
Created attachment 164801 [details]
proposed patch
Comment 4 Alexander Simon 2017-07-20 14:02:24 UTC
applied patch:
http://hg.netbeans.org/cnd-main/rev/c481f44f349e
Comment 5 Quality Engineering 2017-07-22 02:32:17 UTC
Integrated into 'main-silver', will be available in build *201707220001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/c481f44f349e
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixing Bug #271076 Slow parse of remote ccfe 12.6