This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
When remote file system reads directory that has not been cached yet, 72-77% of time is spent in FilesystemInterceptorImpl.refreshRecursively. The scenario was as follows: I had a remote host that has been already set up, but file system cache was empty. The host was really slow (~400ms ping). I pressed "connect" button and waited until it gets green. As soon as it got green I pressed "open remote project" button and waited until file chooser is filled with the content of NetNeansProjects directory. Then I stopped measurements and quitted IDE. When I switched remote VCS off via -J-Dremote.vcs.suport=false, it took 4-5 seconds (from pressing "open remote project" till the moment when file chooser was filled). With VCS on, it took 56-58 seconds. Well, I haven't yet found all this 10x slowness. But I found about half of it. I used statistic measurements in code. The stack is as follows: at org.netbeans.modules.remote.impl.fs.RemoteFileSystemTransport.stat(RemoteFileSystemTransport.java:181) at org.netbeans.modules.remote.impl.fileoperations.spi.FileOperationsProvider$FileOperations.isDirectory(FileOperationsProvider.java:175) at org.netbeans.modules.remotefs.versioning.spi.FileProxyProviderImpl$FileOperationsImpl.isDirectory(FileProxyProviderImpl.java:124) at org.netbeans.modules.versioning.core.api.VCSFileProxy.isDirectory(VCSFileProxy.java:280) at org.netbeans.modules.versioning.core.DelegatingVCS.hasMetadata(DelegatingVCS.java:360) at org.netbeans.modules.versioning.core.DelegatingVCS.getTopmostManagedAncestor(DelegatingVCS.java:178) at org.netbeans.modules.versioning.core.VersioningManager.getOwner(VersioningManager.java:452) at org.netbeans.modules.versioning.core.VersioningManager.getOwner(VersioningManager.java:348) at org.netbeans.modules.versioning.core.filesystems.VCSFilesystemInterceptor.getRefreshInterceptor(VCSFilesystemInterceptor.java:494) at org.netbeans.modules.versioning.core.filesystems.VCSFilesystemInterceptor.listFiles(VCSFilesystemInterceptor.java:391) at org.netbeans.modules.remotefs.versioning.spi.FilesystemInterceptorProviderImpl$FilesystemInterceptorImpl.refreshRecursively(FilesystemInterceptorProviderImpl.java:264) at org.netbeans.modules.remote.impl.fs.RemoteDirectory.updateChildren(RemoteDirectory.java:1362) at org.netbeans.modules.remote.impl.fs.RemoteDirectory.getDirectoryStorageImpl(RemoteDirectory.java:1150) at org.netbeans.modules.remote.impl.fs.RemoteDirectory.getDirectoryStorage(RemoteDirectory.java:532) at org.netbeans.modules.remote.impl.fs.RemoteDirectory.getFileObject(RemoteDirectory.java:410) at org.netbeans.modules.remote.impl.fs.RemoteFileObject.getFileObject(RemoteFileObject.java:421) at org.netbeans.modules.remote.impl.fs.RemoteDirectory.getFileObject(RemoteDirectory.java:402) at org.netbeans.modules.remote.impl.fs.RemoteFileSystem.findResourceImpl(RemoteFileSystem.java:565) One of the problems is that at org.netbeans.modules.versioning.core.VersioningManager.getOwner(VersioningManager.java:452) VersioningSystem[] vs = getVersioningSystems() the array contains 8 instances of VersioningSystem, including local ones. DelegatingVCS.hasMetadata calls VersioningSystem.getMetadataFolderNames() and then operates with these folders via VCSFileProxy. The log (with flags -J-Dnativeexecution.support.logger.level=0 -J-Dremote.support.logger.level=0) shows that each folder is asked 8 times for stat/lstat. That's exactly for that reason. What is probably the worst here is that these 8 lstat calls are asked *before* remote file system returns children to the caller.
Created attachment 163118 [details] A patch with statistics - I haven't yet decided whether to push it; I think i should use profiler instead
Created attachment 163119 [details] A grepped IDE log with statistics
It is a remote file system problem. I've told about separation remote file system from remote file artifices.
fixed in 63b7c38a886c, bcaf05cdfc7b on enum/release82
(In reply to Vladimir Kvashin from comment #4) > fixed in 63b7c38a886c, bcaf05cdfc7b on enum/release82 Sorry, right change set is bcaf05cdfc7b (although it also contains fix for issue 268361)
Integrated into 'main-silver', will be available in build *201612080001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/2b227b168889 User: Vladimir Kvashin <vkvashin@netbeans.org> Log: fixing #269205 - 72-77% of time is spent in FilesystemInterceptorImpl.refreshRecursively (transplanted from 63b7c38a886c3b030385178c25e1c6de4b6090f8)
Integrated into 'main-silver', will be available in build *201612110001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/f880e5e70140 User: Vladimir Kvashin <vkvashin@netbeans.org> Log: addfix for #269205 - 72-77% of time is spent in FilesystemInterceptorImpl.refreshRecursively - use SPI instead of hard-coded extensions (transplanted from 6e92ef9fb836bb7867961f21dba4e6105eb0ac93)