This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 268024 - SimpleFileOwnerQueryImplementation.getOwner() is another cause of major NetBeans slownes
Summary: SimpleFileOwnerQueryImplementation.getOwner() is another cause of major NetBe...
Status: NEW
Alias: None
Product: ide
Classification: Unclassified
Component: Performance (show other bugs)
Version: Dev
Hardware: PC Windows 7
: P2 normal (vote)
Assignee: Tomas Hurka
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-09-14 14:18 UTC by NukemBy
Modified: 2016-09-14 14:18 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description NukemBy 2016-09-14 14:18:33 UTC
SimpleFileOwnerQueryImplementation.getOwner() is another cause of major NetBeans slownes

- - - 

I think this issue is one of the root causes for that issue (https://netbeans.org/bugzilla/show_bug.cgi?id=267969) and it also causes significant slowness in many other use-cases - most of them are happening in background threads, therefore not explicitly noticible - no lags on UI, but background actions are running much longer than needed.

    (... there are 2 potential solutions provided below ... )

Problem #1: 

As far as I see many scenarions of work with а particular file object require identification of it's parent Project - therefore SimpleFileOwnerQueryImplementation.getOwner()->(returns)->Project is the VERY FREQUENTLY called method. 

In turn it calls NbProjectManager.findProject(), what causes rescan of entire parent tree of target file object to find the parent project.
For each of these parents NbProjectManager.findProject() is called recursively - until either folder containing some project is found or tree root is reached. For each of these nodes NbProjectManager tries to construct the project - and just verification of "isProject" is very resource consuming operation (mostly because of reasons described here https://netbeans.org/bugzilla/show_bug.cgi?id=267990, even without "assert" and here https://netbeans.org/bugzilla/show_bug.cgi?id=65135):

	(excerpt from self-profiler when starting a unit test)
		 (... here it tries to load all kinds of supported projects ...)
		 org.netbeans.modules.ide.ergonomics.fod.FeatureProjectFactory.loadProject () - 807ms
		 org.netbeans.modules.project.ui.convertor.ProjectConvertorFactory.loadProject () - 557ms
		 org.netbeans.modules.project.ant.AntBasedProjectFactorySingleton.loadProject () - 347ms
		 org.netbeans.modules.groovy.grailsproject.GrailsProjectFactory.loadProject () - 118 ms
		 org.netbeans.modules.maven.NbMavenProjectFactory.loadProject () - 30ms
		 org.netbeans.gradle.project.NbGradleProjectFactory.loadProject () - 10ms

So what's happening - in many scenarios above mentioned file information is requested for files in Ivy/Maven repositories or temporary directories, e.g.:

      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches/modules-2/files-2.1/org.gradle/gradle-base-services-groovy/2.13/e1fc0f8d0bb9f9ab3eff1fea7c6060c297c2de4"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches/modules-2/files-2.1/org.gradle/gradle-base-services-groovy/2.13"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches/modules-2/files-2.1/org.gradle/gradle-base-services-groovy"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches/modules-2/files-2.1/org.gradle"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches/modules-2/files-2.1"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches/modules-2"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle/caches"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.gradle"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/"
      "RepositoryUpdater.worker": NbProjectManager.createProject  ""
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2/repository/org/eclipse/jetty/jetty-util/8.1.14.v20131031"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2/repository/org/eclipse/jetty/jetty-util"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2/repository/org/eclipse/jetty"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2/repository/org/eclipse"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2/repository/org"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2/repository"
      "RepositoryUpdater.worker": NbProjectManager.createProject  "C:/Users/User/.m2"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp/User/1/vcs-1473765871521"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp/User/1/vcs-1473765871521"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp/User/1"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp/User/1"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp/User"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp/User"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp"
      "Diffsidebar long tasks" loadProject "C:/Temp/~Temp"
      "Diffsidebar long tasks" loadProject "C:/Temp"
      "Diffsidebar long tasks" loadProject "C:/Temp"

There are no project in such directories and NEVER can be - that's why code goes this way (simplified version)
  
    public class SimpleFileOwnerQueryImplementation implements FileOwnerQueryImplementation {
        public Project getOwner(FileObject f) {
            if( !forbiddenFolders.contains(f.getPath()) ...
                p = ProjectManager.getDefault().findProject(f);
                if (p != null) {
                    projectCache.put(fldr, rp);
                    return p;
                }

    --> i.e. ... if project is found for an object - it is cached, otherwise - no cache entry is generated and each consequitive call for same 'bad' file will case entire rescan of parent tree all over again, and this does happen FREQUENTLY over and over again.

Solution #1: put special 'null reference' into cache even when project was not found - this will prevent consequitive false-positive rescans.

...
	
Problem #2: 

    Another reasons for rescans is lookup for folder in the places where they NEVER can be

	- local MAVEN repo   - projectDir.contains(".m2/repository")
	- local Gradle repo  - projectDir.contains(".gradle/caches")
	
	   (in general case paths can be redefined - so in correct implementation it should be configurable)

	- temp files in TEMP folder  - projectDir.startsWith(TEMP_DIR); where TEMP_DIR is like following
	
            private static final String TEMP_DIR; static {
                String tmpDir = System.getProperty("java.io.tmpdir");
                if( tmpDir != null )
                    tmpDir = tmpDir.replace('\\', '/');
                TEMP_DIR = tmpDir;
            }
	
Solution #2: 

add pre-conditional verification of the 'f.getPath()' in SimpleFileOwnerQueryImplementation.getOwner(FileObject f) to completely avoid lookup for files from special 'non-project' directories. 

Note1: FileObject.getPath() reconstructs the String from path parts each time the method is accessed. 
I think it needs to either cache 'path' as String at construction time or add method FileObject.isBelow(String|FileObject root) to optimize performance.

Note2: SimpleFileOwnerQueryImplementation already has 'Set<String> forbiddenFolders', but it is empty, at least this code does nothing

String forbidden = System.getProperty("project.forbiddenFolders", System.getProperty("versioning.forbiddenFolders", "")); //NOI18N