This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | Computing of NB project packages is slow | ||
---|---|---|---|
Product: | profiler | Reporter: | Jesse Glick <jglick> |
Component: | Base | Assignee: | Jiri Sedlacek <jis> |
Status: | RESOLVED FIXED | ||
Severity: | blocker | Keywords: | PERFORMANCE, THREAD |
Priority: | P3 | ||
Version: | 6.x | ||
Hardware: | All | ||
OS: | All | ||
Issue Type: | DEFECT | Exception Reporter: | |
Attachments: | Thread dumps |
Description
Jesse Glick
2006-08-03 01:43:01 UTC
Created attachment 32473 [details]
Thread dumps
Profiling a module project directly does not appear to produce this problem. Profiling a module project with more deps does produce this problem, but the IDE unfreezes itself about thirty seconds or so. You should not block EQ indefinitely like this. According to the thread dumps the real problem is caused by slow project infrastructure. Almost all time is spend in org.netbeans.spi.project.support.ant.SourcesHelper$SourcesImpl.getSourceGroups() FYI: indefinitely != thirty seconds But a progress dialog would be much prettier than blocked ui there. :) Sure. It's "indefinitely" in the sense that the more projects you have as subprojects of the selected one, the longer the freeze (as far as I can make out). BTW do you really need to get source groups for every recursive subproject to begin with? (Cf. my other bug report re. meaning of "only this project" filter.) Optimizing SourcesHelper is probably also possible, though I can't get the profiler to work yet so it will be tough to know which part is slow... So I did some performance measurements in a standalone app. public static void main(String[] args) throws Exception { Project p = ProjectManager.getDefault().findProject(FileUtil.toFileObject(new File("/space/src/nb_all/nbbuild"))); Set<Project> shown = new HashSet<Project>(); long start = System.currentTimeMillis(); show(p, shown); long end = System.currentTimeMillis(); System.err.println("time: " + (end - start) + "msec"); } private static void show(Project p, Set<Project> shown) throws Exception { if (!shown.add(p)) { return; } for (SourceGroup g : ProjectUtils.getSources(p).getSourceGroups(JavaProjectConstants.SOURCES_TYPE_JAVA)) { System.out.println(FileUtil.toFile(g.getRootFolder())); } SubprojectProvider spp = p.getLookup().lookup(SubprojectProvider.class); if (spp != null) { for (Project sub : spp.getSubprojects()) { show(sub, shown); } } } prints all Java source roots in the NB standard distro. It runs for me in between 7 and 11 seconds. Probably could be somewhat faster, but remember that this is loading cca. 200 module projects including project.properties and project.xml. The freeze I saw was certainly longer than this, so maybe something else is going on. Please read my comment in Issue 81769. Could you suggest better Profile Only Project Classes filter behavior or other improvements that would fit your needs? Note that profiling all NetBeans modules at once this way is not a good idea in general, you should use Part of Application profiling (define explicit root methods) and as much restrictive instrumentation filter as possible for profiling large applications. Right, profiling nb_all/nbuild as main project is not a good idea and probably something you don't want anyway. You should select module you want to profile as main project or select it and use context menu to invoke 'Profile Project'. Computing of java source roots was improved. Lowering priority to P3. "you should select module you want to profile" - I have a list of around 10 modules I want to profile. I don't know which is primarily responsible for the slow behavior that has been reported. (If I knew that much, I probably would not need the profiler!) I have gotten it to work by selecting a couple of root methods in java/project and selecting Profile Project on that; apparently the profiler will gather timing information from methods in other projects. It is not clear from the UI what I would do if I had root methods in several projects; would I just pick one of those projects to "profile" arbitrarily? Does it actually make any difference which project I select to profile, other than that if I select a low-level project the dialog is responsive whereas if I select nbbuild it is unusable? A bit confusing. Once you invoke "Profile Project", all packages of this project and all its subprojects recursively are computed, that's why it's so slow and the dialog is blocked. This will be fixed in this issue. If you select just java/project, it's faster than invoking profiling on all NetBeans modules because it contains just a small subset of all NetBeans packages. You can define any root methods from any opened projects for profiling in context of a project. These root methods are stored in context of the profiled project - this means that if you want to profile whole NetBeans, you have to define separate set of root methods for the suite, root method's aren't the sum of all defined root methods from all subprojects (modules). The same applies for your case. Root methods define where profiling starts, not where it "ends", that's why you can see data from other modules. This is what the instrumentation filter does - it defines which classes will be instrumented (== profiled). If you want to profile just java/project, then invoke profiling of this module and use Profile Only Project Classes filter. Could you please describe in more details what's not clear from UI about the root methods? There is a list of root methods that will be used for the profiling session, do you think it's not enough? If you think that current profiler documentation is not good enough, could you please file an issue about it and list the areas which should be improved? Thanks. To repeat my question of Jun 27, it is not clear to me what I would do if I wanted root methods in, say, both java/project and openide/masterfs. Which one would I select for profiling? Whichever one I choose, the IDE will start up with the same arguments, but it is not obvious whether the set of root methods will be affected, assuming I have already selected root methods using the editor's context menu. The only natural-seeming project to profile is nbbuild, but as I have found, that is unacceptably slow. I have no idea what you mean by "root method's aren't the sum of all defined root methods from all subprojects (modules)" but apparently I am not understanding something. For each project a separate set of root methods for profiling is defined. If you are adding root methods from editor, the root method is added to the project the current source belongs to. If you want to add root methods from different projects, you have to use the Select Root Methods dialog with Show All Projects checkbox selected (Profile Project/CPU Part of Application/define.../Add From Project...). When using CPU Part of Application mode, it doesn't matter which NB module you choose to profile unless you want to use Profile Only Project Classes instrumentation filter - that's because the owning suite is always used to start NB. Projects, subprojects, source roots and all packages are now processed lazily after selecting profiling task & settings in separate thread. New instrumentation filter "Profile Project & Subprojects Classes" introduced. As for your computation of resolving source roots - the profiler is searching for all packages in all subprojects, that's why it takes more time than your example. All possible optimizations have been implemented on profiler side but it will always take some time for large projects like NetBeans. Fixed for Beta1. |