Bug 197811 - Project settings dialog should warn the user if the files contained in the project folder exceed a certain limit
Project settings dialog should warn the user if the files contained in the pr...
Status: NEW
Product: editor
Classification: Unclassified
Component: Parsing & Indexing
7.0
All All
: P3 (vote)
: 7.2
Assigned To: Tomas Zezula
issues@editor
: PERFORMANCE, PLAN
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-04-16 09:51 UTC by paolosca
Modified: 2012-02-24 09:31 UTC (History)
0 users

See Also:
Issue Type: ENHANCEMENT
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description paolosca 2011-04-16 09:51:55 UTC
It happens sometimes that the number of files in the project folder are so many that Netbeans becomes unresponsive during the project scanning. 

Specifically, a project folder may contain a link to a directory containing a large amount of files not needed by the project.

A few times I accidentally set /var or ~/ as the project's tests folder and netbeans became unresponsive within minutes.

I suspect that a lot of complains I read in the forum about project scanning that leads to poor performance may in fact be due to misconfigured project or tests folders or links to external directories that expand the scope of the project to large parts of the file system.

A possible solution could be to count the number of files in the project before saving the project's settings and eventually warn the user. The count should include Project folder, Sources folder, Tests folder and, where present, the include paths (i.e. in php). 

I have no idea of how many files can be handled before the performance starts degrading, my /var directory contains about 20GB in over 100k files and it made the IDE unresponsive.

I'm not sure if the total size of the files scanned and their type may affect the project scanning. In example, a 1GB zip file is ignored while a 1GB text file (i.e. a database dump) is parsed. If this is the case the file type (binary or text) and size could be included in the calculation.

Also, I noticed that it is possible to put the project scanning in an infinite loop using links in the project folder, in example a project in ~/projects/testproject that contain a link to ~/projects.


Thank you,


Paolo.
Comment 1 Jesse Glick 2011-04-19 21:13:49 UTC
Could probably display a count of Java sources detected in a newly configured source root in the Sources panel.

I do not think this is the right fix, though. The IDE should not become "unresponsive" just because you configure a project to use the wrong source root (or just open a project thus misconfigured). It will of course spend a long time running its classpath scan, but if you made a mistake you ought to be able to see that there is a mistake (based on messages in the progress bar), and after fixing the project configuration expect the scanning of that root to be canceled. Needs to be determined to what extent this is already true, and what if anything needs to be fixed.
Comment 2 Tomas Zezula 2011-04-20 07:22:42 UTC
Currently (dev) the indexing is cancellable after root scan.
This means that when you change the classpath (or close the project or project group) after a single root is scaneed no other roots from the same project (project group) is scanned.
Canceling of running indexer as also possible from parsing.api point of view. It will require api change (no problem) and also cooperation of indexers which needs to react on cancel and may need to do some roll back if they stored some data.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo