This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 198617 - Netbeans (all versions) are unusably slow especially 'scanning projects'
Summary: Netbeans (all versions) are unusably slow especially 'scanning projects'
Status: RESOLVED INCOMPLETE
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 7.0.1
Hardware: PC Windows 7 x64
: P1 normal (vote)
Assignee: Jan Lahoda
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2011-05-16 05:34 UTC by javacodauk
Modified: 2011-06-06 09:52 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description javacodauk 2011-05-16 05:34:49 UTC
I have developed a large Swing project (around 300 classes) over a number of years and the Netbeans IDE (all versions) is now so slow that I am sometimes waiting 10 minutes for it to finish 'scanning projects'. Scanning (re)starts after every minor change as I am typing (even without saving the file I am working on) and is making Netbeans completely unusable!

I have found that if I don't wait for scanning (and whatever is using the CPU even after it has stopped scanning) to complete and get on with something else, then scanning takes even longer - assuming that it doesn't completely lock up the entire PC at 100% CPU usage and requires killing from task manager.

Large source files are the real killers, but I have noticed that as the the number of classes in the project has grown this is starting to become the main culprit. I have also noticed that if you make a change that produces many errors in multiple files this has a very bad effect on the time taken to complete scanning. 

My problems are not due to lack of memory (8G), disk performance (SATA with fast read / write speed), garbage collection settings (I have tried all configurations), or particularly large files (all less than 10,000 lines after a complete rewrite).

The first thing, in my opinion, that should be done would be to add some intelligence to the various background processes so that they work nicely together. Here are some suggestions which I am sure would improve things:

The scanning process should be rewritten using native code (or heavily optimised somehow) - it is just too slow!!

The 'Navigator View' should stop building the current view if it hasn't finished by the time you switch tabs to edit or open another file in the IDE. If you switch back and forth rapidly between open files in the IDE the Navigator View completely blocks the UI while it sorts itself out. This problem is particularly crippling with large files open (5,000+ lines).

A way to manually suspend the scanning would be nice. This would hopefully give the developer an opportunity to make a batch of changes without the scanning process permenently running in the background slowing everything to a snails pace (or slower).

Once a long scan has started it should block all other background processes until it has completed. It seems that one or more background processes running together combine to degrade the performance of the scaning process making the IDE totally unusable for very long periods.

Netbeans should automatically detect the performance of the PC it is running on and configure itself for optimal memory settings etc. - it is confusing when increasing the memory (in netbeans.conf) seems to slow the IDE down after a certain point.

I would be willing to work with someone to provide feedback as necessary to get these long standing problems resolved if that would be of any help.
Comment 1 Marian Mirilovic 2011-05-16 07:14:45 UTC
this is generic performance issue - so reassign to performance module. I would say the best way should be to share your project with us (internally), so we can use it for tuning/testing purposes.
Comment 2 javacodauk 2011-05-16 08:49:21 UTC
Thanks for your response. I could share the code with you, however as it is a commercial product, would you be willing to sign a non-disclosure agreement?
Comment 3 ayermolayev 2011-05-23 20:37:17 UTC
It looks very bad that the Scanning Project problem initially reported so many versions ago is still not resolved!
In its current state scanning is completelty unacceptable! I wonder why at least a very simple solution -- just make it optional, triggered explicitly by user -- is not yet implemented? This solution has been mentioned so many times in so many bug reports!
Please, make it explicit!
Comment 4 Tomas Hurka 2011-06-02 09:48:28 UTC
Since this is swing project, I would like to ask java team for evaluation. It would be great to get reporter's project for testing purposes.
Comment 5 Jan Lahoda 2011-06-02 15:33:06 UTC
Having the project and being able to reproduce the problem would of course be awesome and would ease out finding the problem. Marian, Petr or David, could you please help with formal aspects of this? Thanks.

Before that, we might be able to find out interesting stuff by generating thread and memory dumps. Could you please take a few thread dumps when the IDE is scanning for a long time (or use the self-profile button) and also when the IDE is blocked after switching the tabs and attach it here?

How to generate thread dumps is here:
http://wiki.netbeans.org/GenerateThreadDump
Self profiling:
http://wiki.netbeans.org/FitnessViaPartnership

The thread dumps might contain general memory overview, but if they don't (it depends on JDK version, I think), could you please try to generate heap statistics and/or heap histogram using "jmap -heap <pid>" or "jmap -histo:live <pid>", respectivelly. That might also help. Also, could you please attach your ${netbeas-userdir}/var/log/messages.log?

Thanks.
Comment 6 Jan Lahoda 2011-06-02 20:03:28 UTC
One though on the Navigator: could you please check that you have selected the "Members View" panel in the navigator? In development builds, there are several debugging panels (e.g. "Trees (NetBeans Development)"), which are not really efficient. But these are intended to be used only when developing NetBeans modules and on small sources. These panels are not enabled in release builds (i.e. when assertions are disabled).

Thanks.
Comment 7 David Strupl 2011-06-06 09:52:48 UTC
I am marking the report as incomplete. Please reopen after adding some relevant information - be it the project itself, relevant logs, snapshots from self profiling or similar. To add one more resource to the already mentioned links to the wiki is the following one:

http://wiki.netbeans.org/FaqScanningAndIndexingIssues

and I assume from the comment that the following link was consulted first:

http://wiki.netbeans.org/FaqScanningAndIndexingPerformanceHints

Thanks for your patience and after we know more details please reopen and we will continue investigating.