This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 149086 - Poor performance of IDE makes it unusable for large unconfigured projects
Summary: Poor performance of IDE makes it unusable for large unconfigured projects
Status: RESOLVED FIXED
Alias: None
Product: cnd
Classification: Unclassified
Component: -- Other -- (show other bugs)
Version: 6.x
Hardware: All Linux
: P2 blocker (vote)
Assignee: Alexander Simon
URL:
Keywords: PERFORMANCE
Depends on: 149538
Blocks:
  Show dependency tree
 
Reported: 2008-10-03 15:49 UTC by kgoertz
Modified: 2008-10-10 05:58 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
NetBeans configuration file (1.53 KB, text/plain)
2008-10-03 17:10 UTC, kgoertz
Details
Terminal capture: start netbeans, wait, 10 thread dumps, exit (218.40 KB, text/plain)
2008-10-06 15:20 UTC, kgoertz
Details
Log files after enabling " -J-Dcnd.highlighting.times=true -J-Dcnd.identifier.error.provider.times=true" (23.19 KB, text/plain)
2008-10-06 17:58 UTC, kgoertz
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kgoertz 2008-10-03 15:49:22 UTC
A few of us have been using the latest development release of NB 6.5.  We require the development release because it has
a fix to enable "pending breakpoints" for GDB debugging in C++.  This was fixed after the initial 6.5 beta.

However we have had to switch to another IDE since the performance is extremely poor.  For example opening new files,
browsing source code, and editing code all involve long pauses.  It can take several seconds after typing before the
editor responds to the input.  

This occurs when working with a large project (in this case the open source project VLC).  
I assume some of this is likely the result of using a debug version of the code, however I thought it would be a good
idea to submit just in case this wasn't expected.

I can fill in more details if this problem is not expected to be fixed.

Kevin
Comment 1 Petr Chytil 2008-10-03 16:53:10 UTC
Hi kgoertz,
could you be more specific please? What project types do you usually use? I assume that you use C++ (because of that
GDB). Do you use remote development or something like that? If these long pauses are always reproducible on your
projects, then thread dumps in those points would be very useful for us [1].

Reassigning to cnd for now, please feel free to reassign when other project types are involved.

[1] http://wiki.netbeans.org/GenerateThreadDump
Comment 2 kgoertz 2008-10-03 17:08:24 UTC
This is with a C/C++ project.  It should be reproducible by downloading the vlc source and creating a project based
around it.
The performance problem is present just while editing files.  GDB debugging is also slow, however the performance issue
is present even when not building/debugging etc.  I have shut down all windows except the file being edited and the CPU
is still pegged.

The problem is present in NetBeans IDE Dev 200809251401 however we have tried multiple versions since early September
and all versions show the issue.

From the system monitor it shows that Java is always consuming at least 50% of the dual-core AMD X2-64-5000 CPU.  It
really appears that some part of the app is stuck in an infinite loop.  The CPU never appears to decrease even after
waiting > 10 minutes.

We do have a few C++ sources included on top of the regular VLC source.  I will attach my NetBeans conf file as well as
the vlc project file.
Running NB6.1 on the same project runs completely fine.  




Comment 3 kgoertz 2008-10-03 17:10:49 UTC
Created attachment 71131 [details]
NetBeans configuration file
Comment 4 soldatov 2008-10-03 19:42:40 UTC
Need other info.
===========================================================
When the NetBeans IDE freezes or becomes sluggish for no apparent reason, it's a recommended to generate a thread dump
and attach file to the reported bug. Please use 'Create a new attachment' link in the 'Attachments' section of the
IssueZilla header. Do not copy the thread dump into the description field!
The Thread Dump
... is a textual dump of all active threads and monitors of Java apps running in a Virtual Machine.
To generate thread dump:

    * Unix, Linux, Mac OS X
          o press Ctrl-\ in the terminal console you used to start the IDE
Comment 5 _ gordonp 2008-10-03 21:42:49 UTC
> it's a recommended to generate a thread dump

In this case, several thread dumps would be even better. If something is stuck in some
repeating loop, comparing several thread dumps could make it easier to see than a single
thread dump.
Comment 6 Leonid Lenyashin 2008-10-03 21:45:53 UTC
Thank you very much for reporting this, kgoertz,

We will look at this as it sounds pretty serious to us. We have a few similar bugs that are fixed or are being fixed. To
make sure if it is already known or a new issue, it would be very useful if you provide the info that soldatov requested.

Thank you,
Leonid. 
Comment 7 kgoertz 2008-10-06 15:20:22 UTC
Created attachment 71213 [details]
Terminal capture:  start netbeans, wait, 10 thread dumps, exit
Comment 8 Vladimir Voskresensky 2008-10-06 16:01:31 UTC
Could you, please, try and turn OFF [1] semantic highlighting for "fields", "methods", "unused variables" and
"unresolved identifiers"?
Is it better now?

[1] Tools->Options->C++->Higlighting
Comment 9 kgoertz 2008-10-06 16:18:32 UTC
I have disabled:
-highlight class' fields
-highlight function's name
-highlight unused variables
-highlight unresovled identifiers

The problem didn't disappear immediately (I changed settings and waited several minutes), but it did go away after a
netbeans restart.
CPU for netbeans is now ~0 at idle.  Looks like the problem.
Comment 10 Alexander Simon 2008-10-06 16:20:39 UTC
Could you provide us information about opened in IDE file(s).
Comment 11 kgoertz 2008-10-06 16:40:00 UTC
I just did the following:
- re-enabled all syntax highlighting
- closed all files
- restarted NB
- started NB
- CPU looked reasonable 
- loaded a .c file (actually vlc.c)
- CPU still reasonable
- closed the .c file and loaded a .cpp file containing classes.  The file is ~700 lines long and contains a full class
implementation 
- CPU spikes to ~50% measured by the Gnome system monitor on a dual core machine (ie infinite loop)
- closed the .cpp file
- CPU falls back to zero




Comment 12 Alexander Simon 2008-10-06 17:30:50 UTC
Features: semantic highlighting for "fields", "methods", "unused variables" and "unresolved identifiers"
require an additional work to resolve each identifier in code.
Now C/C++ internal resolver can resolve about 500-1000 identifiers per second.
So the features can process only 250-1000 lines of code per second.
Resolver start when file is open in editor and restart with delay after file modification.
It is a normal if resolver works 5-10 second on a long file.
Bug if resolver works more then 30 second.
If you can find such file let us know.
Comment 13 Alexander Simon 2008-10-06 17:44:05 UTC
Could you add two additional flags:
-J-Dcnd.highlighting.times=true -J-Dcnd.identifier.error.provider.times=true
and attach log with this flags?
Comment 14 kgoertz 2008-10-06 17:58:51 UTC
Created attachment 71224 [details]
Log files after enabling " -J-Dcnd.highlighting.times=true -J-Dcnd.identifier.error.provider.times=true"
Comment 15 kgoertz 2008-10-06 18:01:06 UTC
Done.  However I didn't have time to check the answer.  I will be away from my desk for the next few hours.

You should be able to reproduce locally though.  If you download the source to VLC, manually create a project, and then
add any .cpp file to the project it should (hopefully) show the issue.  No need to build/configure etc.
Comment 16 Alexander Simon 2008-10-06 18:06:07 UTC
I cannot find file src/amm/AdaptiveMediaManager.cpp
I downloaded sources vlc-0.9.3
Comment 17 kgoertz 2008-10-06 18:46:05 UTC
That one was one of our files, and unfortunately I can't let out any of our source even if it is irrelevant.  If you
replace it with a file that just contains some C++ code it will hopefully work.
Comment 18 Alexander Simon 2008-10-07 10:57:26 UTC
Could you answer:
- how many lines do you have in src/amm/AdaptiveMediaManager.cpp?
- is project configured?
- is code assistance configured?
- does file have a lot of red annotations?

Comment 19 kgoertz 2008-10-07 15:03:21 UTC
- how many lines do you have in src/amm/AdaptiveMediaManager.cpp?
748

- is project configured?
Yes...but minimally.  I just created a NB project and added every source file under the vlc tree

- is code assistance configured?
default

- does file have a lot of red annotations?
I am not sure what that refers to, but I don't think so.  Just C++ comments, and class functions.
Comment 20 Alexander Simon 2008-10-07 15:38:16 UTC
>- does file have a lot of red annotations?
>I am not sure what that refers to, but I don't think so.  Just C++ comments, and class functions.
It is a red waiver in text ant red signs in right side of the editor pane.
They are appeared if you switch on semantic highlighting for "fields", "methods", "unused variables" and
"unresolved identifiers".
Comment 21 kgoertz 2008-10-07 15:48:40 UTC
Good catch.  Yes there are many, many annotations.   Probably about 300 (one for every two lines in the file).  Although
that itself is a problem since there aren't any real problems with the file.  
The errors include
- it can't find include files which are in the same directory as the source
- doesn't understand the keyword "try {"
- can't resolve references to most of the variables etc (likely because it hasn't found the header files)
Comment 22 Alexander Simon 2008-10-07 17:05:32 UTC
fixed in release65_cnd_freeze branch, change set:
http://elif.russia.sun.com/hg/release65_cnd_freeze/rev/c46ebcfeadd0
fixed a low highlighting performance for unconfigured project:
- do not resolve twice of unresolved references
- immediately stop resolving on editor activity
Expected that fix improves performance in 3-5 times on unconfigured project.
Comment 23 Alexander Simon 2008-10-07 17:06:57 UTC
please, review fix
Comment 24 Vladimir Voskresensky 2008-10-07 17:25:08 UTC
I have reviewed the fix and agree with it, but would like to see it tested by QA + no regressions in inaccuracy tests
Comment 25 Alexander Simon 2008-10-07 17:38:24 UTC
additional fix:
http://elif.russia.sun.com/hg/release65_cnd_freeze/rev/a5c95d0cfd74
fix for load from repository ;-)
Comment 26 Alexander Simon 2008-10-08 07:41:17 UTC
Test case is vlc-0.8.6 project.
Project was configured and build on Lunix.
Tested on Solaris (so there are many unresolved include directives and identifiers).
To test user scenario the following tho files was included in project (remove exclude from project flag):
vlc-0.8.6i/activex/dataobject.cpp
vlc-0.8.6i/activex/vlccontrol2.cpp
Confirmed low performance on unconfigured c++ files.
Highlighting time before fix:
dataobject.cpp - 6 seconds
vlccontrol2.cpp - 10 secons
Highlighting time after fix:
dataobject.cpp - 1 second
vlccontrol2.cpp - 2 secons


Comment 27 Vladimir Kvashin 2008-10-08 08:40:37 UTC
We, myself and Alexander, reviewed the changes; the changes are correct.
Comment 28 soldatov 2008-10-08 10:07:24 UTC
verified in build from endif
Yes. 'Error Highlighting' works more quickly. But because original scenario is not clear, opinion of kgoertz I want to
know also (when patch will appear in public build).
Comment 29 Alexander Simon 2008-10-08 10:37:56 UTC
integrated in main trunk, change set:
http://hg.netbeans.org/main/rev/f7a5b70e1dab
Comment 30 Quality Engineering 2008-10-09 05:58:11 UTC
Integrated into 'main-golden', will be available in build *200810090201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/f7a5b70e1dab
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixed: IZ#149086:Poor performance of IDE makes it unusable for large projects
Comment 31 Alexander Simon 2008-10-09 12:39:12 UTC
One problem still exists:
- if file has a lot of error highlighting annotations the UI is frozen when annotations is showing. See related IZ#149538.
For example:
Let file has 1500 errors. UI is frozen about 2 seconds.
It will be 8 seconds for 3000 errors (quadratic dependence).
This is a critical issue for unconfigured c/c++ source files.
So we need work around.
Comment 32 Alexander Simon 2008-10-09 13:05:18 UTC
fixed in release65_cnd_freeze branch, change set:
http://elif.russia.sun.com/hg/release65_cnd_freeze/rev/1d1f59c187b4
Editor show first 100 error annotations.
Limit can be changed by flag:
    -J-Dcnd.highlighting.error.limit=200
For testing you can remove limit by:
    -J-Dcnd.highlighting.error.limit=-1

Please, review fix
Comment 33 Egor Ushakov 2008-10-09 13:59:35 UTC
Review comments:
+ MAX_ERROR_LIMIT constant better be moved to IdentifierErrorProvider
+ better avoid getErrorLimit and just use MAX_ERROR_LIMIT
+ use more readable form if (MAX >=0 && foundError >= MAX)
Comment 34 Egor Ushakov 2008-10-09 14:32:09 UTC
fix reviewed and approved
Comment 35 Alexander Simon 2008-10-09 14:38:36 UTC
fixed in release65_cnd_freeze, change set:
http://elif.russia.sun.com/hg/release65_cnd_freeze/rev/e8444ce6b341
Comment 36 soldatov 2008-10-09 15:51:58 UTC
Verified in build from endif
Comment 37 Alexander Simon 2008-10-09 16:04:26 UTC
fixed in main trunk, change sets:
http://hg.netbeans.org/main/rev/e84ce6791c00
http://hg.netbeans.org/main/rev/b149af030412
Comment 38 Quality Engineering 2008-10-10 05:58:19 UTC
Integrated into 'main-golden', will be available in build *200810100201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/e84ce6791c00
User: Alexander Simon <alexvsimon@netbeans.org>
Log: fixed: IZ#149086:Poor performance of IDE makes it unusable for large projects
- limit number of errors