Bug 183201

Created attachment 96358 [details]
snapshot 1

Created attachment 96359 [details]
snapshot 2

*** Bug 183237 has been marked as a duplicate of this bug. ***

Jardo, please have a look at this issue, it is extremely annoying to restart the IDE (with all the NB modules loaded) several times a day.... Thanks a lot.

null

changeset   : 166215:234c4f0d6291
author      : Jaroslav Tulach <jtulach@netbeans.org>
date        : Mon Apr 05 19:16:22 CEST 2010
summary     : #183201: At most one CC profiling shall be in progress

Integrated into 'main-golden', will be available in build *201004060201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/234c4f0d6291
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #183201: At most one CC profiling shall be in progress

Jardo, are you sure that the fix is correct? Cannot happen that one profiling thread will be running forever?

*** Bug 183621 has been marked as a duplicate of this bug. ***

Created attachment 96862 [details]
Snapshot of NetBeans hogging CPU, suspect CC Sampling again?

NetBeans nightly 201004070201 still slows down tremendously, hogging all CPU. I don't know if it's the same cause, but if I interpret the data correctly, a lot of time is still taken up by logger-completion threads, like in previous snapshots I uploaded to a defect marked as duplicate of this one -- which is why I'm uploading the new snapshot here.

Almost all time seems to go into java.lang.ProcessImpl.waitFor()?

As an additional comment, I didn't really notice any slowdowns when I was just copy-pasting most changes, using alt-/ for variable name completions.

Only when I started to make changes that invoked proper code-completion, triggering it myself manually a couple of times, NetBeans started to slow down.

So I do suspect code completion is still involved.

I'm however not sure that it's exactly the same problem as the original problem report so I'm hesitant to re-open the defect myself.

Tomáši, after the http://hg.netbeans.org/main/rev/234c4f0d6291, there is a singleton profiler for completion. But still the snapshot shows three logger-completion threads running. Is not somethign broken in the SelfProfileAction itself?

I do not see nothing wrong in SelfProfileAction and there were no changes in that area for some time and it looks like this issue appears in recent builds.

I did not have this problem in the build from today (at least I did not notice), however, as I wrote in comment #8:

> Jardo, are you sure that the fix is correct? Cannot happen that one profiling
> thread will be running forever?

I think that the problem is that the profiling thread is not ended correctly, so lowering their number to one is great improvement but probably not the best fix :) Or am I missing something?

> Or am I missing something?

The reality is somehow being missed. Nobody knows why there are multiple threads doing the profiling. Code inspection does not seem to help.

So is it ok to have this issue marked as FIXED?

Actually, since I still experience this problem, I don't think it can be marked FIXED yet?

Only since I'm not a member of the project, I don't want to reopen it myself, since I can't decide if my problem is different from this defect or still the same cause as the original defect.

*** Bug 183821 has been marked as a duplicate of this bug. ***

Reopening.

*** Bug 183621 has been marked as a duplicate of this bug. ***

Created attachment 97034 [details]
snapshot

Is this snapshot OK or not?

Product Version: NetBeans IDE Dev (Build 100409-1a1477cad8d5)
Java: 1.6.0_16; Java HotSpot(TM) 64-Bit Server VM 14.2-b01
System: Linux version 2.6.31-20-generic running on amd64; UTF-8; cs_CZ (nb)

(In reply to comment #21)
> Is this snapshot OK or not?

I guess NOT because my IDE uses 100% of my CPU and I need to restart it.

Tomas Hurka said in a comment to #183993 that this one is P1 and that what I saw is in fact this problem.

BTW I have a heap dump from today's morning build with OOME if it is useful for anything (I guess not).

Addressed in bug 183832. For 6.9 beta I suggest to disable this code (it will be disabled for release too):

diff -r c4b4eaaa2f9e editor.completion/src/org/netbeans/modules/editor/completion/CompletionImpl.java
--- a/editor.completion/src/org/netbeans/modules/editor/completion/CompletionImpl.java  Fri Apr 09 10:05:40 2010 +0200
+++ b/editor.completion/src/org/netbeans/modules/editor/completion/CompletionImpl.java  Wed Apr 14 10:13:42 2010 +0200
@@ -1752,7 +1752,7 @@
         if (profiler == null) {
             return;
         }
-        profile = new Profile(profiler, when);
+// disable for beta:        profile = new Profile(profiler, when);
     }

     private static final class Profile implements Runnable {

Tondo, you have the 6.9 beta clone, can you comment the above line in it? Thanks.

Fixed in Beta:

http://hg.netbeans.org/release69_beta/rev/055321af1257

[1] somewhat improves the situation, although even with this fix I saw logger-completion not being terminated and running forever (until I killed the IDE). IMO there are exit paths in the CC code that do not call CompletionImpl.stopProfiling().

[1] http://hg.netbeans.org/jet-main/rev/e92a68613f37

Since this problem is NOT satisfactorily resolved in trunk I am reopening this report.

I know that after Tonda's fix in release69_beta this is not an issue for 6.9beta builds, but I don't know how to express this state in Bugzilla. However, I DO remember Netbeans releases and practices when looking at a bug report one could have determined where exactly the problem was fixed (eg. in trunk only, in both trunk and the release branch, on the release branch only). Unfortunately we don't seem to care much about such practices anymore.

For the follow-up on trunk reassigning back to Jarda.

What about the fix Jarda did in issue 183832? I.e. http://hg.netbeans.org/core-main/rev/0b27d337bc62. Isn't Vita's [1] in conflict with that?

Anyway these two issues look like the same thing now.

(In reply to comment #29)
> What about the fix Jarda did in issue 183832? I.e.
> http://hg.netbeans.org/core-main/rev/0b27d337bc62. Isn't Vita's [1] in conflict
> with that?

Yes, it is. After a quick look Jarda's fix looks better though. On top of the changes that I did he also found and fixed some places where stopProfiling() should have been called, but was not. So, whoever happens to be resolving this conflict should probably just choose Jarda's fix and ignore mine, which will hopefully make their life easier.

conflict resolved in: http://hg.netbeans.org/core-main/rev/af1cde3fc070

*** Bug 183482 has been marked as a duplicate of this bug. ***

(In reply to comment #31)
> conflict resolved in: http://hg.netbeans.org/core-main/rev/af1cde3fc070

thanks

This still bogs my install down considerably.  In instances where the number of files is very large on PHP prodjects, such as with Magento, the IDE becomes nearly unusable because these checks are triggered on what seems to be nearly every change I make.  The only fix is to turn off the checking altogether.

Here are some thoughts/suggestions.
1) Why are we re-checking the whole project when changes are made in one place.  Would a more tiered approach to caching help here (ie only check based on a more limited criteria)?
2) Can we do something to thread this and throttle it's CPU usage?
3) Can we add a setting that would limit the number of re-scans per specified time period (ie only scan 10 times per hour).

I know this is not an easy task as dealing with a large number of files is no simple thing.  However, the 100% CPU usage and downtime is really annoying.  Thanks for looking into this.

(In reply to comment #34)
> This still bogs my install down considerably.  In instances where the number of
> files is very large on PHP prodjects, such as with Magento, the IDE becomes
> nearly unusable because these checks are triggered on what seems to be nearly
> every change I make.  The only fix is to turn off the checking altogether.
> 
> Here are some thoughts/suggestions.
> 1) Why are we re-checking the whole project when changes are made in one place.
>  Would a more tiered approach to caching help here (ie only check based on a
> more limited criteria)?
> 2) Can we do something to thread this and throttle it's CPU usage?
> 3) Can we add a setting that would limit the number of re-scans per specified
> time period (ie only scan 10 times per hour).
> 
> I know this is not an easy task as dealing with a large number of files is no
> simple thing.  However, the 100% CPU usage and downtime is really annoying. 
> Thanks for looking into this.

This is different issue. Please file a separate one. Thanks.

Wow, I built my new work comp with a Core i7-950, 8GB memory, and an SSD and this is the first I've noticed any type of slowness on it!  It is halting and not allowing cursor movement, scrolling etc. for several seconds delay.  Meanwhile other programs are fine.  I notice javaw.exe is using up to 16% CPU during this and almost 700Mb of memory (netbeans is just 1,104K).  I haven't noticed this before.  I only started using NetBeans after building the new comp 3+ weeks ago however I have put a lot of my code into it.  The file that this started on is an 800 line JSP with lots of Java code and HTML and I was going through and retabbing all the HTML.  I'm reporting it immediately so haven't had time to notice it happening on anything else, but I have edited other large JSPs perhaps not as heavily with no effect.

I exited and restarted NetBeans and it did it again.  However I went to a different file for editing and then back and now it seems to be ok.  Strange.

NetBeans v6.9.1
Windows 7