Bug 185714 - Code completion in PHP with large classes causes 100% cpu usage
Code completion in PHP with large classes causes 100% cpu usage
Status: VERIFIED FIXED
Product: php
Classification: Unclassified
Component: Editor
6.x
PC All
: P1 with 5 votes (vote)
: 6.x
Assigned To: Petr Pisl
issues@php
69patch3-verified
: PERFORMANCE
Depends on: 191842 191885
Blocks:
  Show dependency treegraph
 
Reported: 2010-05-07 16:12 UTC by ss10sb
Modified: 2010-12-14 10:03 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
Messages.log (5.61 KB, application/x-gzip)
2010-05-07 16:13 UTC, ss10sb
Details
thread dump 1 (15.30 KB, text/x-log)
2010-05-07 17:57 UTC, ss10sb
Details
thread dump 2 (11.18 KB, text/x-log)
2010-05-07 17:57 UTC, ss10sb
Details
thread dump 3 (15.68 KB, text/x-log)
2010-05-07 17:57 UTC, ss10sb
Details
thread dump 4 (nb 6.9.1) (25.99 KB, application/octet-stream)
2010-11-01 14:59 UTC, apric
Details
thread dump (769.85 KB, application/octet-stream)
2010-11-12 14:38 UTC, nbphpuser
Details
php class (359.76 KB, text/plain)
2010-11-12 14:42 UTC, nbphpuser
Details

Note You need to log in before you can comment on or make changes to this bug.
Description ss10sb 2010-05-07 16:12:52 UTC
This may be the same bug as Bug #180375 but I'm not sure.
I have two large libraries that are included in most of my projects: Doctrine and Zend.  Zend code completion seems to work, but any time I use any of the Doctrine classes, the Slowness Detector pops up, the code completion never completes and cpu use hits 100% on one of the cores.  It stays at 100% and gradually uses all the memory available to the Java VM.  The editor, etc is still usable (technically) although terribly slow.  I've had to go back to Eclipse to do any work with anything referencing Doctrine which is really depressing!  I like Netbeans!!
I'm currently running Netbeans Dev 201004280200 w/ Sun Java 1.6.0 18 32 bit.
I've tried it with Netbeans 6.8, 6.9M1 in various combinations with Sun 32 bit and OpenJDK 64 bit.  It does it on both my home and work systems.  Both are running Ubuntu 10.04 64 bit.
I've attached the messages.log file, although I don't know if it will help at all.
--Scott
Comment 1 ss10sb 2010-05-07 16:13:19 UTC
Created attachment 98643 [details]
Messages.log
Comment 2 ss10sb 2010-05-07 17:57:12 UTC
Created attachment 98649 [details]
thread dump 1
Comment 3 ss10sb 2010-05-07 17:57:35 UTC
Created attachment 98650 [details]
thread dump 2
Comment 4 ss10sb 2010-05-07 17:57:52 UTC
Created attachment 98651 [details]
thread dump 3
Comment 5 ss10sb 2010-05-07 17:58:29 UTC
A bit more information and some thread dumps:
I've found I can duplicate the issue everytime doing these steps:
$query = Doctrine_Query::create()->select()->from()->where();
Everything is fine up through ::create(), then cpu usage spikes and code completion stops working (td2 is just this step).
Then after the $query variable is created, anytime you try to access any of its methods, code completion never gets past "please wait..." and cpu hits 100%. (td1 is this step).
Combining both steps (instantiate $query and then try to add a method to $query) will spike the cpu at 100% after ::create() and then gives the Slowness Detector message on the $query->something. (td3)
Comment 6 apric 2010-10-14 09:45:37 UTC
Reproducable on 6.9.1 in Win Vista 32-Bit, JDK 1.6_u21, also on MacOSX and Linux.

Using the Doctrine ORM (doctrine-project.org):

$foo = Doctrine_Query::create()
                ->from('foo')
                ->innerJoin('bar')
                ->innerJoin('baz')
                ->andWhere('whatever')
                ->andWhere('yes')
                ->addSelect('no')
                ->addSelect('now it will hang')
                ->

After about 7 successful method chainings the code completion does not work anymore and Netbeans begins to consume 100% CPU.

This bug drives us nuts, our Doctrine Queries are sometimes a chain of 20 or more methods, and we have to restart Netbeans multiple times a day because of this specific bug.
We think it exists since 6.8.
Comment 7 HJM 2010-11-01 10:59:50 UTC
same here. Hope to fix in 6.9.2 or 7.0.

(In reply to comment #6)
> Reproducable on 6.9.1 in Win Vista 32-Bit, JDK 1.6_u21, also on MacOSX and
> Linux.
> 
> Using the Doctrine ORM (doctrine-project.org):
> 
> $foo = Doctrine_Query::create()
>                 ->from('foo')
>                 ->innerJoin('bar')
>                 ->innerJoin('baz')
>                 ->andWhere('whatever')
>                 ->andWhere('yes')
>                 ->addSelect('no')
>                 ->addSelect('now it will hang')
>                 ->
> 
> After about 7 successful method chainings the code completion does not work
> anymore and Netbeans begins to consume 100% CPU.
> 
> This bug drives us nuts, our Doctrine Queries are sometimes a chain of 20 or
> more methods, and we have to restart Netbeans multiple times a day because of
> this specific bug.
> We think it exists since 6.8.
Comment 8 apric 2010-11-01 14:59:38 UTC
Created attachment 102748 [details]
thread dump 4 (nb 6.9.1)

snapshot contains waiting 20 seconds for code completion (just a frame of 20 seconds, will not complete any time soon)
Comment 9 Petr Pisl 2010-11-04 14:59:58 UTC
The next week I plan to spent my time on the performance problem.
Comment 10 Petr Pisl 2010-11-11 16:48:01 UTC
I can reproduce this and it seems like a bug in counting types in the chain
method.
Comment 11 Petr Pisl 2010-11-11 16:56:27 UTC
Fixed in the web-main. The fix is not hard. Probably it can be merged in the beta if it still possible. 

http://hg.netbeans.org/web-main/rev/d1df8e27abb4
Comment 12 Petr Jiricka 2010-11-11 17:13:46 UTC
Marian?
Comment 13 Filip Zamboj 2010-11-11 17:52:27 UTC
(In reply to comment #11)
> Fixed in the web-main. The fix is not hard. Probably it can be merged in the
> beta if it still possible. 
> 
> http://hg.netbeans.org/web-main/rev/d1df8e27abb4

If you can make it to the beta to be verified in beta tomorrow than I'd go for it.
Comment 14 Filip Zamboj 2010-11-11 18:03:42 UTC
@ppisl: would you run new bertram build because this one doesn't seem to be in latest build? I cannot update sources because of uncommitted changes (and I can't commit). I'd be able today that way. thanks.
Comment 15 Filip Zamboj 2010-11-11 21:31:08 UTC
Product Version: NetBeans IDE 7.0 Beta (Build 201011110000)
Java: 1.6.0_20; Java HotSpot(TM) 64-Bit Server VM 16.3-b01-279
bertram build. 

verified based on example from user.
@reporter: could you verifiy, please? 
@ppisl: I hope this fix is safe to be transplanted to beta.
Comment 16 Petr Jiricka 2010-11-11 22:11:58 UTC
Merged to release70_beta: http://hg.netbeans.org/releases/rev/76b46ad128e7
Comment 17 Petr Pisl 2010-11-12 00:26:20 UTC
Thanks to Petr J. to commit it to the beta branch.

To Filip: I'm not sure for 100% that it will not introduce any problem, the change is in low level util method, that is used in more case then just this one. On the other hand it's used many times in the tests, every code completion test and also other tests use it and because no test fails, it should be ok.
Comment 18 Filip Zamboj 2010-11-12 11:00:57 UTC
(In reply to comment #17)
> Thanks to Petr J. to commit it to the beta branch.
> 
> To Filip: I'm not sure for 100% that it will not introduce any problem, the
> change is in low level util method, that is used in more case then just this
> one. On the other hand it's used many times in the tests, every code completion
> test and also other tests use it and because no test fails, it should be ok.

well, it's beta, so it's time to find out if it is all ok :). thanks for committing to beta.
Comment 19 nbphpuser 2010-11-12 14:38:05 UTC
Created attachment 102940 [details]
thread dump

opened IDE and big PHP class. started profiling on autocompletion
Comment 20 nbphpuser 2010-11-12 14:42:08 UTC
Created attachment 102941 [details]
php class


I'm not sure if I should open new issue. I've downloaded the build this bug was fixed and created a big PHP class, where I've tried to do CC and it is still slow. CPU usage is about 50%. See profiler dump on previous comment and used PHP class in attachment.
Comment 21 Filip Zamboj 2010-11-12 15:47:17 UTC
(In reply to comment #20)
> Created an attachment (id=102941) [details]
> php class
> 
> 
> I'm not sure if I should open new issue. I've downloaded the build this bug was
> fixed and created a big PHP class, where I've tried to do CC and it is still
> slow. CPU usage is about 50%. See profiler dump on previous comment and used
> PHP class in attachment.

but it's definitely better you say. Let developers evaluate your thread dump and eventually reopen this issue.
Comment 22 ss10sb 2010-11-15 17:02:44 UTC
Finally had a chance to test it out.  Code completion with Doctrine is quick, CPU and memory usage stays reasonable, awesome!  I didn't play too much, just added Doctrine back into my library path and chained together several methods that used to cause it to die a horrible death.  But it looks like it works great.

Thanks so much for working on this.
Comment 23 apric 2010-11-16 12:26:51 UTC
Cannot verify as _completely_ "fixed":

I'm still having trouble with chains about the size of 10 or more calls.
The IDE won't stand still completely any more, but there are still heavy issues with the CC and its performance:

- up to ~10 chain calls the completion takes up to 10 seconds (the dev. build from 11/11/2010 wants me to send bug/performance reports each time i press "space"!)

- when trying to CC more than ~10 calls, CC breaks completely and does not try to find CC matches any more (but only local to the chain, outside of it the CC works as expected)

reproducable:

(again, using PHP Doctrine Library)

$query = Doctrine_Query::create()
          ->from('foo')
          ->innerJoin('bar')
          ->addSelect('baz')
          ->andWhere('abc')
          /* ... repeat the above 3 lines ... */
          ->{PRESS SPACE HERE}
Comment 24 apric 2010-11-16 18:22:31 UTC
i meant version 20101116* (daily dev build)
Comment 25 nbphpuser 2010-11-17 07:50:38 UTC
not only CC is slow in big files. Writing simple comment line or whatever causes CPU load about 20-30%. Of course, one shouldn't create such big files.
Comment 26 ss10sb 2010-11-17 14:27:13 UTC
Strangely, I'm having none of the mentioned issues.  I chained 15 Doctrine query methods together and it never had a problem.  Documentation (I'm assuming you mean the /** shortcut since a standard // or /*blah*/ hits about 2% cpu) works fine for me as well. Both hit about 10 - 15% CPU usage and drop back to 0 as soon as the CC list pops up or the doc template fills out. GC continues to work and memory usage will go up and back down during the CC/doc process.

I tried out the insanely large Article class and didn't have any problems with CC, although I can see that there might be a problem with the docs since it pushed cpu usage to 30% when I hit /** inside the class above one of the methods.

Perhaps it's my setup.  Now I'm afraid to change it and find out. :)
I'm running Ubuntu 10.10 64-bit (2.6.35-22).
Netbeans Dev 201011150001
Sun Java 1.6.0_22; Java HotSpot(TM) 64-Bit Server VM 17.1-b03
Comment 27 nbphpuser 2010-11-17 14:33:37 UTC
I'm on W7, latest java and latest IDE and have always had problems with very big files. Functions like editing, search, replace, cc and other were always slow if file is bigger than 8k lines (about 500KB)
Comment 28 ss10sb 2010-11-17 15:29:40 UTC
Thought I'd be smart and install Netbeans dev on my Win 7 virtualbox.  With the Article class, typing anything causes it to stop responding with 100% cpu usage, maxed out memory, and much unhappiness.  Granted, my virtualbox machine doesn't have much ram allocated (512 MB), but I only managed to type // before it stopped responding completely.
Comment 29 Petr Pisl 2010-11-18 10:26:09 UTC
Please don't mix more issues together. This is about method chain and it should be fixed. I have tested this with more then 30 calls. The other slow things should be threated separately. 

To apric: The described behavior is the behavior before the fix. Are you sure that you use a build with the fix. I have just try it with 30 methods in the chain and no problem.
Comment 30 nbphpuser 2010-11-18 11:34:49 UTC
Opened new issue #192097 for slowness in big file
Comment 31 apric 2010-12-05 02:09:09 UTC
Seems to be fixed in the nightlies so far.
There are still some issues, but nothing regarding this specific bug.
-> fixed
Comment 32 Petr Pisl 2010-12-09 10:01:35 UTC
Yesterday also committed to thea NB 6.9 patch 3 branch
Comment 33 Marian Mirilovic 2010-12-14 10:03:59 UTC
NetBeans 6.9/.1 Patch 3 has just been released. Please update your installation to get this issue fixed in your NB installation.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo