This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Well it's a famous issue, code completion is extremely slow. I don't actually know the underlying code that is used to make up the code completing feature. All I know is that even if all the items in a code completion database count up to more than a million, a simple auto-suggest query shouldn't take more than a blink of an eye to give results, so, why is this happening? :) Maybe a whole other solution should be used for this code completion feature. I know when you add a project into netbeans projects, it takes quite some time (depending on how big the project actually is) to index it up! Well, I'm sure everyone would be more than happy to have this indexing process get longer, but have the code completion feature work instantly. So, maybe the indexing part should be tweaked a little. And by the way, isn't it kinda weired when sometimes searching a certain code in the whole project actually works faster than the code-completion itself?!
batch reassigning
Created attachment 102214 [details] Thread Dump after memory spike and slowdown Attached is my thread dump after the slowdown started. The memory went from 210MB to around 500MB, and everything slowed down. At the time, I was doing an static call like class:: (the code completion box started then "Please wait...") and had to restart the IDE
Using NB 7.0 M2 201010151215 MacBookPro 10.6.4 Slow for me also in PHP, for class constants, properties, local vars, global vars, also code templates. Is NB just a pig WRT memory use?
Is it slow from beginning when you start the IDE, or it get to slow after a while?
It's slow all the time when the library gets a little big. It's really easy to reproduce this situation. Just add Zend Framework to your php include path along with another library like PEAR, and then start typing in code and you'll see how frustratingly slow it gets.
So I have looked at this together with Tomas and we found mainly three problems. Basically every file has corresponding source structure, that is used for parser. The parser produces parser-result that is used for building a model. Shortly there should be at a time always one file -> one source -> one parser result -> one model. The problems that we found are: 1) More sources are created for one file. Then every source is parsed and the model is built for each one. This looks like a problem in parsing api infrastructure. 2) The source is often invalidated and then is reparsed again. It's not always but sometimes the source is invalidated during calling cc twice or even more times. That means that the parser is called more times and the model is build more times during cc. The same happens, when you call cc in non changed document. This has to be evaluated more and find out, why the source is invalidated even if there is no change. 3) Building model from parser AST is too expensive. For tcpdf.php file the building model tooks 4.5 second. It's huge file, more than 16 thousands lines, but it seems to me slow. The model is built via model visitor. And for examle the empty visitor is done during 4 ms for this testing file and the formatter visitor that builds formatting changes is done during 300 ms. I'm not sure, whether I will be able to speed up the model visitor, but IMHO 4.5 seconds is too much. 4) When user hit enter in cc, the whole file is reformatted and only closest changes are applied around the inserting text from code completion item. For the testing tcpdf.php file it tooks 2.5 second. For the testing file tcpdf.php the parsing takes now 300 ms and building model around 4.5 second. The time until cc is displayed can be expressed now as y * (4.5s + 300ms) + x * (4.5s + 300ms) where the y is usually 2 due to the problem #1 and x is a random number due to the mentioned problem #2. When we fix #1 and #2 then the result time will 4.5s + 300ms.
I agree, 4.5 seconds is a lot! PHP itself would parse that file and give all kinds of reflection info about it in less than a second. (Haven't tested that though!) Do we have these problems in other langs too?
Basically yes, the problem #1 and #2 are in a parsing infrastructure. Problem #3 its in PHP, but the question is whether it's a fixable. It's a big big file and the file with 160000 lines is not very common.
Could someone please check with new dev build whether there is a reasonable improvement?
I downloaded a new PHP build "NetBeans Dev 201011250001", and I do not see any noticeable difference. MAYBE the difference is about 10% for me, because I can't say for sure. I compared with 7.0m2 build with absolutely same settings. In empty PHP project without any libraries like Zend, time of code completion invoking is about 2-5 seconds. Typing by fingers is much faster! :(
Thanks zvk77. There were some fixes, but they doesn't have to be too much visible. I'm now trying do a bigger change to improve it, but need more time.
Integrated into 'main-golden', will be available in build *201012070001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/67c206fd0597 User: Petr Pisl <ppisl@netbeans.org> Log: #189889 - Code completion is extremely slow Small improvement in the performance
Apparently, there are different bottlenecks that cause this performance problem. But I think one of them makes the most noticeable slowdown. Here is how you can reproduce a situation where this bottleneck shows itself: 1. Put Zend Framework library in your project's include_path config in netbeans. 2. Your project will be a simple php file that tries to instantiate a ZF class like Zend_Db_Table_Row_Abstract. 3. Start typing: $row = new Zend_Db_Table_Row(); >> as you may have noticed, it's only one line of code, and none of the framework files are changed, cause they are just in the include path, but yet, the code completion is really slow and we are still better off typing the class name manually. What this tells us, is that even if the problems like duplicate sources and models taking too long to be built didn't exist, the code completion would still be really slow. Can anyone tell us what happens (overally) when the developer types in $var = new Zend_ and then hits space? I mean, suppose all the files in the project are fully parsed and all the models are built, and we are just typing in an empty file; what functions are called when netbeans tries to give the user a list of suggestions for the auto complete?
This is good usecase. Thanks cgalive. So what happens? Basically when you write new Zend_ , then the ide looks for all constructors that can starts with Zend_ . The counting constructors is slow, because the constructor doesn't have to be declared in the class, but in an extend class, which doesn't have to starts with Zend_ . So basically the algorithm is in pseudo language: 1. give me all classes that starts with Zend_ 2. for every founded class 3. If has a constructor continue 4. If doesn't have a constructor, try to find recursively constructor in extended class until a constructor will not be find Because there is many classes that starts with Zend_ the algorithm is slow. So I think the fix for this case, can be that the code completion will suggest in this case just name of classes and when a user choose a class, then the constructor is counted just for the selected class. The disadvantage of this is that you will not see parameters of constructor. Do you agree with this solution
So I did the fix, I will commit probably tomorrow. Measuring on my computer: - Finding all class that starts with Zend_ : 102ms - Counting constructors takes before the fix: 1,7s - 2,3s So after the fix the cc appears in the constructor case during 102 ms.
Petr, thanks for your help, but what about my case (see comment #10) - I did not use Zend or any other libs, and OOP even. And on empty php project code completion take time at least 2 seconds. For example, if I try to type something from base PHP functions like "array_*", I have same result. Currently, I totally switch off "Code Completion" in auto mode because of slowness, and invoke it only in rare cases by Ctrl+Space. I prefer to take care on my nervous system... I think, that Code Completion feature must provide 2 things for developers: 1) more simple code writing; 2) speed! IMHO code completion must work "very fast", "immediately", "instantly". Without speed, Code Completion can't simplify code writing, because each time on CC invoking you must wait approximately same time, which you spend on typing code by hands. It's useless. I have an idea to make additional version of Code Completion, let's name it "Stupid Code Completion", something like Notepad++ does. This means NO objects, NO constructors, NO variables scope, NO function parameters etc - only words from files! Yes, it's stupid (I say it before), but it must be very, very fast! And for me, "Stupid Code Completion" will be better, than present very smart, but very slow Code Completion. If I have enough amount of time and power, I'll try to do it...
@zvk77: I will evaluate your case. Could I ask what is your machine set up?
For the constructor case I have created new issue #193040.
@zvk77: Could you more specify your case? I don't see any specific information. I need to somehow reproduce it. Do you have a file with many functions? I'm sorry, but I need to find out what is exactly slow. You can also create and attach a snapshot as I have described here: http://blogs.sun.com/netbeansphp/entry/is_the_ide_slow_send . It can be helpful.
2 Petr Pisl My computer is dual core Intel Pentium 3GHz , 1.5GBytes of memory, WinXP SP2, optimized system performance, I give big amount of memory to NB (-J-Xmx512m -J-Xms256m, and usually NB use approx. 100MB of 256 heap) Now I use NB 7.0m2 PHP, in other versions same situation. shortcut is netbeans.exe -J-Dorg.netbeans.core.TimeableEventQueue.report=35000 -J-Dorg.netbeans.core.TimeableEventQueue.quantum=30000 -J-Xverify:none -J-Xmx512m -J-Xms256m -J-Dawt.useSystemAAFontSettings=gasp Examples and snapshots i'll make later.
@zvk77: Your computer is not slow one:). It would be great if you can attach the profile report. It's better to create the report that cover really short time, like a few seconds during the slow action, because if the report will profile more then just the action, then it's almost impossible to find out useful information.
Created attachment 103843 [details] Snapshot of a problem in build trunk-nightly-201012080001-ml-php-windows
Created attachment 103847 [details] See how fast autocomplete works for interfaces? Because interfaces are just a bunch of names that point to some file addresses!
@cgalive: I'm now confused:). ad Snapshot of a problem in build trunk-nightly-201012080001-ml-php-windows Is the problem that there is now Zend classes in the code completion? Does have the project Zend framework on the include path, because the two constructors that are displayed at the picture comes from php runtime, not from zend framework. ad See how fast autocomplete works for interfaces? Because interfaces are just a bunch of names that point to some file addresses! I'm sorry, but I don't understand. Does it mean that it's slow for you the see the code completion in this case. I have measure it and it took 58 ms. But probably my fixes are not in the main repository. The build that you refer is created from main repository and I developed over the web-main repository and the changes are pushed to the main repository automatically. But yesterday it was broken due failing tests in core area. Could you please a build from bertram continual server (http://bertram.netbeans.org/hudson/job/PHP-build/ ). Thanks
Hi Petr I was just writing a comment to make the problem more clear, but it's taking me too long to write. I attached those two images just to include their urls in my comment. I will post the comment as soon as I can, hopefully tomorrow. But anyway, thanks for your efforts, we really appreciate it.
@Petr Pisl I did many experiments with various number of files in PHP project, and I do not understand yet completely the source of slowness. I take my existing project, which contains: 5 PHP files (< 60K total) 3 CSS files (< 20K total) 18 JS files ( ~ 460K total) 580 HTML files ( ~ 870K total) I create empty project, and copied existing project to new project file-by-file, each time reloading NetBeans and invoking Code Completion on string "arra". First I copied PHP files, then CSS, etc. All of 580 HTML files is placed in one folder, and this folder included in project "Ignored folders". Here is results of my experiments. 1. NOT surprising fact: more files in project - slower Code Completion. 2. Surprising fact: with installed "Scan On Demand" module, Code Completion works significantly slower!!! In my project, when "Scan On Demand" module is active, first Code Completion on string "arra" takes 10 or more seconds! I tried to use "Refresh All" action of "Scan On Demand" BEFORE using CC, and this has no effect. With switched off "Scan On Demand" module, first CC on "arra" takes ~1 sec. Maybe, problem in "Scan On Demand" module? Now, I don't know what to do next. If I switch ON "Scan On Demand" module, I'll get slow CC If I switch OFF "Scan On Demand" module, I'll get slowness by annoying "Scanning projects...". Hmmmmm! :[
Petr, thank you so much for the fix. I downloaded the right nightly build version and tested it for a few minutes and it works amazing. It's just so much faster than the last one. After the 'scanning project' is finished, you can start typing in class names and and most the times it gives the results in less than a second. So exciting!! Now that the fix works, I have got many of the answers that I was gonna ask in my incomplete comment. Code completion has yet a lot to improve. Especially in 'scanning projects' process, but this fix simply addresses the biggest problem users have had with it. I have to add that I haven't tested it thoroughly, so please consider my comment based on a few minutes experience. I recommend everyone to download the new build. (Although it woraks a bit buggy. Include Paths act weird and you simply have to reset the include paths, and some of the options don't get imported from the stable version)
Uh, Petr, Is it possible to push this fix to one of the stable releases? I just updated to 6.9s latest patch and this fix isn't in its release note. If this fix doesn't break anything, then it wouldn't hurt to have it in a stable release. I download netbeans php's latest builds, but they are buggy. If that isn't possible, is there anyway we could manually push this fix to our own installation?
@cgalive: Which bugs prevents to use dev build? I'm afraid that porting the fix in the old source tree can take more time than fix the new bugs:). I will look at this later. And which fix do you have in mind?
Another big improvement of code completion response time will be to limit the number of results. From what I can tell this can be done from the index module. You will never need nor use the > 1000 classes that are suggested when you type in "Zend_". A result limit of 100 (maybe configurable) will speed up things considerably on this edge use cases.
I'm converting this issue to the task. For the particular case, needs to be filed separate issues. The mentioned here should be already fixed.
As Petr wrote, this should be fixed.
4 years and still this problem not fixed ? Srsly? I trying to adaptate myself netbeans but it so slow code completion it is making me slow too. It must be instant. And dont tall computer issue because it is not. I have i5, 8gb ram, ssd Make core html, css, php functions-codes instant. Netbeans must be more fast.
Don't reopen 4 years old issue. If you are facing some problem, create new one, attach a sample project, write exact steps to reproduce and attach a profiling snapshot.