Bug 179727 - [69cat] Code completion is slow
[69cat] Code completion is slow
Status: CLOSED FIXED
Product: ide
Classification: Unclassified
Component: Performance
3.x
All All
: P2 with 19 votes (vote)
: TBD
Assigned To: issues@performance
issues@performance
EXCEPTIONS_REPORT perf-profileme
: 7.0_WAIVER_APPROVED, PERFORMANCE
Depends on: 180230 192625 192628 193561 195279 197267 197270 197271
Blocks:
  Show dependency treegraph
 
Reported: 2010-01-21 00:35 UTC by jmichelberger
Modified: 2014-02-16 11:24 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
nps snapshot (256.00 KB, application/nps)
2010-01-21 00:35 UTC, jmichelberger
Details
profiler dump (74.79 KB, application/octet-stream)
2010-12-01 17:15 UTC, gholmer
Details
NetBeans log file (1.66 MB, text/x-log)
2010-12-01 17:16 UTC, gholmer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description jmichelberger 2010-01-21 00:35:08 UTC
Build: NetBeans IDE Dev (Build nbms-and-javadoc-4704-on-100118)
VM: Java HotSpot(TM) Server VM, 16.0-b13, Java(TM) SE Runtime Environment, 1.6.0_18-b07
OS: Windows XP

User Comments:
GUEST: invoked cc in test



Maximum slowness yet reported was 9875 ms, average is 5572
Comment 1 jmichelberger 2010-01-21 00:35:14 UTC
Created attachment 93441 [details]
nps snapshot
Comment 2 Jaroslav Tulach 2010-01-21 05:12:19 UTC
The first few reports indicate the CC is blocked by not all projects being open:
http://statistics.netbeans.org/exceptions/exception.do?id=328527

The other shows that the CC waits for 
org.netbeans.modules.java.source.tasklist.TaskCache.refreshTransaction()
and
org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$RootsListeners.add()
http://statistics.netbeans.org/exceptions/exception.do?id=328565
Comment 3 Quality Engineering 2010-02-03 21:49:36 UTC
Integrated into 'main-golden', will be available in build *201002040200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/0661f74f1891
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #179727: Don't start the code completion self profiling when 'scanning in progress...'. It is known that currently no (java) completion is shown while the scanning is running and it is useless to gather more information about such case. As it is hard to find out from editor.completion module that parsing.api is scanning, the actual method relies on 'polite wait message' being provided by the completion provider.
Comment 4 iceman81 2010-02-26 08:01:05 UTC
I have observed that with large files (even for a 1000 line file) this is very evident. Also, (not sure if this is related but) along with code complete, the inspect Members function is equally slow.
Comment 5 Tomas Zezula 2010-02-26 08:10:05 UTC
In the NB 6.9 M1 there was a regression in the persistent caches used by code completion to show param names im methods of class files (rt.jar, libs) which may cause significant slowdown.
The caching was recently fixed: http://hg.netbeans.org/jet-main/rev/f1c0e7896b92
Comment 6 iceman81 2010-02-26 08:17:49 UTC
Do you know if this fixed has made it to the nightly build -> 201002260200?
Comment 7 iceman81 2010-02-26 08:18:17 UTC
Do you know if this fix has made it to the nightly build -> 201002260200?
Comment 8 Tomas Zezula 2010-02-26 08:21:10 UTC
No, the fix was integrated today into jet-main repository.
It will take some time to propagate into the night build.
Comment 9 iceman81 2010-02-26 08:27:11 UTC
Thanks. Will test this out once it is available in the nightly.
Comment 10 Michel Graciano 2010-03-04 05:25:09 UTC
Anyone tested it again? The patch was already integrated [1] and should be available after Feb 26.

[1] http://hg.netbeans.org/main/rev/f1c0e7896b92
Comment 11 iceman81 2010-03-04 08:30:35 UTC
I have tested this. Code completion is a lot better now but inspect members function still seems to be slow. Its hard for me to quantify how slow.
Comment 12 stefan79 2010-03-05 04:10:55 UTC
I´ve tested it at build 201003040200.
I´ve still some slowness-Detections.
Please have a look at report http://statistics.netbeans.org/analytics/exception.do?id=348021.
Comment 13 akochnev 2010-03-05 07:38:27 UTC
I just tested this w/ 201003050200 and there is still a significant slowdown on the groovy code. I have a Grails project : in Groovy code, when I hit "." for code completion, it takes 2-3 seconds for the initial popup to show up. Then, if I type the first couple of characters in the method , it takes additional 2-3 seconds to display the matching subset. I'll file a separate issue on the Grails side. 

In java code, code completion in the same project is instantaneous. 
Overall, it is very unusable.
Comment 14 Alexandr Scherbatiy 2010-04-01 11:16:48 UTC
The issue has 504 duplicates and affects NB JavaFX plugin
Comment 15 Tomas Pavek 2010-04-01 15:36:45 UTC
This issue is just a placeholder (pool) for the automatically collected performance problems related to code completion, as a base for further analysis. For specific problems separate issues should be filed.

If you can see snapshots related to JavaFX code completion, please file a separate issue for that and link the snapshots (or better separate them from this report to their own).
Comment 16 Tomas Pavek 2010-04-01 15:50:26 UTC
There are some infrastructure problems that should be fixed:
(1) The sampling may not stop sometimes - issue 183201.
(2) Since about 100226, the snapshot have absurdly high spend time shown in the table (like millions ms). The snapshots themselves are good.
(3) There seems to be no minimum time the sampling should run to produce reasonable snapshot. It starts with a delay, but if CC appears short after that the snapshot may be too short. Such snapshots should not be probably reported. E.g. saw a snapshot of 56ms which is way too little to say anything useful. I guess the minimum time should be something like 300ms at least.
Comment 17 Tomas Pavek 2010-04-14 17:11:48 UTC
The mentioned 3 problems have been addressed by fix of issue 183832.

Now the self-profiling starts after 750ms, a problem is reported and snapshot created if it exceeds 2s (i.e. should have at least 1250ms long snapshots).
Comment 18 Jaroslav Tulach 2010-04-14 17:22:19 UTC
We will analyze the snapshots and conclude what shall be fixed for post 6.9 version.
Comment 19 disciple 2010-07-07 23:01:51 UTC
It is slow more often than not. Very very slow.
Makes Groovy so unbearable when using code completion?
Comment 20 tbrunhoff 2010-09-03 20:50:00 UTC
Not sure where you guys are on this, but the current version below is pretty slow. Over the last month I've been back to writing c++ and the "code completion is slow" popup occurs about 1 in 4 times I hit a dot or a pointer. And the 3 out of 4 times it does not popup it is never instantaneous. Usually over one second.

I'll bump this to P2. I spend about half my time writing code with VS 2010 on a VIAO laptop with a ultra low power 1.6Ghz processor, and code completion there is always instantaneous. Netbeans, on the other hand, I run on a 2.5 Ghz core 2 quad.

Product Version: NetBeans IDE Dev (Build 201008280001)
Java: 1.6.0_18; Java HotSpot(TM) 64-Bit Server VM 16.0-b13
System: Linux version 2.6.32.9-70.fc12.x86_64 running on amd64; UTF-8; en_US (nb)
Userdir: /home/toddb/.netbeans/dev
Comment 21 b166er 2010-09-20 22:29:45 UTC
remove me from CC List
Comment 22 virtualeyes 2010-10-15 17:43:31 UTC
Just downloaded Sun JDK 1.6 v22 and NetBeans 6.9.1

Groovy code completion is painfully slow, far faster to type everything out manually. 

Of course, just typing any text within a Groovy class rockets the CPU to 80-100%, so looks like code completion is not the sole problem.

Fedora 13 x64 on Sony Vaio F series laptop: Intel quad-core extreme + 8GB RAM.

I love NB, but this is a show stopper, can't waste time coding long-hand.
Comment 23 velodiver 2010-10-23 22:17:28 UTC
Have you considered netbean's cache as a source of this problem?  

~ .netbeans/6.9/var/cache

I added a bug recently about cache corruption because code completion completely failed a few times in 6.9.1.  (#190832).

Don't know how the cache is designed, but it appears to either become corrupted or just get too large. 

Delete that directory (netbeans can not be open). NB will rebuild and performance will improve for a while.

My cache is now: 250 MB 31,000 files, 6000 folders.
Delete and reopen the same project. Rebuild cache is 110 MB.
Comment 24 hantsy 2010-11-20 04:11:27 UTC
I think the cache is the problem...NetBeans must include a high perfermance cache system.
The cache is not only for code completion, but also for scan, go to, find usage...

1. why scan the jar files when I opened netbeans everytime. For a java project, it need scan the jar once when I added it to the project. Other jar files, such as Java Platform(java 6 sdk, java 7 sdk),Java Libraries and Java EE server, only scan once when added it to NetBeans IDE. They can be share for all Projects, in a project, only need add a reference to the cached result.

2. the maven "go to" feature to find the maven artifact is 10 time slower than the Eclipse m2eclipse.

3. the Maven jar scan is another problem, why is scanned periodly. The central maven repository is shipped with the cached info about the whole repository. Download it and reuse it. In NetBeans 7.0, I noticed that it declared it can monitor external changes effetively. Why not only scan changed jar in local maven repository?
Comment 25 Jesse Glick 2010-11-20 15:10:43 UTC
#1 - when a JAR has been scanned before, NB will subsequently only check its timestamp to make sure it has not since changed. That is not related to this issue.

#2 probably unrelated; do not discuss here.

#3 also unrelated; see discussion in bug #190675.
Comment 26 seanja 2010-11-21 18:11:49 UTC
To be fair I just hit the short cut to autocomplete the code without typing anything in. This was in a drupal project so there is a lot of stuff to bring back.
Comment 27 hantsy 2010-11-22 08:02:01 UTC
Although the problem are not related to those I motioned...I think all of them are caused by the cache system of NetBeans(I do not know if there is a cache system in NetBeans for all), if there have no effective algorithm in cache system...currently patch and patch again, it can not resolve the problem at all.
Comment 28 stefan79 2010-11-23 07:13:35 UTC
I think the most slowness-reports are while "Scanning projects".
While this task runs, many other features are very slow or still not available (Code-Completion, GoToClass, ...).

Is there no way to make a dirty-read, while scanning projects?

At 6.8 or 6.8 was a big point "Make the IDE useable while scanning projects".
This point is much better, but the dirty-read-feature is missing?
Comment 29 gholmer 2010-12-01 17:15:04 UTC
Created attachment 103504 [details]
profiler dump

Had only one project open, IDE seemed quiescent. Several times today I've seen CPU usage jump up very high. Build number 201012010001. Will also attach log file.
Comment 30 gholmer 2010-12-01 17:16:37 UTC
Created attachment 103505 [details]
NetBeans log file

NB was running with -J-Dorg.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.level=FINE
Comment 31 Jiri Kovalsky 2011-01-05 14:22:26 UTC
Just for the record, according to Dusan Balek the snapshot from Glenn has nothing to do with code completion as well as 20 latest exception reports.
Comment 32 idle 2011-01-08 13:48:45 UTC
Yestarday somtimes code completion worked perfect, but today i must wait up to 10 sec.
Comment 33 zburnham 2011-02-21 14:35:08 UTC
I'm seeing this sort of thing on Mac OS X as well.  It does seem to be happening during the 'Scanning projects' phase of having NB open a project.
Comment 34 fabriziogiudici 2011-03-06 19:09:14 UTC
I?m seeing this today while writing XML code in a DocBook file. It's really annoying as it stops every few seconds, and BTW it's completely unable to auto-complete the XML tags. I've uploaded a snapshot by means of the IDE.
Comment 35 fabriziogiudici 2011-03-06 19:39:46 UTC
Removing var/cache didn't solve the problem. It reappeared immediately after NetBeans was restarted.
Comment 36 _ tboudreau 2011-03-09 00:08:33 UTC
It seems fairly obvious that the locking model for code completion is too coarse.
Comment 37 cmygeHm 2011-03-10 14:57:33 UTC
I don't know. It is weakly computer and many processes. That's all...
Comment 38 David.m Beer 2011-03-12 12:24:39 UTC
It seems that any long running task or quite intensive task is blocking other activities. Yesterday while the IDE was updating the maven repository index the whole IDE just ground to a halt and stopped me from working. Especially code completion.
Comment 39 Marian Mirilovic 2011-03-25 23:21:10 UTC
Please evaluate : if you do not plan to fix it for NB 7.0 ask for waiver ASAP.
Comment 40 narve 2011-03-26 00:47:47 UTC
My 50 cents: Netbeans is currently utterly un-usable (for my projects at least) because of performance problems. These performance problems surely can not be delayed to after 7.0 release??
Comment 41 m_potociar 2011-03-26 01:04:27 UTC
+1 to the previous "utterly unusable" comment.
Comment 42 rcasha 2011-03-26 04:13:06 UTC
Note to everyone: It has been suggested on the NetCAT mailing list that we should re-file this bug since the slowness may have different causes depending on the type of file being edited.

In my case it's the JSF editor, but others have reported slowness with the PHP editor etc. It seems that the exception reporter automatically assigned these as duplicates of one another when they're not.

Ideally, test this using the latest RC build.
Comment 43 troodon 2011-03-26 09:19:08 UTC
I have been using the daily php builds and can tell that code completion does not work for most of the time. Further typing in the php editor is annoyingly slow. For example scrolling up and down using the arrow keys in php files containing 2000 or more lines is really slow and the editor is barely usable.
Comment 44 stosh 2011-03-26 13:12:52 UTC
I've been using 7.0 beta 2 with Java code and it is great.  The code completion works every time.
Comment 45 jag 2011-03-26 15:48:37 UTC
It's really erratic for me.  And depends heavily on how long NetBeans has been running.  It'll be great for a while when I first fire it up.  But after a few hours it becomes unusable.  Long (20 second-ish) pauses every 10-ish keystrokes.  I'd call this a P1 show-stopper
Comment 46 Franc.Esco 2011-03-28 15:25:49 UTC
Code completion has always been *VERY SLOW*, taking 3-5 secs each time autocompletion is called.
Please fix it, it's the most annoying issue of NB so far.
Comment 47 Marian Mirilovic 2011-03-30 11:58:14 UTC
dstrupl sent request for waiver to nbdev@netbeans.org . 
So removing 70 fix candidate and change Target Milestone to TBD.
Comment 48 greggwon 2011-03-30 14:16:07 UTC
It seems pretty clear at this point that the NBDEV team has no ability to provide performant code completion.  It seems like it is time to rip out everything that uses the cache, as designed and just start over.  This issue seems to be the #1 reason why people choose to NOT use netbeans.  It's a major roadblock and I am completely convinced after 5 years of complaining about it and opening issue after issue both explicitly and through the automated mechanisms this issue is tracking that there is no solution that will ever happen.

The NBDEV folks are stuck in their ways, convinced they know everything, refuse to listen to the users and just can not imagine how we can be having problems with their golden GEM of an IDE.

There is no way to implement a cache without its access being completely seamless for both read and write.  Blocking readers because writers are busy and the like has been demonstrated in java.util.concurrent related research as impossible to be performant.

I really don't know how else to say this.  It's hard to be this critical given all the time and energy that the NBDEV team does put into Netbeans.  But the place where the project is at now, is just terrible.

There is just too much stuff that wrong with the source code scanning and the symbol lookup mechanisms.  It's a lot of software, and all the micro tuning being done is just not fixing much of anything it seems.
Comment 49 zburnham 2011-03-30 14:27:45 UTC
greggwon, your comments are neither constructive nor useful.  While I share your frustration at the poor state of code completion in current versions of NB, it's clear that the team is aware of the situation and will fix it if they can.  If they can't fix it, you are welcome to try another editor.

You can always fix the problem yourself and submit a patch.
Comment 50 dallstar 2011-03-30 14:38:45 UTC
What I believe gregwon is trying to say is that there may be no solution other than rewriting whole lot.

I know the pain as I am working on a massive codebase (some of which I cant access with NetBeans due to slowness of code completion with larger files) and I can understand that some things are unfixable due to initial design and approach of developers. 

So as much as I really like NetBeans I hope something can be done about it's occasional (for me) slowness because it really renders it useless at times.

My suggestion - introduce an easily accessible button on the panel somewhere (next to memory usage graph?) 'Disable Code Completion' or anything that is slow but not 100% necessary. Then I will be able to easily switch it off for large files and continue to use other features of NetBeans without CC. My argument is - I can code without CC but I cant code with 2 sec intervals between every letter I type.

Cheers!
Comment 51 ssanders 2011-03-31 12:19:12 UTC
There should be an OS option that's just "Windows," as this affects at least XP through 7.
Comment 52 David.m Beer 2011-03-31 12:36:01 UTC
I changed the OS option to All as this seems to affect all OS systems.
Comment 53 simon_orange 2011-03-31 14:14:36 UTC
just trying to remove myself from this behemoth
I just can't cope with it...
Comment 54 Marek Fukala 2011-04-01 08:58:46 UTC
exception report 434322 => Bug 197314 - CSS completion blocked by CSS parser error analysis
Comment 55 Marian Mirilovic 2011-04-02 06:06:02 UTC
All : do not change Target Milestone, Platform.
Comment 56 _ tboudreau 2011-04-19 01:33:43 UTC
Performing a service to humanity and removing 503 people from the cc list for this bug.  Anyone who actually *wants* notification every time something inconsequential changes here, feel free to re-add yourself.

If anyone admining the site needs a copy of the original cc list, ask me for it.
Comment 57 alied 2011-04-19 03:43:53 UTC
(In reply to comment #56)
> Performing a service to humanity and removing 503 people from the cc list for
> this bug.  Anyone who actually *wants* notification every time something
> inconsequential changes here, feel free to re-add yourself.
> 
> If anyone admining the site needs a copy of the original cc list, ask me for
> it.

Thanx
Comment 58 Jesse Glick 2011-04-19 22:25:40 UTC
(In reply to comment #56)
> If anyone admining the site needs a copy of the original cc list, ask me for
> it.

Available from: http://netbeans.org/bugzilla/show_activity.cgi?id=179727
Comment 59 Marian Mirilovic 2011-05-18 11:08:39 UTC
We improved the way Slow Code Completion reports are treated and there shouldn't be a new report marked as duplicate of this issue anymore. All newly reported issues will go directly to particular areas.
Comment 60 Jackie_Rosen 2014-02-16 11:24:34 UTC
SPAM - Removed by Administrator


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo