This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 258444 - [newcodemodel]Slow code completion for even a toy project
Summary: [newcodemodel]Slow code completion for even a toy project
Status: NEW
Alias: None
Product: cnd
Classification: Unclassified
Component: Code Model (show other bugs)
Version: 8.1
Hardware: Macintosh Mac OS X
: P3 normal (vote)
Assignee: Vladimir Voskresensky
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-03-19 16:26 UTC by keithyip
Modified: 2016-09-02 11:36 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
video and sample project (5.81 MB, application/zip)
2016-03-19 16:26 UTC, keithyip
Details
profile (574.94 KB, application/octet-stream)
2016-03-20 02:34 UTC, keithyip
Details

Note You need to log in before you can comment on or make changes to this bug.
Description keithyip 2016-03-19 16:26:07 UTC
Created attachment 158926 [details]
video and sample project

The code completion window needed 6 seconds to show up. There was 1 GB free RAM (truly free, not occupied by the slow file cache management system). CPU ramped up to 200%.

In the attachment, there is an video and the project shown in the video. To use the project, boost and cpp-netlib are needed.

There are many slow code completion bug fixed. I am not sure if 6 seconds is counted as slow. Can't imagine the time needed for the code completion window to show up for a real project.

System:
OS X 10.11.3 (15D21)
MacBook Pro (Retina, 13-inch, Early 2015)
2.9 GHz Intel Core i5
16 GB 1867 MHz DDR3
Comment 1 Vladimir Voskresensky 2016-03-19 17:14:33 UTC
Hello,

Could you, please, provide Profile-me snapshots during completion invocation?
http://wiki.netbeans.org/FaqProfileMeNow

Btw, one file which includes boost can touch 2000 boost+stl files :-)
So, the project become not so toy.

I'd like to as you the following:
Could you,please, have a look in the IDE Help->About.
There is a line
Cache directory: /tmp/nb_cache_dev

remember this directory, close the IDE, clean this folder completely, start the IDE with extra properties:

netbeans -J-Dcnd.modelimpl.timing=true

and send me the message log (or View->IDE Log) when parsing progress is finished and you will see in message log smth like in mine:
==========
Parsing statistic of JConvert:
Parsed 258 KLines, Time 30 seconds, Speed 8 KLines/second, Max Memory 1140 Mb
Work, %        Speed, KLines/second    Memory, Mb
    5        2        255
    10        6        236
    15        6        257
    20        9        596
    25        23        293
    30        38        384
    35        46        459
==========
Btw, it would be helpful if you send message log
http://wiki.netbeans.org/FaqLogMessagesFile
Comment 2 keithyip 2016-03-20 02:34:14 UTC
Created attachment 158930 [details]
profile

After I opened nb, I record 3 operations with 2 seconds in between.

1. Typed std::
2. Typed #include (not important but just funny to see how slow "include" is)
3. Typed #include <str>
Comment 3 keithyip 2016-03-20 02:41:19 UTC
Parsing statistic of code-completion:
Parsed 345 KLines, Time 44 seconds, Speed 7 KLines/second, Max Memory 435 Mb
Work, %		Speed, KLines/second	Memory, Mb
	5		3		172
	10		3		224
	15		5		211
	20		8		362
	25		5		360
	30		2		272
	35		22		362
	40		15		301
	45		10		269
	50		2		316
	55		15		255
	60		29		351
	65		13		300
	70		10		305
	75		18		331
	80		17		354
	85		32		337
	90		15		346
	95		27		350
	100		29		435
Comment 4 keithyip 2016-03-21 06:28:57 UTC
cpp-netlib is the problem. However in Xcode, the code completion popup appeared instantly, after a very long initial indexing and the first time 3 seconds delay in the popup. Is it because Xcode more aggressively cache the data?
Comment 5 Vladimir Voskresensky 2016-03-22 02:36:38 UTC
Alexander, can you have a look, please?
Comment 6 Vladimir Voskresensky 2016-03-22 02:49:19 UTC
(In reply to keithyip from comment #4)
> cpp-netlib is the problem. However in Xcode, the code completion popup
> appeared instantly, after a very long initial indexing and the first time 3
> seconds delay in the popup. Is it because Xcode more aggressively cache the
> data?
Yes, it could be. You can try the following:
create RAM disk (mount 1Gb of RAM)
Then start NB with 
netbeans --cachedir /path/to/ram/disk
Comment 7 Alexander Simon 2016-03-22 08:41:49 UTC
Investigation.

Profiling snapshot has 3 cycle of parsing file:
/Users/keithyip/Documents/Workspace/code-completion/main/src/main.cpp
Each parsing consumes about 13 seconds.
Hot spot is ClankDriverImpl.preprocessImpl() - 40 seconds.
The most time consuming method is java.io.UnixFileSystem.canonicalize0[native] () - 24 seconds.
For example that method is called from:
ClankFileObjectBasedFileSystem.getFileObject()
Comment 8 Vladimir Voskresensky 2016-03-22 11:30:22 UTC
Thanks, Alexander. What are the clients which cause 3 parsing?
Can we collect all data in one parsing pass and cache for future use by these 3 clients?
Comment 9 Alexander Simon 2016-03-22 12:25:28 UTC
(In reply to Vladimir Voskresensky from comment #8)
> Thanks, Alexander. What are the clients which cause 3 parsing?
> Can we collect all data in one parsing pass and cache for future use by
> these 3 clients?
see comment #3
Comment 10 Alexander Simon 2016-03-22 12:26:09 UTC
(In reply to Alexander Simon from comment #9)
> (In reply to Vladimir Voskresensky from comment #8)
> > Thanks, Alexander. What are the clients which cause 3 parsing?
> > Can we collect all data in one parsing pass and cache for future use by
> > these 3 clients?
> see comment #3
I mean comment #2
Comment 11 keithyip 2016-03-22 14:29:29 UTC
(In reply to Vladimir Voskresensky from comment #6)
> (In reply to keithyip from comment #4)
> > cpp-netlib is the problem. However in Xcode, the code completion popup
> > appeared instantly, after a very long initial indexing and the first time 3
> > seconds delay in the popup. Is it because Xcode more aggressively cache the
> > data?
> Yes, it could be. You can try the following:
> create RAM disk (mount 1Gb of RAM)
> Then start NB with 
> netbeans --cachedir /path/to/ram/disk

It is even slower. I used this command to create a RAM disk
diskutil erasevolume HFS+ 'RamDisk' `hdiutil attach -nomount ram://2097152`