This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Created attachment 138583 [details] IDE Profiling Output during unresponsive state As soon as code assistance is enabled, Memory and CPU load rise until IDE is unusable and takes many seconds to react to any input. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 32035 sascha 20 0 9144m 6.5g 14m S 363.5 85.4 96:06.72 java from netbeans.conf: netbeans_default_options="-J-client -J-Xmx6G -J-Xss2m -J-Xms512M -J-XX:PermSize=32m -J-Dapple.laf.useScreenMenuBar=true -J-Dapple.awt.graphics.UseQuartz=true -J-Dsun.java2d.noddraw=true -J-Dsun.java2d.dpiaware=true -J-Dsun.zip.disableMemoryMapping=true -J-Dsun.awt.disableMixing=true" java is: java version "1.7.0_25" Java(TM) SE Runtime Environment (build 1.7.0_25-b15) Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) There are roughly 1,1 GB of source files in the project, and about 5 GB of pre-compiled stuff and images. Some Files have 10-20 thousand lines.
Please, answer on following questions: How many CPU cores do you have? How many files do you have in the project? How many source code files (c/c++/headers) do you have in the project? How many lines of source code contains your project? Please, attach message log.
more /proc/cpuinfo | grep "model name" model name : Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz everything without svn directories: find trunk -type f | grep -v '.svn' | wc -l 70000+ only c/c++ files: find trunk -type f | egrep '(\.cpp$|\.c$|\.h$|\.hpp$|\.cxx$|\.c++$|\.h++$)' | wc -l 27000+ linecount in c/c++ files: find trunk -type f | egrep '(\.cpp$|\.c$|\.h$|\.hpp$|\.cxx$|\.c++$|\.h++$)' | xargs -i wc -l {} | sed -e 's/^\([0-9]*\) .*/\1/g' | paste -sd+ | bc 7570000+ Did you more than the above information by "Please, attach message log"? Or is the above info what you needed? BTW: I know these numbers are bad, but Qt Creator can load them without thinking twice. Qt Creator has fewer useful features, of course.
Thanks for the information about your project. It is really huge. Have you seen the notification from IDE that it face memory lack? (If you click on parsing progress bar you should see red "Low memory" message). For your application (27K files/7.5MLoc) it's worth to try and give more memory (we are working on reducing memory for huge projects). For 5MLoc I had to use 6G => for 7.5MLoc try 9G (if you have enough physical memory)
Btw, what is the memory consumption for Qt Creator when it load/parse your project?
(In reply to Vladimir Voskresensky from comment #3) > Thanks for the information about your project. It is really huge. Have you > seen the notification from IDE that it face memory lack? > (If you click on > parsing progress bar you should see red "Low memory" message). Not that I can remember. I saw that warning frequently but raised memory (by -J-Xmx ) until it was gone. > For your application (27K files/7.5MLoc) it's worth to try and give more > memory (we are working on reducing memory for huge projects). For 5MLoc I > had to use 6G => for 7.5MLoc try 9G (if you have enough physical memory) I re-raised from 6 to 7G. But I have only 8GB in my machine. This seems to help a bit already. > Btw, what is the memory consumption for Qt Creator when it load/parse your > project? Qt Creator 2.7.0 after opening and parsing the project (< 30 Seconds) PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 19106 sascha 20 0 3650m 516m 55m S 0.0 6.6 0:32.81 qtcreator(In reply to Vladimir Voskresensky from comment #4)
(In reply to Vladimir Voskresensky from comment #3) > For 5MLoc I had to use 6G => for 7.5MLoc try 9G (if you have enough physical memory) As mentioned I assigned 7G but to no avail. After some minutes NB hung again with high cpu load. At that time no visible parsing was active and no memory warning visible.
> As mentioned I assigned 7G but to no avail. After some minutes NB hung again > with high cpu load. At that time no visible parsing was active and no memory > warning visible. I looked at profiling snapshot: - a subversion client+file system consume a lot of resources. Probably it is a duplicate of the bug #225028. To be sure we need a memory heap dump. Could you also make experiment without subversion: - copy project with sources - remove all subversion folders - reproduce problem on the project without subversion. >Please, attach message log It is View->IDE log. It also can help.
*** Bug 233900 has been marked as a duplicate of this bug. ***
As an owner of bug which considered a duplicate of this I can confirm that I use network filesystem too (but not nfs, afs, apple sharing). And too have very high cpu utilization and at the same time high transfer rate between NetBeans and storage. Eventually this leads for me to broken CodeAssist parsing: system exludes a lot of files from CodeAssist, and even you include them manually doesn't see them. Plus any change leads to project reparse, which makes it sequential process. Looks like CodeAssist cannot successfully parse relatively big project from network share and continuously tries to do that occupying a lot of CPU and network bandwidth.
OverSun and gnSascha, could you, please. provide info from Help->About Thanks! Vladimir.
(In reply to Vladimir Voskresensky from comment #10) > OverSun and gnSascha, could you, please. provide info from Help->About Product Version: NetBeans IDE 7.3.1 (Build 201306052037) Java: 1.7.0_25; Java HotSpot(TM) 64-Bit Server VM 23.25-b01 Runtime: Java(TM) SE Runtime Environment 1.7.0_25-b15 System: Linux version 2.6.32-358.2.1.el6.x86_64 running on amd64; UTF-8; en_US (nb) User directory: /home/sascha/.netbeans/7.3.1 Cache directory: /home/sascha/.cache/netbeans/7.3.1 Thank you.
Product Version: NetBeans IDE 7.4 Beta (Build 201307092200) Java: 1.7.0_25; Java HotSpot(TM) 64-Bit Server VM 23.25-b01 Runtime: Java(TM) SE Runtime Environment 1.7.0_25-b15 System: Mac OS X version 10.8.4 running on x86_64; UTF-8; en_US (nb) User directory: /Users/dodintso/Library/Application Support/NetBeans/7.4beta Cache directory: /Users/dodintso/Library/Caches/NetBeans/7.4beta I've actually installed 7.4 beta yesterday, everything exactly the same was for 7.3.1
(In reply to Alexander Simon from comment #7) > > As mentioned I assigned 7G but to no avail. After some minutes NB hung again > > with high cpu load. At that time no visible parsing was active and no memory > > warning visible. > > I looked at profiling snapshot: > - a subversion client+file system consume a lot of resources. > Probably it is a duplicate of the bug #225028. To be sure we need a memory > heap dump. > > Could you also make experiment without subversion: > - copy project with sources > - remove all subversion folders > - reproduce problem on the project without subversion. > > >Please, attach message log > It is View->IDE log. It also can help. BTW: There was no heap dump created, although there's defined: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/kremers/.netbeans/7.3.1/var/log/heapdump.hprof Can/should I force a heap dump, when Netbeans starts to get sluggish?
(In reply to gnSascha from comment #13) > BTW: There was no heap dump created, although there's defined: > > -XX:+HeapDumpOnOutOfMemoryError > -XX:HeapDumpPath=/home/kremers/.netbeans/7.3.1/var/log/heapdump.hprof > > Can/should I force a heap dump, when Netbeans starts to get sluggish? You can set heap size to 3G and probably java will generate a dump. Or you can force a heap dump.
Sascha, thanks for the logs. Please, follow http://wiki.netbeans.org/NativeFileNotifications and increase number of allowed max watched files On my system I have :-) #ulimit -n 900000
(In reply to Vladimir Voskresensky from comment #15) > Sascha, thanks for the logs. > Please, follow > http://wiki.netbeans.org/NativeFileNotifications > and increase number of allowed max watched files > > On my system I have :-) > #ulimit -n > 900000 % cat /proc/sys/fs/inotify/max_user_watches 900000 % ulimit -n 1024 it seems to delay the problem. But a Netbeans session started and left over night will be unresponsible in the morning. The heapdump.hprof is 11GB. Can I produce a smaller one and how?
> The heapdump.hprof is 11GB. Can I produce a smaller one and how? Set smaller java heap size (for example by option -J-Xmx2g)
(In reply to Alexander Simon from comment #17) > > The heapdump.hprof is 11GB. Can I produce a smaller one and how? > Set smaller java heap size (for example by option -J-Xmx2g) I uploaded a small heapdump (~ 288 MB).