This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I have seen a number of comments on this. On startup, it seems that the IDE spawns as many threads as it deems necessary to scan all the projects. On a dual core machine, I had a test running which had the load averages on Linux at exactly 1.00. This means a single core was fully used and another was available. Starting NetBeans, load average climbs to over 8 and the machine becomes close to unresponsive until scanning is complete, which takes a few minutes. Background scanning also causes freezes and jumpy interaction. Looking at the NetBeans process, I saw that NetBeans had 45 active threads while scanning. This is ridiculous - spawning a separate thread for every job in parallel without regard to the number of code paths on a machine is actually slower because of contention. If their are the equivalent of 2 cores free, the 2 threads working through a queue sequentially will actually improve performance. This is assuming that the threads are full active the entire time, which appears to be the case. If the threads were waiting for certain events in an idle state, then I could see the point, but that is not happening.
Tomas, please review this.
NetBeans 7.1.1 has single threaded scan. Single thread does the scan of all roots. One additional thread is used internally by Lucene to do merges, if you look at thread dump the scan thread is Parsing & Indexing Loop. Other threads are not belonging to scan. In Nb 7.2 where the index is transactional and allows editor features during the scan (the editor sees last committed state) there is a separate thread to do scanning which is called RepositoryUpdated.worker. This thread is taken only for a scan time and returned back to thread pool after the scan. On system with higher throughput 4 CPU threads the scan also can took 2 additional threads doing the IO prefetch as these threads block on IO subsystem or select. In this case there are 3 threads involved in scan where 1 of them is waiting for IO completion at given time. The ideal load for IO intensive process is 2 threads per CPU.
Unfortunately, this does not coincide with the evidence on my machine. I am not looking at any Java instrumentation to get the thread count, I am looking at /proc/[pid], which is where I saw 45 threads during the scan. "The ideal load for IO intensive process is 2 threads per CPU". Yes, if those cores are not otherwise engaged and NetBeans is the only thing running. Debating the number of threads aside, the fact remains that NetBeans pushes my load avg to over 8 during the scan phase and makes the machine unusable for minutes until it has completed. I attempted to offer some reasons as to why this happens based on my observations. Perhaps I was incorrect, or maybe there is a bug somewhere. You can choose to ignore the problem (which is a complaint made by others on the forums as well) as marking it "invalid", but that won't make it go away. The only thing "invalid" might have been my conjecture as to the cause of the issue, but the root issue is still there.
You can use jstack <jvm_pid> to get the java thread list including the thread state, it would be interesting to find out the other running threads. You can also use the IDE self profile action to generate the nps file (thread dumps in time period). If you can do it and attach it to this issue we can look what are the other active threads. The load is CPU load or composite load including the IO load? Thanks
Just to be sure: what is your project type/language? Most languages use the common infrastructure, which runs the scanning mostly single threaded (as Tomas described), but the C/C++ support uses their own infrastructure, that uses more cores, AFAIK. Thanks.
An offline message from the reporter: the projects are C++. Thanks for the answer.
rabinnh, http://wiki.netbeans.org/GenerateThreadDump is needed to analyze. workaround: -J-Dcnd.modelimpl.parser.threads=1 btw, default value = number of CPUs. Btw, do you have hyperthreading? I cause Runtime.getRuntime().availableProcessors() return 2xCPUs
Created attachment 118421 [details] Netbeans output as requested I hit ctrl-\ while it was "frozen" before it said it was doing anything on the status bar. I entered "kill -QUIT pid" while it was scanning.
Thanks for thread dumps! there are 3 of them. The first one could be like issue #208441 The second one was recently fixed as issue #207812 The third one shows no activity in UI, but has CPU/IO activity in "Refresh-After-WindowActivated" which is run to check if changed files appeared between IDE loosing/gaining focus. Can you provide IDE version info you have from Help->About? Can you try the latest daily build of IDE?
Here's the versions: Product Version: NetBeans IDE 7.1.1 (Build 201203012225) Java: 1.6.0_22; OpenJDK 64-Bit Server VM 20.0-b11 I'll snapshot the machine and try the daily. I'm on a really hard deadline for the end of next week, so I can't chance anything disrupting the system.
Tried "Product Version: NetBeans IDE Dev (Build 201204170400)" Big improvement. Here are some observations: First start up: It asked (I replied yes) and imported settings from 7.1.1. I also had it import additional plugins when requested (JUnit). However, there was no "Code assistance" menu item and syntax checking was missing. "Slowness detected" messages multiple times. Waiting until it finished and shut it down. Second start: Very slow. Load average went almost to 7, but syntax checking was now working. Third and subsequent starts: Much faster. Load average never went above 2. Was able to start working in 30 seconds or so instead of minutes. You may as well disable all UI controls except "Close Window"; until it validates tools and scans projects any UI interaction results in "Slowness detected" and lost mouse clicks, etc. I was just trying to click "Help|About" to copy the version number. All in all, the biggest problems seem to have been solved with this version, although I would recommend display a modal dialog with a "Cancel" button until the UI has validated tools and scanned projects. Also, I would prefer that each project be scanned on access. If I have 30 projects open and I need to change one source file, I dread opening the UI and waiting for all the projects to be scanned after I do a full batch build just to make a simple change. Great progress though.