This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I've got a C++ project consisting of +/- 22,400 files. When Netbeans starts it takes quite a while to parse the project. Which is okay by me - the project's way too big (but why parsing it all at EACH start ?) The problem is that I can't work on the other project because the IDE is not very responsive while scanning the project. So basically, I can't use Netbeans while parsing this project. Personally, I wouldn't mind that, if the repetitive parsing can not be avoided, the parsing takes longer to be more responsive.
Could you, please, send you ${userdir}/var/log/message.log file? One recommendation could be: try to use 32-bit JVM and give IDE more memory to do that you can #netbeans -J-d32 -J-Xmx2G For big projects increasing memory limit can show huge difference (i.e. 10 times faster parse)
It is a memory consumption problem indeed. From the moment it starts scanning the individual files the memory consumption skyrockets. Looking with VisualVM to what Netbeans is doing. :-) even with 2GB of heap space the garbage collector is very busy. Do you want me to send a heap dump ?
Created attachment 111413 [details] log file
2GB of heap just made it through. :-) However, the memory consumption remains quite high. Even after extensive GC the heap topped off at 1.2 GB. And I'm editing nothing. Just started Netbeans and let it run through its scanning process.
22K of files is really big project. As I see you use 64bit JVM could you try 32-bit (which uses almost twice less memory)
I can have a try with that yes. Didn't thought about the reference size for 64bits indeed. Thanks ! In the meantime, another thing that might interest you: Removed a lot of source folders to keep the memory consumption down. But it does only when restarting Netbeans. Removing source folders doesn't lower the memory consumption.
(In reply to comment #1) > Could you, please, send you ${userdir}/var/log/message.log file? > > One recommendation could be: try to use 32-bit JVM and give IDE more memory > to do that you can > #netbeans -J-d32 -J-Xmx2G > > For big projects increasing memory limit can show huge difference (i.e. 10 > times faster parse) Can we integrate this knowledge into the product? It seems much more user friendly if once we have information on project size, parsing speed, memory used and memory available (as well as bitness of JVM) we provide user with a meaningful advice. Does not look too difficult to implement, does it?
That's a good idea indeed - no other IDE does this by my knowledge. Netbeans' profiler already does something similar; to indicate the burden of the profiling on the system.
Last, but not least, it broke something in the indexing of the open projects. Some classes can not be found by the "Open Type" (CTRL-O) dialog. Removing the user directory and restarting did the trick. Although it's probably overkill to delete it all.
You might want to reassign to Alexander
(In reply to comment #9) > Removing the user directory and restarting did the trick. Although it's > probably overkill to delete it all. You don't have to delete whole userdir. it's enough to delete userdir/var/cache/cnd/model Other way is: in project's context menu select Code Assistance->Reparse Project
Thanks ! I just didn't know these options.
Btw, after switch to 32bit jvm and increasing memory limit, what is responsiveness of IDE? How long does it take to parse your sources?
One more note: normally once parsed we do not reparse project next time you open IDE, we only check for changed files and reparse them and files dependent on them.
Alexander, please, implement notification based on number of files, parse time (i.e. how long does it take to go 50% -> 60%) and memory used at checkpoints.
fixing, change set: http://hg.netbeans.org/cnd-main/rev/6447c245b999 - show low memory warning in parsing progress message
Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/6447c245b999 User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - show low memory warning in parsing progress message
fixed, change set: http://hg.netbeans.org/cnd-main/rev/a60c66a0cbe5 IDE shows a warning in the right bottom corner of the main frame if code model parser needs more memory. On click on the warning IDE shows dialog with an explanation of the problem and with a way how to fix problem. Dialog is sown once for IDE session. Implemented following strategy to detect low memory: - IDE consume more then 90% of all available heap. - estimated time of remaining parsing is more then 1 minute.
I think information about bitness should be checked as well and in case of 64bit JVM we can propose to run (install) 32bit version
(In reply to comment #19) > Implemented following strategy to detect low memory: > - IDE consume more then 90% of all available heap. > - estimated time of remaining parsing is more then 1 minute. Isn't 1 minute a too aggressive threshold?
additional fix, change set: http://hg.netbeans.org/cnd-main/rev/c33c6ebac12a - add advice about using 32-bit JVM
remove debug trace: http://hg.netbeans.org/cnd-main/rev/eaf469a4eb62
Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/a60c66a0cbe5 User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - show dialog if code model parser needs more memory
Integrated into 'main-golden' Changeset: http://hg.netbeans.org/main-golden/rev/c33c6ebac12a User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #202967 Parsing performance impact - add advice about using 32-bit JVM