This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I'm working on a huge C++ project. The project has about ~3 million lines of code. I noticed that the process for parsing the source code, have some serious problems. When I used the default settings of the netbeans for memory, after I opened the project, the IDE complains about memory when the source code indexed process reached at around 30% or something like that. Then I changed -Xmx16g (my machine has 32g RAM), this time it does not complain about lack of memory. The parsing process goes smoothly and looks like it is working very well. The progress bar smoothly goes forward from 0% ...10%...20%....50%..., and finally to 100%. Then the problem comes out, the progress bar stopped at 100% and never disappear even after one day! When I tried to shut down the IDE, a popup window asked me if I really want to close the application, since other process is not finished yet and in the dialog window, it shows the other process, the parser! Anyway I hit OK, the IDE does closed. But I found another problem, when I check the processes using the Linux top command. I found a java process is still running ( I'm sure there is no other process using Java), which used about 16g memory. So I have to kill it by hand. I think normally, when netbeans exit, the java process should automatically exits too. I'm using RHEL 5. I don't know how to debug this issue. If somebody can give me some instruction, I would be very happy to help to find and fix this issue if possible.
Thanks for the report. About memory: We did a lot with memory in 7.3 and I hope we solved your problem as well. (See i.e. issue #197297). Would be great if you can try 7.3 to verify on your source base. As about never ending parse (let's consider this is the topic of this issue): Please, attach thread dump when parser hangs. wiki.netbeans.org/GenerateThreadDump Thanks! Vladimir.
I tested on 7.3Beta yesterday. Looks like it is faster than 7.2.1. But after the progress bar reaches 100%, it just stops there and the progress bar then never disappear. When try to close netbeans, the popup dialog confirmed that the parser is still running. Then force to quit, namyly, choose "Yes" from the dialog, netbeans closed. But then check the processes running with Linux "top" command, see java is taking over ~16g memory and have to kill it manually. I also noticed that the parser parsed all the files (around 40,000). on the progress bar, it shows (4x,xxx/4x,xxx files parsed, 100%). I'll try make a dump later.
getting threads dump on Linux is very easy (just run NB from command line and press Ctrl+\). But it is essential for investigating this issue. 16g can be due to your startup parameters. Please, start NB from command line with options -J-Xmx2G to reduce memory down to 2Gb (btw in this case even on 64-bit system pointers will be still 32bit, so it reduces memory almost twice comparing to NB running with more than -J-Xmx3Gb). Thanks! Vladimir.
Created attachment 125586 [details] profiler dump after the parse is done (100%). this is the memory dump from netbeans 7.3Beta. Noticed that the parser parsed all the files at around 8:12-13. but the the progress bar never gone away.
Created attachment 125587 [details] screen shot of netbeans (especially the progress bar) netbean Screenshot. Note that the progress bar shows 100% (41857 of 41857) done, but somehow, it never goes away. also see "Capture2.png" to confirm that the parser is still running.
Created attachment 125588 [details] when exit IDE , it shows the parser is still running.
Thank you for information. I will analyzer
When tried to cancel the "parsing" process, it does not allow me to do so either,even it shows as 100% completed.
Do you use fortran?
fixed potential infinite loop in FortranLexicalPrepass.matchAttrStmt hopefully fixed: http://hg.netbeans.org/cnd-main/rev/96ff83d5773f
Yes. There are some fortran codes, also some C#, F#, and .NET managed C++ codes, not too many though. Thanks,
thanks for confirmation. It would be great if you can verify fix. i.e. when official daily build will contain this fix. Or try the candidate which already has the fix from: http://bertram-tst.netbeans.org:8080/job/cnd-main/lastSuccessfulBuild/artifact/nbbuild/ I'm still eager to know if -J-Xmx2G is enough now in 7.3 to handle your project size with still reasonably good parse time :-) Thanks! Vladimir.
Integrated into 'main-golden', will be available in build *201210130002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-golden/rev/96ff83d5773f User: Vladimir Voskresensky <vv159170@netbeans.org> Log: fixed #219592 - parser process never finish although progress bar shows 100% done. - fix potential infinite loop in FortranLexicalPrepass.matchAttrStmt
Vladimir, Thanks a lot for your work! I tested with this one: http://bertram-tst.netbeans.org:8080/job/cnd-main/lastSuccessfulBuild/artifact/nbbuild/ With the following parameters: -J-Xms384m -J-Xmx2g -J-XX:PermSize=128m It parsed all the files in around ~22 minutes. That's really amazing. And after all the files parsed. The progress bar is gone! You fixed the bug! Though when it is parsing the code, I still noticed that the "Lack of Memory" yellow icon showed in the process for a while beside the progress bar. But it was gone later. I also saw a lot of IndexOutOfBoundsException s when parsing some Fortran code. And some errors of matchClosingParen(): Missing closing paren on line xxx. And many NullPointerException s too. Though those does not affact the platform visually. Thanks for you great work!
Thanks for your verification! Could you, please, file a separate bug and attach your message.log to it. We will fix exceptions as well. Big trace into message.log slows down parse as well. Can you help us and try to find the minimal memory requirement for your project? Can you try with -J-Xmx2800m? (it is still short pointers)
tested with 2800m still has the lack of memory problem. But it was fin 2850m, 2900m. And when I increase the limits to and above 2850m, I also noticed that the parser runs also faster, takes only ~10 minutes to parse all the code. Thanks.