This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
This is just a flyby bug report created during my search for an IDE that will handle the Mozilla source code nicely. Eclipse handles Mozilla's source pretty well, but it's a poor user interface. NetBeans looks a lot nicer, but I'm sorry to find that it really can't handle Mozilla. The Mozilla source is pretty easy to build nowadays, especially on linux and mac, but if anyone working on NetBeans is interested in working out NetBeans problems on a big project like Mozilla then I'd be happy to help in getting Mozilla set up and in discussing the issues.
Can you add some additional comments? For me NetBeans has not problems with Firefox. I described my investigation on my site, but on russian language only. Google translate: http://translate.google.com/translate?js=n&prev=_t&hl=ru&ie=UTF-8&layout=2&eotf=1&sl=ru&tl=en&u=http%3A%2F%2Fnetbeans.mojgorod.ru%2Fnetbeans_firefox.html
Hi soldatov. Sorry, I got really busy just after filing this bug and everything else got dropped. I'm still pretty busy, but I'll run through things again in the next couple of weeks and get back here with more details. I'll probably create separate bugs for some separate issues and mark them as dependencies of this bug so their relationship can be tracked.
The instructions given in the article linked in comment 1 won't work with current Mozilla code. Alexander, I'll go through the process of building Mozilla and attempting to get NetBeans working with it again, and I'll take careful notes for this bug, but it would probably be a good idea to do all that on the same platform as you if you're going to be working on this bug. I can build on Mac 10.6, Ubuntu, or Windows 7. What would your preference be? (It's a bit of a pain to build on Windows, so I'd rather try on one of the other two if possible, but I'll do it on Windows in you prefer.)
I built mozilla browser 4 on Ubuntu. You are right the mozilla browser is a challenging project for NB 7.0.
Alexander, what is so special about Mozilla? How many of our big customers may face the same issue? How come Eclipse is doing better in this case? Isn't it a P2?
jwatt, Could you clarify: - How did you create NB project? - What was the biggest problem in NetBeans? - Are you going to file separate bugs? (or you gave up NetBeans) - What does Eclipse handle better then NetBeans? Your answers (experience) help us to improve next NetBeans release. Thanks, Alexander
Hi. No, no, I've not given up on NetBeans. I've just not yet made the time to reconstruct the steps I took last time and write things up. Most likely I'm not going to manage to do that this evening, but I'll block out tomorrow evening for that and get back to you then.
I've created a wiki page here: https://developer.mozilla.org/En/Developer_Guide/Using_NetBeans_with_Mozilla Hopefully that's enough to start with for any of you guys that are interested to reproduce the problems, and hopefully figure out how to improve things.
Can you specify Firefox build and your hardware? Can I use Firefox 4.0.1?
I'm building tip (HEAD) from mozilla-central - so the code that will become Firefox 7. (That's the code I'm interested in seeing working, since that's what developers are working with nowadays.) I get the "NetBeans is ususably unresponsive" problems on both a MacBook Pro with OS X 10.6 and Thinkpad with Ubuntu 11.04 installed. Both have 8 GB RAM and fast processors, so I'm pretty sure if you follow the instructions as given on the wiki page you'll be able to reproduce the problems on any Mac or Ubuntu machines. Solaris maybe not, but I'd suspect so there to.
I really, really wouldn't recommend building Firefox 4.0.1 for this investigation, but if that's what you really want to do for some reason, the best way to do it depends on where you're at. If you already have a checkout of Mozilla central, then you can pull the changesets from the release 2.0 (Firefox 4.0) repository into your mozilla-central repo: hg pull http://hg.mozilla.org/releases/mozilla-2.0/ Or if you don't have mozilla-central, then you can just clone that entire release repo: hg clone http://hg.mozilla.org/releases/mozilla-2.0/ You can then get a working copy of the 4.0.1 source by checking out the 4.0.1 release tag using: hg co -r FIREFOX_4_0_1_RELEASE Another way to get the 4.0.1 source is by downloading the .tar.bz source snapshot from the applicable releases 'source' directory from ftp.mozilla.org as indicated in the wiki page. I'd warn that the build instructions for 4.0 are probably a bit different to mozilla-central though, so expect to have to change things a bit. (Also even if 4.0.1 happens to work okay in NetBeans roughly following the wiki instructions (actually that's doubtful), that's not really all that helpful since it's now a dead branch.)
I think memory is problem of slow parsing. I had to switch to 32bit JVM and give it -J-Jxm2G to be able to be parsed withing 200 sec.
What I did originally: preparing firefox for Netbeans on Ubuntu. I use the latest 7.0.1 Netbeans bits. #sudo apt-get build-dep firefox #sudo apt-get install mercurial libasound2-dev libcurl4-openssl-dev libnotify-dev libxt-dev libiw-dev mesa-common-dev autoconf2.13 yasm #hg clone http://hg.mozilla.org/mozilla-central/ #cd mozilla-central created file .mozconfig in mozilla-central folder with the content: ----------- . $topsrcdir/browser/config/mozconfig mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-debug #I don't have the newest yasm, so disabled some modules as ac_add_options --disable-webm ac_add_options --disable-libjpeg-turbo #use parallel build mk_add_options MOZ_MAKE_FLAGS="-j 8" ------------ Have *not* issued build from command line, but instead prepared project first as: started Netbeans with -J-Xmx2G 1. Click the New Project button, select "C/C++ Project with Existing Sources", then click "Next". 2. Specify mozilla-central as the root source directory, select the configuration mode "Custom", then click "Next". 3. Specify "Using an existing makefile" and point to mozilla-central/client.mk. Unselect the "Clean and Build after Finish" option, and click "Next". 4. Leave everything as is ("Build Command" to be "make -f client.mk", the "Clean Command" to be "make -f client.mk clean"), then click "Next". 5. Leave everything as is and click Next 6. Leave "Automatic Configuration" and click "Next". 7. Change "Project Location" path to be out of sources, i.e. /home/user/NetBeansProjects 8. press Finish When project is opened in Project tab => right click on project and turn off 'Code Assistance -> C/C++ Code Assistance' to save time on parsing not yet configured project. Now build firefox directly from IDE: btw, to speed up I specified parallel build using "-j" in .mozconfig and special compiler flags are not needed if building directly from IDE using Clean & Build actions. So, invoked Clean & Build from project context menu and after finished build (it took about 25 min on my laptop ). After that progress did not stopped and I saw a lot of exceptions in console[1], but when it stopped => reviewing properties of source files I can see that IDE configured all options used during build. After that I closed IDE and restarted it to eliminate possible memory leaks occurred during build process and following configuration of code assistance. I have 64 bits jvm => started with -J-Xmx4G When project is opened waited more till CPU gone to 0% (exceptions like [1] again in console) Then turned on Code Assistance from context menu and it parsed about 200 secs on my laptop (with cpu time as 20 min). IDE ate 2.7Gb Running IDE for parse with 2G caused stop of parsing at about 80% and almost not going further.
(In reply to comment #13) > created file .mozconfig in mozilla-central folder with the content: Not using the debug options is kinda cheating, since that will reduce the amount of memory NetBeans will have to use. ;) > started Netbeans with -J-Xmx2G I just tried this, and it made a big, big difference, even with my debug Firefox build. NetBeans flew through the first 70% of parsing probably roughly the same sort of speed that Eclipse does, although it did seem to slow down after that. In the end it took about 8 minutes to finish, which is a huge improvement. Scrolling performance is now also no longer a problem. That said, I'm still having responsiveness problems with some things. * The "Go to File" dialog takes about 2-3 seconds searching for "*SVGFrame" for example (Eclipse's "Open Resource" dialog is instantaneous), and typing seems to freeze as you're typing in. * The "Go to Symbol" dialog seems clunky, since as I try to type in "BuildDisplayList" the typing freezes repeatedly. The search results appear no slower than in Eclipse's "C/C++ Search" though, so that's good (well, still room for improvement in both IDEs there ;) ). I'm out of time to experiment with this further today, but I'll report back on any further responsiveness problems. I did notice another problem with correctness: * When I open file nsFrame.cpp, select the symbol "GetStyleDisplay" and from the context menu select "Go To Declaration" or "Go To Implementation", NetBeans seems to completely make up a result because I end up in nsIFrame.h line 142, which is utterly wrong. > btw, to speed up I specified parallel build using "-j" in .mozconfig and > special compiler flags are not needed if building directly from IDE using > Clean & Build actions. > After that progress did not stopped and I saw a lot of exceptions in > console[1], but when it stopped => reviewing properties of source files I can > see that IDE configured all options used during build. I think you just got lucky with the files you checked the properties on. If you use -j, then the build output from the various make processes gets jumbled and build output parsers will not be able to make complete sense of all of the combined build output. Even if (a big if) all the individual lines from the various make processes remain intact in the combined build output (i.e. without lines from one make process interrupting in the middle of lines from other processes), and even if you use -w to get the "Entering/Leaving" lines, the interleaving of "Entering/Leaving" lines from various make processes will mean the closest "Entering" line proceeding a gcc line building a source file will not always be the correct one. In this case the build output parser will not necessarily be able to know which source file the gcc line is actually building, or from which CWD, preventing it from always correctly resolving relative include paths specified on the command line, for example. This problem is further compounded by the fact that Mozilla has plenty of source files with the same name in various parts of the source tree, and some source files (particularly under the 'nss' directory) are not specified using absolute paths. In short I think avoiding -j is a good idea while trying to work though the problems NetBeans has with Mozilla, since it avoids one source of bogus problems. > Then turned on Code Assistance from context menu and it parsed about 200 secs > on my laptop (with cpu time as 20 min). IDE ate 2.7Gb > Running IDE for parse with 2G caused stop of parsing at about 80% and almost > not going further. Hmm, although things slowed down for me at about 70-80%, they didn't stop. [Side rant - why does Java even require memory to be pre-allocated? I don't see why it can't do what all modern software does and dynamically allocate more!]
(In reply to comment #14) > [Side rant - why does Java even require memory to be pre-allocated? I don't see > why it can't do what all modern software does and dynamically allocate more!] I think aggressive garbage collector is a Achilles' heel of Java
(In reply to comment #14) > > Not using the debug options is kinda cheating, since that will reduce the > amount of memory NetBeans will have to use. ;) [skipped] > > I think you just got lucky with the files you checked the properties on. If you > use -j, then the build output from the various make processes gets jumbled and > build output parsers will not be able to make complete sense of all of the > combined build output. Even if (a big if) all the individual lines from the > various make processes remain intact in the combined build output (i.e. without > lines from one make process interrupting in the middle of lines from other > processes), and even if you use -w to get the "Entering/Leaving" lines, the > interleaving of "Entering/Leaving" lines from various make processes will mean > the closest "Entering" line proceeding a gcc line building a source file will > not always be the correct one. In this case the build output parser will not > necessarily be able to know which source file the gcc line is actually > building, or from which CWD, preventing it from always correctly resolving > relative include paths specified on the command line, for example. This problem > is further compounded by the fact that Mozilla has plenty of source files with > the same name in various parts of the source tree, and some source files > (particularly under the 'nss' directory) are not specified using absolute > paths. > > In short I think avoiding -j is a good idea while trying to work though the > problems NetBeans has with Mozilla, since it avoids one source of bogus > problems. Thanks for your explanations, potential issues and good understanding of IDE internals :-) Fortunately I'm still correct - starting from upcoming 7.0.1 you don't have to specify any special option, because Netbeans has differentiating feature which you can see in Tools->Options->C++->Project Options It is "Use smart build analyzer to configure code assistance" (in 7.0 it is off by default) It does all the magic when Clean&Build is called from ide and handles all potential issues you've mentioned above :-)
(In reply to comment #14) > > > started Netbeans with -J-Xmx2G > > I just tried this, and it made a big, big difference, even with my debug > Firefox build. NetBeans flew through the first 70% of parsing probably roughly > the same sort of speed that Eclipse does, although it did seem to slow down > after that. In the end it took about 8 minutes to finish, which is a huge > improvement. What about bitness of your java? If it is 64bit JVM, can you, please, try -J-Xmx4G? Anyway I recommend to use 32bit JVM which uses 2 time less memory (due to pointers size) without any real performance penalty on NB
(In reply to comment #16) > Fortunately I'm still correct - starting from upcoming 7.0.1 you don't have to > specify any special option, because Netbeans has differentiating feature which > you can see in Tools->Options->C++->Project Options > It is "Use smart build analyzer to configure code assistance" (in 7.0 it is off > by default) > It does all the magic when Clean&Build is called from ide and handles all > potential issues you've mentioned above :-) Intriguing. I'd be really interested to know more about how it manages to handle the problems I mentioned. Maybe that would be better talked about in private email off this bug though. :) (In reply to comment #17) > What about bitness of your java? 32-bit.
(In reply to comment #16) > Thanks for your explanations, potential issues and good understanding of IDE > internals :-) > Fortunately I'm still correct - starting from upcoming 7.0.1 you don't have to > specify any special option, because Netbeans has differentiating feature which > you can see in Tools->Options->C++->Project Options > It is "Use smart build analyzer to configure code assistance" (in 7.0 it is off > by default) > It does all the magic when Clean&Build is called from ide and handles all > potential issues you've mentioned above :-) Is the smart build analyzer still part of NetBeans 7.1, and still turned on by default? I can't find any option to turn it on/off in the IDE (Tools > Options doesn't exist any more, and the Configure Code Assistance Wizard doesn't mention it in its "Select analyzer" dropdown), and if I build like normal a lot of the options the compiler is invoked with are missing.
(In reply to comment #19) > Tools > Options doesn't exist any more On MacOSX Tools|Options == NetBeans|Preferencies... for a lot of times
Oops. Thank you!
some progress: http://hg.netbeans.org/cnd-main/rev/b38e65cfc466 http://hg.netbeans.org/cnd-main/rev/4c9645ca0ddf http://hg.netbeans.org/cnd-main/rev/8c2ae21e594d
Integrated into 'main-golden', will be available in build *201203170400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-golden/rev/ad794a7c7a0e User: Vladimir Voskresensky <vv159170@netbeans.org> Log: fixing #197297 - NetBeans is unusable with the Mozilla source code - reuse field when only one macro is defined/undefined in snapshot, no need for collection
Integrated into 'main-golden', will be available in build *201203200400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-golden/rev/67181412c942 User: Vladimir Voskresensky <vv159170@netbeans.org> Log: fixing #197297 - NetBeans is unusable with the Mozilla source code - generate empty macro replacement tokens by request only
*** Bug 199697 has been marked as a duplicate of this bug. ***
*** Bug 202247 has been marked as a duplicate of this bug. ***
significant memory reductions: http://hg.netbeans.org/cnd-main/rev/c96509dcb108 http://hg.netbeans.org/cnd-main/rev/9f08e8ab7570 http://hg.netbeans.org/cnd-main/rev/9a2d715169fe
Hi Vladimir. Sounds great! I don't know anything about non-official builds of NetBeans, but if you can let me know when and where I can find builds with these fixes I'll happily test them out on the Mozilla source.
try bits from http://bits.netbeans.org/dev/nightly/latest btw if you use oracle's jdk 6/7 then 2Gb heap is enough even on 64bit version
The recent work reduced memory consumption to 750-900Mb, so now mozilla works fine with -J-Xmx1G heap and no any tricks with 32/64 modes are needed.
of course -J-Xmx2G can speed up intensive use of Find Usages, because caches will live longer comparing to -J-Xmx1G
Thanks for your work Vladimir. I've just tried the nightly netbeans build and so far it's by far the best I've seen something handle the mozilla code base. Is there a way to use a header for macro definitions? We use -include <obj>/dist/include/mozilla-config.h which contains ~100 #define I don't want to maintain manually.
(In reply to comment #32) > Thanks for your work Vladimir. I've just tried the nightly netbeans build and > so far it's by far the best I've seen something handle the mozilla code base. This is great. Btw, what is your Xmx limit you use now? Our measurements shows that after the last changes -J-Xmx1G should be enough, but the confirmation from real user is very welcomed. > > Is there a way to use a header for macro definitions? We use -include > <obj>/dist/include/mozilla-config.h which contains ~100 #define I don't want to > maintain manually. If you clean&build your project just at least once from IDE => It will auto detect everyting and "-include" as well. No need to do anything manually. See nbproject/private/Default.properties file to realize the trick. Btw, after that you'll have for source files Compile File (F9) in editor calling exactly what is supposed to be called for this compilation unit if you'd like just to check compile errors.
Just to record here the simplest way to put Mozilla into IDE: It could be like: 1) adjust .mozconfig content after checkout 2) no need to build from command line after 1) 3) run NB and use wizard File->New Project->C/C++->Project from existing sources. on the first step of wizard I specified top level source dir and switch to Custom mode on the second I specified client.mk as make file on the third I change clean command to be rm -rf obj-debug then next, next, next, finish and firefox is built and parsed.