This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 156593 - NB 6.5 takes forever to scan opened projects and then dies for large projects
Summary: NB 6.5 takes forever to scan opened projects and then dies for large projects
Status: RESOLVED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Web Opener (show other bugs)
Version: 6.x
Hardware: Macintosh All
: P3 blocker (vote)
Assignee: Milan Kubec
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-01-12 08:21 UTC by tomsve
Modified: 2009-04-21 11:55 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Stack dump made with jstack. (34.20 KB, text/plain)
2009-01-22 11:48 UTC, tomsve
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tomsve 2009-01-12 08:21:21 UTC
I have a "project" made of up 24 maven projects in a hirarchy and consisting of 3739 java source files alltogheter. 

In NetBeans 6.1 it takes a little under 5 minutes to scan the projects on startup and then I can work. During the
scanning the ide is semi responsive.

In NetBeans 6.5 opening exactly the same projects the scanning goes far beyond 5 minutes and finally hang the IDE (have
to be killed with the kill command) or sometimes just crashes the Java VM. During the scanning the IDE is not responsive
at all! 

This problem problems occur:
- on a 2.4 GHz Core2 Duo HP laptop running Mandriva Linux where projects reside on external USB disk. 
- on a 2.5 GHz Core2 Duo MacBook Pro running Mac OS X 1.5 where projects reside on an external firewire800 disk.
- on a 2.8 GHz Mac Pro 2 x QuadCore (8 cores) where projects reside on a network disk (Netgear ReadyNAS) accessed over
gigabit ethernet (and I am the only user of the machine and the NAS!).

Running the IDE with Java5 or Java6 does not seem to matter (I need to use Java5 due to DataSource conflicts in Java6). 

I can add that after the latest update I did manage to open the projects on the Mac Pro (8 core machine) with the
project scanning taking under 5 minutes and not hanging or crashing, but it still fails on the MacBook Pro (2 cores).

NetBeans 6.1 still works fine on all of these machines, so I'm sticking with that for now, but it would be nice to be
able to use 6.5 and future versions of Netbeans.
Comment 1 Milan Kubec 2009-01-12 08:59:06 UTC
Please, could you try this usecase also with projects stored on local disk? All your usecases use some kind of external
storage and creating FileObjects over network can be expensive, but it shouldn't hang the IDE. Thanks.
Comment 2 tomsve 2009-01-12 17:27:55 UTC
I now also tried copying my projects to local disk and opening them in NetBeans 6.5 in my MacBook Pro. Same thing. It
finally hangs. 

I'm not sure why I specified the disks since that is really not relevant. The important point here is that it works
perfectly fine in all cases with NetBeans 6.1, but not with NetBeans 6.5.

One interesting observation is that this problem existed early in the release candidates for 6.5 (didn't have the time
to bug this then, hoped it would be solved in final release). Then however it worked every other time if you let the
machine be until it finnished scanning. It took around 15 minutes, but after that you could work for that session. Now
in the final release it hangs everytime and on the mac have to be killed manually with "kill <process id>". Trying to
close the window does not work. 

You can se that after a while the "moving" progress bar for the scan stops moving and after that the whole GUI is
completely dead.

In netbeans 6.1 it take less than 5 minutes to do the scan and the whole IDE is sluggish during the scan but never
completely unresponsive.

Comment 3 tomsve 2009-01-12 17:37:15 UTC
Update: In the last try to start NB 6.5 on my MacBook Pro I left it after it hanged, and did other things. After about
20 minutes it become responsive again, and the scanning progressbar went away. So I haven't been patient enough. But
this is not OK anyhow. I should not have to wait 20 minutes for the IDE to start, not being able to do anything during
that time. There are still an extreme difference between NB 6.1 and NB 6.5. 

Since it went much faster on the Mac Pro, processor power do make a difference. But an 8 core CPU with single user
should probably not be the minimun requirement :-)

Comment 4 Milan Kubec 2009-01-13 11:46:22 UTC
Thanks for additional info. Please could you also get couple of thread dumps when IDE hangs and attach them to this
issue. And also some thread dumps from initial scanning when IDE is semi responsive. Thanks.
Comment 5 tomsve 2009-01-14 12:34:47 UTC
Only if you tell me how to do these thread dumps. I assume there must be some external command for that since when it
hangs there is absolutely nothing what so ever that can be done through the application window!
Comment 6 Milan Kubec 2009-01-14 12:44:24 UTC
See howto: http://wiki.netbeans.org/GenerateThreadDump
Comment 7 tomsve 2009-01-22 11:48:54 UTC
Created attachment 76132 [details]
Stack dump made with jstack.
Comment 8 tomsve 2009-01-22 12:13:43 UTC
Sorry it took a while, but I have now attatched a stacktrace made with jstack. This trace was done on my 8 core Mac Pro
running NB 6.5 with a JDK 1.6 64 bit server vm (only server vm:s are available for 64 bit). jstack was run when the
processor usage was around 50%. Note that that is 50% of 8 cores! That would be 200% of 2 cores, which is probably why
the MacBook pro laptop seems completely dead for a long while. 

Interestingly enough Netbeans 6.5 is the only program that has been able to load the processor to the level of 50% so
far, and it does so singlehandedly :-) When I bought the machine I thought "Yeah 8 cores! You can't have to much
processor power!". But I have never been able to reach 50% before, and I'm running quite a lot and have several VMWare
machines running. 

Currently, around 15+ minutes after opening the 24 projects and starting scanning the processor usage is down to normal
and the IDE GUI is completely dead. The scanning progessbar is no longer moving,  just frozen in place. But on this
machine I did manage to get through it without hanging at least once before. So it is a bit random if it works or not
even if you have the processor power.


Comment 9 tomsve 2009-01-22 12:23:53 UTC
It occured to me that Netbeans itself probably is developed using netbeans and must be larger than 3739 java files.
Maybee not as many projects though. 

Another thing I realized and haven't mentioned since it didn't occur to me that it might be relevant is that these 24
projects are maven projects using the netbeans maven plugin. But as far as I understand the maven plugin it should only
supply classpaths, not replace or affect the project scanning code.
Comment 10 Milan Kubec 2009-01-22 12:46:15 UTC
From the attached thread dump I can only say that disk reading is going on there. When did you created that thread dump?
When the IDE was responsive or after it went dead? To identify possible problem please create more thread dumps in a row
with delay of couple seconds when the IDE is semi responsive and when the IDE looks hung. Are those projects on local or
network disk? Did you try to work with the same configuration of projects on other platform than Mac OS?

Would be possible to run the IDE on non server JVM? I personally develop NetBeans on Mac Book Pro and have open more
than 24 projects with much more than 3800 java files and have never seen such problem.
Comment 11 tomsve 2009-01-22 14:56:25 UTC
As I said, even the netbeans code must be larger than the code in my projects. 

So maybee it is maven that causes the problem. Are you using maven for the project you mentioned ? I have no other
projects of the same size to test with. But such a test could be created both using maven and not.

It could also possible be some strange code in the projects that the scanning has difficulty with and where the scanning
bugs out. We have a lot of young less experienced developers in the project, and no time allocated for code review ...

If this was a general performance problem then there would be a lot more bugs reported on it, and it would probably
already be fixed by now :-) 

I have a feeling more stack traces will not pinpoint the problem. I also feel that I need to try to reproduce this
problem with other code than the one I'm having problems with now, to do some test setups, to get more meat on the bones
as we strangely say in Sweden :-) 

I suggest we close this issue, and I will make a new if I can come upp with better information on the problem. It is
going to take some time since I don't have much of that right now.
Comment 12 Milan Kubec 2009-01-23 07:42:40 UTC
I think that the issue can be left open, just adding INCOMPLETE kw. Please add as much details as you can. Thanks.
Comment 13 Milan Kubec 2009-04-21 11:55:00 UTC
Closing the issue as suggested. If there is more info, please file new issue.