This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 26113 - PERFORMANCE: After large checkouts/updates ide is overloaded by performing Refreshed
Summary: PERFORMANCE: After large checkouts/updates ide is overloaded by performing Re...
Status: VERIFIED FIXED
Alias: None
Product: obsolete
Classification: Unclassified
Component: vcscore (show other bugs)
Version: 3.x
Hardware: PC Linux
: P2 blocker (vote)
Assignee: Martin Entlicher
URL:
Keywords: PERFORMANCE
Depends on: 28708
Blocks: 25926 26581
  Show dependency tree
 
Reported: 2002-07-30 10:26 UTC by dmladek
Modified: 2003-07-30 17:52 UTC (History)
0 users

See Also:
Issue Type: TASK
Exception Reporter:


Attachments
here is FTD (139.03 KB, text/plain)
2003-02-05 12:13 UTC, dmladek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description dmladek 2002-07-30 10:26:32 UTC
Product Version       = NetBeans IDE 3.4 RC 1
  IDE Versioning        = IDE/1 spec=2.23
impl=200207252340
  Operating System      = Linux version 2.4.18
running on i386
  Java; VM; Vendor      = 1.4.1-rc; Java
HotSpot(TM) Client VM 1.4.1-rc-b18; Sun Microsystems
Inc.
  Java Home             =
/usr/java/j2sdk1.4/sun/jdk1.4.1/jre
  System Locale; Encod. = cs_CZ; ISO-8859-2
==================================================================

I'm not satisfied with how the (at least)
vcsgeneric module (I use G-CVS FS for such type of
work only) behaves after you perform
large update/checkout (eg. cvs co . on
cvs.netbeans.org or you could update c. 10modules
with -d -P)

Then, after update is finished, the huge
Refreshing action will begin which almost freez
the ide for aproximately 5,10 minutes to 1hour or
more...sometimes leads to OOME:-(Also increased
CPUs activity you could notice)
If you're not patient user you think ide freez and
kill it.

I very please you, if you could somehow
coordinate/optimalize those neccesseary taks to
update vcscache to not to make ide unusable for
this time... otherwise it push me to look arround
other cvs-client alternativity...
Comment 1 dmladek 2002-07-30 10:29:12 UTC
adding PERFORMANCE keyword.
Comment 2 Martin Entlicher 2002-07-30 14:48:24 UTC
You're right. We need to spend some time on improving the performance.
Comment 3 dmladek 2002-07-30 15:27:20 UTC
Thank you;-)

Also when performing only Refresh Recursively on such huge Working dir
(netbeans sources) OOME is thrown and ide must be killed:-(
Comment 4 Martin Entlicher 2002-09-24 15:42:36 UTC
While fixing issue #25364 I've found several problems, that can lead
to this behavior.

The files should be refreshed while the checkout is running and all
files from a single folder should be updated together. This grouping
will significantly speed up disk cache storage. Access to the disk
cache should be generally minimized.

Comment 5 Tomas Pavek 2003-01-20 14:38:26 UTC
postponing to milestone 2
Comment 6 Martin Entlicher 2003-02-03 10:04:59 UTC
Started to work on it...
Comment 7 Martin Entlicher 2003-02-03 16:46:45 UTC
Fixed in the main trunk:

More efficient file status reporting to the filesystem:
/cvs/vcscore/src/org/netbeans/modules/vcscore/cmdline/ExecuteCommand.java,v 
<--  ExecuteCommand.java
new revision: 1.57; previous revision: 1.56

Filesystem refreshing improved so that only really existing
FileObjects are refreshed:
/cvs/vcscore/src/org/netbeans/modules/vcscore/versioning/VersioningFileSystem.java,v 
<--  VersioningFileSystem.java
new revision: 1.21; previous revision: 1.20
/cvs/vcscore/src/org/netbeans/modules/vcscore/VcsVersioningSystem.java,v 
<--  VcsVersioningSystem.java
new revision: 1.35; previous revision: 1.34
/cvs/vcscore/src/org/netbeans/modules/vcscore/VcsFileSystem.java,v 
<--  VcsFileSystem.java
new revision: 1.196; previous revision: 1.195

Cache improved to refresh only existing folders and create&refresh
intermediate folders (those between the currently existing and updated
from VCS commands):
/cvs/vcscore/src/org/netbeans/modules/vcscore/caching/VcsCache.java,v 
<--  VcsCache.java
new revision: 1.51; previous revision: 1.50

Status information of updated files is provided during the checkout or
update command execution for the efficiency (some fixes for correct
status reports are included):
/cvs/vcsgeneric/profiles/cvsprofiles/src/org/netbeans/modules/vcs/profiles/cvsprofiles/commands/CvsUpdate.java,v 
<--  CvsUpdate.java
new revision: 1.5; previous revision: 1.4
Comment 8 dmladek 2003-02-05 11:15:25 UTC
Well Martin,

It is much better than it used to be, but still there are some
weaknesses :-( Mainly when Memory progress bar is closed to maximum
used RAM... Then when ide or me performing GC, then it takes very long
time, for which you can't do anything with the ide...

I forgot to create FTD:-(( and latter I couldn't get into this state
again....

So definitely it is better and much more robusness;-)
I think I could say that it is fixed on the build #2003-02-05-0100


Comment 9 dmladek 2003-02-05 12:03:00 UTC
Well Martin,

I'm changing my mind and the best think is to reopen this bug.

-It seems to me that you improve the part of the stand alone checkout
and paraler Refreshes. This part is done well.

-But Afte checkout is finished, my ide is still overweighted by many
cvs tasks in background .... Now, I've made a 2 FTDs, so I hope 
that it might you solve it in some resonable way....

One more think about my test environment:
I was checkouting to the empty working the aliased module:
standard form our cvs-pserver cvs.netbeans.org

Try to manage yourself on the same TC, please;-)


And there is still one strange thing:
If I'm performing only stand-alone Refresh recursively on huge
dir-hierarchy (have the same sources like for checkout)...
I don't get into such torubles as I have now (CPUs are used for almost
100%, I don't have almost free RAM)

Comment 10 dmladek 2003-02-05 12:13:04 UTC
Created attachment 8807 [details]
here is FTD
Comment 11 dmladek 2003-02-05 13:24:48 UTC
Because neither Refresh Recursively after ide's restart isn't so robus 
as I've expected and after a time it ended for OOME I entred a new
ISSUE# 30686 for it. MEybe they're related....
Comment 12 Tomas Pavek 2003-02-11 16:17:11 UTC
Probably related to issue 27713, will be fixed together for M3.
Comment 13 Martin Entlicher 2003-02-12 14:45:18 UTC
The problem with background tasks should be fixed by the fix of issue
#27713. So the checkout is hopefully O.K. now.

The problem with the OOME when doing Refresh Rec. is most probably a
completely different story and will be solved is issue #30686.

So I'm resolving this as fixed.
Comment 14 dmladek 2003-07-30 17:52:56 UTC
I think I can verify this issue as FIXED on 

  Product Version       = NetBeans IDE Dev (Build 200307300100)
  Operating System      = Linux version 2.4.20-18.9 running on i386
  Java; VM; Vendor      = 1.4.2; Java HotSpot(TM) Client VM 1.4.2-b28;
Sun Microsystems Inc.
  Java Home             = /usr/local/java/j2sdk1.4.2/jre
  System Locale; Encod. = cs_CZ; ISO-8859-2
  Home Dir; Current Dir = /usr/local/home/delphym;
/usr/local/forte/forte3/NBdev-last/netbeans/bin
  IDE Install; User Dir = /usr/local/forte/forte3/NBdev-last/netbeans;
/usr/local/home/delphym/.netbeans/dev
================================================================

I perform checkout of r:release35 of standard module
and hugu update of trunk all modules on netbeans.org repository.

I didn't noticed any serious overloads of the IDE.
But also I have more powerfull machine now;-)

Thanx Martin