This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 186655 - NamingFactory.collectChildren() is not effective
Summary: NamingFactory.collectChildren() is not effective
Status: RESOLVED DUPLICATE of bug 184854
Alias: None
Product: platform
Classification: Unclassified
Component: Filesystems (show other bugs)
Version: 6.x
Hardware: PC Linux
: P1 normal (vote)
Assignee: Jaroslav Tulach
URL:
Keywords: PERFORMANCE
: 186482 186791 186890 186897 187358 187403 188856 189006 (view as bug list)
Depends on: 184854
Blocks:
  Show dependency tree
 
Reported: 2010-05-25 11:35 UTC by Tomas Mysik
Modified: 2010-08-16 15:12 UTC (History)
11 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
snapshot (15.70 KB, application/octet-stream)
2010-05-25 11:35 UTC, Tomas Mysik
Details
one more snapshot (13.52 KB, application/octet-stream)
2010-05-28 10:51 UTC, Tomas Mysik
Details
one more snapshot (11.71 KB, application/octet-stream)
2010-06-02 11:47 UTC, Tomas Mysik
Details
application demonstrating issue of slowness (1.03 MB, application/x-tar)
2010-06-03 16:22 UTC, Vladimir Voskresensky
Details
Speed up (3.86 KB, patch)
2010-06-04 09:51 UTC, Jaroslav Tulach
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tomas Mysik 2010-05-25 11:35:17 UTC
Created attachment 99429 [details]
snapshot

Have a look at the snapshot.

Product Version: NetBeans IDE Dev (Build 100521-ea4985d4f1ae)
Java: 1.6.0_20; Java HotSpot(TM) 64-Bit Server VM 16.3-b01
System: Linux version 2.6.32-22-generic running on amd64; UTF-8; cs_CZ (nb)
Comment 1 Jaroslav Tulach 2010-05-25 12:23:58 UTC
I'll see what I can do with this when fixing bug 184854.
Comment 2 Jaroslav Tulach 2010-05-27 11:06:30 UTC
*** Bug 186791 has been marked as a duplicate of this bug. ***
Comment 3 Jaroslav Tulach 2010-05-27 11:06:59 UTC
Another 60s NPS: http://netbeans.org/bugzilla/attachment.cgi?id=99531
Comment 4 Tomas Mysik 2010-05-28 10:51:28 UTC
Created attachment 99598 [details]
one more snapshot
Comment 5 Tomas Mysik 2010-06-02 11:47:02 UTC
Created attachment 99754 [details]
one more snapshot
Comment 6 Vladimir Voskresensky 2010-06-03 13:00:56 UTC
Start traversing FS using FileObject map 1
ls time: 15443ms; 183612 wrappers where 38118 for dirs
Start checking time stamps using FileObjects map 1
time 1: 3487
time 2: 3366
time 3: 3486
time 4: 3508
time 5: 3422

-------------
FS.refresh statistics (183,401FileObjects):
  FileSystem refresh: 1 calls in 353091ms
  Invocation of FileChangeListeners: 0 calls in 0ms
  Folder refresh: 0 calls in 0ms
  File refresh: 328680 calls in 342364ms
FS.refresh statistics (38,125FileObjects):
  FileSystem refresh: 1 calls in 33565ms
  Invocation of FileChangeListeners: 0 calls in 0ms
  Folder refresh: 0 calls in 0ms
  File refresh: 243276 calls in 24808ms
Comment 7 Vladimir Voskresensky 2010-06-03 16:22:00 UTC
Created attachment 99791 [details]
application demonstrating issue of slowness
Comment 8 Vladimir Voskresensky 2010-06-03 21:06:39 UTC
353091ms is a good candidate to be fixed asap
Comment 9 Jaroslav Tulach 2010-06-04 09:51:17 UTC
Created attachment 99811 [details]
Speed up

I can speed the implementation a bit and make it O(n) instead of O(n^2). However the problem is not fixed, I am afraid. The time needed to find even small number of children is still quite high, as it depends on the number of FileObject in memory. Rather the implementation shall change and time time shall be proportional to number of existing children.

The numbers below show that it does not matter how many children we have, the time is always the same, guided by those 170000 of existing files.

[NamingFactory]: 318 children for /nb/main/wlm.editor/src/org took 36 all names: 176746
[NamingFactory]: 17 children for /nb/main/cnd.navigation/test/unit took 39 all names: 176713
[NamingFactory]: 21 children for /nb/main/j2ee.ant/build took 45 all names: 176704
[NamingFactory]: 6 children for /nb/main/j2ee.ant/src took 40 all names: 176683
[NamingFactory]: 12 children for /nb/main/j2ee.ant/antsrc took 69 all names: 176677
[NamingFactory]: 2 children for /nb/main/j2ee.ant/nbproject took 53 all names: 176665
Comment 10 Jaroslav Tulach 2010-06-04 09:56:40 UTC
*** Bug 186482 has been marked as a duplicate of this bug. ***
Comment 11 Jaroslav Tulach 2010-06-04 10:02:55 UTC
*** Bug 186897 has been marked as a duplicate of this bug. ***
Comment 12 Jaroslav Tulach 2010-06-04 10:09:15 UTC
*** Bug 186890 has been marked as a duplicate of this bug. ***
Comment 13 Jaroslav Tulach 2010-06-10 15:20:34 UTC
*** Bug 187358 has been marked as a duplicate of this bug. ***
Comment 14 Jaroslav Tulach 2010-06-10 15:23:25 UTC
*** Bug 187403 has been marked as a duplicate of this bug. ***
Comment 15 Jaroslav Tulach 2010-06-10 15:56:20 UTC
I believe this is fixed by #8 build of
http://deadlock.netbeans.org/hudson/job/prototypes-immutable-file-name-184854/
at least when running Vladimir's test application, the CPU is not occupied any more, and most of the time is spend by I/O operations.
Comment 16 Jaroslav Tulach 2010-06-14 06:15:38 UTC
My tests seem to indicate that my fix for bug 184854 also reduces the CPU usage in collectChildren method (by deleting it). Marking as duplicate, please verify.

*** This bug has been marked as a duplicate of bug 184854 ***
Comment 17 Quality Engineering 2010-06-15 03:26:52 UTC
Integrated into 'main-golden', will be available in build *201006150001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/3c7f40a1c126
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #186655: Effectively eliminating slowness of collectChildren method by deleting it.
Comment 18 Vladimir Voskresensky 2010-06-15 05:58:48 UTC
Jarda, you have closed bug as duplicate, but commit message was put here. Please, resolve which one to integrate into 6.9.1 
Thanks,
Vladimir.
Comment 19 Jaroslav Tulach 2010-06-15 13:33:24 UTC
As soon as stabilized, all commits from branch immutable-file-name-184854 will be the ones to backport.
Comment 20 Tomas Mysik 2010-06-15 13:43:32 UTC
I can confirm that this issue seems to be fixed.
Comment 21 Vladimir Voskresensky 2010-06-16 10:18:34 UTC
great! now current numbers are the same for both runs (the first and following) and it is the same as was before for the second run. Speed is still now good (original P3?)

--------------
Start traversing FS using FileObject map 1
ls time: 17704ms; 183612 wrappers where 38118 for dirs
Start checking time stamps using FileObjects map 1
time 1: 4834
time 2: 3432
time 3: 3467
time 4: 3524
time 5: 3435
attach recursive listener to FileObject...10101
Done traversing FS using FileObject map 1
-------------------------------------
FS.refresh statistics (183,614FileObjects):
  FileSystem refresh: 1 calls in 28188ms
  Invocation of FileChangeListeners: 0 calls in 0ms
  Folder refresh: 0 calls in 0ms
  File refresh: 329107 calls in 30541ms
FS.refresh statistics (183,614FileObjects):
  FileSystem refresh: 1 calls in 25906ms
  Invocation of FileChangeListeners: 0 calls in 0ms
  Folder refresh: 0 calls in 0ms
  File refresh: 329107 calls in 29266ms
Comment 22 Vladimir Voskresensky 2010-06-16 10:20:35 UTC
(In reply to comment #21)
> great! now current numbers are the same for both runs (the first and following)
> and it is the same as was before for the second run. Speed is still now good
> (original P3?)
I meant that imho speed is *not* good (original P3?), because it's 6 times slower than could be :-)
Comment 23 Jaroslav Tulach 2010-07-26 08:02:16 UTC
*** Bug 188856 has been marked as a duplicate of this bug. ***
Comment 24 Jaroslav Tulach 2010-08-16 15:12:56 UTC
*** Bug 189006 has been marked as a duplicate of this bug. ***