This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 229746 - Slow versioning via favorites via FolderChilden with YAGL
Summary: Slow versioning via favorites via FolderChilden with YAGL
Status: VERIFIED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Data Systems (show other bugs)
Version: 7.3
Hardware: PC Windows 7 x64
: P2 normal (vote)
Assignee: Jaroslav Havlin
URL:
Keywords:
: 227997 233118 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-05-14 17:30 UTC by matthias42
Modified: 2013-08-19 14:08 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Profiler Snapshot from low performance (6.70 KB, application/octet-stream)
2013-05-14 17:30 UTC, matthias42
Details
new profiler snapshot (sampling profile) (17.87 KB, application/octet-stream)
2013-07-08 13:35 UTC, matthias42
Details

Note You need to log in before you can comment on or make changes to this bug.
Description matthias42 2013-05-14 17:30:39 UTC
Created attachment 134440 [details]
Profiler Snapshot from low performance

At work I often need to edit files on our user facing webserver. The document root is exposed for the developers as a samba share. The share is mapped to a drive letter on my machine.

For this use-case I do mostly short edits. I used to use the favorites window of netbeans 7.2 to browse the directory tree and open the relevant files. With 7.3 I'm often blocked by the IDE. Expanding the directory nodes takes a long time. I would expect this with directories containing many files, but in most cases the relevant directories contain only few files.

When doing a packet trace in parallel with these blocks (looks like the AWT is blocked, the reporting tool reacts in these situations), I see directories scanned, I did not open and for which the associated project is not opened in the IDE. 

I'll attach the profiler snapshot created by the low-performance tool.

Currently I always keep an native explorer window open to at least be able to continue to work. But I really need a solution for this...

Please note: The effect is not directly visible - I need to work with the IDE and at some point this happens...
Comment 1 Jaroslav Tulach 2013-05-16 07:45:12 UTC
FolderChildren calls Favorites.VisQ under Children.MUTEX. Favorites.VisQ calls into versioning API which performs FileUtil.normalizeFile and that takes ages.

Probably cleanest way is to fix FolderChildren.
Comment 2 Jaroslav Havlin 2013-06-14 13:19:16 UTC
http://hg.netbeans.org/core-main/rev/e0b297a9b38c
Favorites.VisQ is now called off Children.MUTEX.readAccess.
Matthias, please verify that the fix works properly. Thank you.
Comment 3 Quality Engineering 2013-06-15 03:02:43 UTC
Integrated into 'main-golden', will be available in build *201306142301* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/e0b297a9b38c
User: Jaroslav Havlin <jhavlin@netbeans.org>
Log: #229746: Slow versioning via favorites via FolderChilden with YAGL
Comment 4 matthias42 2013-07-08 11:11:44 UTC
Sorry - I just installed nightly build 201307072319 and the problem remains.

I had a wireshark open while this was happening and I think I know the problem: The whole directory tree (in this case two levels down there is a huge amount of PDF document) is read. You can see a massive amount of packages listing these files.

So - what I asume/deduce/whatever:

- When opening a folder (not sure whether in all cases, but in this case the folder was a svn checkout) a recursive decend happens, reading all directories
- Interactive usage does not get priority over this scanning (or what ever that is)
- The root problem is probably the dog slow implementation of getCanonicalFile on windows, that is called in FileUtil.normalizeFileOnWindows.

Maybe this helps...
Comment 5 Jaroslav Havlin 2013-07-08 11:26:25 UTC
Matthias, can you please attach new profiler snapshot? Thank you.
Comment 6 matthias42 2013-07-08 13:35:06 UTC
Created attachment 136825 [details]
new profiler snapshot (sampling profile)

Attached you find the profile I took today.
Comment 7 Jaroslav Havlin 2013-07-08 14:31:30 UTC
Thank you very much for the new snapshot.

The EDT seems to work fine, the operation is run outside of the lock.

Thread "Data System Nodes" really spends most of time in o.n.m.versioning.core.VcsVisibilityQueryImplementation.isVisible(), but I don't know how to fix it.

As a workaround, if the PDFs have a common file format, you might try to add them to ignored file list, in Menu -> Tools -> Options -> Miscellaneous -> Files.

Ondra, please, any ideas?
Comment 8 Jaroslav Havlin 2013-07-08 14:49:36 UTC
(In reply to comment #7)
> As a workaround, if the PDFs have a common file format, you might try to add
> them to ignored file list, in Menu -> Tools -> Options -> Miscellaneous ->
> Files.
>
> Ondra, please, any ideas?
I'm sorry, Ondra's just clarified it to me, no recursive descend is performed by VcsVisibilityQueryImplementation, so the only problem is the slow FileUtil.normalizeFile() method.
Comment 9 Jaroslav Havlin 2013-07-09 12:05:57 UTC
> ... so the only problem is the slow FileUtil.normalizeFile() method.
FileUtil.toFile normalizes all files since fix for bug 217919 was applied.

This was corrected by Jaroslav Tulach in
http://hg.netbeans.org/core-main/rev/5b4232e00531

Please check whether it works fine now for you.
Thank you.
Comment 10 Quality Engineering 2013-07-11 04:11:03 UTC
Integrated into 'main-silver', will be available in build *201307102300* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/5b4232e00531
User: Jaroslav Havlin <jhavlin@netbeans.org>
Log: #229746: Do not call normalizeFile in FileUtil.toFile
Comment 11 matthias42 2013-07-13 11:04:38 UTC
I used the nightly build for one and half day it looks very good. I'll stay on the nightly build for now and keep an eye on the behaviour. For now the feeling is back to the one I was used to in netbeans 7.2.

Thank you all very much!
Comment 12 Jaroslav Havlin 2013-07-22 06:40:13 UTC
*** Bug 233118 has been marked as a duplicate of this bug. ***
Comment 13 Jan Peska 2013-08-19 14:08:04 UTC
*** Bug 227997 has been marked as a duplicate of this bug. ***