Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 171330 - Enter in Java editor generates 31 disk touches
Enter in Java editor generates 31 disk touches
Status: RESOLVED FIXED
Product: platform
Classification: Unclassified
Component: Filesystems
6.x
All All
: P3 (vote)
: 6.x
Assigned To: Jiri Skrivanek
issues@platform
: PERFORMANCE, TEST
Depends on: 171953
Blocks: 65135
  Show dependency treegraph
 
Reported: 2009-09-02 15:44 UTC by Jaroslav Tulach
Modified: 2009-10-15 10:26 UTC (History)
7 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
Test to count the number of touches after enter in editor (13.63 KB, patch)
2009-09-02 15:47 UTC, Jaroslav Tulach
Details | Diff
New patch that yields those ~50 touches (4.32 KB, patch)
2009-09-09 10:05 UTC, Jaroslav Tulach
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jaroslav Tulach 2009-09-02 15:44:45 UTC
An enter in Java Editor seems to generate 220 disk touches. This is slightly too much. More info at
http://wiki.netbeans.org/FitnessWithoutTouches
Comment 1 Jaroslav Tulach 2009-09-02 15:47:07 UTC
Created attachment 86976 [details]
Test to count the number of touches after enter in editor
Comment 2 Jaroslav Tulach 2009-09-02 15:51:01 UTC
The highest number of disk touches seems to come from AddOverrideAnnotation currently (through getSourceLevel method).
Comment 3 Max Sauer 2009-09-07 13:23:47 UTC
Fixed, source level can now by obtained through CompilationInfo, which should eliminate unneeded touches. Hope it helps, unfortunately the test does not 
run on mac properly, causes an error (java.lang.AssertionError: No files touched exp: 0 was: 2).
---
http://hg.netbeans.org/jet-main/rev/b097452d3cd3
Comment 4 Quality Engineering 2009-09-08 12:26:17 UTC
Integrated into 'main-golden', will be available in build *200909080527* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/b097452d3cd3
User: Max Sauer <msauer@netbeans.org>
Log: #171330: Enter in Java editor generates ~220 disk touches
Comment 5 Jaroslav Tulach 2009-09-09 08:50:53 UTC
Improved. But there is still room for making it better. I have modified the page 
http://wiki.netbeans.org/FitnessWithoutTouches
with my current results. There is still over 40 of touches, some of them from sidebars, but some of them still request 
the source level. Please eliminate more.
Comment 6 Max Sauer 2009-09-09 09:10:22 UTC
OK. Not so serious now imho => P3.
Comment 7 Jaroslav Tulach 2009-09-09 10:05:12 UTC
Created attachment 87338 [details]
New patch that yields those ~50 touches
Comment 8 Jaroslav Tulach 2009-09-09 10:10:51 UTC
I am not saying this all needs to be fixed in java.editor. The stacktraces show that the Java editor can improve 
especially in getSourceLevel queries. Please evaluate this, the sooner the better.

I am more worried by other kinds of touches. Why these need to happen (this is not maven project, neither cnd or 
mobility one, so this all seems quite unnecessary):
    org.netbeans.modules.cnd.makeproject.MakeProjectFileOwnerQuery.getOwner(MakeProjectFileOwnerQuery.java:71)
    org.netbeans.api.project.FileOwnerQuery.getOwner(FileOwnerQuery.java:101)
    org.netbeans.modules.mobility.project.J2MEProjectUtils.getProjectForDocument(J2MEProjectUtils.java:118)
    org.netbeans.modules.mobility.editor.PPCompletionProvider.getAutoQueryTypes(PPCompletionProvider.java:114)
    org.netbeans.modules.editor.completion.CompletionImpl.insertUpdate(CompletionImpl.java:303)
    org.netbeans.lib.editor.util.swing.PriorityDocumentListenerList.insertUpdate(PriorityDocumentListenerList.java:79)
or
    org.netbeans.modules.maven.queries.MavenFileOwnerQueryImpl.getOwner(MavenFileOwnerQueryImpl.java:196)
    org.netbeans.api.project.FileOwnerQuery.getOwner(FileOwnerQuery.java:101)
    org.netbeans.modules.mobility.project.J2MEProjectUtils.getProjectForDocument(J2MEProjectUtils.java:118)
    org.netbeans.modules.mobility.editor.PPCompletionProvider.getAutoQueryTypes(PPCompletionProvider.java:114)

Because of these alien touches I want to keep this higher priority bug. Please evaluate and fix what you can in 
java.editor and let's pass the bug to mobility.

Comment 9 _ tboudreau 2009-09-09 22:40:44 UTC
Took a brief look at the code - probably 
org.netbeans.modules.mobility.project.J2MEProjectUtils.getProjectForDocument(J2MEProjectUtils.java:118):

        J2MEProjectUtilitiesProvider utils = Lookup.getDefault().lookup(J2MEProjectUtilitiesProvider.class);
        if (utils == null){
            return null;
        }
        final FileObject fo = utils.getFileObjectForDocument(doc);
        if (fo != null)
            return FileOwnerQuery.getOwner(fo);

Probably either the implementation of J2MEProjectUtilitiesProvider.getFileObjectForDocument (which I could not find), or
this method could detect if the document belongs to a Mobility project somehow, and return null if not, without invoking
FileOwnerQuery (or saving that as a last resort, needed only if it really is a J2ME project).

However, unless there is some object from mobility.editor (J2MEDataObject?  But that is probably not used for all .java
files under a J2ME project - not sure) which can be tested for against the Document, to make the negative test fast, the
code will probably need to walk the parent tree of the FileObject and do *something* that touches the disk, to determine
if it is looking at a J2ME project.  I'm assuming that this code can simply return null if the project is not a mobility
project, and that there is probably a way to detect that situation based on what is already in memory, so that the
negative test does not do I/O.  Petr?
Comment 10 Max Sauer 2009-09-10 09:48:25 UTC
http://hg.netbeans.org/jet-main/rev/3c8315b39f5a should eliminate disk touches originating in getSourceLevel queries. Reassigning to mobility.
Comment 11 Jaroslav Tulach 2009-09-10 16:33:33 UTC
There is few calls via
    org.netbeans.modules.mobility.project.J2MEProjectUtils.getProjectForDocument(J2MEProjectUtils.java:118)
    org.netbeans.modules.mobility.editor.PPCompletionProvider.getAutoQueryTypes(PPCompletionProvider.java:114)
what could be done is to remember the document -> project mapping in some cache (document property, maybe) and query 
just once. The test measures second enter, so if you store this mapping during first enter, your code will do not 
additional touches the second time.
Comment 12 _ tboudreau 2009-09-10 21:54:09 UTC
> remember the document -> project mapping in some cache

That would work;  however, I'd like to hear from Petr S. first if there is some in-memory data that can determine if we
even need to call FileOwnerQuery, before we implement a cache.  

*If* PPCompletionProvider.getAutoQueryTypes() should return no result if it is invoked against a file that does not
belong to a mobility project, and *if* we can determine from the Document or the FileObject for the document that it
does not belong to a mobility project (J2MEDataObject? some Document property? Something in the FileObject's
DataObject's Lookup?), we can simply skip the call to FileOwnerQuery completely.

Later, if there is too much I/O happening for *legitimate* calls (i.e. it really is a mobility project), we can
implement a cache if necessary.

Petr, I don't really know what PPCompletionProvider does or if there is a way to know from the Document or FileObject if
we can reliably decide if we can return nothing from PPCompletionProvider.getAutoQueryTypes(), so probably you should
take this from here.
Comment 13 Petr Suchomel 2009-09-11 07:56:04 UTC
In case of AddOverrideAnnotation, is this still connected with Mobility? Should we split the issue?
Comment 14 Jaroslav Tulach 2009-09-11 08:33:22 UTC
If you wish, feel free to report some separate subissues and keep this as umbrella. But I am updating the list of 
touches on http://wiki.netbeans.org/FitnessWithoutTouches so I personally don't see much need for more issue. Also 
this issue somehow captures the main objective: typing in Java editor causes huge and unnecessary I/O storm.
Comment 15 Max Sauer 2009-09-11 08:50:39 UTC
psuchomel: There are no more touches from AddOverrideAnnotation anymore, please see the test output or wiki page.
Comment 16 Quality Engineering 2009-09-11 21:54:24 UTC
Integrated into 'main-golden', will be available in build *200909111401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/3c8315b39f5a
User: Max Sauer <msauer@netbeans.org>
Log: #171330: Enter in Java editor generates 49 disk touches
Comment 17 Jaroslav Tulach 2009-09-15 07:04:59 UTC
I've updated the page with new results. Right now we are down to 31 touches:
http://wiki.netbeans.org/FitnessWithoutTouches
The project queries are now gone (thanks to fixes in JavaSource level, probably). The remaining issue involve 
mercurial (because we are under NetBeans Hg root), DiffSideBar and CloneableEditorSupport. Passing to Mercurial guys 
for evaluation. Lowering priority as there was some progress and the current results are influenced by test setup.

Enabling test: ergonomics#017889e557fe
Comment 18 _ tboudreau 2009-09-15 17:35:12 UTC
Looks like a fix for issue 126156 such as Jesse and I suggest might eliminate some of the touches as a side-effect,
although it would be non-trivial to implement.
Comment 19 Ondrej Vrabec 2009-09-16 10:15:40 UTC
mercurial cache fix: cdev #134821f9889c
diffsidebar fix: cdev #289075e6a9e7
Comment 20 Tomas Stupka 2009-09-16 12:59:23 UTC
file access 20 times in mecurial, most of them from: 

Mercurial.getRepositoryRoot() - 10
fixed by extending the already existent caching at that place, so that file.io significantly reduced. should also have
effect in other scenarios as the method is frequently called from many places - #22d6f3114be6

StatusCache - 8 
most of them triggered by diffsidebar - fixed by ovrabec

Comment 21 Tomas Stupka 2009-09-16 13:40:24 UTC
reassigning to CloneableEditorSupport for further evaluation
Comment 22 Quality Engineering 2009-09-18 22:52:01 UTC
Integrated into 'main-golden', will be available in build *200909181401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/017889e557fe
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #171330: Enabling test to ensure that the number of disk touches after enter is not significantly increase over our current state (~31)
Comment 23 _ tboudreau 2009-09-19 17:15:11 UTC
31 still seems like about 31 too many... The user pressing a key does not imply that anything has changed on disk; 
therefore if the IDE is touching the disk, something is still doing useless work.
Comment 24 Jaroslav Tulach 2009-09-21 12:16:30 UTC
I have fixed the test in ergonomics#bb5d5057a474 to eliminate unrelated touches. Now we are down to two. Imho caused 
by unnecessary conversion of FileObject to URI in parsing API and java.source. See 
http://wiki.netbeans.org/FitnessWithoutTouches
Can these two be eliminated as well?
Comment 25 Jan Lahoda 2009-09-21 14:03:08 UTC
Hm, I guess you want me to pass the FileObject embedded in the Indexable directly and not convert it to and from URL. I
do not think this is feasible - there is intentionally no getFileObject method on the Indexable for performance reasons.
Adding one would require a full API review - the architectural impact of such a change is not-obvious and such change is
controversial, and so the fasttrack review cannot be used for it. I do not think the method would be accepted in the
review anyway, as it would likely be massively abused and would bound us to use of FileObjects for indexing (preventing
us from using j.l.File forever).

An alternative solution is to change PersistentClassIndex to work over URL and add a URL cache into FileObject (ideally,
the cache could be shared even by the URLMapper, for the benefit of all IDE, but I won't require that for now). This
would solve these two accesses.

I have a prototype of the change in the PersistentClassIndex, but I cannot test it properly, as your change is not part
of jet-main yet.
Comment 26 Jan Lahoda 2009-10-12 16:39:43 UTC
I have changed the PersistentClassIndex to keep the URL and convert it to FileObject/JavaSource when needed, instead of
converting it eagerly:
http://hg.netbeans.org/jet-main/rev/6f49fa94772a

For the remaining disk touch (call to FileObject.getURL()), I do not think this can be actually fixed in
java.source+parsing.api, as noted above. Would be better, IMO, to cache the result on filesystem level, so that only the
first call to getURL would touch the disk, and the subsequent calls would simply return the cached value. This may be
helpful for many parts of the IDE, not only this single usecase.
Comment 27 Jiri Skrivanek 2009-10-14 08:01:19 UTC
I eliminated disk touch in FileObject.getURL() by overwriting file.toURI(). We already know whether given file is folder
or not.
core-main #71d55a1c829c
Comment 28 Quality Engineering 2009-10-15 10:26:41 UTC
Integrated into 'main-golden', will be available in build *200910150201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/71d55a1c829c
User: Jiri Skrivanek <jskrivanek@netbeans.org>
Log: #171330 - Eliminate disk touch by overwriting file.toURI() because we know whether given file is folder or not.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo