This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 222330 - Classpath for the listed files is retrieved by NB when showing "Open Recent File" and Favorites
Summary: Classpath for the listed files is retrieved by NB when showing "Open Recent F...
Status: RESOLVED FIXED
Alias: None
Product: projects
Classification: Unclassified
Component: Generic Projects UI (show other bugs)
Version: 7.2.1
Hardware: PC All
: P3 normal with 1 vote (vote)
Assignee: Milos Kleint
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-11-18 11:59 UTC by attila.kelemen
Modified: 2013-07-10 12:56 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
stacktrace of the call (531 bytes, text/plain)
2012-11-19 08:53 UTC, attila.kelemen
Details
Hovering recent files triggers the project loading (474.97 KB, image/png)
2012-11-19 08:59 UTC, vaskar
Details

Note You need to log in before you can comment on or make changes to this bug.
Description attila.kelemen 2012-11-18 11:59:32 UTC
When a user hovers the mouse over "File/Open Recent File", NetBeans will attempt to retrieve the classpath for each of the listed files. This is undesirable because retrieving the classpath might be an expensive operation. This is also true for the Favorites (classpath is queried for each node) and seems to be due to the same reason.

From the callstack, I guess that the classpath is queried in order to show error badges on the nodes. If this is the case, can the showing of error badges be disabled in these menus?

Also, you may see this reported issue:
https://github.com/kelemen/netbeans-gradle-project/issues/25#issuecomment-10479999
Comment 1 Milos Kleint 2012-11-19 08:21:09 UTC
(In reply to comment #0)
> When a user hovers the mouse over "File/Open Recent File", NetBeans will
> attempt to retrieve the classpath for each of the listed files. 

do you have the stacktrace around? can you attach it? is is the one from 
https://github.com/kelemen/netbeans-gradle-project/issues/25#issuecomment-10480260 ?
what types of files did you have in the recent files? (screenshot please)



> This is
> undesirable because retrieving the classpath might be an expensive operation.
> This is also true for the Favorites (classpath is queried for each node) and
> seems to be due to the same reason.

if either of those is called from AWT then it's a problem. Otherwise I believe it's working as intended. In netbeans projects don't have to be explicitly opened in order to be loaded. Many cases like that. Eg. when a open project directly depends on a non-opened one, the non opened one will be loaded as well.
You can limit the functionality provided from OpenProjectHook which is called when project is opened and closed. but the core functionality should be always around. Like classpath, nodes etc.


I'm reassigning to utilities which I believe to be the owner of the recent files item. We should not be loading projects from that menu. But technically speaking it could be also an issue in the data loaders of affected files. I know of some that wrongfully use FileOwnerQuery to load the owning project.

the use if favourites is most likely correct AFAIK. but we can reassign there later.
Comment 2 attila.kelemen 2012-11-19 08:53:02 UTC
Created attachment 128037 [details]
stacktrace of the call

I have attached the stacktrace and yes, it is the same one as in the issue above.

> what types of files did you have in the recent files?

I only had .java files in the menu, I will attach a screenshot if you really need one for this.

> if either of those is called from AWT then it's a problem.

They are not called from the event dispatch thread and it wouldn't be a problem for me anyway. I know projects can be loaded anytime but even then fetching the classpath for a large multi-project build could take minutes and there is little gain for fetching them. This can be quite annoying because it blocks other projects from being loaded.

Getting the classpath in favorites is less of a problem but still undesirable and since NB only requests the classpath for the listed nodes (and not the unfolded one) it couldn't really deduce if the folder contains files with errors anyway (at least, not for folders).
Comment 3 vaskar 2012-11-19 08:59:46 UTC
Created attachment 128038 [details]
Hovering recent files triggers the project loading

Here is the screenshot.

As Attila said, there were only java files in the recents menu, and it triggered the project loading.
Comment 4 attila.kelemen 2012-11-19 09:01:16 UTC
> not the unfolded one

should have been "not the folded ones".
Comment 5 Jaroslav Havlin 2012-11-19 10:24:53 UTC
The issue is quite complex. I'm not aware of any other way to get icon for a file. Only solutions that I can think of are:

- Introduce a command-line switch that turns off icons in the recent files list.
- Create an API for getting icons based only on file extension.
- Provide an API providing method e.g. isOwnedByOpenProject(FileObject).
  (then we could show icons only for files from open projects).
- Add a method e.g. DataObject.findInCache(FileObject).
- Do not show icons in recent files list (probably too radical, UI regression).

Jarda, can you please help to evaluate the issue. Thank you.
Comment 6 Jaroslav Tulach 2012-11-20 11:58:59 UTC
As far as the "Open Recent File" menu goes - we can (and probably should) apply the same solution as in "Open Recent Project". The system remembers the icons at the time of closing and persists them for subsequent starts. With a system like this there need to be no I/O when showing the "Open Recent File" submenu.
Comment 7 Jaroslav Havlin 2012-11-20 16:28:33 UTC
> As far as the "Open Recent File" menu goes - we can (and probably should) apply
> the same solution as in "Open Recent Project". The system remembers the icons
> at the time of closing and persists them for subsequent starts.
http://hg.netbeans.org/core-main/rev/37063b26d565
Thank you for the advice!

Reassigning to Favorites. Ondra, can you please evaluate it? Thank you.
Comment 8 Ondrej Vrabec 2012-11-21 09:01:18 UTC
Regarding the same problem in Favorites view. The classpath is scanned here:
"AWT-EventQueue-1" prio=10 tid=0x00007f81e803c800 nid=0x2dc6 at breakpoint[0x00007f8218483000]
   java.lang.Thread.State: RUNNABLE
        at org.netbeans.modules.parsing.impl.indexing.errors.ErrorAnnotator.enqueue(ErrorAnnotator.java:255)
        at org.netbeans.modules.parsing.impl.indexing.errors.ErrorAnnotator.isInError(ErrorAnnotator.java:281)
        - locked <0x00000000d1433000> (a org.netbeans.modules.parsing.impl.indexing.errors.ErrorAnnotator)
        at org.netbeans.modules.parsing.impl.indexing.errors.ErrorAnnotator.annotateIcon(ErrorAnnotator.java:124)
        at org.netbeans.modules.masterfs.filebasedfs.FileBasedFileSystem$StatusImpl.annotateIcon(FileBasedFileSystem.java:349)
        at org.openide.loaders.DataFolder$FolderNode.getIcon(DataFolder.java:1223)

The Favorites view is not directly involved, only DataSystems and Explorer (responsible for annotating the icon) and Parsing. I doubt it can be somehow disabled only for the Favorites view and anyway, at this point you're already browsing a project's area, so something (if not everything) must be already known about the project. Passing to parsing to evaluate - is there a way to turn error badge annotations for files under closed project when browsing views? And does it even make sense?
Comment 9 Tomas Zezula 2012-11-21 09:20:59 UTC
Unfortunately in NB project system there is no difference among opened and closed but loaded project, the only difference is the presence in Project UI tab. The projects are automatically loaded when their files are being accessed. For lang infrastructure there is no way how to find out if project is opened and not load it.
It's even questionable if it's desired.
Comment 10 Milos Kleint 2012-11-21 10:31:22 UTC
ok, recent files are fixed, and the general agreement is that favourites behaviour is correct. AFAIK there are no plans to change the scope for project loading and recognition, so project owners have to cope with the fact that project is to be loaded even if it's not opened. Usually via FileOwnerQuery I suppose.
Comment 11 vaskar 2012-11-21 11:11:26 UTC
Is there any way to make NB forget about the projects, as if they have never been opened? By removing some part of the userdir or something like that?
Comment 12 Milos Kleint 2012-11-21 11:25:24 UTC
(In reply to comment #11)
> Is there any way to make NB forget about the projects, as if they have never
> been opened? By removing some part of the userdir or something like that?

not really. it's all just in memory. Eg. FileOwnerQuery will (via one of the implementations) for a given file iterate the parent chain and for each directory attempt to load the project. If it succeeds it's the project owning the file. Just the basic picture. 
Known projects are kept in ProjectManager and eventually released when noone is using them, but via FOQ can be recreated at any time.
Comment 13 Quality Engineering 2012-11-21 13:42:23 UTC
Integrated into 'main-golden', will be available in build *201211211016* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/37063b26d565
User: Jaroslav Havlin <jhavlin@netbeans.org>
Log: #222330: Classpath for the listed files is retrieved when showing Open Recent File
Comment 14 attila.kelemen 2013-05-19 21:05:44 UTC
I have been recently working on making the Gradle support extensible (the first candidate extension is NBAndroid). This required me some refactoring, so that I can no longer hack around the issue regarding "Favorites". Therefore, I ask you to please reconsider the resolution of this issue.

Reading the comments, I can see some misunderstanding. The problem is not that NetBeans loads projects not opened but that it loads projects simply because their folder are displayed on the "Favorites" panel. However, the only thing NetBeans asks the Project implementation is the classpath for the displayed nodes. In my opinion this is not useful. Especially not useful when requesting the classpath for a project directory (which should be null for all projects).

An acceptable compromise (in which case I would consider this issue fixed) would be to not ask for the classpath for directories which are project directories themselves (i.e., ProjectFactory.isProject returns true).
Comment 15 Milos Kleint 2013-05-20 04:48:40 UTC
unfortunately there are project types that have source roots in project root, on top of that when you expand a project folder in favourites the classpath will likely be asked for all of them as well.

you get same effect upon ide restart with files opened from unopened projects.

it's a problem for maven projects as well, i've got several slowness issues related to the problem of us attempting to handle unopened projects. while upon opening projects we have a fairly robust loading ui, request to load a non opened project can come from any thread at any time and is xpected to be fast.
Comment 16 attila.kelemen 2013-05-20 11:17:47 UTC
If some project types have their source roots in the root folder, is it feasible that the "Favorites" window does not query the classpath of any node? As far as I could tell, it just does so for the error badge which seems unimportant to me (in the Favorites window).

By the way, how fast loadProject is expected to be? I mean it is allowed to throw IOException which implies that it may do some I/O. Can it be called from the Event Dispatch Thread?

Regardless, I do relatively small work in ProjectFactory.loadProject (check if some file exist or not). However, loading a project will trigger a *slow* background load of models for the project. This is unavoidable. Especially, since I will have extensions, as they will need the model. Atleast to know if they have to be active or not.

So, all in all, loadProject itself is fast but will trigger some slow background task. This will also block subsequent (legitim) project opens (loads) until it completes.
Comment 17 Milos Kleint 2013-05-20 14:36:19 UTC
(In reply to comment #16)
> If some project types have their source roots in the root folder, is it
> feasible that the "Favorites" window does not query the classpath of any node?
> As far as I could tell, it just does so for the error badge which seems
> unimportant to me (in the Favorites window).
> 
> By the way, how fast loadProject is expected to be? I mean it is allowed to
> throw IOException which implies that it may do some I/O. Can it be called from
> the Event Dispatch Thread?
> 

unfortunately for unopened projects the answer is yes.
basically anything calling FileOwnerQuery will be affected here. Among others that's for example FIleEncodingQuery called on opening documents and i'm sure there are more. 


> Regardless, I do relatively small work in ProjectFactory.loadProject (check if
> some file exist or not). However, loading a project will trigger a *slow*
> background load of models for the project. This is unavoidable. Especially,
> since I will have extensions, as they will need the model. Atleast to know if
> they have to be active or not.

there are some tricks that could help, like have a ModelProvider in project's lookup that will be used by extensions to get the model (or not) and listen for events like model loaded, model obsolete.

> 
> So, all in all, loadProject itself is fast but will trigger some slow
> background task. This will also block subsequent (legitim) project opens
> (loads) until it completes.

ProjectFactory.load should be only called by ProjectManager once per project instance, any subsequent calls to ProjectManager should be returning the original instance until CGed.
Comment 18 attila.kelemen 2013-05-20 15:57:12 UTC
I still can't see how I can prevent model loading when NB requests for the ClassPathProvider (without *very* dirty hacks). That is, I somehow has to determine which extension is providing the ClassPathProvider and I can only make decision after seeing what models are available by loading the models from Gradle.

The only thing which comes to mind is that I subclass Lookup, check if "ClassPathProvider" was requested. If so, I return a ClassPathProvider which first checks if the project directory is queried, if not then falls-back to actual queries. Also, I need to consider listeners registered with Lookup.Result. This is something I really want to avoid knowing that NB is very sensitive for anything related to ClassPathProvider, so I can totaly expect that something will break even after seemingly minor changes.

Is it really necessary to show error badges in the "Favorites" window or is something hard to be removed?
Comment 19 Milos Kleint 2013-05-20 16:36:40 UTC
(In reply to comment #18)
> I still can't see how I can prevent model loading when NB requests for the
> ClassPathProvider (without *very* dirty hacks). That is, I somehow has to
> determine which extension is providing the ClassPathProvider and I can only
> make decision after seeing what models are available by loading the models from
> Gradle.
> 

you can make the claaspathprovider model loading aware, if model is not loaded, just return empty classpaths, you get error markers in java files then but once the model loads fire achange from CPP and errors disappear


in the long term i would personally like to have favourites less project aware or have some additional level in project loading that would make FIleQwnerQuery and it's Simple implementation walking up parent folders less eager to load unknown projects.
Comment 20 attila.kelemen 2013-05-20 16:59:18 UTC
(In reply to comment #19)
> 
> you can make the claaspathprovider model loading aware, if model is not loaded,
> just return empty classpaths, you get error markers in java files then but once
> the model loads fire achange from CPP and errors disappear
> 

I do not have a single ClassPathProvider. Rather, depending on what models are available either NBAndroid or the JavaExtension will return the ClassPathProvider. Apart from that, even in this case I still need the wrapped Lookup hack (as I said before) because otherwise I have to make sure that the Lookup will be propagated with instances from extensions (because if I don't wrap, I cannot know what instances are requested and some requested might only be available after full model load).

Also, providing empty classpaths (rather than null) caused problems in the past. I don't know why but this is something Jan Lahoda told me and it was indeed the problem.

So as I can see, fixing this on my side seems way too much effort (considering how much time I will need to hack around problems which will arise due to classpath errors) and more importantly, needs some serious hacking. I don't know how much effort needed for "fixing" the Favorites but if that is too much as well, I would rather tell users not to use Favorites (or live with the fact that every project will be loaded).
Comment 21 Milos Kleint 2013-05-20 18:41:07 UTC
> 
> So as I can see, fixing this on my side seems way too much effort (considering
> how much time I will need to hack around problems which will arise due to
> classpath errors) and more importantly, needs some serious hacking. I don't
> know how much effort needed for "fixing" the Favorites but if that is too much
> as well, I would rather tell users not to use Favorites (or live with the fact
> that every project will be loaded).

just remember that favourites is not the only place loading projects out of the open project loop
Comment 22 attila.kelemen 2013-05-20 18:44:37 UTC
(In reply to comment #21)
> just remember that favourites is not the only place loading projects out of the
> open project loop

Yes, I know but this is the most surprising. Other places I didn't find unexpected. (e.g.: if a file of a project is opened, I don't think there is anything should be done).
Comment 23 Milos Kleint 2013-07-10 12:28:43 UTC
if I got it right changeset http://hg.netbeans.org/core-main/rev/37063b26d565 fixes the issue of recent file icons. Correct? 


so this issue can be closed, the follow up talk about FileOwnerQuery and projects is more generic and future plans oriented and is also covered in some other issues.

for merging of ClassPathProviders, please see LookupMerger api class that allows one to have one instance appear to be in the lookup from the outside, but have multiple implementations that dynamically change or complement themselves during the lifetime of the projects.
Comment 24 Jaroslav Havlin 2013-07-10 12:56:40 UTC
(In reply to comment #23)
> if I got it right changeset http://hg.netbeans.org/core-main/rev/37063b26d565
> fixes the issue of recent file icons. Correct?
Yes.

> [...] so this issue can be closed [...]
I agree.