Bug 191334

I consider this feature as really important. I always have to switch between Eclipse or IDEA, if I need to know which maven projects use a certain method or class. We work with more than 180 Maven projects, it's not possible to keep open all those projects in the IDE, which would end in endless "scan for changes" scans..

I'd like to contribute a patch for this issue.
It seems like this feature requires modifications in WhereUsedQueryUI & ClasspathInfo.
I think there should be another combobox entry in the "find usages" dialog, e.g "Transitive dependencies" which will search for dependend jars in the local repository.
I wonder if there is a possibility to find out if the containing project is a maven one, so I could enabled/disable this combo entry for non maven projets. WDYT?

Afaik there is no API for detection of maven projects, Tondo? Anyway..any patch is welcome.

Jesse needs to comment on this.

BTW, note that local repository can become very large over time...

(In reply to comment #3)
> Afaik there is no API for detection of maven projects, Tondo? Anyway..any patch
> is welcome.

following two possibilities are really error prone:

private boolean isMavenProject(final Project project) {
        if (project != null && project.getProjectDirectory() != null) {
            if (project.getProjectDirectory().isValid() && project.getProjectDirectory().isFolder()) {
                final FileObject projectPom = project.getProjectDirectory().getFileObject("pom", "xml");
                if (projectPom != null && projectPom.isValid()) {
                    return true;
                }
            }
        }
        return false;
    }

same with project.toString().contains("Maven"). Perhaps there's a better solution..

After some researching it looks like that this feature is not that easy to implement. It's curious that the problem (finding all transitive dependencies) was already solved in JavaTypeProvider#computeTypeNames (GoTo Type ) but is differently and incompletely implemented in FileSearchAction#getFileNames() (GoTo file: finds only non transitive files) and ClassIndex#getResources(find usages: finds only non transitive usages), which both require cache indices for external jar's and / or their content to be found.

are external jar's even cached by design?

So far I've tried to set the transitive SOURCE jar's into the source classpath of the find usages context but with no luck perhaps because the jar's weren't cached.. 

Do you have any suggestions?

(In reply to comment #4)
> Jesse needs to comment on this.
> 
> BTW, note that local repository can become very large over time...

That's true.
a solution could be to extract all dependencies to non opened maven projects from all loaded maven projects and search ONLY those URL's in the local repository.

First of all, bug #55291 is already open for finding usages among the classpath entries of the open projects - which would include any declared or transitively implied dependencies of your projects. The Maven support already supplies the usual information about the locations of sources for these JARs if asked, so there is nothing specific to Maven to implement there.

So (and I hope I am reading the reporter's intent from comment #1 correctly!) I would rephrase this as a request to know what Maven projects "out there" might be using a given class (method, etc.) which would typically be defined in an open project. That is a potentially unbounded search which would never be offered for Ant-based projects (where you simply have no clue what is downstream of a given project exposing an API), but in the Maven universe you can at least say that anything present in the local repository is a potential candidate. Any search hits could be presented in a way that the user could open the source file from the corresponding source project, if one is known; or else at least open an artifact viewer window (this offers the option to check out sources, for example). Such a feature would be tremendously useful for developers working on large multimodule projects in which it is not necessarily obvious where a particular API is being called.

This would I think be implemented as another FU provider, just like e.g. apisupport.refactoring does when searching layer files. Cannot be implemented in java.source since that knows nothing of Maven.

NexusRepositoryIndexerImpl currently creates an index of the local repository using all standard registered IndexCreator's, plus a NbIndexCreator which also records (transitive) artifact dependencies. This extra set of fields is absent from remote downloaded indices, but is used to implement a special Find Usages action on artifact nodes in the Maven Repository Browser.

A standard index creator records a list of all classes in each JAR, but there is no existing index of class -> class dependencies. With such an index, the FU implementation could jump directly to candidate users of a Java element, perhaps filtering out false positives (e.g. classes using a different method in the same API class). It would not be hard to write an index creator which does this - org.netbeans.nbbuild.VerifyClassLinkage.dependencies shows an optimized version with very little code - though I fear that performance of creating the local index would suffer somewhat due to (1) extra overhead of parsing every *.class in every JAR, (2) a substantially larger index with all these additional fields. Could be prototyped to see if the overhead is substantial. Recording just a CRC32 (say) of referenced class names might reduce index size, since there is no need to check substrings.

Another option would be to use the existing artifact dependency index to find all downstream artifacts depending on the owner of the selected Java element, and then search their classes for references. This effectively shifts the heavy processing from index creation time into the Find Usages action. That would be effective if most API modules were reasonably small - so a large fraction of the users of the module would in fact be referencing the given element in it - and if most dependencies on the API module were really required for compilation and not just included by transitivity. (If you routinely run dependency:analyze and fix all of its warnings then it would suffice to consider only direct dependencies, but this is not a safe assumption in general.) But if these conditions are not met, and I suspect they would not be in typical big systems which just dump a standard list of dependencies into the classpath of every project, then this option would be too slow to be useful.

(In reply to comment #7)

> So (and I hope I am reading the reporter's intent from comment #1 correctly!) 

You read the intention correctly. :-)

So from your very extensive comment I assume two possible solutions.

1:
Implement a new FU provider in the refactoring module which accesses the repository index (NbIndexCreator) to find all transitive dependend artifacts in the local repository on the index. After that, parse all artifacts and search for the required class / method and return the filtered results.
Which would result in a lot of parsing effort and would also cause long wait times for the user.

2:
Write or extend a jar indexer which indexes all *.class files in a jar and adds at least the fields:
ElementKind.CLASS 
ClassIndexImpl.UsageType.TYPE_REFERENCE ClassIndexImpl.UsageType.METHOD_REFERENCE
ClassIndexImpl.UsageType.FIELD_REFERENCE
to the lucene document index of the found classes. With such an index, it should be possible to use JavaWhereUsedQueryPlugin with an extended classpath to find all referencing classes. The classpath could be extended with all directly dependend jar archives.
This solution, like you said would result in longer indexing times and a bigger index but is much faster for the user to use.

WDYT?

onmomo,
In your first comment you mentioned, that you don't want to open all your 180 projects to avoid scanning, but in your latest comment you suggest to extends scanning to scan all those projects even if they are not open...

(In reply to comment #9)
> onmomo,
> In your first comment you mentioned, that you don't want to open all your 180
> projects to avoid scanning, but in your latest comment you suggest to extends
> scanning to scan all those projects even if they are not open...

NB scans every opened project for changes, tasks, compile errors/warnings or even more if you have optional code quality plugins like findbugs, PMD installed, which I don't.. In addition, is every maven-project "installed" by the embedder to check if the projects are valid. All those scans seem very slowy and take a lot of time to complete, every time the user starts NB.

The extended scan i suggested should only occur after the first load of a project and only after changes in the dependency section of the project, not after every startup of NB. Furthermore, I don't believe indexing takes as much time as the full scans I mentioned above. But this should be prototyped like Jesse already told.

[various fields inexplicably set to incorrect values as part of comment #8]

(In reply to comment #8)
> Implement a new FU provider in the refactoring module

It would be in Maven support, not in the generic refactoring module.

> Write or extend a jar indexer which indexes all *.class files in a jar and adds
> at least the fields:
> ElementKind.CLASS 
> ClassIndexImpl.UsageType.TYPE_REFERENCE
> ClassIndexImpl.UsageType.METHOD_REFERENCE
> ClassIndexImpl.UsageType.FIELD_REFERENCE
> to the lucene document index of the found classes.

Not exactly sure what you are suggesting here. I was suggesting an addition to the Maven repository indexer; there would be no change to generic Java infrastructure (including classpath scanning upon project open).

(In reply to comment #11)
> [various fields inexplicably set to incorrect values as part of comment #8]
> 
> (In reply to comment #8)
> > Implement a new FU provider in the refactoring module
> 
> It would be in Maven support, not in the generic refactoring module.
> 
> > Write or extend a jar indexer which indexes all *.class files in a jar and adds
> > at least the fields:
> > ElementKind.CLASS 
> > ClassIndexImpl.UsageType.TYPE_REFERENCE
> > ClassIndexImpl.UsageType.METHOD_REFERENCE
> > ClassIndexImpl.UsageType.FIELD_REFERENCE
> > to the lucene document index of the found classes.
> 
> Not exactly sure what you are suggesting here. I was suggesting an addition to
> the Maven repository indexer; there would be no change to generic Java
> infrastructure (including classpath scanning upon project open).

Ok thx, I'll look what I can do.

I managed to register a new find usages refactoring provider to get noticed if find usages is called on a maven project. (ActionsImplementationProvider --> doFindUsages(), canFindUsages())

well despite of the different index search, the maven FU implementation looks quiet simular to the java refactoring module impl. it seems like I have to create a new refactoring module for maven refactorings which dublicates code from java refactoring module which is really nasty... but I don't know how to do it with less code duplication. WDYT?

(In reply to comment #13)
> the maven FU implementation looks
> [quite similar] to the java refactoring module impl. it seems like I have to
> create a new refactoring module for maven refactorings which [duplicates] code
> from java refactoring module

What sort of code would be duplicated? There may be some opportunity for extracting common utilities into e.g. JavaRefactoringUtils.

I do not think F.U. on a type (class, ...) should have much overlap with refactoring.java, since such queries could be satisfied directly from the augmented Maven index. F.U. on a member (field, method, constructor) is trickier because you need to analyze candidate classes using the containing type and check to see if they are really using that member.

Of course you could start by supporting the extended F.U. only on types; the user could choose to open some or all source projects corresponding to search hits, and rerun F.U. on the member over the now-open projects to get exact results.

You could also do conservative filtering over candidate usages found in repository JARs: scan the constant pool for an identifier matching the simple name of the member being searched. This could produce some false positives, such as usages of a different method overload. For example, F.U. on org.w3c.dom.Node.getAttributes would mistakenly match:

  org.w3c.dom.Node n = ...;
  // ...later:
  javax.print.PrintService ps = ...;
  ps.getAttributes().add(...);

but I think such false positives should not be so frequent as to be bothersome.

There are also some exotic use cases where a simple type->type reverse index will miss some usages. I am thinking of things like

  // in an open project:
  public interface A {void m();}
  public abstract class B implements A {}
  // in a repo artifact:
  public class C extends B {public void m() {}}

where C.class makes no mention of A, despite implementing one of its methods; so F.U. on overriders of A.m() will not produce repo hits. Fixing that would require much more sophisticated bytecode analysis (dealing with the Java type hierarchy), which is probably not worth it.

well what I've got so far:

I've implemented an additional index in "NexusRepositoryIndexerImpl" which adds to every artifact "Document" an new Field which contains for all classes inside this artifact an class to referringClass index. e.g

classes: foo.Bar is used by ref.Foo and ref.Bar
classes: foo.Bar2 is used by ref.Foo2 and ref.Bar2
field "nbdc" contains value: nbdc:foo.Bar<ref.Foo,ref.Bar,\nfoo.Bar2<ref.Foo2,ref.Bar2,\n

I was necessary to set the field to Store.YES to read the field value afterwards.
after parsing the field value, I'm able to return all referring classes in the local repository.

The index size increased from approx. 4MB to 34MB (150MB repository).

>Of course you could start by supporting the extended F.U. only on types; the
>user could choose to open some or all source projects corresponding to search
>hits, and rerun F.U. on the member over the now-open projects to get exact
>results.

This is what I would like to achieve. But I'm stuck right now, I don't know in what Type I have to return the referring classes. I tried it as FileObject, but I can't create a FileObject from a class within a jar archive..

(In reply to comment #15)
> field "nbdc" contains value:
> nbdc:foo.Bar<ref.Foo,ref.Bar,\nfoo.Bar2<ref.Foo2,ref.Bar2,\n
> 
> The index size increased from approx. 4MB to 34MB (150MB repository).

You could make the field much more compact using a binary format, e.g.:

nbdc = refs*            -- for all classes in JAR, in alpha or ZIP order
refs = sum* 0x00000000  -- classes referred to, then separator
sum  = byte{4}          -- CRC-32 sum of class FQN

Then to find referrers of a particular class, just search nbdc for substring matches on the (binary) CRC-32 of that class's FQN. Parse each result into 4-byte chunks to find the referrers (this will omit the rare false positives caused by matches not aligned on 4-byte boundaries, though you will still get even rarer false positives from hash collisions). Then correlate the referrer index to an actual class in the JAR and include it in the result.

> I don't know in what Type I have to return the referring classes.

I guess you would want to return a SimpleRefactoringElementImplementation holding a *.class FileObject, where openInEditor would just look for OpenCookie on the corresponding DataObject - this ought to open the *.java from the corresponding source project where known, else use the *-sources.jar when this has been downloaded, else fall back to showing a skeleton. To be fancy you could try to find the actual line(s) of code using the queried class, but this may be overkill.

> I can't create a FileObject from a class within a jar archive

File jar = ...;
String fqn = ...;
FileObject fo = URLMapper.findFileObject(new URL("jar:" + jar.toURI() + "!/" + fqn.replace('.', '/') + ".class"));
if (fo != null) ...

> > I can't create a FileObject from a class within a jar archive
> 
> File jar = ...;
> String fqn = ...;
> FileObject fo = URLMapper.findFileObject(new URL("jar:" + jar.toURI() + "!/" +
> fqn.replace('.', '/') + ".class"));
> if (fo != null) ...

Thanks, this was the last missing puzzle! It's possible now to find/display and to open source or skeleton classes, which are located in the local repo.
Is it possible to open a correlating project after a interessting class was found? Are there any information on closed projects? I guess it won't work if the project was never opened in the IDE..

> You could make the field much more compact using a binary format, e.g.:
> 
> nbdc = refs*            -- for all classes in JAR, in alpha or ZIP order
> refs = sum* 0x00000000  -- classes referred to, then separator
> sum  = byte{4}          -- CRC-32 sum of class FQN
> 
> Then to find referrers of a particular class, just search nbdc for substring
> matches on the (binary) CRC-32 of that class's FQN. Parse each result into
> 4-byte chunks to find the referrers (this will omit the rare false positives
> caused by matches not aligned on 4-byte boundaries, though you will still get
> even rarer false positives from hash collisions). Then correlate the referrer
> index to an actual class in the JAR and include it in the result.
> 

This looks interesting, but I'm not excately sure how I should implement this binary format. I'm also not able to retrieve the matching ArtifactInfo for the queried index field (only field value), without the JAR, It's not possible to resolve the referrer CRC32 because there's nothing to match against. 

Should I upload the patch? it would be nice if you could review it and give me some feedback, plz.

I have to correct myself, was late...

> Is it possible to open a correlating project after a interessting class was
> found? Are there any information on closed projects? I guess it won't work if
> the project was never opened in the IDE..

It's possible to open an already closed project with "/Navigate/Select in projects" after opening a referring class. Unfortunately, the referring class is editable before opening the closed project, looks like a bug in the IDE.

If the project was never ever opened in the IDE before, the "select in projects" would like to open the "NB-build module" probably because It was started from the IDE.

(In reply to comment #17)
> Are there any information on closed projects? I guess it won't work if
> the project was never opened in the IDE.

Right, MavenFileOwnerQueryImpl makes the association when a project is first opened and remembers it thereafter.

> I'm not exactly sure how I should implement this binary format.

Any particular issue?

> I'm also not able to retrieve the matching ArtifactInfo for the
> queried index field (only field value)

Perhaps you need to prefix the field with the groupId|artifactId|version| of the artifact defining it.

> Should I upload the patch?

Sure. Could at least put it on a branch.

(In reply to comment #18)
> the referring class
> is editable before opening the closed project, looks like a bug in the IDE.

There is no law saying you cannot edit a source file in a non-open project.

> If the project was never ever opened in the IDE before, the "select in
> projects" would like to open the "NB-build module" probably because It was
> started from the IDE.

For a skeleton class, the IDE opens e.g. $userdir/var/cache/index/s1625/java/14/gensrc/junit/framework/Assert.java; this would only be "owned" by any project in case you started the IDE using 'ant tryme' so that $userdir = nbbuild/testuserdir, but normally not. For an actual source file like junit/framework/Assert.java in ~/.m2/repository/junit/junit/4.8.2/junit-4.8.2-sources.jar, this would not likely be considered "owned" by anything.

> > I'm not exactly sure how I should implement this binary format.
> 
> Any particular issue?
> 
> > I'm also not able to retrieve the matching ArtifactInfo for the
> > queried index field (only field value)
> 
> Perhaps you need to prefix the field with the groupId|artifactId|version| of
> the artifact defining it.
> 

Well, to optimize the index I tried something like this:

classes: foo.Bar is used by ref.Foo and ref.Bar
classes: foo.Bar2 is used by ref.Foo2 and ref.Bar2

field "nbdc" contains value:
nbdc:groupId|artifactId|version|CRC32(foo.Bar)<CRC32(ref.Foo),CRC32(ref.Bar),\nCRC32(foo.Bar2)<CRC32(ref.Foo2),CRC32(ref.Bar2),\n
 
Then searched for e.g CRC32(foo.Bar) and parsed artifactId's and CRC32 classes ref.Foo and ref.Bar from the field value. To resolve the CRC32 classes back to FQN, i tried to match em with the contained classes in the parsed artifact. Unfortunately won't work because a class could have a dependency to another artifact in the repo. So it would be necessary to check every class in every artifact in the repo for this solution to work...

Could you be more specific on the optimation you recommended?

nbdc = refs*            -- for all classes in JAR, in alpha or ZIP order
refs = sum* 0x00000000  -- classes referred to, then separator
sum  = byte{4}          -- CRC-32 sum of class FQN

sum is CRC32 of every class? What do you mean with sum* 0x00000000 and refs*?

(In reply to comment #20)
> To resolve the CRC32 classes back to
> FQN, i tried to match em with the contained classes in the parsed artifact.

Right...

> Unfortunately won't work because a class could have a dependency to another
> artifact in the repo. So it would be necessary to check every class in every
> artifact in the repo for this solution to work...

I am afraid I lost you here. You have determined, say, CRC32(ref.Foo). So you resolve the artifact named at the head of the record to a JAR file and list its classes, searching for one with the same CRC-32.

(Could perhaps also use the JarFileContentsIndexCreator.FLD_CLASSNAMES field for this, if there is any advantage to delaying download of the referring artifact; certainly if the user wants to open a hit you will need to download it anyway.)

I actually suggested a format that is even more compact by omitting even the name of the referring class, relying on an external order to infer which class that is. Whether this saves enough space to be worth the extra effort, I am not sure; depends on the average number of outgoing references per class, which might seem high until you consider small anonymous inner classes such as event handlers.

> sum is CRC32 of every class?

Right, as a four-byte sequence.

> What do you mean with sum* 0x00000000

Zero or more checksums (of referred-to class FQNs), followed by a sequence of four null bytes.

> and refs*?

Zero or more reference lists.

(In reply to comment #21)
> Could perhaps also use the JarFileContentsIndexCreator.FLD_CLASSNAMES field

Or rather FLD_CLASSNAMES_KW, which is what is actually stored. This field omits all nested classes (*$*.class), so for it to be useful you would need to aggregate references from any nested classes into that of their top-level class... which might not be a bad idea anyway, since you want to present hits to the user on the level of a Java source file, and most references from nested classes will either be to an enclosing class or to the same APIs that the enclosing class refers to. (The _target_ of references may well be nested classes; for example, you might wish to search for usages of JComponent.AccessibleJComponent as distinct from usages of JComponent generally.)

You presumably need not store references to classes in the same artifact; regular "Find Usages" on an open source project would already be returning these, and they likely constitute a large fraction of all class references so omitting them should save space. You can also omit references to classes known to be in the JRE.

Another optimization: rather than prefixing the field value with the full GAV (group|artifact|version|), simply prefix the SHA-1 of the artifact, from which you can easily find the artifact (ArtifactInfo.SHA1). Maven Indexer stores this field as a hex string (40 bytes), but you could store it as binary (20 bytes). Or there is probably some way in the Lucene API to find the full document corresponding to a given record; then you just need to find the ArtifactInfo.UINFO record in the same document as your match, and the "class references" record need not mention the containing artifact at all.

I am assuming that Lucene can store, and index, binary content (an arbitrary hex sequence), but I do not know much about this yet. If it does not deal well with binary data, then you could still use a compact alternative such as Base-64.

> > Unfortunately won't work because a class could have a dependency to another
> > artifact in the repo. So it would be necessary to check every class in every
> > artifact in the repo for this solution to work...
> 
> I am afraid I lost you here. You have determined, say, CRC32(ref.Foo). So you
> resolve the artifact named at the head of the record to a JAR file and list its
> classes, searching for one with the same CRC-32.
> 

Well, if I store a field from the content of a JAR by putting all classes and all referring classes of this JAR into the field. There will be propably classes in the JAR which are referring to classes outside of this JAR. So I'm not able to resolve those entries in this field because the matching classes are located in another jar.

>You presumably need not store references to classes in the same artifact;
>regular "Find Usages" on an open source project would already be returning
>these, and they likely constitute a large fraction of all class references so
>omitting them should save space. 

This would save space. Currently it would not return the classes in the artifact because I implemented the local repo search as an additional scope..

>You can also omit references to classes known to be in the JRE. 
This would differ from the Java find usages feature which finds JRE dependencies.

Created attachment 109278 [details]
Maven find usages feature which supports usages search on types in local maven repository

This is my first code contribution to NB. I hope the patch works.

Comment on attachment 109278 [details]
Maven find usages feature which supports usages search on types in local maven repository

- As a fist step, the patch contains FU without index optimizations.

(In reply to comment #24)
> Created an attachment (id=109278) [details]

The patch reverts a bunch of recent changes in maven.indexer. Mercurial misuse, I guess. I will try to clean it up and make a branch out of it for further testing. Also copying RetoucheCommit is likely unnecessary, and copying WhereUsedPanel seems wrong - rather want a plugin to the standard feature (bug #199779 may be helpful there).

(In reply to comment #26)
> (In reply to comment #24)
> > Created an attachment (id=109278) [details] [details]
> 
> The patch reverts a bunch of recent changes in maven.indexer. Mercurial misuse,
> I guess. I will try to clean it up and make a branch out of it for further
> testing. Also copying RetoucheCommit is likely unnecessary, and copying
> WhereUsedPanel seems wrong - rather want a plugin to the standard feature (bug
> #199779 may be helpful there).

> The patch reverts a bunch of recent changes in maven.indexer. Mercurial misuse,
> I guess.

I did following on main: 
hg pull -u
hg diff -g > 191334.diff

I'm new to mercurial, this is what I would do in subversion, what's wrong?

> ... and copying WhereUsedPanel seems wrong

With #199779 it's not necessary if maven FU would support ALL standard feature search types (comments, subtypes etc.), I guess.

(In reply to comment #27)
> I did following on main: 
> hg pull -u
> hg diff -g > 191334.diff

Well somehow it went wrong. You would want to use MQ or branches rather than just leaving uncommitted local modifications sitting around. Anyway I have converted it to a branch (managed by the PBranch extension) in core-main: maven_find_usages_191334

> With #199779 it's not necessary if maven FU would support ALL standard feature
> search types (comments, subtypes etc.)

Even without #199779 I do not think anything special is needed, since the standard Where Used should add in contributions from various modules that offer it. At least this works for apisupport.refactoring.

(In reply to comment #28)
> Even without #199779 I do not think anything special is needed, since the
> standard Where Used should add in contributions from various modules that offer
> it.

The UI seems to be basically working in the branch. Presentation is poor and needs some work.

Getting closer. I trimmed the index to not store references to classes in the same JAR. Also it will no longer store references to JRE classes - way too much overhead, and rarely useful. This takes the local index size to within a factor of 2 or 3 what it would be otherwise. Have not yet tried using a binary index.

Presentation is still ugly. Maybe need to define a custom refactoring element tree factory; need to learn more about the API. Also need to check whether it is possible to offer a special scope so the feature can be offered only when it really makes sense. Also can consider limiting results to just the newest version of a given artifact.

(In reply to comment #30)

The branch changes look quite good!

> Presentation is still ugly. Maybe need to define a custom refactoring element
> tree factory; need to learn more about the API. 
Would be nice if the search hits are listed by their artifacts and not by their packages.
artifacts-GAV|package|SimpleClassName|Hits in Class body

> Also need to check whether it is possible to offer a special scope so the feature can be offered only when it really makes sense. 

I would prefer something simular to my original implementation. For example an additional scope "Local Maven Repository" could appear if FU is used for Maven projects.

>Also can consider limiting results to just the newest version of a given artifact.

Really useful if you have a lot of snapshots in the repository.

(In reply to comment #31)
> Would be nice if the search hits are listed by their artifacts and not by their
> packages.

Agreed, I am just not yet sure how to accomplish that. I merely return a FileObject for the hit and some other code produces the tree view for it. See my earlier comment about a tree factory.

> I would prefer something simular to my original implementation.

Your original patch had just a couple of unreferenced UI classes, so I am not sure what that would be. Anyway copying implementation from refactoring.java is not maintainable. I still need to look at bug #199779.

> an
> additional scope "Local Maven Repository" could appear if FU is used for Maven
> projects.

Right, this would be fine if it is possible.

(In reply to comment #31)
> if the search hits are listed by their artifacts

Working now in branch; can be refined.

> additional scope "Local Maven Repository" could appear

This appears to be impossible.

(In reply to comment #33)
> (In reply to comment #31)
> > if the search hits are listed by their artifacts
> 
> Working now in branch; can be refined.
> 
I like the local repo as root element! 

> > additional scope "Local Maven Repository" could appear
> 
> This appears to be impossible.

Well, I my original impl. I modified WhereUsedQueryUI, WhereUsedPanel/WhereUsedPanel.form  to feature a "Local Maven Repository" scope. The scope disabled not supported Maven search types if it was selected. Furthermore, only hits in the local repo were displayed. If it wasn't selected, the maven search was disabled and the results of the original feature were shown.
So it was seperated from the original feature and would have offered the possibility to add additional modifications to the UI, like a combo "Display only newest snapshots" etc..

But yes, It is less maintainable and a lot harder to read than using the same UI, but more flexible. Do you think of a lot of modifications to the original feature in the future?

(In reply to comment #34)
> Do you think [there will be] a lot of modifications to the original
> feature in the future?

Yes.

Anyway when all the indexer hits are grouped together under one "Maven Repository: local" node, I think it is much less important to suppress this feature when not specifically requested. This tree view also makes it unnecessary to suppress hits from older artifacts: the newest version should show up first under its artifact node, and you can simply ignore older hits if you are not interested.

BTW I just pushed a fix to display icons. You will get a project icon for a version node when there is a known source project association, typically for snapshot versions.

I just pushed a fix to use a more compact format for nbdc. I could not find a way to actually store binary CRC-32s, so I had to use Base-64; and I also could not find a way to perform a custom tokenization, so all the 6-character tokens are separated by spaces instead (even though the tokenization is implied by character positions). Nonetheless, the overhead seems to be pretty modest; the space consumed is typically a fraction of that used by ArtifactInfo.NAMES (from which it is keyed), so it only seems to add maybe 30% to the size of the local repo index, which I think is acceptable given the usefulness of the information. Still need to test more thoroughly.

(In reply to comment #36)
> the overhead seems to be pretty modest

With my current local repo, from 9.5M to 12M to add the class usage index.

(In reply to comment #7)
> at least open an artifact viewer window (this offers the option to check out
> sources, for example)

Unfortunately this does not seem straightforward. The Refactoring API offers openInEditor on the RefactoringElementImplementation, i.e. leaf; the local Maven repo impl will open the source file from the local project, if known, else from the *-sources.jar if that can be located, else falling back to showing the pseudo-disassembly of the class. But there is no API to offer an open action, or any other UI customizations, to parent nodes such as that representing the artifact. (CheckNodeListener.mouseClicked does check for userObject instanceof Openable, but this does not seem to apply also to processing the Enter key, so it is not accessible.)

(In reply to comment #14)
> you could start by supporting the extended F.U. only on types

This is what is done for now.

Merged: core-main #906cf147a360

Integrated into 'main-golden', will be available in build *201107130600* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/906cf147a360
User: Jesse Glick <jglick@netbeans.org>
Log: #191334: Find Usages on class references from (non-open) Maven artifacts.

(In reply to comment #37)
> (In reply to comment #36)
> > the overhead seems to be pretty modest
> 
> With my current local repo, from 9.5M to 12M to add the class usage index.

Nice, really impressive index optimization!

(In reply to comment #14)
> you could start by supporting the extended F.U. only on types

> This is what is done for now.

I guess the implementation of other search options may be pretty much the same as in java.refactoring, which has used Java source API.

(In reply to comment #41)
> I guess the implementation of other search options may be pretty much the same
> as in java.refactoring, which has used Java source API.

Perhaps, or using the approximations given in comment #14 if that is easier.