This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 134992 - Non-Java source file handling issues
Summary: Non-Java source file handling issues
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Tomas Zezula
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-05-13 22:00 UTC by jessholle
Modified: 2008-08-18 15:34 UTC (History)
1 user (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jessholle 2008-05-13 22:00:48 UTC
We are experiencing issue #121853.

As I look at other bugs being linked to this bug I think there is a bigger underlying gap/issue here that I'd like to
raise for discussion.  [I did so on nbuser (as suggested by those manning the NetBeans booth at JavaOne) to no apparent
avail, so I'm recording this as an issue.]

The issue:

    NetBeans's editor (error marking, code completion, etc) has no out-of-the-box or easy-to-add means of handling
references to classes which result from non-Java source files.

Sure NetBeans now has great handling for all sorts of languages from Ruby to JavaScript.  Sure via some Ant tweaks you
can convince "compile" and "build" actions to do most anything.  The gap is that there's no systematic recognition that
regular Java projects may contain non-Java source files that produce Java classes which other classes in those projects
or others are written against.

What non-Java sources are involved is somewhat irrevelant -- the issue remains the same.  The workaround is to use
-J-DCacheClassPath.keepJars=true in one's netbeans.conf, but that does not actually suffice as per issue #121853.  Also
this assumes the non-Java-sourced classes have been built into the project compilation classpath -- the editor will show
errors in any case until this is done.

The point issue cited above should be addressed, but the larger issue is that NetBeans should provide an easy means (I
include authoring a *simple* module in this category) by which one can tell the IDE that certain non-Java files produce
Java classes and what those classes look like.

This issue applies to everything from JAXB to various custom code generation scenarios and is an ever more pressing
issue for our continued use of NetBeans.  We're probably ~80% Eclipse users the way it is, but the 20% is likely to
shrink rather than grow if this is not addressed.
Comment 1 Tomas Zezula 2008-05-14 15:58:53 UTC
There is currently (NB 6.5) such an API sketch, currently not in the official API packages yet. I am not sure if it's exactly what you need. This "API" was 
designed for to support groovy & scala files. When the Java module creates index & caches for source root it consults the VirtualSourceProviders which may 
create a virtual java files. These files are indexed and they will be available in the code completion and so on.
The interface is: http://hg.netbeans.org/main/file/e1b6c99e4370/java.source/src/org/netbeans/modules/java/source/usages/VirtualSourceProvider.java
The interface has 2 methods:
1) Returns the extension for which the second method should be called.
2) Translates given files into tuples {qualifiedName, content} which are added into given result.
Is this interface what you need?
Comment 2 jessholle 2008-05-14 16:07:18 UTC
That looks like what I need.  If it works for groovy and scala use cases, I'd bet it would work for my (far simpler) use
case.
Comment 3 Tomas Zezula 2008-05-14 16:38:05 UTC
OK, in this case you need to implement this interface and register it into the META-INF/services.
If you have any problems or questions feel free to ask.
It's better for me to use this issue or mail than the nbdev, for some reason mails I send to this mailing list come with several days delay.

Comment 4 jessholle 2008-05-14 16:51:27 UTC
That sounds pretty easy.  If I could take a gander at groovy and/or scala usages of this API (so as to be sure I
understood intended usage) that would probably help.

Also this is only in 6.5, right?  [I just moved to 6.1.]
Comment 5 Tomas Zezula 2008-05-14 16:53:47 UTC
Yes, it's 6.5 and it's not yet final, so it may change during the 6.5 development if someone comes with some new use case.
Comment 6 jessholle 2008-05-14 16:59:51 UTC
I'll probably let the groovy and scala folk take the lead here then, as my stuff should be pretty easy to write and
won't be terribly useful until 6.5 is further along -- so there's little call for me to write against the first draft
API available.
Comment 7 jessholle 2008-06-27 13:38:44 UTC
Are they any examples of using VirtualSourceProvider in NB 6.5 yet?

[For that matter is VirtualSourceProvider in NB 6.5 yet?]

I'd like to try to hack out my own VirtualSourceProvider, but I'd really like an example to go from -- as the Javadoc on
this class is a bit sparse at the moment.  I can guess at how to use the Result interface but a simple blurb on the
meaning of each parameter and/or an example would help a lot...
Comment 8 Tomas Zezula 2008-06-27 14:00:26 UTC
Yes.
The groovy support is using it.
Path: groovy.editor/src/org/netbeans/modules/groovy/editor/parser/GroovyVirtualSourceProvider.java
Comment 9 jessholle 2008-06-27 17:25:08 UTC
Okay I completely coded my VirtualSourceProvider (it is pretty short and simple), but I can't compile it because I can't
figure out what module VirtualSourceProvider comes from.  The NetBeans editor seems to know about this class, but when I
try to build my NetBeans module it does not.  Which module(s) do I need to add to my dependency list?  So far I have
"Java Source" and "File System API", but I can't build...
Comment 10 Tomas Zezula 2008-06-27 17:32:10 UTC
For now you need an implementation dependency on the java.source module, not only API dependency.
The class in not yet part of the API.
Comment 11 jessholle 2008-06-27 17:37:39 UTC
Thanks.

How do I add the necessary META-INF/services entry via a module project?

In free-form projects (which I normally use) I just add this to my source as if it were a Java package, but the NB
module projects seem to prevent this.  What's the secret?
Comment 12 jessholle 2008-06-27 17:42:09 UTC
Never mind -- I figured it out.
Comment 13 Tomas Zezula 2008-06-27 17:42:48 UTC
You need to create META-INF folder (other/folder) under the source root. The secret is that you need to create it as a folder not as a package because the META-INF is not valid package name.
Comment 14 jessholle 2008-06-27 20:11:47 UTC
Well, the good news is my VirtualSourceProvider implementation seems to work.

The bad news is that NB 6.5 seems to ignore its results sometimes.  Opening a new project I find error badges on files.
 When I open the files to see the errors, I see no errors in the editor.  The error badge on the file may then disappear
-- only to reappear again later.  After several go arounds of this, the error badge finally seems to stay gone.

This is accompanied by a lot of: "Incorrect error badges detected" and "Going to recompute root" messaging on the
org.netbeans.modules.java.source.tasklist.IncorrectErrorBadges logger in the IDE log.

I'm thus assuming my implementation is good (as it is very simple) and that there is some overall NB 6.5 stability work
needed in this area.
Comment 15 jessholle 2008-06-28 16:33:54 UTC
Besides the fact that the scan seems to miss the data from the VirtualSourceProvider until I open various sources in the
editor [I wonder if this is something as silly as my extension being "rbInfo" and thus after "java"...], I note another
gap in expected functionality here.

I would expect that "Go to Source" or "Go to Declaration" would take me to the appropriate (non-Java) source file here.
 Clearly VirtualSourceProvider gives sufficient information to make this possible.  Specifically it provides a mapping
between the non-Java source files and classes, though it does not provide enough information to allow navigation to the
appropriate line of the non-Java source file.  Despite this, the editor says "Cannot perform Go to Declaration here". 
This should be rectified (though this is not as critical as getting the source scan to initially treat references to
such classes properly).
Comment 16 Tomas Zezula 2008-06-30 10:17:52 UTC
Goto source - it's known issue, will be fixed in the NB 6.0.
The errors seems strange to me, there is no similar report from groovy or scala. I am adding Martin, the groovy plugin developer, to the cc list.
Can you provide more info about the errors?
Are they in files referencing the virtual sources or in virtual sources themselves?
Thanks
Comment 17 jessholle 2008-07-05 22:09:00 UTC
After applying a VirtualSourceProvider implementation, the errors are marks in the trees, not in the editor and are
against files which reference the virtual sources.
Comment 18 jessholle 2008-07-06 17:12:16 UTC
If the same issue (error marks in the tree views) is not seen in Groovy or Scala use cases, I have one guess as to the
cause.  References to my virtual classes are almost entirely of 2 types, xxx.class or xxx.SOME_CONSTANT -- and in almost
all cases these reference types exist together.  I'm therefore guessing that the xxx.class references are failing given
that they're part of the static class initialization for the referring class.  Just a guess, though.

I have added enough logging to my VirtualSourceProvider to allow me to dump my virtual sources and ensure that they
properly compile, etc, so that's not the issue.

I assume that all classes provided by VirtualSourceProviders are considered together with regular Java sources so that
references in both directions are supported, right?  I only currently need references from Java sources to my virtual
sources, but I am wondering about the future.  For instance, NetBeans has alarmingly poor support for AspectJ, though I
suspect much more than a VirtualSourceProvider is necessary here.
Comment 19 jessholle 2008-07-10 15:57:58 UTC
Note that the behavior does not get any better (or worse) with build 200807080009.
Comment 20 Tomas Zezula 2008-08-18 15:16:23 UTC
Fixed by  VirtualSourceProvider.

Comment 21 jessholle 2008-08-18 15:22:09 UTC
I agree that this is now fixed (at least in build 200808120201).  The extra error marks have gone away, etc.

There is, however, one issue: currently I still have to rebuild my plug-in with each daily build I move to because this
API is internal and I'm not a "friend" of it.

That works fine for my usage, but prevents me from rolling this module out to others for their (easy) consumption.
Comment 22 Tomas Zezula 2008-08-18 15:27:44 UTC
I will change the impl version from auto generated to fixed token, it will stay the same until some noncompatible change. Please can you fill such an issue on 
java/source.
Thanks.
Comment 23 jessholle 2008-08-18 15:34:59 UTC
That would be great!  I'll file an issue along these lines.