This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
I have two J2SE projects, A and B. A depends upon B's output jar, as specified in A's Libraries panel in its project properties dialog. A full clean and build of A works just fine -- it cleans A and B, builds B, and then builds A with no errors. However, NetBeans editors show that *some* packages and classes present in B's output jar cannot be found. After some experimentation, I've determined that those packages and classes that cannot be found are those that are being injected into B's jar (as part of B's build process, in a -post-jar ant target hook) from sources other than the source code associated with B (some of those classes come from other jars, and we need to merge them into B's output jar; other classes are generated by some code-generation processes that go straight to bytecode, such as Clojure's gen-class facility, scalac, asm-generated code, etc). So, it appears that the editors in Netbeans don't actually resolve the classfiles present in output jars of dependent projects -- they only resolve classes that have corresponding source files in those dependent projects. If this is really the case, I'd say it's a bug -- there's absolutely no reason (at least that I can come up with) for Netbeans not to use the actual output jar from dependent projects, as opposed to speculating on what will be in those output jars based on the Java source files contained in those projects. The only workaround I know of is to manually add references to interim jars we could build that contain those classfiles that are injected into project B's output jar. This strongly subverts the notion of a project-level dependency, and breaks the encapsulation between projects. I've attached a sample project that exemplifies the problem. As described above, project A is dependent upon the nested project B; the latter's build process injects some classes into project B's output jar (in a real-world project, this could be from any of a dozen different entirely legitimate sources). The com.foo.app.Main class in project A refers to those injected classes, which the Netbeans editor flags as errors; however, the build of project A completes successfully, and running that main class works just fine: [catapult:Development/ProjectA] chas% java -cp dist/ProjectA.jar:ProjectB/dist/ProjectB.jar com.foo.app.Main org.apache.commons.logging.impl.LogFactoryImpl@c672d0 [catapult:Development/ProjectA] chas% Interestingly, a run configuration to do the same fails with a compiler error, so it looks like this limited visibility into what project B is actually producing goes further than just the editor: run: Exception in thread "main" java.lang.ExceptionInInitializerError Caused by: java.lang.RuntimeException: Uncompilable source code - package org.apache.commons.logging does not exist at com.foo.app.Main.<clinit>(Main.java:9) Java Result: 1
Created attachment 73624 [details] sample project, with nested dependency
The inter project dependency is always source to source, there are several reasons for this. One of this is that the editor has to be independent on the resulting jar file as it may be cleaned or not even compiled. You can solve the problem by copying the output jar out of the project and add it as jar file.
I understand that in the situation where the output jar file doesn't exist, the editor needs to go against the source files present in the project dependency. However, if the jar file *does* exist, shouldn't editors rely upon that jar, rather than the source code? That *seems* like an easy win-win sort of approach. Is there a downside to *falling back* on known source code, rather than referring to it exclusively? Especially as other languages flourish that target the JVM by emitting classfiles, it's very counterproductive to assume (mandate?) that a project dependency's jar only contains classfiles that map to known Java-language source files. Yes, I can work around this problem by copying the dependency's jar outside of the project directory and linking to that upstream. That workaround is sub-par, if only because then I need to manually set up the chained build.
>However, if the jar file *does* exist, shouldn't editors rely upon that jar, >rather than the source code? That *seems* like an easy win-win sort of approach. Is there a downside to *falling back* >on known source code, rather than referring to it exclusively? No, it's much worse. You modify something in project A and you don't see it in project B until you rebuild the project A, this is unacceptable behavior. >Especially as other languages flourish that target the JVM by emitting classfiles, it's very counterproductive to assume >(mandate?) that a project dependency's jar only contains classfiles that map to known Java-language source files. As far as I understand you use pure j2seproject which produces class files from java sources. If not the language should provide the needed IDE hooks as for example groovy does.
>>However, if the jar file *does* exist, shouldn't editors rely upon that jar, >>rather than the source code? That *seems* like an easy win-win sort of approach. Is there a downside to *falling back* >>on known source code, rather than referring to it exclusively? >No, it's much worse. You modify something in project A and you don't see it in project B until you rebuild the project A, >this is unacceptable behavior. My apologies, I misworded my proposal. I am not proposing referring to the output jar file exclusively or primarily; rather, refer to the Java source code first, and then fall back on the output jar file (if it exists) if the class being searched for is not found in the Java source code. >>Especially as other languages flourish that target the JVM by emitting classfiles, it's very counterproductive to assume >>(mandate?) that a project dependency's jar only contains classfiles that map to known Java-language source files. >As far as I understand you use pure j2seproject which produces class files from java sources. If not the language >should provide the needed IDE hooks as for example groovy does. There are dozens of languages that target the JVM, and most of their communities simply do not have the resources or wherewithal to implement full-fledged IDE support. Further, it is very common for users of these other JVM languages to mix Java sources with source files of those other languages, as they can interoperate very readily in most cases. In the base case where the j2se project's jar contains only classfiles compiled from Java source files, the change I'm proposing has no impact; however, for those who are using a non-Java JVM language, the proposed change would provide a great deal of benefit (i.e. bringing functionality up to the basic level of all-Java projects).
>refer to the Java source code first, and then fall back on the output jar file Doesn't work either, imagine you have deleted a class in sources without rebuilding the project A => the class, even it doesn't exist any more, is visible to project B.
>>refer to the Java source code first, and then fall back on the output jar file >Doesn't work either, imagine you have deleted a class in sources without rebuilding the project A => the class, even it >doesn't exist any more, is visible to project B. Seems like an edge case, especially since (a) removing classes is a relatively rare event and (b) the "shadow" classfile would be eliminated on the next clean build. Just to clarify: it is expected that any JVM language that generates classfiles must have a NetBeans plugin in order for its classfiles to be usefully recognized by the IDE?
>Seems like an edge case, especially since (a) removing classes is a relatively rare event and (b) the "shadow" classfile >would be eliminated on the next clean build. Deleting a class is an edge case? It's a P1 defect! You force all the IDE features to create invalid code. And there are another P1 related to this change, like rebuild of dependent project on each rebuild of the project A, jar file opening, etc. >Just to clarify: it is expected that any JVM language that generates classfiles must have a NetBeans plugin in order for >its classfiles to be usefully recognized by the IDE? Or it can create an ordinary library jar file.
BTW you don't need to add an IDE language support just implement VirtualSourceProvider (2 methods) is enough.
I'll look at VirtualSourceProvider, but working on IDE features one language at a time is a lot more work than adding a few subant calls and a copy task to some ant builds. Isn't this a broader issue? There's lots of processes/tools that generate classfiles that are entirely unrelated to programming languages; consider parser generators, tools like asm, MDA tools, etc. It seems that gracefully handling such generated resources in a general way would be extremely beneficial. Perhaps my suggestions are off the mark (obviously, mine being a user's perspective), but maybe there's another approach that would satisfy all use cases. Two auxiliary issues: (a) when I drop the project dependency, and set a jar dependency on projectname/dist/projectname.jar, the generated classfiles are still not picked up. If I move projectname.jar to projectname/projectname.jar, and set the jar dependency to that path, all's well. It seems that the IDE detects that projectname/dist/projectname.jar is the output of a j2se project and decorates editors based on that project's source files, even though the dependency is explicitly set on the jar, and not on the project. (b) given the way that project dependencies work, perhaps it would make sense to not indicate in the Libraries panel of a project's properties dialog that a project's output jar is going to be used to resolve classes, when it's that project's source files that will be used again. That UI hint is very misleading.
>It seems that the IDE detects that projectname/dist/projectname.jar is the output >of a j2se project and decorates editors based on that project's source files, even though the dependency is explicitly >set on the jar, and not on the project. Unfortunately yes, the project system uses Queries, in this case the SourceForBinaryQuery which provides sources for the binary (jar). The query works with URL, the only information how it can find out the answer is a match with the binary jar file. This causes a fact that even if you add the project output as a jar file it's handled as a project. Can you attach more details which tool do you use to generate these class files? Maybe we can find some simple solution of your problem.
We're using Clojure pretty heavily (http://clojure.org), which uses ASM (http://asm.objectweb.org/) to generate bytecode (usually at runtime, but optionally during a build process if one needs classfiles that can be statically linked). Perhaps a new directory in projectname/build could be established? We could configure our clojure compilation process to drop all output there (and other tools could be configured similarly); that directory could then be monitored by the IDE for editor decoration, and folded into the output jar file for normal ant builds.
Yes, this will work. I will download the closures and try.
A potential course of action was suggested after the issue was marked as WONTFIX, which tzezula indicated had promise. However, the issue was never reopened. This is probably obvious to those who are familiar with the details, but the described behaviour is also present in NB RCP modules.
The virtual source provider works fine used by groovy and others. Someone needs to create the implementation for clojures, which does not belong to the j2seproject. Anyway you can try to use the -J-DCacheClassPath.keepJars=true command line option to force NB to use the jar files in addition to source files but with all the problems (delete file...) I've described above.
-J-DCacheClassPath.keepJars=true appears to do exactly what we want, thank you! (I was completely unaware of it.) Having to do a rebuild after deleting a file is a small price to pay to get the UI to report errors properly the other 99% of the time, IMO. Clojure is simply our particular use-case -- this issue will affect anyone generating classfiles from non-java sourcecode (such as when using certain parser generators, ASM in general, etc). It seems that supporting an optional subdirectory in build within projects where generated classfiles can be deposited is a good general (and default) solution, regardless of the tooling being used to generate that code. Do you consider that to be a worthwhile enhancement possibility?
Yes, it's a valid enhancement. We already have such generated folders for sources. Adding similar folders for classes sounds as a valid requirement.