Bug 218658 - Generated Sources for JPA metamodel are not recognized in source packages
Generated Sources for JPA metamodel are not recognized in source packages
Status: REOPENED
Product: javaee
Classification: Unclassified
Component: Persistence
8.2
All All
: P3 with 19 votes (vote)
: Dev
Assigned To: Sergey Petrov
issues@javaee
:
: 251130 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-20 07:27 UTC by marcincinik
Modified: 2017-05-12 01:45 UTC (History)
13 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
compilation issues (100.96 KB, image/png)
2012-09-20 07:27 UTC, marcincinik
Details
files *.rapt and *.sig (26.01 KB, application/octet-stream)
2015-10-20 18:59 UTC, FiruzzZ
Details

Note You need to log in before you can comment on or make changes to this bug.
Description marcincinik 2012-09-20 07:27:41 UTC
Created attachment 124617 [details]
compilation issues

Please consult the screenshot first.

I am using JPA generated metamodel which in turn uses metamodel processor from eclipse link (this is standard mechanism used in java based on META-INF/"services").

Metamodel is generated properly during compilation phase and the generated classes show up in "Generated Sources (annotations)" (also properly).

The problem:
----------------
Classed belonging to "Source packages" (normal classes) in general can't see any classes which show up in "Generated sources (annotations)".

The project build is based on maven. Maven build succeeds, however Netbeans shows some imports which cannot be resolved which leads to compilation failures reported by Netbeans. Those imports are coming from the "Generated sources (annotations)".

The problem persists also in dev 201209200001
Comment 1 cheesus 2013-10-13 19:56:56 UTC
experiencing the same problem on 7.4RC1s (same project, several people on different OSses).

strange enough, adding build/generated-sources/ap-source-output to src path and removing it again after one good compile did indeed helped...!?
Comment 2 cheesus 2013-10-14 09:26:16 UTC
The safer way seems to be:

stop netbeans
rm -rf  ${HOME}/.cache/netbeans/7.4rc1/
(or Library/Caches/NetBeans/7.4rc1)
start netbeans
clean build
Comment 3 pierrecaron 2013-10-24 17:23:43 UTC
I also have the same problem.

I cannot find the ${HOME}/.cache/netbeans/7.4rc1/ (or library/Caches/NetBeans/7.4rc1) folder (I'm running on Windows XP).
Comment 4 luca.adamo 2014-01-30 17:56:31 UTC
It is still happening to me, both on version 7.2.1 and version 7.4. The same exact project compiles fine on another workstations with 7.4. I tried to delete the cache and restart with no luck. The only workaround is to manually add the build/generated-sources folder to the src PATH
Comment 5 chriswesdorp 2014-06-06 11:22:45 UTC
Running into the same issue working with Product Version: NetBeans IDE 8.0 (Build 201403101706)

Clearing the cache seems to remove the problem for the first start, but the problem returns after changes to the entities.
Comment 6 ender01 2014-08-25 20:04:59 UTC
I am seeing the same behavior with generated classes from the cxf-codegen-plugin. 
Maven still builds/compiles everything fine but the editor complains about the classes not being found.
Comment 7 fistons 2014-12-30 12:39:18 UTC
Same problem here, with Netbeans 8.0.2. Cleaning the cache seems to work with Linux but not with Windows 7
Comment 8 FiruzzZ 2015-02-23 12:23:21 UTC
Bug remains on
Product Version: NetBeans IDE 8.0.2 (Build 201411181905)
I tried changing the lib, switching between eclipselink 241 > 252, didn't work, now the IDE doesn't find more generated meta model than before
Comment 9 FiruzzZ 2015-02-23 12:36:46 UTC
(In reply to FiruzzZ from comment #8)
> Bug remains on
> Product Version: NetBeans IDE 8.0.2 (Build 201411181905)
> I tried changing the lib, switching between eclipselink 241 > 252, didn't
> work, now the IDE doesn't find more generated meta model than before

PD: removing the cache folder /local/netbeans/cache solved the issue, but the IDE threw a few problems notifications on its start up
Comment 10 naskaom 2015-03-03 14:33:03 UTC
Clearing the .cache/netbeans/8.0.2 didn't work for me. I had to add generated metamodel classes to the src path and not remove them. Confused.
Comment 11 RayDeCampo 2015-03-03 14:41:06 UTC
I was experiencing this issue on a Maven project where the JPA implementation was Hibernate.  I solved it by adding hibernate-entitymanager and hibernate-jpa-2.1-api to the pom.xml as provided dependencies.  HTH
Comment 12 naskaom 2015-03-03 18:58:02 UTC
Mine is not a maven project. And I use eclipselink.
Comment 13 Petr Hejl 2015-04-16 07:46:42 UTC
*** Bug 251130 has been marked as a duplicate of this bug. ***
Comment 14 koarl 2015-08-11 08:16:02 UTC
Same here (NB 8.0.2, Maven, JPA metamodel generated by Hibernate). Clearing the cache works (at least on Linux), but this is not an option when working with large projects on a daily basis. 

I'm using NB for over a decade now (each and every workday), but this one is driving me crazy. Please fix it.
Comment 15 Dusan Balek 2015-09-08 13:11:45 UTC
Unfortunately, I cannot reproduce the problem in the current dev build. I have tried to create a simple Maven project with eclipselink and it seems to work as expected. Could you please create a simple test project demonstrating the problem and attach it to the issue? Also, cloud you please check that your source root's cache directory contains sources generated by the eclipselink's annotation processors? (go to var/cache/index directory located in your NB userdir, in Segments file find which segment belongs to your source root and go to the proper segment directory. There should be directory java/14/sources containing all sources generated by the annotation processors). Thanks.
Comment 16 FiruzzZ 2015-09-11 10:59:08 UTC
(In reply to Dusan Balek from comment #15)

java/14 has
  /classes/projectname/test (with a few random testing classes)
  /refs
      (3 segments, .gen, .cfs)
  /sources (empty)
  /whitelist (empty)
  attributes.properties
  fqn2files.properties
  checksums.properties

Nothing related to "generated model" or any classes
Comment 17 rogerio-163 2015-10-07 12:55:29 UTC
same issue with the tutorial of code generation using Annotations Processor (https://netbeans.org/kb/docs/java/annotations-custom.html)

Any solution? I need it for a college project, please, help me.

By adding auto-generated folder in the library and then remove, the error stop, but if i save any changes in any other file, or perform a rebuild, the issue back.
Comment 18 Abrham 2015-10-13 15:34:04 UTC
Also having this problem. I can provide a little more background on what is happening in my situation.

Using EclipseLink 2.6 and Netbeans 8.0.2

The problem for me only happens when I have extended a @MappedSuperclass on my original @Entity.

When using the MetaModel Netbeans will only not recognize the @MappedSuperclass variables.

However, I can sometimes empty the cache, or clean/build to re-generate the MetaModel which sometimes resolves the problem but not all the time.
Comment 19 Dusan Balek 2015-10-20 09:38:52 UTC
To FiruzzZ: As can be seen from your caches, there were no sources generated by the annotation processors while indexing your project. Please check that annotation processing is enabled in editor for your project.

To rogerio-163: Unfortunately, I cannot reproduce the problem. I walked through the tutorial (step-by-step) and everything worked as described.

To all: Could you please create a simple test project demonstrating the problem and attach it to the issue? Thanks.
Comment 20 FiruzzZ 2015-10-20 18:56:10 UTC
(In reply to Dusan Balek from comment #15)
> ..Also, cloud you please
> check that your source root's cache directory contains sources generated by
> the eclipselink's annotation processors? (go to var/cache/index directory
> located in your NB userdir, in Segments file find which segment belongs to
> your source root and go to the proper segment directory. There should be
> directory java/14/sources containing all sources generated by the annotation
> processors). Thanks.

I double checked and result that I wasn't looking at right folder, I found the one for the project and there just a few (6 files) generated from Entities, I looked around and I found in the path:
../classes/jgestion/entity/
*.rapt (<--- those files got their proper metamodel generated)
but those with the extension *.sig (doesn't, 84 files)
I will attack those in case those could be helpful for something.
Comment 21 FiruzzZ 2015-10-20 18:59:00 UTC
Created attachment 156855 [details]
files *.rapt and *.sig
Comment 22 silva_fabio 2015-11-24 18:28:48 UTC
I had this problem on the past and didn't remember how to solve that.

After two days headache i found it!

In the persistence.xml file i removed the instruction below: 
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>

Maybe it works on old netbeans versions. Actually my persistence.xml file didn't had this line on previous versions of my project, it was added when editing persistence.xml in design mode when i changed "Persistence Provider" value and back to the same.

Someone else can confirm that?
Comment 23 silva_fabio 2015-11-25 16:58:04 UTC
Still missing something in my solution :-(

It works for my office computer, but to the home no.

I uninstalled and installed the same versions (7.3 and 8.1) at home who worked in the office and used the same project that was using at the office. I have the same project using the same versions of netbeans working in the office but not at home :-(
Comment 24 silva_fabio 2016-08-03 23:00:45 UTC
*** Solution that worked for me ***

1. Click right on the project and select "Properties";
2. On the left side select "Libraries";
3. On the right side remove all occurrences from "EclipseLink (JPA 2.1)" and "EclipseLink-ModelGen (JPA 2.1)" that you find in "Compile" and "processor" tabs and click "Ok";
4. Open your file "persistence.xml" (this is the most important part, without it will not work), make any changes (even a space at the end of the file, because we want Netbeans know that there was a change in this file) and save.
5. Repeat steps 1 and 2;
6. On the right side in tab "Processor" includes first the library "EclipseLink (JPA 2.1)" and then "EclipseLink-ModelGen (JPA 2.1)" and click "OK";
7. Wait for the Netbeans do indexing (follow the footer message) and everything will work.


*** Explanation ***

In the cache properly generated for a project that uses JPA metamodel we have:
- ${DEFAULT_CACHEDIR_ROOT}/8.1/index/s###/java/15/classes -> *.sig, *_.sig e *.rapt
- ${DEFAULT_CACHEDIR_ROOT}/8.1/index/s###/java/15/sources -> *_.java

In the defective cache of a project that uses JPA metamodel we have:
- ${DEFAULT_CACHEDIR_ROOT}/8.1/index/s###/java/15/classes -> *.sig
- ${DEFAULT_CACHEDIR_ROOT}/8.1/index/s###/java/15/sources -> empty

The cache is only generated properly when Netbeans identifies any change in persistence.xml. This occurs when the file ${DEFAULT_USERDIR_ROOT}/8.1/var/filehistory/##/some-folder/some-DATA-file-thats-represents-persistence.xml-file is updated / changed. At this point the Netbeans is the proper generation of cache files eliminating the problem.


*** One of the ways I found to do the problem occurs ***

1. Use a new installation of Netbeans or delete the user and cache directories;
2. Open a cleaned project(without the build and dist directories) that uses JPA metamodel in the code;
3. Wait for index (follow the footer) and see the problem occurs.


*** Tested Versions ***

- 7.1;
- 7.3.1;
- 8.1;
- dev-20160729-e0576ad941f3


Please let me know if it worked for someone else ;-)
Comment 25 smil 2016-08-17 15:06:13 UTC
Hi,

I have the same problem with our own annotation processor. So I think this has nothing to do with JPA meta-model. 

I believe that the "testcase" Fabio provided is correct. I remember starting the IDE without compile just before the problem happens, so it must be the trigger.

In our annotation processor case, the source code generated rarely change, so maybe this is why IDE does not pick them up for recompile.

ps.
Why NB reads this cache on startup anyhow? Cleaning the whole directory (except maven caches) makes my IDE start faster and more correct, without any apparent loss of historical data etc...
Comment 26 puce 2016-10-20 20:33:55 UTC
Similar issue here with a custom annotation + annotation processor (NetBeans 8.1):

@SomeAnnotation(someFile = "META-INF/someDir/somefile.xml", someClass = SomeType.class)

The error message is that
<user-home>/.cache/netbeans/8.1/index/s21/java/15/classes/META-INF/someDir/somefile.xml
could not be found.

The file is available however in the Maven project at:
src/main/resources/META-INF/someDir/somefile.xml
Comment 27 sviperll 2016-12-21 13:11:26 UTC
I've got this behaviour in 8.2. The workaround that worked for me was using any other directory for generated sources, but not generated-sources/annotations. I've set up javac to use /target/generated-sources/javac instead of /target/generated-sources/annotations in maven's pom.xml and Netbeans has happily indexed all generated sources. My pom.xml is like this

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <!-- ... -->
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>
    <!-- ... -->
    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <configuration>
                        <showDeprecation>true</showDeprecation>
                        <showWarnings>true</showWarnings>
                        <compilerArgs>
                            <arg>-s</arg>
                            <arg>${project.build.directory}/generated-sources/javac</arg>
                            <arg>-Xlint:all</arg>
                        </compilerArgs>
                    </configuration>
                </plugin>
                <!-- ... -->
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.4</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <configuration>
                            <tasks>
                                <mkdir dir="./target/generated-sources/javac"/>
                            </tasks>
                        </configuration>
                        <goals>
                            <goal>run</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>            
            <!-- ... -->
        </plugins>
    </build>
    <!-- ... -->
</project>
Comment 28 netbeans.89423 2016-12-22 09:21:48 UTC
I have the same problem.

I'm trying

            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.0</version>
                <configuration>
                    <generatedSourcesDirectory>${project.build.directory}/generated-sources/anno</generatedSourcesDirectory>
                </configuration>
            </plugin>

for now as a nicer way of doing what the previous commenter suggested.

Could this be a race between Netbeans' compile-on-save and the maven compilation or sth like that?
Comment 29 netbeans.89423 2016-12-22 12:41:39 UTC
Somehow CoS can mess up the netbeans cache for generated sources when one triggers it in a clean state (ie after mvn clean).

Here is how to reproduce it:

https://github.com/jjYBdx4IL/bug-testcases/tree/master/netbeans218658
Comment 30 kwrobel 2017-02-17 22:10:31 UTC
I'd like to throw my 2c into the mix:

First things first, this is an Editor problem, not a compile-time problem. Second, this is not a Maven issue, but also non-Maven.
Third: it is annoying, annoying, annoying that this does not work (yet again) when using something other than standard configurations, such as GlassFish.

Here's how anyone can replicate the issue:

1. Download, install and configure Apache TomEE 7.0.2 Web Profile
2. Add Apache TomEE 7.0.2 Web Profile to NetBeans as a Server
3. Create a new Web Application targeting Apache TomEE server. (Java EE 7)
4. Start the Netbeans, built-in JavaDB Server (a.k.a. Derby)
5. Run the Entities from Database wizard. Create a new Data Source (e.g. jdbc/sampleds) and point to the JDBC connection that points to SAMPLE on Derby. Select all entities, provide a package name (e.g. entity) and use default settings for everything else. Hit Finish.

... at this point, if this was a Web Application targeting Glassfish or Wildfly, you'd be able to Clean & Build, then see a "Generated Sources" node appear in grey containing all the Metamodel classes for the entities. That does not happen with TomEE and so we need to set up the Annotation Processor properly and compiler options.

6. Go to Project Properties, Libraries and click on Processor tab
7.1. Remove the erroneously added EclipseLink (JPA 2.1) library from the Processor-Path. (Why is *this* happening?)
7.2. Click on Add JARs... navigate to your TomEE installation directory, then into "lib" directory and add the openjpa-2.4.1.jar (if you are on 7.0.2 Web Profile as instructed above)
7.3. Click on Add JARs... navigate to your TomEE installation directory, then into "lib" directory and add the javaee-api-7.0-1.jar (if you are on 7.0.2 Web Profile as instructed above)
8. Click on "Compiling" node under "Build". Make sure the "Enable Annotation Processing" and "Enable Annotation Processing in Editor" are both checked (default). Inside the "Additional Compiler Options:" add the following String (without double-quotes): "-Aopenjpa.metamodel=true"
9. Click "Ok" button to apply changes to the project.
10. (Optional) Fix project.properties file as follows: Click on "Files" tab and navigate to nbproject. Open the "project.properties" file. Change "annotation.processing.processor.options=-Aeclipselink.canonicalmodel.use_static_factory=false" to "annotation.processing.processor.options=". Or you can also try to add the -Aopenjpa.metamodel=true flag as shown above, but it won't have any impact.
11. Right-click on project and choose "Clean and build..."

... at this point, we can see "Generated Sources (ap-source-output)" appear. Now on to the actual issue being discussed here in the thread, which is NetBeans Editor not seeing those generated sources.

12. Run the New Session Beans from Entity Classes wizard. Add all entities. Put them in a package called "facade". Click Finish.
13. Open the facade.CustomerFacade.java file
14. Add the following method code to the source:

    public List<Customer> findCustomersWithoutPurchaseOrders() {
        CriteriaBuilder cb = em.getCriteriaBuilder();
        CriteriaQuery<Customer> cq = cb.createQuery(Customer.class);
        Root<Customer> fromCustomer = cq.from(Customer.class);
        cq.select(fromCustomer);
        
        cq.where(cb.isEmpty(fromCustomer.get(Customer_.purchaseOrderCollection)));
        TypedQuery<Customer> qry = em.createQuery(cq);
        
        return qry.getResultList();
    }

15. Fix imports. you will notice immediately that "Customer_" has a "Cannot be resolved" red warning. Click ok.

... at this point you will notice that the Editor throws up an error on the line with "cq.where". It is because the editor cannot find a symbol for "Customer_" as stated above. "Customer_" is a Metamodel class for the "Customer" entity which you will need in order to create truly type-safe Criteria Queries.

16. Manually add "import entity.Customer_;" on the line right below "import entity.Customer;". (mind the underscore)

... The editor will still flag errors. However, a "Clean and Build" will compile successfully!

If you read and followed my comment this far and you are wondering if this happens with other servers, I can confirm this also happens with WildFly 10.1.0-Final. The only difference there is that I didn't have to fix anything in the project in order for the Metamodel classes to get generated. That's all set up properly. But the code above with the Criteria API Query that relies on the Metamodel Customer_ class, that will still fail in Editor but not compiler.

Contrast that to setting up a project with Glassfish. Everything works inside the editor, including finding the Metamodel classes that get generated into the *same locations*!

If someone could please, please and pretty please look into this? After all, Criteria API was created in order to avoid making JPQL errors since they are simple string representations of a query. And most of NetBeans own entity facade code is utilizing Criteria Queries as well. So it just stands to reason to make this Editor issue go away.

I am willing to spend some time on testing, but I am not a NetBeans developer and would need some guidance on what classes or modules to look at.
Comment 31 kwrobel 2017-02-17 22:37:07 UTC
Btw: I'm using:
Product Version: NetBeans IDE 8.2 (Build 201609300101)
Java: 1.8.0_45; Java HotSpot(TM) 64-Bit Server VM 25.45-b02
Runtime: Java(TM) SE Runtime Environment 1.8.0_45-b14
System: Mac OS X version 10.12.3 running on x86_64; UTF-8; en_US (nb)

Also testing with:
Product Version: NetBeans IDE 8.2 (Build 201609300101)
Updates: NetBeans IDE is updated to version NetBeans 8.2 Patch 1
Java: 1.8.0_111; OpenJDK 64-Bit Server VM 25.111-b14
Runtime: OpenJDK Runtime Environment 1.8.0_111-8u111-b14-2~bpo8+1-b14
System: Linux version 3.16.0-4-amd64 running on amd64; UTF-8; en_US (nb)

Same results.
Comment 32 kwrobel 2017-02-17 22:47:28 UTC
A small addition to my post above (#30):

I claimed above that using a WildFly-targeted web project would throw the same error (and I still stand by that). However, there is a difference with Wildfly over TomEE:

After clearing out the cache for NetBeans 8.2 (from $HOME/Library/Caches/NetBeans/8.2 on macos), I was able to reopen my previous Web Project targeting WildFly and it did *not* throw anymore errors. However, the Apache TomEE targeted Web Project still flags code error in the editor.

This bug has been open since 2012. That's 5 years now. Can we get this ball rolling, please? I am willing to test. Dusan Balek is assigned to this bug. If you are reading this, can you please give me pointers on where I should start looking? I've already compared project.properties files, project.xml, build-impl.xml.... I just cannot find any "Aha!" setting that is obviously missing or misconfigured.

Any advice where I should start looking?

Also, I want to say that the bug is currently miscategorized as a "Compiler" issue. It's not. This is an Editor issue! I can prove it because the compiler just builds the project fine, even if the Editor thinks it's not.
Comment 33 kwrobel 2017-02-20 01:50:31 UTC
After a bit more trial and error, I came to the conclusion (as others have here) that deleting the Netbeans cache files gets rid of the problem, even for the Apache TomEE-targeted Web Application project. So to me that indicates that maybe the issue may not necessarily be with the Editor itself or the compiler, but that the sources don't get re-indexed after compiling. The question is: when does code get indexed? Does it get indexed as you create code in the editor? I mean, that would be obvious as you are creating classes and variables that get referenced by other source code. Then maybe it is part of the build script? After all, the metamodel classes get generated during Build and the editor is not involved at that point. Does anyone from Netbeans have more insights about when a project get indexed? Can it be triggered manually? If so, how? Some help would be much appreciated.
Comment 34 koarl 2017-02-20 11:04:08 UTC
For me, this is the single most annoying NB bug for years now. I am also willing to test, or even dig into the code and have a look myself, but therefore I would appreciate any hints where to look. Thank you!
Comment 35 ledj56 2017-04-12 01:37:31 UTC
I solved this problem when using Maven as a build system by simply reconfiguring the destination directory name of the generated sources. 

https://stackoverflow.com/questions/9847483/why-doesnt-netbeans-ide-see-the-generated-sources/43331054#43331054

Now, I am trying to figure out the same approach with Gradle.
Comment 36 Dusan Balek 2017-05-10 12:03:35 UTC
Thanks to the reproducible test case posted in the comment #30, now I see that the problem is in the Web Application project that does not pass annotation processor options (e.g. "-Aopenjpa.metamodel=true") to the nb-javac instance running in editor.

Fixing "Project Properties -> Build -> Compiling" customizer similarly to the one provided by J2SE Project.

http://hg.netbeans.org/jet-main/rev/eb8b575ce137

Please set the "openjpa.metamodel=true" via Annotation Processor Option instead of using Additional Compiler Options in step 8 of comment 30. With this change, generated sources should be accessible both via editor and via external build.

The remaining question is why Java Persistence support uses EclipseLink as default persistence provider (steps 7.1, 7.2, and 7.3 of the comment 30). Reassigning to J2EE Persistence for evaluation.
Comment 37 netbeans.89423 2017-05-11 14:27:16 UTC
The example https://github.com/jjYBdx4IL/bug-testcases/tree/master/netbeans218658 as listed in comment #29 does not make of any non-basic netbeans features as does the example in comment #30 though. And it still does not work. No web service, just a stripped down example of hibernate persistance metamodel gen.
Comment 38 Quality Engineering 2017-05-12 01:45:15 UTC
Integrated into 'main-silver', will be available in build *201705120001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/eb8b575ce137
User: Dusan Balek <dbalek@netbeans.org>
Log: Issue #218658 - Generated Sources for JPA metamodel are not recognized in source packages - annotation processor options should be passed to nb-javac.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo