will do more in depth evaluation, but most likely an issue with jar maven plugin, all the parameters you mention end up in manifest, the rest of the jar stays the same

it's actually also happening with ant based projects. the cause is the following code in CreateModuleXML.java ant task that skips the regeneration when the file exists.

File xml = new File(xmldir, codenamebase.replace('.', '-') + ".xml");
if (xml.exists()) {
                    // XXX should check that the old file actually matches what we would have written
                    log("Will not overwrite " + xml + "; skipping...", Project.MSG_VERBOSE);
                    return;
}


reassigning to apisupport harness

Is it so problematic to do clean build? There are some changes that require that and I see no problem invoking clean build on such circumstances. If clean build helps, then I don't see a reason to spend time on fixing issues of this kind (we can't fix them all anyway).

> I'm pretty sure this also affects
> new top components or anything which will register anything

If a creation of new top component required clean build, then it would be a different story. That happens frequently. But switching eager on/off is very  infrequent change.

@ServiceProvider registration requires a clean as well. The build time difference is around a minute btw as I have a big module.

It could definitely be more modularized and that would make this better, but I'm on a team, and that as a requirement is low on the pole. It really is a pain in the butt to do a clean. Over the course of the day if one uses lookup to really decouple things, then various providers are added often, and the pain is a little obscured.

I'm refactoring a good bit of code, and this comes up a lot. Once I get the providers in then I can better break this up, but for now this is eating a lot of time.

Imagine one is working on various files and adds a single provider reg and then forgets they haven't cleaned that go around...they run it...get an unexpected state...go hmm...ah...so they not only rebuild, but forget something so minute, ran, and had to wait not only on the build but the application to start and close as well.

I already use the command line to build only the modules I'm working on plus use nbm:cluster-app because of the difference in my day and time it makes. I reopened this because of this. If I need to break things down into separate problems I suggest we change the summary of this to reflect an overarching issue and do the initial gathering here and then file separate issues for the specific details after that. That will at least keep the conversation in one place should it be needed for now.

Even for things such as eager however it is still a pain. One forgets if it doesn't affect them over the course of a month or two. Then when you do you try a couple things before you go...oh yeah...needed to clean and build. For new people it is worse because they don't know they need to clean because of such a change. Heck, its not even new people. I have to help team members with such things often, and is one of those things where you just have to be on your toes. 

They spin their wheels and finally call me and I can set them straight, but that is a big loss in man hours because the tool doesn't inform you of that, and I don't think that is clearly documented in pom.xml comments generated by the IDE, nor in the project properties dialog any where, or any where else I have seen. It impacts the quality of the IDE from the users experience in my opinion.

This seems even worse when one uses the message annotations. I find if I do not do a clean build on a module using @Messages, I will get all kinds of errors about the resource not being able to be found. That means every time I need to rerun the application, then I can't do a simple mvn install of the module, but am forced to do mvn clean install.

The bit about @Messages happens on Windows and Linux both.

OK, if you want to move this issue forward, I need you to set an Ant based project up, attach it and I am ready to modify a single file and observe that something is broken. If there are multiple things that can go wrong, we can go piece by piece or you can create separate bugs for each case.

Incomplete until I will get the test case.

Jarda, I'm reopening. I'm going to attach a couple projects. One is Ant based, and the other is Maven. This maybe needs assigned back to Milos. I don't know. These projects I'm going to attach do not get into the moduleType, but instead deal with @ServiceProvider. In the Ant project, if you add a new "FakeProvider" class, and you will see, I suggest you add one called JardaFakeProvider, and then build and run the app, then you will see that show fine in the provided top component.

In the Maven project if you add a new FakeProvider to the netbeans-221781-maven-main project, lets call it JardaFakeProvider or MilosFakeProvider and do something similar to what I have done there, such as return "Jarda" or "Milos" from the getName method, right click and build netbeans-221781-maven-main, then right click and build netbeans-221781-maven-parent, then right click and run netbeans-221781-maven-app, you will not see the new FakeProvider(s) show up in the provided top component.

So, it seems other than the moduleType issue, everything else is specific to Maven based NB RCP projects. Whether a module is eager or not etc...sure perhaps one can live with having to clean to change that, and of course it is a pain to figure out since nothing really tells you...hey if you change this clean and build, but these other things make it nearly unbearable for a good sized app where things are being added and refactored all the time. The issues with @Messages, @ServiceProvider, as well as new top components all appear to be mvn RCP related and require "mvn clean install" before they will show up.

You can try to add a new top component to netbeans-221781-maven-main and can see what I'm talking about. I'm using NetBeans 7.2.1 and JDK 7 along with Maven 3.0.4.

I will try to put together use cases which demonstrate how the @Messages annotations causes issues in mvn based NB RCP projects as well.

Created attachment 130341 [details]
Ant based project showing that new classes annotated with @ServiceProvider work with just a build versus a clean and build

Created attachment 130342 [details]
Maven project showing that new classes added with @ServiceProvider require a clean install or a clean and build before they show up or have the annotations processed at build time

I did a clean before I attached the projects, but I accidentally left the private folder in the nbproject folder of the Ant project...just a heads up if it has any impact at all. Thanks.

I went ahead and reassigned this to Maven. moduleType perhaps can be lived with not updating, but the rest of the issues here are hard to live with considering full rebuilds can be time consuming overall especially for service provider and top component registrations. I still need to figure out better use cases for the @Messages problems. I did notice though that uses of @Messages with a Maven based project results in quite a few issues if one does an "mvn install" versus an "mvn clean install", and this seems to not affect the Ant based projects.

do you have compile on save on or off? that's the only possible place where netbeans IDE support could be involved, otherwise it's either maven itself (via some of the plugins) and/or the annotation processor codebase.

I've tried with CoS off.
1. on the main project level, adding a @provider annotation gets correctly processed (META-INF/services/FakeProvider file gets updated. However removing the annotation does not update it. That seems correct to me. The update appears to get transfered correctly from target/classes to jar to nbm (that's something you can verify on your side as well)
2. on the app project level, we compare the dependency's nbm file timestamp  - File.lastModified() -against the .lastModified timestamp file in the cluster folder. If the nbm file is newer it's content gets copied into the application's clusters. you can verify that by enabling debug message in maven build and looking for something along these lines:
Copying org.netbeans.modules.tests.issue-221781-maven:netbeans-221781-maven-main:nbm-file:1.0-SNAPSHOT to cluster netbeans221781maven
The .lastModified files are touched after all the nbm files are updated.
This also seemed to work for me fine and the nbm file appeared to get properly copied over.
3. the rest is runtime stuff, maybe some caches are in the way. I didn't try that one. You can check this by deleting the user directory in application/target/ after building the app and before running it.

I should have some time in the next couple days to try some things out Milos. Thanks.

Milos, just curious, since you mentioned caches and not trying that one, had you built once before adding a new provider? I noticed you said removing the annotation didn't remove it which would be a rebuild. If you add a provider, mvn install, then add another provider, mvn install, and then mvn nbm:cluster-app, and then run what do you get? Which maven version are you using?

(In reply to comment #15)
> Milos, just curious, since you mentioned caches and not trying that one, had
> you built once before adding a new provider? 

yes.
mvn install with one provider. check output directory. Add another provider, mvn install, check output directory again.

> I noticed you said removing the
> annotation didn't remove it which would be a rebuild. 

Yes, only mvn clean install removes the generated content. 

> If you add a provider,
> mvn install, then add another provider, mvn install, and then mvn
> nbm:cluster-app, and then run what do you get? Which maven version are you
> using?

3.0.4 I guess. 

important is also my note about CoS, always make sure that the maven build output contains note about compiling some files, if none are compiled, maybe CoS compiled it for you without processing the annotation.. (just guessing here)

OK, so I had done a clean. I came in. I did a clean install on the main parent POM. I added a provider. mvn install on the module, mvn install on application, and then ran. It showed up. Next, I added another provider, saved, mvn install, checked under META-INF/services the file org.netbeans.modules.tests.issue221781maven.FakeProvider and the new providers class name is not there. For the module, I checked compile on save, and it was set to "for application execution only". I have not disabled it, and will try the entire process over.

"I have not disabled it"...should have read..."I have NOW disabled it" ... and will try over.

I didn't start this time with cleaning, but I am still not getting any new providers placed into the META-INF/services file. I am going to try by cleaning with compile on save off, and then will see what happens. I will try this exact same project on a completely different machine later tonight. That machine will have Windows 7 and JDK 6 I think. This setup is Ubuntu 12.04 and JDK 7. I'm using mvn 3.0.4.

OK, so it seems it is indeed related to compile on save. The thing is...that is the default apparently. First, I did have to do a clean to get things to start working correctly after I turned off compile on save. I only had to turn it off for the module; at least for the changes to appear in the META-INF/services file. Next, I went back and turned compile on save back on. I did a clean, I added a class, build, shows up in the services file. I added another, build, and it did not show up in the services file. Too, I did check the output window, and it had a message there was nothing to compile. It seems the annotation processing of the compile on save feature is not working correctly.

I have not tried to see if the other issues are related to this or not, but I will. Specifically one of them that gives me the most trouble other than service provider is the messages annotation. If I don't perform a clean and build I will get all kinds of sporadic errors at runtime. Some value won't be in the Bundle etc.

Milos, can you confirm on your side whether or not compile on save has such an effect for you? I feel like for Maven projects that should not be the default if it causes such problems. I have not yet checked to see what happened in the Ant based project with regard to compile on save, but will check it now; at least for service provider.

OK. It has been a while since I have worked with Ant based projects in the IDE...at least RCP ones, and there is not option under the project properties to turn compile on save off or on. I suppose that would explain why it didn't have the same problem.

OK, so I tested my theory on needing to do a clean and build after turning off compile on save to have the effect of actually not compiling on save or compiling on save, and that seems to be the case indeed. I changed the property in my project properties; in this case I went from NOT compiling on save to compiling on save. I changed a source file. I saved. I then did a build. I see "compiling 1 file"... I made more changes, saved, same thing. Next, I did a clean and build. I then made more changes, and I see there is "nothing to compile". So, that seems odd too.

There seem to be multiple issues listed here in this issue, and apparently they are all related to compile on save.

I just tested messages. I added a few providers I had already created. I turned compile on save off, and did a clean and build (mvn clean install). Next, I changed a couple to use messages like:
package org.netbeans.modules.tests.issue221781maven;

import org.openide.util.NbBundle.Messages;
import org.openide.util.lookup.ServiceProvider;

/**
 *
 * @author wade
 */
@ServiceProvider(service=FakeProvider.class)
@Messages({"BooProvider.name=Boo"})
public class BooProvider implements FakeProvider {

    @Override
    public String getName() {
        return Bundle.BooProvider_name();
    }
    
}

I did a build as I changed each one, and I ran the application. Everything ran fine. I turned compile on save BACK ON, and then I made some changes, saved, built, etc. I then changed the above example to:
package org.netbeans.modules.tests.issue221781maven;

import org.openide.util.NbBundle.Messages;
import org.openide.util.lookup.ServiceProvider;

/**
 *
 * @author wade
 */
@ServiceProvider(service=FakeProvider.class)
@Messages({"BooProvider.name_value=Boo"})
public class BooProvider implements FakeProvider {

    @Override
    public String getName() {
        return Bundle.BooProvider_name_value();
    }
    
}

Then saved and built. At runtime it produced the following trace:
java.util.MissingResourceException: Can't find resource for bundle org.openide.util.NbBundle$PBundle, key CTL_FakeProviderDisplayAction
	at java.util.ResourceBundle.getObject(ResourceBundle.java:393)
[catch] at org.netbeans.core.startup.layers.BinaryFS$AttrImpl.getValue(BinaryFS.java:719)
...

Which is a resource from the top component FakeProviderDisplayTopComponent which is in the project:
@Messages({
    "CTL_FakeProviderDisplayAction=FakeProviderDisplay",
    "CTL_FakeProviderDisplayTopComponent=FakeProviderDisplay Window",
    "HINT_FakeProviderDisplayTopComponent=This is a FakeProviderDisplay window"
})
public final class FakeProviderDisplayTopComponent extends TopComponent

So, it seems with compile on save turned on, the @Messages annotation processing gets messed up, and I assume the Bundle.properties file is being stepped on. Next, I went back, turned compile on save OFF again, and sure enough I have not been able to reproduce the problem again. I have been changing values in @Messages annotations for a while where, and I build the module, and then run the application after each change, and it all seems to be working fine.

So, definitely, there are issues with compile on save and annotation processing.

I went ahead and changed the summary to reflect the biggest problem and nature of the issue.

compile on save is off by default in 7.3, with exception of web related packagings that were explicitly requested by the j2ee team.

*** Bug 225285 has been marked as a duplicate of this bug. ***

jlahoda: is there a way to enable Compile on save infrastructure to include the generated non-class files in the target/classes folder of the project? I don't see any other way to fix the issue. Maybe apart from warning users someplace in ui that after changing annotations of any kind, clean build is required.

How are the classes themselves included? Is external logic such as a compiler putting them there for you? What about file system listeners if so? What would not work with that approach? If not, at least for NB annotations some kind of an annotation context could be used that could detail created files that need to be copied over. Wouldn't help necessarily with 3rd party annotations though, so not the best solution. I mean it could be an open API for those using NB, but still doesn't solve whole thing. It seems like files just need to be monitored some how. 

As far as putting them in target/classes goes isn't there an expectation that before releasing one should clean and build with compile on save turned on? What happens on compile on save when new classes are generated by annotations? How are those being included?  Are they causing the same type issue, or is there special logic for them already? I am trying to understand the difference between the way they are treated versus say a generated .properties file.

I might have found a "solution".

http://maven.apache.org/plugins/maven-compiler-plugin/compile-mojo.html#staleMillis

running a non-clean build with -DlastModGranularityMs=X where X is a high enough NEGATIVE number will force a re-compile of the java files matching the range, typically the compile on save ones (or all)

The question is how to determine the best value for X.

(In reply to comment #29)
> I might have found a "solution".

well, the solution only applies to running Build project instead of Clean Build project. Unless we find a way to determine when to suppress skipping lifecycle phases in Run/Debug/Profile. So far I have no idea how to figure that an annotation processor processes a non-java file.

> Is it so problematic to do clean build?

As someone who has lost a few days of my life to compile-on-save problems (if it is on, you never can be sure what you are building against, why tests that cannot possibly fail anymore still are failing, so you change other things...), the answer to this question is YES.

So...

Detect annotation changes underneath a Maven project, and set a flag to not use compile-on-save but run a real clean-and-build the next time some action might require one.

Or fix it right - I don't care.

> So far I have no idea how to figure that an annotation processor processes a non-java file.

Ugly thought:  Find a way to hook into javac so that you can be notified when annotation processors run at all, and a way to hook into the filesystems API to detect what files are read.  Hmm, I wonder if you could do the first hook with an annotation processor which you inject into the classpath which consumes all annotations... then...do you need to know what was read or what was written?

(In reply to comment #31)
> > Is it so problematic to do clean build?
> 
> As someone who has lost a few days of my life to compile-on-save problems (if
> it is on, you never can be sure what you are building against, why tests that
> cannot possibly fail anymore still are failing, so you change other things...),
> the answer to this question is YES.
> 
> So...
> 
> Detect annotation changes underneath a Maven project, and set a flag to not use
> compile-on-save but run a real clean-and-build the next time some action might
> require one.

Please remember that the project where the annotation change occurs is different from the project being run. There might not even be a way to execute a build across both of the projects.

> 
> Or fix it right - I don't care.

The more likely alternate is that I'm going to disable CoS on nbm+application projects again.

The bug is still existing ;-)

I get time by time a

java.util.MissingResourceException: Can't find resource for bundle org.openide.util.NbBundle$PBundle, key Plugins
	at java.util.ResourceBundle.getObject(ResourceBundle.java:450)
	at java.util.ResourceBundle.getString(ResourceBundle.java:407)
	at org.openide.util.NbBundle.getMessage(NbBundle.java:642)

COS is a feature by the IDE and I can choose it for maven projects. If I disable COS, I get warnings on running test cases.

IMHO it should work or completly disabled for maven projects with @Messages.

(In reply to _ tboudreau from comment #31)
> > Is it so problematic to do clean build?
> 
> As someone who has lost a few days of my life to compile-on-save problems
> (if it is on, you never can be sure what you are building against, why tests
> that cannot possibly fail anymore still are failing, so you change other
> things...), the answer to this question is YES.

I second tboudreau's comment, and wadechandler's frustrations as well, especially in connection with @Message annotations. Figuring out the exact situations when a 3-minute "Clean & Build" was necessary was a major stumbling block for me when I started learning the platform. I found it especially deceptive that a _clean_ build of the entire project should be necessary. In my platform app this has the side effect of resetting my development user directory, meaning I need to reconfigure my app the next time I run it etc.

The problem is even bigger for beginners because all the tutorials tend to go through steps that involve service annotations, creating forms in the form designer (which causes internationalization bundle changes), etc. In the long term of a project such changes might be rare, but when you're first setting up and trying to learn what the different settings do, it happens all the time.

(In reply to ebakke from comment #34)
> I found it
> especially deceptive that a _clean_ build of the entire project should be
> necessary. In my platform app this has the side effect of resetting my
> development user directory, meaning I need to reconfigure my app the next
> time I run it etc.

Yes, that's the main reason I don't invoke Clean & build so often. There should be at least an option to clean the project without deleting testuserdir.
Another reason is that it takes quite a few minutes to rebuild the project consisting of almost 100 modules.