A related idea (P5!): it would be cool to be able to use various tokens across
different file types consistently; e.g. not *.java but also *.html, *.xml, ...
right now you would have to copy your stuff to every such module's settings
(assuming they all supported this stuff, which they don't). Idea: in system FS,
if there is some file TemplateTokens/SingleLine/FOO, then __FOO__ is treated as
a token automatically by all compliant modules, with value taken from the file.
If it is in TemplateTokens/MultiLine/FOO, then similar, but each line of
contents is prefixed by appropriate comment delimiter for that file type. Thus
in Java:

/*
__COPYRIGHT__ * __NAME__.java
 *
 * Created on __DATE__, __TIME__
 */

where TemplateTokens/MultiLine/COPYRIGHT reads:

         Sun Public License Notice
Blah blah blah...
[trailing newline]

also in XML:

<!--
__COPYRIGHT____NAME__.xml
Etc.
-->

(no delimiter needed here) and in properties:

__COPYRIGHT__# __NAME__.properties
# etc.

(with "# " inserted before every line).

Or have just one type of token, and if the value happens to contain newlines,
then if the containing line was "${prefix}__TOKEN__${suffix}" then for every
line in the token value (possibly none) insert
"${prefix}${thatline}${suffix}\n". This might be more natural since this would work:

/*
 * __COPYRIGHT__
 * __NAME__
 */

without wasting a blank line etc. if __COPYRIGHT__ is not defined, and giving
more control over formatting to the user.

__USER__ should probably be defaulted to ${java.user} if the file does not
exist, as now.

Module settings would have to be modified to read/write these files instead of
the system property as now. Or I guess it would need to be a global setting
eventually.

Does this sound like a good idea? It could be implemented just in Java module on
an experimental basis and other modules could copy as needed and try to share
the setting node (utilities module? not needed for reading, note). If it became
popular and well-liked then a subclass of FileEntry.Format could hold the
reference implementation. (Yarda long ago vetoed any API change for this purpose
however.)

Jesse, that sounds *very* good. Thanks :-) 

Target milestone -> 3.3.1.

It is not Java specifics, properties, XML, CSS and HTML needs it too.
We need a shared copyright file that can be processed by various
formatters/tokens in templates.

Now I see that Jesse already expressed it :-).

Cleaning up before 4.0 planning

Target milestone was changed from not determined to TBD

License settings should be shareable see also issue #31836.

David K. - FYI. Whoever works on templates should think about this
(probably as a future option, not to be implemented right away).

Thanx for pointing this to me Jesse. I filed issue 31849 for myself to
do not forget about this.

I've implemented issue 31836 (which adds auto-insert-copyright when
the tasklist/docscan module is running). If/when this issue is
implemented, I should change the tasklist to use the new copyright
setting instead, so I'm marking this issue as blocking 31836 (even
though 31836 has been fixed - I wish there was an "affects" relation
ship, not just blocks or depends on :)

Just wanted to add that the __DATE__ macro, and possibly others,
doesn't work for JSP files. I think they should.

I suggest we address at least the simple solution in the original
description for 4.1 (since waiting for the "complete" solution could
take too long) - it should be really simple.

*** Issue 36520 has been marked as a duplicate of this issue. ***

*** Issue 19896 has been marked as a duplicate of this issue. ***

Touching even other areas, yet especially this (maybe this is off topic and
needs to be it's own issue...please let me know if needed I'll move it into it's
own separate issue, but it affects this one as well): If there were a way to
express particular pieces of a given token, the token, and replacement strings
(what if the token has say a */ inside of it's real value and will add this to
the given template for a .java file and will be inside a comment?).  Basically
what I mean is if the token is to be inside a comment: as Jesse suggested
placing *, #, or what ever in front of each line if it is a comment, then some
how this could be expressed by the module or which ever code requesting a token
to be expanded.

What I'm proposing is a system level Template and Code Template API with a
manager so it makes changes like this easier on the different module developers
so changes are more likely to be adopted faster, and some changes can then be
directly controlled and manipulated by the TemplateManager without changes to
every module or even a single one (once they manager and supporting classes are
adopted and used that is)...doing this it could then be possible to allow
developers to manipulate through APIs other modules templates just by a given
token being used.  I suppose we could have global tokens that get searched if a
given module isn't going to expand a given token.  I'm not sure which way is
better yet...global takes precedence or module does....what if the user of the
IDE wants to do something with all templates? I suppose we could always have a
global editor that adds GLOBAL_ to all tokens added by the user through some
token manager in advanced options, but what if they want to affect the same
token in all modules and it doesn't begin with GLOBAL_?  Maybe it could just be
documented in user documentation this will have to be replaced with
GLOBAL_WHATEVERTOKEN or __GLOBAL_WHATEVERTOKEN__ what ever the case....  

Basically the general template or template token functionallity could be in a
global API where which ever requesting module requests a given template token to
be expanded.  Tokens could be registered sort of like what happens with
URLMapper or JDBC just a little different.

Doing something sort of like JDBC (conceptual classes of course):

TemplateTokenExpander instances are registered with TemplateTokenManager.  The
different types are associated with different expanders.  The first expander to
"say" it will process a given token for a type gets to do so.  With either all
modules taking precedence over the global or the global taking
precendence....ideas here?  So: 

TemplateTokenManager.register(String type /* .java or .whatever or maybe just
use identifiers of some sort java, xml, html...I suppose file extensions works
best or some how integrate with the MIME type stuff to figure out
extensions???*/, TemplateTokenExpander expander)
TemplateTokenExpander[] TemplateTokenManager.getExpanders(String type)
String TemplateTokenManager.expandToken(String token)
String TemplateTokenManager.expandToken(String token, String context)
String TemplateTokenManager.expandToken(String token, String startValue, String
endValue, String lineStartValue, String lineEndValue)
void addTemplateTokenStore(String type, TemplateTokenStore store)
void addTemplateTokenStore(String type, String context, TemplateTokenStore store)
static TemplateTokenManager getDefault()

and

boolean TemplateTokenExpander.isHandlingToken(String token) /* self explanatory */
boolean TemplateTokenExpander.isHandlingContext(String context) /* basically
expanders for different modules can use the modules context API or named context
contracts i.e. java will have comment, javadoc, singlelinecomment, source, maybe
others...so this can be used to help the manager figure out who will handle a
request to expand a token and other modules can more easily target other modules
TemplateTokens if the context are documented for any given module */
String TemplateTokenExpander.expandToken(String token)
String TemplateTokenExpander.expandToken(String token, String context)
String TemplateTokenExpander.expandToken(String token, String startValue, String
endValue, String lineStartValue, String lineEndValue)
TemplateTokenStore[] getTemplateTokenStores()
void addTemplateTokenStore(TemplateTokenStore store)
void removeTemplateTokenStore(TemplateTokenStore store)

and

abstract String TemplateTokenStore.getTokenValue(String token)

So the manager would be able to pass off the method call to the selected
expander based on whether the expander can handle the request or not.  Then
individual modules such as say the java project module could use the manager and
this way other modules or the global system could add expanders or stores which
can be used indirectly by the module.

...

I think that would be pretty clean and allow for a global template token
replacement system where all modules could affect tokens in the entire system
and obviously other modules.  So, the java editor, xml editor, etc used in say
java projects could be affected by say the J2SE project module simply by it
registering stores and expanders for the different types.  The replacement in
TemplateTokenExpander.expandToken(String token, String startValue, String
endValue, String lineStartValue, String lineEndValue) could be pandered off to a
utility class or make TemplateTokenExpander an abstract class instead of an
interface to pander this off to a real method.  Well I suppose
TemplateTokenExpanderInterface or ITemplateTokenExpander could be the super
through implements then pander off the real work in an abstract method in
TemplateTokenExpander then change the above methods to use the interfaces.  This
way it's very configurable.  expandToken(token, context) is pretty obvious and
could be coded up how ever it needs to be for any given expander though the real
value really needs to come from the store so values can truely be affected
system wide.  Modules also need to not directly access their expander which will
be registered, but instead need to use the manager so this remains system
wide....maybe the interface methods need to be declared protected.

Anyways, that's the general idea obviously needing to be discussed and thought
about a bit more.  I suppose this comes from me personally not liking the idea
that token replacement and template handling is so module dependent at the
moment when it in itself is a separate idea with pieces only being specific.

Obviously remove methods need to be available in the manager class to allow
module clean up.

Reiterating what I indirectly stated in another comment:
This really needs to be available at projects level.  To be able to set the
license used in files created for a given project.

JavaProject, APISupport, some J2EE code do this or similar in their own ways.

Let's start a discussion of what is necessary to improve the templating 
system: http://openide.netbeans.org/tutorial/reviews/opinions_13250.html

Usage of Velocity sounds OK to me. But be careful about semantic API "leakage".
The context map is a Map<String,Object>? Is there a concise definition
(independent of Velocity) about what the map can contain? Or about the format of
the macro language?

Possibly related issues which could be listed as blockers: issue #88962, issue
#91548, issue #34381, others?

Thanks for review, the reviewed version is here:
http://www.netbeans.org/source/browse/openide/www/tutorial/reviews/opinions_13250.html?rev=1.8&view=markup
I'll start with extending DataObject.createFromTemplate. There is a branch 
templates_13250 in openide rooted at templates_13250_root. To try it use:

cvs co -r templates_13250 openide

The code is ready for another review. The review document 
http://openide.netbeans.org/tutorial/reviews/opinions_13250.html
is updated:
http://www.netbeans.org/source/browse/openide/www/tutorial/reviews/opinions_13250.html?rev=1.9&view=markup

I'll attach the current diff from the templates_13250 branches of openide and 
libs. Please review during one week. We'll close the review someday next week.

Created attachment 37843 [details]
Changes in openide and libs

Created attachment 37844 [details]
The changes I've done to java templates to show that they can share one license file - needs more consultation with java gurus

The proposed changes look fine to me.
JR01: What about to move Template Manager,e.g. package
org.netbeans.core.favorites.templates in module core/favorites to
openide/templates module to keep them close each other.
JR02: WD.getProperty() could be deprecated when WD.getProperties() covers up it.

I seems ok for me.

PN01: Every template has an associated engine attribute, but you hardcode
freemaker in the handler impl which could be in fact reused for more scripting
languages. Seems like small implementation bug...

Generally looks good to me.


[JG01] The code

for (Map.Entry<String,? extends Object> e : map.entrySet()) {
    all.put(e.getKey(), e.getValue());
}

could probably be simplified to

all.putAll(map);

Similarly for c.param.


[JG02] The code

FileObject fo = null;
boolean done = false;
for (CreateFromTemplateHandler h :
Lookup.getDefault().lookupAll(CreateFromTemplateHandler.class)) {
    if (h.accept(getFile())) {
        done = true;
        fo = h.createFromTemplate(getFile(), f, name,
DataObject.CreateAction.findParameters(name));
        assert fo != null;
        break;
    }
}
if (!done) {
    fo = getFile().copy (f, name, getFile().getExt ());
}

could be simplified to

FileObject fo = null;
for (CreateFromTemplateHandler h :
Lookup.getDefault().lookupAll(CreateFromTemplateHandler.class)) {
    if (h.accept(getFile())) {
        fo = h.createFromTemplate(getFile(), f, name,
DataObject.CreateAction.findParameters(name));
        assert fo != null;
        break;
    }
}
if (fo == null) {
    fo = getFile().copy(f, name, getFile().getExt ());
}

and similarly below.


[JG03] Repeating PN01. The code

ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine eng = mgr.getEngineByName("freemarker");

should I guess be

ScriptEngine eng = engine(template);
assert eng != null;


[JG04] Why the two ways of copying bindings from 'values' into eng.context?


[JG05] Need to define an encoding for OutputStreamWriter and InputStreamReader.
Using default platform encoding is dangerous here.


[JG06] openide/templates/src/org/netbeans/modules/templates/layer.xml should be
deleted and its manifest entry with it.


[JG07] What was the original license of
libs/freemarker/src/org/netbeans/libs/freemarker/FreemarkerEngine.java and
libs/freemarker/src/org/netbeans/libs/freemarker/FreemarkerFactory.java?


[JG08] Where does org.netbeans.libs.jsr223 come from? Is that added to libs/* as
well?

Thanks for your comments.

Here are few that I believe deserve deeper discussion:
http://www.netbeans.org/source/browse/openide/www/tutorial/reviews/opinions_13250.html?r1=1.11&r2=1.12

Also I have solved the formating issue by reusing IndentEngine:
http://www.netbeans.org/source/browse/openide/templates/test/unit/src/org/netbeans/modules/templates/Attic/IndentEngineIntTest.java?rev=1.1.2&view=markup

Here are my answer to comments that I believe I have sufficient answer to:
JR02: probably it is not wise to trade wd.getProperty for 
wd.getProperites().get and I see no reason for doing so. Both methods continue 
to work without problems

PN01&JG03: Fixed.

JG01: If you re-read javadoc for Map.putAll you might get surprise that this 
does not work.

JG02: Simplified.

JG06: Will fix before commit, otherwise commit validation does not pass.

JG07: One that allows its change to anything else when a credit is given.

JG08: It is in libs module in CVS, sorry probably not in the diff. It is a 
simple wrap module around jsr223-api.jar - the actual JAR will likely change 
as soon as its GPL w/CPexception version is released (expected to happen 
before 6.0 release).

See you on Tuesday's review.

[JG01'] I just reread the Javadoc for Map.putAll and I don't know what you are
talking about. What surprise?


[JG04'] I'm just curious about the code

Bindings bind = eng.getContext().getBindings(ScriptContext.ENGINE_SCOPE);
bind.putAll(values);
for (Map.Entry<String, Object> entry : values.entrySet()) {
    eng.getContext().setAttribute(entry.getKey(), entry.getValue(),
ScriptContext.ENGINE_SCOPE);
}

This looks redundant but probably because I am ignorant of the difference
between Bindings.put and ScriptContext.setAttribute. Are these unrelated? Is
there some reason why a single 'values' map is being used to set both? Can you
give an example of what this means?


[JG05'] Use of platform default encoding is dangerous because either the
template or the output file (or both) might require an encoding different from
the platform default encoding. For example, a Mexican Windows user named "Raúl"
with encoding set to Cp1252 tries to instantiate an XML template shipped with
the IDE:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Created on ${date} by ${user} -->
<root/>

Decoding the template in Cp1252 is in this case harmless, and substitution
inside the script engine probably proceeds without issue. But when the result is
then written in Cp1252, his name becomes garbage in UTF-8, possibly even making
the file malformed (causing parse errors).

In the absence of a better API (e.g. issue #19928), the only thing I can think
of is to use EditorCookie to both read the template and write the result. This
would at least give data loaders such as for XML a chance to use the proper
encoding based on information such as the XML declaration.


[JG06'] Why not? (out of curiosity)

Surprise is that sometimes putAll does not work as expected. It happend to me 
when I tried to run it during the testing. I am not sure why, but for example
http://java.sun.com/javase/6/docs/api/java/util/concurrent/ConcurrentHashMap.html#putAll(java.util.Map)
says that putAll is not additional operation, but a replacement. In contrary 
the Map interface defines it as additional operation. Pretty weird. Using 
incremental put(key,value) seems safer.

"Additional"? Did you mean "optional" operation, that is, it may throw
UnsupportedOperationException from putAll?

But when you're talking about "replacement" and javadoc of CHM.putAll, it makes
me think you are affraid that _only_ preexisting keys will get replaced. This is
not the case, not even for CHM. It really copies _all_ the mappings, which also
replaces preexising entries with equal keys, of course.

Accepted with TCA:
http://www.netbeans.org/source/browse/*checkout*/openide/www/tutorial/reviews/opinions_13250.html?content-type=text%2Fhtml&rev=1.13

Jesse, you missed the review and we still need resolution of:
   
[JG04] Why the two ways of copying bindings from 'values' into eng.context?

   Y: What do you means? What should be the fix?

[JG05] Need to define an encoding for OutputStreamWriter and 
InputStreamReader.
Using default platform encoding is dangerous here.

   Y: How?

Please, if you think something of your comments is important, create TCA or 
TCR issue and make it a blocker of this one. Feel free to update the opinion 
document. Otherwise I'll be working on integration tomorrow.

JG04 is simply curiosity; I would like to know what this part of the
implementation does.

I will file a blocker issue for JG05.

Created attachment 38162 [details]
Commit log

Integrated.

*** Issue 97840 has been marked as a duplicate of this issue. ***

I see that we've marked this as fixed, but what about the user-visible feature that inspired this (described 
right in the issue description): Define a copyright in -one- place and have all templates automatically use 
it? It looks to me like the templates still have copyrights or file headers inlined in the templates rather than 
referencing a global preference somewhere. Plus, the options dialog should have a place to set these 
things.  Is there a separate issue for tracking this?

Jarda has delegated license related stuff to the issue #95399.