Bug 13250 - Allow to have token placeholders for templates
Allow to have token placeholders for templates
Status: RESOLVED FIXED
Product: platform
Classification: Unclassified
Component: Data Systems
3.x
All All
: P2 (vote)
: 6.x
Assigned To: Jaroslav Tulach
issues@platform
plan60 nbide
: API, API_REVIEW
: 19896 36520 97840 (view as bug list)
Depends on: 94666 94676 115058 204221
Blocks: 18040
  Show dependency treegraph
 
Reported: 2001-06-27 16:58 UTC by Petr Nejedly
Modified: 2012-05-30 14:38 UTC (History)
16 users (show)

See Also:
Issue Type: ENHANCEMENT
:


Attachments
Changes in openide and libs (106.23 KB, patch)
2007-01-30 17:21 UTC, Jaroslav Tulach
Details | Diff
The changes I've done to java templates to show that they can share one license file - needs more consultation with java gurus (18.54 KB, patch)
2007-01-30 17:22 UTC, Jaroslav Tulach
Details | Diff
Commit log (12.37 KB, text/plain)
2007-02-07 10:05 UTC, Jaroslav Tulach
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Petr Nejedly 2001-06-27 16:58:37 UTC
I propose adding something like __COPYRIGHT__ to the top of all templates,
and creating an empty item in strings table for it (COPYRIGHT='').
Then everybody can simply fill in his copyright notice in one place
and have it available for all newly created classes.

It may look like:
/* __COPYRIGHT__
 * __NAME__.java
 *
 * Created on __DATE__, __TIME__
 */

and I'll fill my COPYRIGHT with:
\n *                 Sun Public License Notice\n * The con.....\n *
Microsystems, Inc. All Rights Reserved.\n *\n * Contributor(s): __USER__.\n *\n

which would be translated as:
/*
 *                 Sun Public License Notice
 *
 * The con..... 
 * Microsystems, Inc. All Rights Reserved.
 *
 * Contributor(s): Petr Nejedly.
 *
 * Test.java
 *
 * Created on 27.06.2001, 17:35
 */

BTW: It will be no-change for users that won't notice it, so it is transparent.
Comment 1 Jesse Glick 2001-06-27 17:51:37 UTC
A related idea (P5!): it would be cool to be able to use various tokens across
different file types consistently; e.g. not *.java but also *.html, *.xml, ...
right now you would have to copy your stuff to every such module's settings
(assuming they all supported this stuff, which they don't). Idea: in system FS,
if there is some file TemplateTokens/SingleLine/FOO, then __FOO__ is treated as
a token automatically by all compliant modules, with value taken from the file.
If it is in TemplateTokens/MultiLine/FOO, then similar, but each line of
contents is prefixed by appropriate comment delimiter for that file type. Thus
in Java:

/*
__COPYRIGHT__ * __NAME__.java
 *
 * Created on __DATE__, __TIME__
 */

where TemplateTokens/MultiLine/COPYRIGHT reads:

         Sun Public License Notice
Blah blah blah...
[trailing newline]

also in XML:

<!--
__COPYRIGHT____NAME__.xml
Etc.
-->

(no delimiter needed here) and in properties:

__COPYRIGHT__# __NAME__.properties
# etc.

(with "# " inserted before every line).

Or have just one type of token, and if the value happens to contain newlines,
then if the containing line was "${prefix}__TOKEN__${suffix}" then for every
line in the token value (possibly none) insert
"${prefix}${thatline}${suffix}\n". This might be more natural since this would work:

/*
 * __COPYRIGHT__
 * __NAME__
 */

without wasting a blank line etc. if __COPYRIGHT__ is not defined, and giving
more control over formatting to the user.

__USER__ should probably be defaulted to ${java.user} if the file does not
exist, as now.

Module settings would have to be modified to read/write these files instead of
the system property as now. Or I guess it would need to be a global setting
eventually.

Does this sound like a good idea? It could be implemented just in Java module on
an experimental basis and other modules could copy as needed and try to share
the setting node (utilities module? not needed for reading, note). If it became
popular and well-liked then a subclass of FileEntry.Format could hold the
reference implementation. (Yarda long ago vetoed any API change for this purpose
however.)
Comment 2 Svata Dedic 2001-06-28 07:52:02 UTC
Jesse, that sounds *very* good. Thanks :-) 
Comment 3 Jan Chalupa 2001-11-27 12:49:39 UTC
Target milestone -> 3.3.1.
Comment 4 _ pkuzel 2002-03-07 16:59:42 UTC
It is not Java specifics, properties, XML, CSS and HTML needs it too.
We need a shared copyright file that can be processed by various
formatters/tokens in templates.

Now I see that Jesse already expressed it :-).
Comment 5 Svata Dedic 2002-05-21 17:49:52 UTC
Cleaning up before 4.0 planning
Comment 6 Marek Grummich 2002-07-19 16:45:03 UTC
Target milestone was changed from not determined to TBD
Comment 7 _ pkuzel 2003-03-10 17:57:59 UTC
License settings should be shareable see also issue #31836.
Comment 8 Jesse Glick 2003-03-10 19:04:32 UTC
David K. - FYI. Whoever works on templates should think about this
(probably as a future option, not to be implemented right away).
Comment 9 David Konecny 2003-03-11 08:45:35 UTC
Thanx for pointing this to me Jesse. I filed issue 31849 for myself to
do not forget about this.
Comment 10 Torbjorn Norbye 2003-03-19 05:51:23 UTC
I've implemented issue 31836 (which adds auto-insert-copyright when
the tasklist/docscan module is running). If/when this issue is
implemented, I should change the tasklist to use the new copyright
setting instead, so I'm marking this issue as blocking 31836 (even
though 31836 has been fixed - I wish there was an "affects" relation
ship, not just blocks or depends on :)
Comment 11 Chris Kutler 2003-06-12 20:28:19 UTC
Just wanted to add that the __DATE__ macro, and possibly others,
doesn't work for JSP files. I think they should.
Comment 12 Martin Matula 2004-11-12 07:36:43 UTC
I suggest we address at least the simple solution in the original
description for 4.1 (since waiting for the "complete" solution could
take too long) - it should be really simple.
Comment 13 Jesse Glick 2005-06-16 11:12:46 UTC
*** Issue 36520 has been marked as a duplicate of this issue. ***
Comment 14 Jesse Glick 2005-06-16 11:13:39 UTC
*** Issue 19896 has been marked as a duplicate of this issue. ***
Comment 15 _ wadechandler 2005-12-28 16:21:55 UTC
Touching even other areas, yet especially this (maybe this is off topic and
needs to be it's own issue...please let me know if needed I'll move it into it's
own separate issue, but it affects this one as well): If there were a way to
express particular pieces of a given token, the token, and replacement strings
(what if the token has say a */ inside of it's real value and will add this to
the given template for a .java file and will be inside a comment?).  Basically
what I mean is if the token is to be inside a comment: as Jesse suggested
placing *, #, or what ever in front of each line if it is a comment, then some
how this could be expressed by the module or which ever code requesting a token
to be expanded.

What I'm proposing is a system level Template and Code Template API with a
manager so it makes changes like this easier on the different module developers
so changes are more likely to be adopted faster, and some changes can then be
directly controlled and manipulated by the TemplateManager without changes to
every module or even a single one (once they manager and supporting classes are
adopted and used that is)...doing this it could then be possible to allow
developers to manipulate through APIs other modules templates just by a given
token being used.  I suppose we could have global tokens that get searched if a
given module isn't going to expand a given token.  I'm not sure which way is
better yet...global takes precedence or module does....what if the user of the
IDE wants to do something with all templates? I suppose we could always have a
global editor that adds GLOBAL_ to all tokens added by the user through some
token manager in advanced options, but what if they want to affect the same
token in all modules and it doesn't begin with GLOBAL_?  Maybe it could just be
documented in user documentation this will have to be replaced with
GLOBAL_WHATEVERTOKEN or __GLOBAL_WHATEVERTOKEN__ what ever the case....  

Basically the general template or template token functionallity could be in a
global API where which ever requesting module requests a given template token to
be expanded.  Tokens could be registered sort of like what happens with
URLMapper or JDBC just a little different.

Doing something sort of like JDBC (conceptual classes of course):

TemplateTokenExpander instances are registered with TemplateTokenManager.  The
different types are associated with different expanders.  The first expander to
"say" it will process a given token for a type gets to do so.  With either all
modules taking precedence over the global or the global taking
precendence....ideas here?  So: 

TemplateTokenManager.register(String type /* .java or .whatever or maybe just
use identifiers of some sort java, xml, html...I suppose file extensions works
best or some how integrate with the MIME type stuff to figure out
extensions???*/, TemplateTokenExpander expander)
TemplateTokenExpander[] TemplateTokenManager.getExpanders(String type)
String TemplateTokenManager.expandToken(String token)
String TemplateTokenManager.expandToken(String token, String context)
String TemplateTokenManager.expandToken(String token, String startValue, String
endValue, String lineStartValue, String lineEndValue)
void addTemplateTokenStore(String type, TemplateTokenStore store)
void addTemplateTokenStore(String type, String context, TemplateTokenStore store)
static TemplateTokenManager getDefault()

and

boolean TemplateTokenExpander.isHandlingToken(String token) /* self explanatory */
boolean TemplateTokenExpander.isHandlingContext(String context) /* basically
expanders for different modules can use the modules context API or named context
contracts i.e. java will have comment, javadoc, singlelinecomment, source, maybe
others...so this can be used to help the manager figure out who will handle a
request to expand a token and other modules can more easily target other modules
TemplateTokens if the context are documented for any given module */
String TemplateTokenExpander.expandToken(String token)
String TemplateTokenExpander.expandToken(String token, String context)
String TemplateTokenExpander.expandToken(String token, String startValue, String
endValue, String lineStartValue, String lineEndValue)
TemplateTokenStore[] getTemplateTokenStores()
void addTemplateTokenStore(TemplateTokenStore store)
void removeTemplateTokenStore(TemplateTokenStore store)

and

abstract String TemplateTokenStore.getTokenValue(String token)

So the manager would be able to pass off the method call to the selected
expander based on whether the expander can handle the request or not.  Then
individual modules such as say the java project module could use the manager and
this way other modules or the global system could add expanders or stores which
can be used indirectly by the module.

...

I think that would be pretty clean and allow for a global template token
replacement system where all modules could affect tokens in the entire system
and obviously other modules.  So, the java editor, xml editor, etc used in say
java projects could be affected by say the J2SE project module simply by it
registering stores and expanders for the different types.  The replacement in
TemplateTokenExpander.expandToken(String token, String startValue, String
endValue, String lineStartValue, String lineEndValue) could be pandered off to a
utility class or make TemplateTokenExpander an abstract class instead of an
interface to pander this off to a real method.  Well I suppose
TemplateTokenExpanderInterface or ITemplateTokenExpander could be the super
through implements then pander off the real work in an abstract method in
TemplateTokenExpander then change the above methods to use the interfaces.  This
way it's very configurable.  expandToken(token, context) is pretty obvious and
could be coded up how ever it needs to be for any given expander though the real
value really needs to come from the store so values can truely be affected
system wide.  Modules also need to not directly access their expander which will
be registered, but instead need to use the manager so this remains system
wide....maybe the interface methods need to be declared protected.

Anyways, that's the general idea obviously needing to be discussed and thought
about a bit more.  I suppose this comes from me personally not liking the idea
that token replacement and template handling is so module dependent at the
moment when it in itself is a separate idea with pieces only being specific.
Comment 16 _ wadechandler 2005-12-28 16:25:33 UTC
Obviously remove methods need to be available in the manager class to allow
module clean up.
Comment 17 _ wadechandler 2005-12-28 16:35:35 UTC
Reiterating what I indirectly stated in another comment:
This really needs to be available at projects level.  To be able to set the
license used in files created for a given project.
Comment 18 Martin Krauskopf 2006-09-05 21:42:54 UTC
JavaProject, APISupport, some J2EE code do this or similar in their own ways.
Comment 19 Jaroslav Tulach 2006-12-03 22:34:32 UTC
Let's start a discussion of what is necessary to improve the templating 
system: http://openide.netbeans.org/tutorial/reviews/opinions_13250.html
Comment 20 Jesse Glick 2006-12-04 18:53:01 UTC
Usage of Velocity sounds OK to me. But be careful about semantic API "leakage".
The context map is a Map<String,Object>? Is there a concise definition
(independent of Velocity) about what the map can contain? Or about the format of
the macro language?
Comment 21 Jesse Glick 2006-12-25 17:47:18 UTC
Possibly related issues which could be listed as blockers: issue #88962, issue
#91548, issue #34381, others?
Comment 22 Jaroslav Tulach 2007-01-16 17:21:26 UTC
Thanks for review, the reviewed version is here:
http://www.netbeans.org/source/browse/openide/www/tutorial/reviews/opinions_13250.html?rev=1.8&view=markup
I'll start with extending DataObject.createFromTemplate. There is a branch 
templates_13250 in openide rooted at templates_13250_root. To try it use:

cvs co -r templates_13250 openide


Comment 23 Jaroslav Tulach 2007-01-30 17:20:03 UTC
The code is ready for another review. The review document 
http://openide.netbeans.org/tutorial/reviews/opinions_13250.html
is updated:
http://www.netbeans.org/source/browse/openide/www/tutorial/reviews/opinions_13250.html?rev=1.9&view=markup

I'll attach the current diff from the templates_13250 branches of openide and 
libs. Please review during one week. We'll close the review someday next week.
Comment 24 Jaroslav Tulach 2007-01-30 17:21:05 UTC
Created attachment 37843 [details]
Changes in openide and libs
Comment 25 Jaroslav Tulach 2007-01-30 17:22:34 UTC
Created attachment 37844 [details]
The changes I've done to java templates to show that they can share one license file - needs more consultation with java gurus
Comment 26 Jiri Rechtacek 2007-01-31 08:53:33 UTC
The proposed changes look fine to me.
JR01: What about to move Template Manager,e.g. package
org.netbeans.core.favorites.templates in module core/favorites to
openide/templates module to keep them close each other.
JR02: WD.getProperty() could be deprecated when WD.getProperties() covers up it.
Comment 27 Michal Skvor 2007-02-01 10:26:13 UTC
I seems ok for me.
Comment 28 Petr Nejedly 2007-02-01 11:09:53 UTC
PN01: Every template has an associated engine attribute, but you hardcode
freemaker in the handler impl which could be in fact reused for more scripting
languages. Seems like small implementation bug...
Comment 29 Jesse Glick 2007-02-01 18:35:26 UTC
Generally looks good to me.


[JG01] The code

for (Map.Entry<String,? extends Object> e : map.entrySet()) {
    all.put(e.getKey(), e.getValue());
}

could probably be simplified to

all.putAll(map);

Similarly for c.param.


[JG02] The code

FileObject fo = null;
boolean done = false;
for (CreateFromTemplateHandler h :
Lookup.getDefault().lookupAll(CreateFromTemplateHandler.class)) {
    if (h.accept(getFile())) {
        done = true;
        fo = h.createFromTemplate(getFile(), f, name,
DataObject.CreateAction.findParameters(name));
        assert fo != null;
        break;
    }
}
if (!done) {
    fo = getFile().copy (f, name, getFile().getExt ());
}

could be simplified to

FileObject fo = null;
for (CreateFromTemplateHandler h :
Lookup.getDefault().lookupAll(CreateFromTemplateHandler.class)) {
    if (h.accept(getFile())) {
        fo = h.createFromTemplate(getFile(), f, name,
DataObject.CreateAction.findParameters(name));
        assert fo != null;
        break;
    }
}
if (fo == null) {
    fo = getFile().copy(f, name, getFile().getExt ());
}

and similarly below.


[JG03] Repeating PN01. The code

ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine eng = mgr.getEngineByName("freemarker");

should I guess be

ScriptEngine eng = engine(template);
assert eng != null;


[JG04] Why the two ways of copying bindings from 'values' into eng.context?


[JG05] Need to define an encoding for OutputStreamWriter and InputStreamReader.
Using default platform encoding is dangerous here.


[JG06] openide/templates/src/org/netbeans/modules/templates/layer.xml should be
deleted and its manifest entry with it.


[JG07] What was the original license of
libs/freemarker/src/org/netbeans/libs/freemarker/FreemarkerEngine.java and
libs/freemarker/src/org/netbeans/libs/freemarker/FreemarkerFactory.java?


[JG08] Where does org.netbeans.libs.jsr223 come from? Is that added to libs/* as
well?
Comment 30 Jaroslav Tulach 2007-02-02 14:09:34 UTC
Thanks for your comments.

Here are few that I believe deserve deeper discussion:
http://www.netbeans.org/source/browse/openide/www/tutorial/reviews/opinions_13250.html?r1=1.11&r2=1.12

Also I have solved the formating issue by reusing IndentEngine:
http://www.netbeans.org/source/browse/openide/templates/test/unit/src/org/netbeans/modules/templates/Attic/IndentEngineIntTest.java?rev=1.1.2&view=markup

Here are my answer to comments that I believe I have sufficient answer to:
JR02: probably it is not wise to trade wd.getProperty for 
wd.getProperites().get and I see no reason for doing so. Both methods continue 
to work without problems

PN01&JG03: Fixed.

JG01: If you re-read javadoc for Map.putAll you might get surprise that this 
does not work.

JG02: Simplified.

JG06: Will fix before commit, otherwise commit validation does not pass.

JG07: One that allows its change to anything else when a credit is given.

JG08: It is in libs module in CVS, sorry probably not in the diff. It is a 
simple wrap module around jsr223-api.jar - the actual JAR will likely change 
as soon as its GPL w/CPexception version is released (expected to happen 
before 6.0 release).

See you on Tuesday's review.
Comment 31 Jesse Glick 2007-02-02 18:59:39 UTC
[JG01'] I just reread the Javadoc for Map.putAll and I don't know what you are
talking about. What surprise?


[JG04'] I'm just curious about the code

Bindings bind = eng.getContext().getBindings(ScriptContext.ENGINE_SCOPE);
bind.putAll(values);
for (Map.Entry<String, Object> entry : values.entrySet()) {
    eng.getContext().setAttribute(entry.getKey(), entry.getValue(),
ScriptContext.ENGINE_SCOPE);
}

This looks redundant but probably because I am ignorant of the difference
between Bindings.put and ScriptContext.setAttribute. Are these unrelated? Is
there some reason why a single 'values' map is being used to set both? Can you
give an example of what this means?


[JG05'] Use of platform default encoding is dangerous because either the
template or the output file (or both) might require an encoding different from
the platform default encoding. For example, a Mexican Windows user named "Raúl"
with encoding set to Cp1252 tries to instantiate an XML template shipped with
the IDE:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Created on ${date} by ${user} -->
<root/>

Decoding the template in Cp1252 is in this case harmless, and substitution
inside the script engine probably proceeds without issue. But when the result is
then written in Cp1252, his name becomes garbage in UTF-8, possibly even making
the file malformed (causing parse errors).

In the absence of a better API (e.g. issue #19928), the only thing I can think
of is to use EditorCookie to both read the template and write the result. This
would at least give data loaders such as for XML a chance to use the proper
encoding based on information such as the XML declaration.


[JG06'] Why not? (out of curiosity)
Comment 32 Jaroslav Tulach 2007-02-03 14:33:02 UTC
Surprise is that sometimes putAll does not work as expected. It happend to me 
when I tried to run it during the testing. I am not sure why, but for example
http://java.sun.com/javase/6/docs/api/java/util/concurrent/ConcurrentHashMap.html#putAll(java.util.Map)
says that putAll is not additional operation, but a replacement. In contrary 
the Map interface defines it as additional operation. Pretty weird. Using 
incremental put(key,value) seems safer.
Comment 33 Petr Nejedly 2007-02-03 15:31:48 UTC
"Additional"? Did you mean "optional" operation, that is, it may throw
UnsupportedOperationException from putAll?

But when you're talking about "replacement" and javadoc of CHM.putAll, it makes
me think you are affraid that _only_ preexisting keys will get replaced. This is
not the case, not even for CHM. It really copies _all_ the mappings, which also
replaces preexising entries with equal keys, of course.
Comment 34 Jaroslav Tulach 2007-02-06 16:55:42 UTC
Accepted with TCA:
http://www.netbeans.org/source/browse/*checkout*/openide/www/tutorial/reviews/opinions_13250.html?content-type=text%2Fhtml&rev=1.13

Jesse, you missed the review and we still need resolution of:
   
[JG04] Why the two ways of copying bindings from 'values' into eng.context?

   Y: What do you means? What should be the fix?

[JG05] Need to define an encoding for OutputStreamWriter and 
InputStreamReader.
Using default platform encoding is dangerous here.

   Y: How?

Please, if you think something of your comments is important, create TCA or 
TCR issue and make it a blocker of this one. Feel free to update the opinion 
document. Otherwise I'll be working on integration tomorrow.
Comment 35 Jesse Glick 2007-02-06 17:19:25 UTC
JG04 is simply curiosity; I would like to know what this part of the
implementation does.

I will file a blocker issue for JG05.
Comment 36 Jaroslav Tulach 2007-02-07 10:05:13 UTC
Created attachment 38162 [details]
Commit log
Comment 37 Jaroslav Tulach 2007-02-07 10:05:52 UTC
Integrated.
Comment 38 Jan Pokorsky 2007-03-14 19:26:14 UTC
*** Issue 97840 has been marked as a duplicate of this issue. ***
Comment 39 Torbjorn Norbye 2007-03-14 20:06:50 UTC
I see that we've marked this as fixed, but what about the user-visible feature that inspired this (described 
right in the issue description): Define a copyright in -one- place and have all templates automatically use 
it? It looks to me like the templates still have copyrights or file headers inlined in the templates rather than 
referencing a global preference somewhere. Plus, the options dialog should have a place to set these 
things.  Is there a separate issue for tracking this?
Comment 40 Jan Pokorsky 2007-03-14 21:38:27 UTC
Jarda has delegated license related stuff to the issue #95399.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo