This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
When I'm put some international character to Licence.txt (via Tools -> Templates) it's input incorectly to new created class file. Problem is, that Templates system store .txt files in system default encoding (windows-1250 for Windows XP) and .java files store in UTF-8.
What template? And all files in the system file system are supposed to be kept in UTF-8 always; bug (I guess in openide/fs, for projects/queries) if not.
I only opened in Templates manager Licences -> Default Licence a put ěščřžýáí to them. When I'm now creating new Class a get this: ěščřžýáĂĂ© I also try import new UTF8 encoding files to Templates. Suffix .java works, but .txt doesn't.
So why is this filed in apisupport?
in solaris ja utf8 locale, I changed a java class template to have multibyte and it looked ok when creating that class, but will try in regular ja (euc) solaris locale since that emulates what was seen by bug filer in windows, ie a non utf8 locale. tprochazka - how can I see the licence file as per your description ? Is it included in the java project ? Jesse - if this is an issue for all templates, does it mean separate one needed for each provider of templates ? ken.frank@sun.com
for a java annotation template, just as an example, in solaris ja locale, when adding multibyte to it, then creating java annotation class, the multibyte does not look ok and this confirms perhaps what filer saw in windows -- and that its ok in ja utf8 locale but not in windows or solaris ja is related to the encoding handling. BTW, and maybe this is separate issue - n ja or ja utf8 locales on solaris, when adding multibyte to template, then saving in editor, then closing editor, get exception attached but when reopen in editor, the multibyte looks ok. (this was for cddl license, but happens with others too)
Created attachment 42866 [details] exception on closing a template file in editor that has mbyte
is a separate issue needed for each template type area ? that is, is it like the project file encoding where each project type implements the feq for their project ? ken.frank@sun.com
This looks to relate to the FreeMarker scripting engine. I was able to reproduce it on WinXP with system charset CP1250. In case I tried to modify templates (licenses.txt or NewClass.java) with Czech characters ěščřžýáí they were written properly in UTF-8. When I tried to create NewClass then Czech characters ěščřžýáí of NewClass.java template were read properly BUT characters of included licence template were corrupted as tprochazka mentioned. The debugger shows proper UTF-8 encoding for both target and source in org.netbeans.modules.templates.ScriptingCreateFromTemplateHandler. So the bug seems to be caused by the misconfigured scripting engine. http://freemarker.org/docs/pgui_misc_charset.html should help to solve this. BTW on Linux it works for me but my system encoding is UTF-8. Reassigning to jtulach who did the FreeMarker integration.
I don't know if this relates to that for nb6 there is/will be, for many project and file types, a sense of project encoding, and that the default value of it will be utf-8 (even though user might start in a locale whose encoding is not utf8, utf8 is supposed to work) when user changes project encoding, it changes how encoding is viewed for next files created in this project and any kind of next projects -- except for projects that have not implemented this project file encoding. Thus I wanted to mentioned this in case there is, as part of this fix or other things related to templates, something that might need to be done. Also, What I don't know is if, for the projects that don't do the project encoding implementation, but who might provide file types/templates that would be used by projects that do use the project encoding, if there might be a problem or conflict. And dont know if that might relate to one template including another template in it when file is created. ken.frank@sun.com
Ken, the template support of course reuses FileEncodingQuery API so in case the target is a project allowing to customize the encoding it is used. I guess the default impl of the query is the system (JVM) encoding. All templates should be UTF-8 encoded. What does not work is including templates (e.g. licences) by other templates (e.g. .java). It looks that some encoding attribute of the scripting engine (FreeMarker) has to be set to the target encoding before creating each new file. But it has to be done by the FreeMarker integration module not clients of the Templates API.
as to comment below: Ken, the template support of course reuses FileEncodingQuery API so in case the target is a project allowing to customize the encoding it is used. I guess the default impl of the query is the system (JVM) encoding. All templates should be UTF-8 encoded. not all templates are utf-8 encoded now as those for html, jsp and xml are/will be seeding the encoding with that of the project encoding ? is this a problem ? for this issue - can it be fixed for nb6 ? ken.frank@sun.com
*** Issue 116986 has been marked as a duplicate of this issue. ***
Ok, I've simulated the problem on windows.
"#104770: Use FileEncodingQuery when reading content of included templates" Checking in nbproject/project.xml; /shared/data/ccvs/repository/libs/freemarker/nbproject/project.xml,v <-- project.xml new revision: 1.5; previous revision: 1.4 done Checking in src/org/netbeans/libs/freemarker/RsrcLoader.java; /shared/data/ccvs/repository/libs/freemarker/src/org/netbeans/libs/freemarker/RsrcLoader.java,v <-- RsrcLoader.java new revision: 1.8; previous revision: 1.7 done Checking in test/unit/src/org/netbeans/api/templates/ProcessorTest.java; /shared/data/ccvs/repository/libs/freemarker/test/unit/src/org/netbeans/api/templates/ProcessorTest.java,v <-- ProcessorTest.java new revision: 1.11; previous revision: 1.10 done
tprochazka, please let us know if this fix solves the situation. ken.frank@sun.com
I changed license template file to have some multibyte then created project, running in ja locale, and the mbyte in license looked ok then I changed project encoding prop to euc-jp and created another project, and the license part looked ok also; however, since the same mbyte being used looks ok in both utf8 and euc-jp project encoding I don't know if my steps really verify this issue. ken.frank@sun.com
from all comments below, seems that it can be verified.