This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 114123 - I18N - The Encoding Issue
Summary: I18N - The Encoding Issue
Status: RESOLVED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Generic Infrastructure (show other bugs)
Version: 6.x
Hardware: PC All
: P2 blocker with 3 votes (vote)
Assignee: Jesse Glick
URL:
Keywords: I18N
: 168517 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-08-29 13:57 UTC by goran
Modified: 2019-07-20 01:10 UTC (History)
8 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
[Open|Save] in encoding actions (22.92 KB, application/octet-stream)
2007-10-23 16:35 UTC, Tomas Zezula
Details

Note You need to log in before you can comment on or make changes to this bug.
Description goran 2007-08-29 13:57:22 UTC
Encoding is such a crucial property of the file that is simple unbelievable that NB team keeps on ignoring it for 
almost 4 years (just check the history of http://www.netbeans.org/issues/show_bug.cgi?id=42638 and all relate bugs). 
Setting the encoding only per project is not going to solve the problem neither -did you ever developed anything in the 
environment where developers are spread all over the world and where final project needs to be assembled out of files 
in different encodings. Using external editors (read UltraEdit) before importing files into IDE is the solution in my 
case , but you can not imaging how annoying is it !... And then Property files, JSP etc .... I hoped so much that with 
NB6 and J2SE6 this will be solved once for good :-/

Don't want to start any religious ware here BUT just an suggestion:
If some of you guys take look around the corner you'll see Eclipse and you will see that they have a recursive system 
of Encoding inheritance - on all the levels: each file (or dir)  can specify its own encoding OR use the encoding of 
its parent (this is the default behavior). It all goes up to the project level, where the Project itself inherits its 
default encoding from the encoding property of IDE itself. Finally, IDE gets its default encoding from the OS.Simple, 
effective and straight forward - IMHO

Why we can not have the same behavior ?! Is it possible that is so marginal issue for NB developers ? NB is great 
product and i really don't understand this attitude ....
Comment 1 Tomas Zezula 2007-08-29 16:14:05 UTC
The project encoding is not the only encoding you have. Indeed the project encoding is the default encoding of project
and encoding of project's java files. The other files like (JSP, HTML,...) can have other encoding. The reason why all
the java files have to have the same encoding is because the java compiler works in this way. If you create two java
files depending on their self each having different encoding the compiler compiles them in single encoding (one is
compiled in wrong encoding or not compiled at all) even if you start two java compilers with different -encoding option.
Can you describe the case you are solving in more details?
Comment 2 goran 2007-08-30 20:11:08 UTC
Fair enough - I wasn't thinking about limitations of javac.... indeed javac should support the encoding per file not 
per invocation (request probably should be logged against this @ SUN)... But until they do so something, we  could to 
do something for NB

Anyhow, as you have pointed out, apart from javac, there are some other cases: I tried to summarize them bellow (pls 
feel free to add some other)

A) Java source code:
like someone proposed here, we can specify encoding in the Open/Save dialog box (and maybe some others) - once when 
text is in the NB it is UTF-16 encoded anyway. This is, IMHO, a nice compromise for the javac problem - in a case when 
source files are coming in a different encoding. When they need to be included in the project, they have to be 
Open/Save-d  (which is always the case, anyway) ... and that's all - not perfect but it works.

B)
XML, HTML, JSP, CSS and others which have information about their encoding included inside the file (in a form of some 
markups etc) :
It is not mandatory, per their specification, to include markups about their encoding, which means that their 
respective editors in NB have to guess something about the  encoding  used: usually they go for system default one and, 
usually, they are right. But they can be wrong as well ! Having encoding specified in the Open/Save dialog box (like in 
the case "A") can only partially solve the problem in this case. This type of files should actually have the Encoding 
Property on their Property page as a permanent attribute. Editors in question should try to respect that property if 
markup attributes are not specified in the file. However, If they are specified --and-- if that value is different from 
the one specified in the Property page then, the Editors, should ask a user (via some dialog box) in which encoding the 
file needs to be saved 

C)
CVS/SVN
Don't know anything about these - my SVN just works :-) ... but i guess they have their own "encoding" story ?

--------
Anyhow, what is your thought on this ? ...  It seems a lot of work but the problem is that NB is not used only as a 
Java editor but also it serves as an "Editor"/Viewer for many other things. Again, it is really annoying to have such a 
powerful tool like NB and still go for some external tools to do such a trivial thing like an encoding conversion .... 
like a Mercedes without air conditioner ... on a hot day :-)
Comment 3 Milan Kubec 2007-09-03 09:35:37 UTC
Tomasi, isn't this rather enhancement for next release of NetBeans than a regular bug. It needs to be treated on
different level than projects IMO.
Comment 4 Tomas Zezula 2007-09-07 13:10:33 UTC
I've finally got to this, sorry for the later answer. The most things you have described here are done. The NetBeans
encoding API is very flexible, it provides API for clients like editor, refactoring and search and SPI provided by
projects, data types (XML, HTML, ...). Each provider can define priority, for example the XML provider has higher
priority than the project since XML may have stored encoding, if there is no stored encoding the encoding system uses
project encoding when the XML is under some project otherwise it uses system encoding.

So for XML, HTML, JSP you have a chance to override the project encoding by specifying the encoding in the file. As you
pointed the problem is when you import sources stored in some other encoding different from project encoding and system
encoding. Fox example you import XML file without encoding property stored in UTF-16 into project which uses UTF-8. The
"Open In Encoding" action should help in this, you will be able to open the file and possibly add the encoding property
or save it in the original encoding using "Save In Encoding" action. I think that all these problems can be solved by
these two actions. Do you agree?

I've added Jano (a netBeans UI engineer) to comment the  suggested actions. I am not sure if these actions will be
accepted by the UI team, but I will at least try to provide module with these actions on Autoupdate.
Comment 5 goran 2007-09-17 07:21:16 UTC
was off for a week - i.e. this is still important thing (imho) :-)

so: those two actions (Open/Close) seem to be sufficient to solve the problem of Java source files, but for the XML, JSP
etc i think poperty attribute should be specified (because the reasons ive described in the section B of my second post).

Anyhow, if those two Open/Save actions can be implemented it would be already a great progress !

Did you hear from Jano & Co ? Any dates etc ?


Comment 6 Tomas Zezula 2007-09-17 17:25:49 UTC
No UI suggestions yet, I will implement the actions during this weekend as contrib module for now.
Comment 7 Ken Frank 2007-09-19 16:34:41 UTC
I think that rfe for allowing changing of encoding prop of xml and jsp can be good; it already exists for html.
I don't know if that is needed as separate rfe than this one.

ken.frank@sun.com
Comment 8 Tomas Zezula 2007-09-19 19:13:54 UTC
As usually I've forgot to delete INCOMPLETE :-)
Comment 9 Tomas Zezula 2007-10-23 16:35:42 UTC
Created attachment 51510 [details]
[Open|Save] in encoding actions
Comment 10 Tomas Zezula 2007-10-23 16:43:45 UTC
I've created a module with Open In Encoding and Save in encoding actions, the sources are located in the Netbeans CVS
contrib/encoding. The actions are placed in the Files menu when invoked they open file chooser in which you select file
to open (save) and desired encoding. In addition to available encodings there is a special item "Default encoding" which
means the encoding which would be used when the user opens (saves) the file using the ordinary open (save). When you
open (save) a file using open (save) in encoding the encoding is bound to this file, it's remembered until you open
(save) it in the default encoding. So, after opening the file in an encoding the file is always opened in this encoding
until you save it or reopen it in the default encoding.

Comment 11 Ken Frank 2007-10-23 17:02:29 UTC
I installed and used it today.

I think this is good that it requires explicit location for saving the file rather than saving in the project by default;
as well as for opening the file it asks for location rather than assuming the project.
since for java files, the need is to have all java files be in same encoding, and this way, if user
has some files in other encoding, they can open them in their existing encoding, and then save them in the other encoding,
and they can then give path to the project files, but at least it requires them to do explicit steps about it.

ken.frank@sun.com

ken.frank@sun.com
Comment 12 goran 2008-02-07 08:40:33 UTC
so, what would be the "Target Milestone" for this one and who has to set it up ... also who has to change the status 
to "FIXED"?  (I hope you didnt forget it ;-)
Comment 13 Tomas Zezula 2008-02-07 08:50:41 UTC
Hi Goran,
No target milestone yet, the Open|Save in encoding are available only on auto update for dev build 6.1. To resolve this
issue I need to put them either to product or stable auto update. The first may be hard from the UI point of view
(adding new actions to files menu), the seconds seems better.
Have you tried these actions?
Tomas
Comment 14 roti 2009-07-31 09:42:38 UTC
I tried this plugin, and it works great. It remembers my choice to the file, so I don't need to open the file with 
custom encoding every time.

I think two enhancement needed:

1. the file encoding should be changeable in file's properties window
http://www.netbeans.org/issues/show_bug.cgi?id=157817

2. in the projects, the default handling of file encoding should stay, but an option is needed in project properties:
a checkbox in Project/Properties/Sources/Encoding "use this encoding for all project files" to enable that all the 
files in the project be handled with the project's encoding.

In this way all the three cases solved:
1. set encoding per file
2. mixed encoding in a project (with unchecked checkbox)
3. one encoding for all project files (with checked checkbox)

Comment 15 Jiri Rechtacek 2009-07-31 10:27:07 UTC
It's a known RFE reported in version 6.0, please do not move Version field forward.
Comment 16 pribyl 2009-10-06 15:25:33 UTC
*** Issue 168517 has been marked as a duplicate of this issue. ***
Comment 17 johnsonlau 2009-10-09 14:58:58 UTC
I've tried the encoding module for a long time.
Honestly, I don't think it is a convient way to solve the file's encoding.

I agree with jrechtacek it's better to have a property to specify file's encoding.
I finished a patch to do this.
See http://www.netbeans.org/issues/show_bug.cgi?id=174083 for the duplicated issue
or
http://www.netbeans.org/nonav/issues/showattachment.cgi/89104/per-file-encoding.patch
just for the patch itself

A screenshot is attached also.
http://www.netbeans.org/nonav/issues/showattachment.cgi/89103/newly_introduced_prop.png
You can see that a new property named Text Encoding was appended to the platform.
It defaults to the encoding of the whole project, or your OS's default one while no project is avaiable for the file,
for example, when you drag a text file just from Windows Explorer.
In my situation, GBK on a simplified Chinese Windows environment.
And you can easily change it into another valid encoding (invalid value would cause a message dialog to come out),
it will reload the document immediately without closing it and new encoding will be used then.
If the document is modified, it will be closed automatically,
then you must save your existed work after the encoding is changed,
or your new encoding will be dismissed.

I didn't think much about saving the file with a different encoding from which it is opened with.
But you can simply do a copy (in encoding A) & paste (in encoding B) to solve it,
as what I've done many times in Eclipse.
Issues like resetting the per-file encoding to whole project's one are not considered,
but not that difficult to resolve.

I think it is meaningful to provide a property for encoding on each file in the platform,
since only the user knows exactly about the encoding of a file.
We could simply warn the user or address that
the per-file encoding property is only for editing, not for the build.
Maybe someday some build would rely on both per-file encoding and project's encoding,
but clearly it is not now.
Comment 18 johnsonlau 2009-10-26 16:45:46 UTC
Hi.

I've created an FOW page for this issue.
(http://wiki.netbeans.org/TextEncodingFOW)

Is there anyone who would review it or give more advices.
Thanks.
Comment 19 Vitezslav Stejskal 2009-12-01 10:18:13 UTC
Also see issue #19928.
Comment 20 matthies 2010-02-02 12:45:45 UTC
It would be really useful for this to get implemented.
Comment 21 Antonin Nebuzelsky 2010-03-29 14:10:25 UTC
Change of default owner.
Comment 22 Jesse Glick 2010-07-02 19:45:46 UTC
No plans for the standard IDE distro to include functionality to set different encodings on individual files within the project (except for formats such as XML which define an encoding in band). Third-party modules can easily support this using various GUIs if really needed. (Does not make any sense for Java sources since these must use a uniform encoding in order to be passed to javac in batch.)
Comment 23 pulzaraider 2011-09-04 19:41:44 UTC
(In reply to comment #22)
> No plans for the standard IDE distro to include functionality to set different
> encodings on individual files within the project (except for formats such as
> XML which define an encoding in band). Third-party modules can easily support
> this using various GUIs if really needed. (Does not make any sense for Java
> sources since these must use a uniform encoding in order to be passed to javac
> in batch.)

So please add support for different file encodings at least in Netbeans for PHP. 
Every professional PHP IDE have this feature (Zend Studio, Eclipse...), so if you - developers of Netbeans - 
think support for PHP language seriously - it's a MUST-HAVE FEATURE.
Project that have all files in one encoding would be great, but in practice with bigger (and older) projects in a lot of companies it's just an utopia!

It would be great if Netbeans would have feature which is described in http://wiki.netbeans.org/TextEncodingFOW

But if it can't be done, fix at least Encoding Support plugin: why it's available only in dev list of plugins? It's still buggy or not finished? Why can't be improved and available in Netbeans PHP by default?
Comment 24 Jesse Glick 2011-09-06 19:25:27 UTC
(In reply to comment #23)
> please add support for different file encodings at least in Netbeans for PHP

This would be an RFE for php/project.

> fix at least Encoding Support plugin

I do not know about status of this plugin but can file issues in contrib/code for tzezula.