This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 166597 - Properties-Sources-Encoding sets errorneous default encoding
Summary: Properties-Sources-Encoding sets errorneous default encoding
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Project (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P3 blocker (vote)
Assignee: Tomas Zezula
URL:
Keywords: I18N, RELNOTE
: 173192 (view as bug list)
Depends on: 173398
Blocks: 24668
  Show dependency tree
 
Reported: 2009-06-05 11:48 UTC by jiai
Modified: 2009-12-18 09:48 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jiai 2009-06-05 11:48:28 UTC
Steps To Reproduce:

1. Open a new project (Windows has in my case cp1252 encoding)

2. Set Properties-Sources-Encoding: UTF-8

3. Create a new Java file, call
System.out.println(Charset.defaultCharset().toString());

4. The API-DOC for Charset.defaultCharset() says:
The default charset is determined during virtual-machine startup and typically
DEPENDS UPON the locale and charset of the underlying operating system. 

5. The Output of this call gives you on Netbeans 6.7 RC1: UTF-8!

On Netbeans 6.5.1 you'll get: windows-1252!

This is a serious bug in my eyes, because every call to external resources
(Textfiles, Excel tables, database access) needs to be refactored for legacy
projects which are using UTF-8 encoding (e.g. Web-Applications).

It is a wrong assumption that the encoding of the source
.java file implies something about the encoding of the files read/written by
the program inside the .java file!

The same situation appeared in Eclipse when version changed from 3.3 to 3.4, see discussion there:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=239593 

Produktversion: NetBeans IDE 6.7 RC1 (Build 200905312038)
Java: 1.6.0_14; Java HotSpot(TM) Client VM 14.0-b16
System: Windows XP Version 5.1 läuft auf x86; Cp1252; de_DE (nb)
Comment 1 Jesse Glick 2009-09-09 15:45:49 UTC
As of issue #24668, running the application uses the same encoding as the source project, which is useful for display of
output in the Output Window, and what people usually expect will happen when "Encoding" is changed for the project.

Obviously well-written programs do not make assumptions about the default charset when opening files on disk but will
specify an explicit encoding when converting bytes to characters or vice-versa. New code should generally use UTF-8 for
all text files, though legacy formats may specify other charsets or even embed the charset to be used elsewhere in the
file, so this depends very much on the application.

You could try passing -Dfile.encoding=cp1252 in the program's run arguments if you wish to override the default charset
at runtime but do not wish to change the project's source encoding. The IDE will not "know" about this encoding so
non-ASCII text printed to stdio will not be correctly displayed in the Output Window, but other streams will be fine.

I recommend WONTFIX.
Comment 2 jiai 2009-09-28 14:21:19 UTC
The same problem will appear on data which is retrieved using the JDBC-ODBC-Bridge (sure, you shouldn't use it for "real
databases") but it is often the only possibility when you are importing data from Access or (without POI) from Excel. In
this cases a -Dfile.encoding=cp1252 is not a workaround.
Comment 3 Jesse Glick 2009-09-29 18:23:21 UTC
*** Issue 173192 has been marked as a duplicate of this issue. ***
Comment 4 Tomas Zezula 2009-09-30 14:16:57 UTC
I've added runtime.encoding property which can be added into private.properties to override the source.encoding.
The runtime.encoding property requires documentation in NB 6.8.
For next release we should add an UI for setting this property and probably change the output window not to use FEQ but a runtime.encoding (requires a new 
API). I've created an enhancement #173398.

jet-main 727f5f6fff58
Comment 5 Jesse Glick 2009-09-30 15:32:38 UTC
This fix will not work for projects with CoS enabled. You need to also patch java.source.ant to interpret such a
property if defined. Of course this ought to become a documented API constant at some point, so file an API DEFECT for that.
Comment 6 Tomas Zezula 2009-09-30 15:48:57 UTC
>This fix will not work for projects with CoS enabled.
Try it :-)
See the changes in the J2SEActionProvider. The j.s.ant.bridge already accepts the runtime.encoding.
Comment 7 Jesse Glick 2009-09-30 15:55:41 UTC
I see, sorry. So this just needs to be documented (like a lot of things in JavaRunner).
Comment 8 Tomas Zezula 2009-09-30 16:03:29 UTC
Right, I will make the constant public and add doc + API change. I've filled a task #173406 for it.
Comment 9 Quality Engineering 2009-10-01 17:26:19 UTC
Integrated into 'main-golden', will be available in build *200910010513* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/727f5f6fff58
User: Tomas Zezula <tzezula@netbeans.org>
Log: #166597:Properties-Sources-Encoding sets errorneous default encoding
Comment 10 Masaki Katakai 2009-12-13 21:11:56 UTC
Hi Tomas and Jesse,

I still don't understand well why platform encoding isn't used and source encoding is used instead. The major issue is that the behavior of user application will be changed when it started from NetBeans and when it starts outside NetBeans.

Actually I got a report from community folks in local mailing list:

 - when I tried my app by starting "java -jar .../JavaApplication.jar",
   it works!
 - but when I start it from NetBeans, it doesn't work! Why?

I agree with Jesse's point,

> Obviously well-written programs do not make assumptions about the default
> charset when opening files on disk but will

However, actually it's not easy for Java beginners. It's not easy for most users to understand the current NetBeans behavior - why the differences happen. Because adding -Dfile.encoding is not usual.

I think adding -Dfile.encoding="source encoding" should be optional. The default behavior should start without -Dfile.encoding. It's natural and we could provide the exact same encoding of OS to user application. If someone wants to change it (e.g. for their output window), he/she can put runtime.encoding into runtime.encoding.
Comment 11 Jesse Glick 2009-12-14 08:46:36 UTC
See bug #173398 regarding a UI to customize the runtime encoding. We felt that the highest priority, especially for new users, was making sure that

public class Main {
    public static void main(String[] args) {
        System.out.println("Що राजः");
    }
}

would display in the Output Window what you saw in the editor. If we defaulted to platform encoding, you would in many cases see

?? ?????

or similar.
Comment 12 Tomas Zezula 2009-12-15 01:42:52 UTC
Hi Masaki,
in the NB 6.8 there is a runtime.encoding property (no UI, only in the properties file) by which you can enforce the runtime encoding. I agree that the non UI properties are not suitable for most of the users.
So the issue #173398 has to be solved.
Comment 13 Masaki Katakai 2009-12-15 17:11:23 UTC
Thank you Jesse and Tomas,

I see. Could you add this topic to user FAQ? I'll translate it to Japanese so that Japanese users can understand the workaround. Because there is another issue other than file encoding and it's serious for Swing application in Japanese environment.

Currently Swing application with -Dfile.encoding=UTF-8 on Windows platform is causing bad font rendering. Chinese fonts are being used instead of Japanese. UI fonts are changed when we add -Dfile.encoding=UTF-8.

I can not say it's JDK bug because JDK does not have much assumption of the usage for Japanese locale. I already have filed an RFE and it will be fixed in u18.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6856390

We still need to use u17 now. So the workaround in FAQ is necessary.
Comment 14 Tomas Zezula 2009-12-16 13:23:46 UTC
David, can you point someone from doc team to add this into documentation.
Thanks
Comment 15 David Strupl 2009-12-17 06:27:18 UTC
I have put Irina on Cc: and I am sending her mail right now.
Comment 16 Irina Filippova 2009-12-18 02:34:17 UTC
I suggest adding a note to the existing FAQ entry: 
http://wiki.netbeans.org/FaqI18nProjectEncoding

Masaki, Tomas, Jesse,
Can you please review this text? It should be added to http://wiki.netbeans.org/FaqI18nProjectEncoding under the passage - "The encoding affects at least: ....."

Starting from NetBeans IDE 6.8, you can also specify the encoding that will be used at runtime. For example, this can be useful when the encoding for the operating system on which the application will run is different from your project's encoding. 

To specify the encoding to be used at runtime:
1. In the Files window for your project, open nbproject > private > private.properties
2. Type runtime.encoding=<encoding> and save the file.
This encoding will override the encoding setting for your project and will be used when running your application.
Comment 17 Jesse Glick 2009-12-18 09:48:12 UTC
FAQ addition sounds fine. I might replace "Type..." with "Add a line..." for clarity.