This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 156729 - I18N - save as does not save in correct encoding and writes string incorrectly into sql editor
Summary: I18N - save as does not save in correct encoding and writes string incorrectl...
Status: RESOLVED WONTFIX
Alias: None
Product: db
Classification: Unclassified
Component: SQL Editor (show other bugs)
Version: 6.x
Hardware: Sun All
: P2 blocker (vote)
Assignee: Jiri Rechtacek
URL:
Keywords: I18N
Depends on:
Blocks:
 
Reported: 2009-01-14 02:25 UTC by Ken Frank
Modified: 2009-03-25 16:12 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
image (2.92 KB, image/gif)
2009-01-14 02:39 UTC, Ken Frank
Details
image (3.29 KB, image/gif)
2009-01-14 02:40 UTC, Ken Frank
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ken Frank 2009-01-14 02:25:11 UTC
running pseudo localized, solaris, ja locale

create javadb table (you don't need to use  mbyte in db name and column name - problem happens just with
data - table will have insert statement of varchar that will have multibyte - see first gif
for how that command looks in sql editor.

choose save as and choose file name - file is saved and also the file contents i think
are written into editor window - see second gif - the mbyte is not correct in this gif,
compare with first gif and with history window.

also in terminal in ja locale, the mbyte not look ok.
(ja locale uses euc-jp encoding.

being in ja utf8 solaris locale instead, and doing same steps, I think problem does not happen
but will leave it for developers to look at more closely.

am guessing here only that perhaps encoding of dbase or utf8 encoding is used
instead of that of locale user is in (unless db module uses utf-8 default project encoding
which in this case not correct since not associated with a project).

again, there is nothing about using mbyte in name of table or column here - that is not needed
to show this and is not part of this at all.
Comment 1 Ken Frank 2009-01-14 02:39:37 UTC
Created attachment 75800 [details]
image
Comment 2 Ken Frank 2009-01-14 02:40:05 UTC
Created attachment 75801 [details]
image
Comment 3 John Baker 2009-01-14 04:51:52 UTC
Is the result the same with "Save" ?
Comment 4 Ken Frank 2009-01-15 03:26:19 UTC
John,

I think this might need to be incomlete for now, but can you try on windows in
ja reg settings with some ja chars as data, as i have done on solaris ?

I'm getting problem sometimes, sometimes no, sometimes on first invocation when
using new userdir (I don't know if userdir gets info from db written to it or not);

it might be also mbyte in name of table and column, which is legal for java db,
vs just in data from insert statement.

one thing also is not clear is how to use this feature, since sometimes it seems
that if the statement has not been executed, it saves the previous statement that had,
I dont think this is related to locale but just about is that expected behavior ?

ken.frank@sun.com
Comment 5 John Baker 2009-01-15 19:32:54 UTC
The correct encoding is not saved when I do the following:

1) Execute Query

e.g.
select * from APP.CUSTOMER where NAME = "фи"
(enter some other locale for the value)

2) Save As

value is removed


I'm not sure if this is db or platform yet
Comment 6 John Baker 2009-01-15 21:12:18 UTC
This appears to be in platform not db.

If you create a Java application in the userdir then the locale is not saved

1) Create a new Java application project in $userdir/JavaApplication1
2) In Main.java insert some characters of another locale
3) Choose File->Save As and save Main.java in the new default user HOME directory

Result:
The characters in another locale are saved as garbage characters in Main.java 
Comment 7 Ken Frank 2009-01-15 21:37:42 UTC
I'm not expert in underlying code but is scenario about saving java file itself
(vs the original db save as) - is it a valid one

since its expected users would use characters valid for the encoding their
project is in (might not be same as locale they start nb in and thats ok,
assuming system fonts can display the characters.

but if insert chars of another locale, assuming those chars are in different
encoding, then I don't think those chars can be saved ok, since there is just
once encoding a file can be saved in, but this case seems there could be
multiple encodings in the file.

whether save as uses current global project encoding or encoding of the locale user
is in, could make a difference in basic case of all chars in file in same encoding,
which is needed, but if different encodings in the file, save as can't handle that case.

there is no project encoding associated per se with db operations as
in this issue, so am guessing db operations uses encoding of locale user is in,
or does it use the current view of global project encoding ?

thats what I thought might be happening when filed issue, that perhaps just the wrong
encoding assumption was being used.

but as mentioned in report, then had problems replicating it regularly, and seemed
perhaps something in uers dir had impact or it happened just once in a session with new userdir.



ken.frank@sun.com
Comment 8 Stanislav Aubrecht 2009-01-16 11:30:24 UTC
jbaker, the scenario with java files you're describing is correct behavior. the default 'save as' implementation creates
an exact copy of the original file. but because it isn't saved in any project folder default os encoding will be used to
interpret multi-byte characters. moving saved-as file back to original project folder and opening it in editor will pick
encoding defined in the project and display multi-byte chars correctly.
in fact, the same might apply for db queries saved to user's home dir. please evaluate again.
Comment 9 John Baker 2009-01-16 18:35:27 UTC
Thanks for the explanation.
SQL Editor files haven't in previous releases been saved in a project.
However, in 7.0, now that SQL templates are supported the project encoding should be used if it isn't

Comment 10 John Baker 2009-01-16 18:39:20 UTC
This use case works is expected:

1) With OS default character set to en, from a db connection, choose Execute Query

e.g.
select * from APP.CUSTOMER where NAME = "фи"
(enter some other locale for the value)

2) Save As

Result:  The value in the select statement will appear as garbage characters since the default OS character set is assumed.
Comment 11 John Baker 2009-01-16 18:59:29 UTC
1) Set OS character set to en

2) Create a new Java Application e.g.
select * from APP.CUSTOMER where NAME = "фи"
(where the value uses ja or other non-en char set)

2) Choose Save As and save the file with a different name in the same folder

Result:
The value in the select statement should retain the ja characters
(as long as the file is saved in the project folders)
Comment 12 John Baker 2009-01-16 19:02:33 UTC
(previous comment was a copy/paste mistake)

1) Set OS character set to en

2) Create a new Java Application then from the project folder, create a new SQL template
and enter the following statement:

select * from APP.CUSTOMER where NAME = "фи"
(where the value uses ja or other non-en char set)

2) Choose Save As and save the file with a different name in the same folder

Result:
The value in the select statement should retain the ja characters
(as long as the file is saved in the project folders)
Comment 13 Ken Frank 2009-01-16 19:21:48 UTC
John,

i'm not sure the last 2 cases are valid in some cases, at least for nb, since if
by en character set means en regional setting or locale that is not a utf-8 one,
then the characters might not be viewable ok in nb in any case, unless its related
to project that has/uses that encoding, and assuming os font paths are set ok.

or it might even be at os level that if in some en locale, the characters of other encoding
might not show ok, and that could be expected.

but using db features not necesarily tied to using a project anyway


as to the save as spec in general, can save as location be anywhere, not just in a project dir ?

ken.frank@sun.com
Comment 14 John Baker 2009-01-16 23:03:34 UTC
> i'm not sure the last 2 cases are valid in some cases, at least for nb, since if
> by en character set means en regional setting or locale that is not a utf-8 one,
> then the characters might not be viewable ok in nb in any case, unless its related
> to project that has/uses that encoding, and assuming os font paths are set ok.
> or it might even be at os level that if in some en locale, the characters of other encoding
> might not show ok, and that could be expected.

makes sense.
note, the use case in desc12 should be disregarded, it was a mistake

> but using db features not necesarily tied to using a project anyway
at least not planned for the next release

> as to the save as spec in general, can save as location be anywhere, not just in a project dir ?

yes it can.

As I understand what saubrecht says is the encoding by the project determines the encoding used by files in the project.
If files are not part of the project then the encoding by the OS determines the encoding of the file.
So, the behavior is expected as described in desc11

Comment 15 Jiri Rechtacek 2009-03-25 16:12:01 UTC
In accordance with saubrecht comment, it's behavior as design in generally speaking.