This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 27547 - I18N - Japanese form file is corrupted with internal CVS client?
Summary: I18N - Japanese form file is corrupted with internal CVS client?
Status: VERIFIED FIXED
Alias: None
Product: versioncontrol
Classification: Unclassified
Component: CVS (show other bugs)
Version: -FFJ-
Hardware: PC Windows XP
: P2 blocker (vote)
Assignee: Martin Entlicher
URL:
Keywords: I18N
: 28398 (view as bug list)
Depends on:
Blocks:
 
Reported: 2002-09-24 18:44 UTC by mokada
Modified: 2007-01-04 17:14 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
sorry again, I failed to attach valid encoded text. this file is my sample files. (2.42 KB, application/octet-stream)
2002-09-24 18:48 UTC, mokada
Details
added one JLabel to JFrame.jar files, and checked in by internal CVS client , then checked out to new directory. (2.43 KB, application/octet-stream)
2002-09-24 19:04 UTC, mokada
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mokada 2002-09-24 18:44:31 UTC
Hello,

I'm using Sun ONE Studio 4.0 Community Edition for
developing a GUI application. I handle my all
source code and .form files with CVS
pserver(version 1.11.1), using internal CVS client
support.

But when I checked out my all source tree in new
folder, the .form files seems to be corrupted (IDE
said so). I checked server's files, and found all
.form files in the repository is corrupted.

In the form files on local disk, 
... 
<Component class="javax.swing.JRadioButton"
name="jRadioButton1"> <Properties> <Property
name="text" type="java.lang.String"
value="<E3><83><86><E3><82><B9><E3><83><88><EF><BC><91>"/>
</Properties> 
... 
(representing Japanese character code parts as
<hex> code. )

the 'Property' element's attribute 'value'
contains Japanese text(UTF-8 encoding). 

It seem to be corrupted in CVS server as follows...

<Property name="text" type="java.lang.String"
value="<E3><83>?<82><B9><E3><83><88><EF><BC>?/>

The last double-quote of 'value' attribute is
missing, and some 1 byte of UTF-8 encoded
character is changed to '?'.

I tried using cygwin's cvs.exe as external cvs
client and commited the form again, then the form
file seems to be valid. I think Sun One Studio's
internal CVS client have a bug when commiting form
file which contains Japanese.

I note that my source files also contain Japanese,
but these are Ok with internal CVS client. Sources
are Shift_JIS encoded ( Japanese Windows' default
), but .forms are UTF-8 encoded. I think this
might be an encoding-related issue. 

I'd like to attach my sample form and source file,
but form file and source file have different 
encoding, then simply attaching two in one message
would cause problems. So I attach two files as a
jar archive, encoded by uuencode. Sorry for
inconvenience.

begin 664 JFrame.jar
M4$L#!!0`"``(`'T4.2T````````````````)``0`345402U)3D8O_LH```,`
M4$L'"``````"`````````%!+`P04``@`"`!]%#DM````````````````%```
M`$U%5$$M24Y&+TU!3DE&15-4+DU&(J\TW,RTQ++2[1#4LM*L[,S[-2,-0SX.5R(B
M+DI-+$E-T76J!`F8Z!G$&Q@J:`27YBGX9B87Y1=7%I>DYA8K>.8EZVGR<O%R
M`0!02P<(YN8^`$<```!'````4$L#!!0`"``(`!(2.2T````````````````+
M````2D9R86UE+F9O<FWE5LUNVD`0OO,4UJI'8VPP!"1(E)"BMLH!E58]+WA#
M-C6[:+T0Z(U$JGJIJOZIR@M4N??4*F^#U![S"IU=V(J\0X-$&A(%45,O;,SL[,(B
M](J\UXO-6=4<\WAD0$E+,:<BP;&81UN$=9MX:>/VMDR\C8V<YDJ@TN>BE#.>Z3(B
M&N*B:S$BVP2SP.IQ;^"3P#H$:_U'V2&WGC0$[A'EX3&(:#MC&-76F,DC(FFG
M*7B?"$E)H/0+5L8&@]TUI-RUZ"L2!S[&0VSA$VGMTQYA*B]D#+$_@*5LV36S
M3L6TS:+I..IRX=DIFXY=,BM;()?5S8V$"EPE)964!9C;H+$=T-KA=J5S0KE4
M!,_@+ENHF%G7,56H2E'MS-MFWE375@%N)=CEAEZTQT(H0QR]7`S#ZB5;J_5R
M7CTK+W`W73=Z@(!YE%N"GB8/J-1$I"AJ<LKD7?2X=]%3MD,:-!UAUMDM<%6R
M58IE[:>8!Q2@+Y;,0CE)B*/1S1Z=.;")WU)`51(J\TN4\[XQAJ$I]SIX\N841@(B
M2=*$M3GWH9%GGJ08D*6=)9MS)4=UPB01*[GR2$"[X.U_?V&JN86#1JT(J\'`+-(B
M(J\=#1PB/,/)\(@RBAADXH\_A)W8<>85UD^#20JD`)(K6A]4+;'43+R.AC->Z@(B
M@L$?3'4P9!R%X6J(C*A4TS%..<Y,"0=XS`?2Z/@X"&X;MAW>ZV.9MWQM'UC[
MN@'VN/"("'U$WEN#=AUL.4N@KW,F,87DXS@J[9$5``-=ZTD3,^*CJ+&.M>2@
M<&>X-Y`"=L?>4EHCS*C^%P#$W;J*DP>AD,@:;#J"]O48N$8`&&Y8&AX5I"/U
M%S!Z1W,)S+EKTQD[N3EZ8NUJ16WXT&V)DH:#X&99HT)$RH6E?8H]RO<&4JI9
M$!4XH7/F"4E_JM,+(J\?"19"3GAHZ/(5A+"OT>146<GKV>GGZ?GKVYNGR?I%&1(B
MMBB0HC*"LC2(J\%+*-@OIP&ZCD0G(.S2M3\PCKUFL2H7J#>+=,I%UMN=1$"DW3(B
M$RFF:C<5,@4IG?I]:G2`VXGIHJ4-5>CCNMIN`:3(J\9B!]VB"DPF8@?=X@)'>=(B
MD"9?IY/SZ>1B>OIV.GFW+E3U(])YN<=',V"Q8JTO58SMZO++/SWV9F1M8/`M
M_#BM<T[(J\.O_Q\]O%/1L/#M(+#HZYV<DQ.FS/VU1SZEB[G?D-4$L'".4.#L1M(B
M`P``9Q```%!+`P04``@`"``2$CDM````````````````"P```$I&<F%M92YJ
M879AS59/:^-&%+_[4PPY*6%7<F3'21P*ZWB=D"6XBV/(PF99QM+8GHVD,9J1
M'7?)H?*E"X4LI:&G_1`]++T5NK`L"X5"S_T`[:'0C]"9T9]H;%GVMCU4`B'I
M_>;WWOSFS;QG;)7`%GATY$,7Z2_@&/)/(J\:?I(\B0#8@'S'+9-,K[AEF]!TRS(B
MOK//[4:I9&Q)X`S-GLP&X?OPK?AZ``,V)#X`+KF$-I3(4=!SL`4L!U(:>P+H
MBB'/ID!XO-+I!'L#/3:]+`%^R0=W$0="@8<FH$](J\-V'@Q`(2DT<_M<UXM+BP(B
MAUF3N"/B(8]1;?-`6JY5]NX04^`B'K,-^)L%'8=/NN(J\3%TPP&V(/L"$"%O$H(B
M(J\P.+\9DQ$GG8D@XP=/`72()$<'IB.V]TVB?MXSIX2$#[\RZ7P\;]*<<))\1&(B
M.O<L>;D,#)!^9$D#26B@,X%3"GPT0![RY8+TIM+;D9"B96,>4N(T%L3'8PX$
M8X+M!0W`2(J\,X;K7O'[:.3]IUU9H*]^(Q])"S#3Z3FBLK)"V)E!+;@38FAP%C(B
MQ,L=D;$KXPJ&Y*!/86])1-*RB#4_`5OY!&QU/6QSB*S+0W*5&W)BS-$C-^P[
M/5(XM.US[-ED<HHI$ZFA):-T.&$Z&O,%U2-$PX8CQ@'9K9'9.#)-)A+9=`CE
M#K5<FI9X!VC,YGG$A:XP$PFI"?N!8KY.OZZS$U#R1J>(=?F)H&U<!)6R51//
MWKY(J\W[L(^OWM[8VL5%%ZZEP#3:%1^->E-C=RLE)P-RR&B5>D[QRB2&`HH8^1(B
M+PX)9,]+'#$52IR$UIAC6E_QK&QYBD5[;*5@E:5K$1$L<IHK.:O%G.8B9V4E
MYTXQ9V61LZIRUB3GCN2L[8GGGBV>U7XQ<U5A3D^"8G(>(J\.Y&[@GR?\S%-+C_(B
M)AM3NIP=K"3/;FT77@0U"^TM78)X5)9I@'B5DX56`+7-"!@-NG<GY"'Q;>2?
MPBD)F-YLM;NM3I9E!*W+M(F(JFBK_7"^ABZ6X.5BK10^(J\G)TTCGKUB7F^5*R(B
M-$S#`#=???=FUOGRZ6P8_A5^S^^/WWQX]5/X-OPM_*&>C?^TL09QSHR6'$7_
M9#[Y5/]Z-@6T:>O7XD5+ME*-T8AO#BB@N4U46MU6%L;%"29CTQF=3?D6=77Q
M7RNKZ929@#(LB3AI"A^,(&]V`?0'-.Y.71=Z-G"PA(J\3?P+UKY]0NF3(^R_@8(B
M<"'VM#/F(J\XHOJ9X^RVYU<<0D/;5.AV22TS^7XA6Z_>7UMZ__^/I'<!_<_GIS(B
M?DM?O<MVF6/H8]AS$%6476PKDWU(J\L!P7+6MZ-*R-+.),$C^S40O0F7Y#[6$*(B
MQLBJD-28-7&5-7%%&F1Q<7R9U0I_OCD)_PQ_OSO*U(4J79?^!E!+!P@Q^$2)
M``0``!P.``!02P$"%``4``@`"`!]%#DM``````(`````````"0`$````````
M````````````345402U)3D8O_LH``%!+`0(4`!0`"``(`'T4.2WFYCX`1P``
M`$<````4`````````````````#T```!-151!+4E.1B]-04Y)1D535"Y-1E!+
M`0(4`!0`"``(`!(2.2WE#@[$;0,``&<0```+`````````````````,8```!*
M1G)A;64N9F]R;5!+`0(4`!0`"``(`!(2.2TQ^$2)``0``!P.```+````````
L`````````&P$``!*1G)A;64N:F%V85!+!08`````!``$`.(J\```"E"```````(B
`
end
Comment 1 mokada 2002-09-24 18:48:24 UTC
Created attachment 7499 [details]
sorry again, I failed to attach valid encoded text. this file is my sample files.
Comment 2 mokada 2002-09-24 19:04:07 UTC
Created attachment 7500 [details]
added one JLabel to JFrame.jar files, and checked in by internal CVS client , then checked out to new directory.
Comment 3 Jesse Glick 2002-12-23 16:36:41 UTC
Consistent use of the I18N keyword.
Comment 4 Martin Entlicher 2003-02-20 13:21:34 UTC
*** Issue 28398 has been marked as a duplicate of this issue. ***
Comment 5 Martin Entlicher 2003-02-24 16:57:11 UTC
Maybe this can be fixed by the fix of issue #11470.

One more I18N fix applied:
/cvs/javacvs/libsrc/org/netbeans/lib/cvsclient/command/PipedFileInformation.java,v 
<--  PipedFileInformation.java
new revision: 1.4; previous revision: 1.3

I will test it a little more with czech characters and will resolve
this if it works.
Comment 6 Martin Entlicher 2003-02-25 12:34:30 UTC
There're still problems with the corrupted form due to a different
encoding (some characters are corrupted). I'm investigating a possible
solution....
Comment 7 Martin Entlicher 2003-02-26 11:15:39 UTC
The problem was, that javacvs could potentionally corrupt any file,
that was written in an encoding different from the current encoding.

Readers and Writers must NOT be used to read and write files, because
they convert bytes to characters according to the current encoding and
thus can corrupt characters from different encodings.

Fixed in the main trunk by usage of byte IO via InputStream and
OutputStream:

/cvs/javacvs/libsrc/org/netbeans/lib/cvsclient/file/DefaultFileHandler.java,v 
<--  DefaultFileHandler.java
new revision: 1.9; previous revision: 1.8
/cvs/javacvs/libsrc/org/netbeans/lib/cvsclient/file/DefaultTransmitTextFilePreprocessor.java,v 
<--  DefaultTransmitTextFilePreprocessor.java
new revision: 1.6; previous revision: 1.5
/cvs/javacvs/libsrc/org/netbeans/lib/cvsclient/file/DefaultWriteTextFilePreprocessor.java,v 
<--  DefaultWriteTextFilePreprocessor.java
new revision: 1.6; previous revision: 1.5
/cvs/javacvs/libsrc/org/netbeans/lib/cvsclient/file/WriteRcsDiffFilePreprocessor.java,v 
<--  WriteRcsDiffFilePreprocessor.java
new revision: 1.7; previous revision: 1.6
/cvs/javacvs/libsrc/org/netbeans/lib/cvsclient/file/WriteTextFilePreprocessor.java,v 
<--  WriteTextFilePreprocessor.java
new revision: 1.4; previous revision: 1.3
Comment 8 Martin Entlicher 2003-02-26 11:22:50 UTC
Please test this intensively so that we can be sure that
commit/update/checkout works correctly. Please try to do various
merges and diffs on both UNIX and Windows platform and with different
encodings of the source files.
Then please verify that the result of checkout/update/merge is really
exactly the same as the committed source.
Thanks.

Comment 9 Jiri Kovalsky 2003-02-26 11:29:50 UTC
Can you Mokada confirm this fix resolved your issue ? Ken, could you
ask somebody from your team to help with this ? Thanks.
Dane, I think regression testing is great task for you. Will you take
care of this ?
Comment 10 Martin Entlicher 2003-02-27 11:46:15 UTC
Just a test comment, please ignore.
Comment 11 dmladek 2003-02-27 11:52:28 UTC
I tested it hardly with Czech locales on my linux.
I didn't meet any problems during it.
I could say that it is finaly fixed...
Comment 12 mokada 2003-02-27 16:19:54 UTC
I'm sorry for slow reply.

I tried fixed IDE in Japanese locale, and found the problem is solved!
My form files were kept sane after committed by IDE's built-in cvs
client. Thank you for improvements.