This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 108562 - I18N - apply diff patch corrupts multibyte
Summary: I18N - apply diff patch corrupts multibyte
Status: RESOLVED FIXED
Alias: None
Product: utilities
Classification: Unclassified
Component: Diff (show other bugs)
Version: 6.x
Hardware: All All
: P3 blocker (vote)
Assignee: diff-issues@utilities
URL:
Keywords: I18N
Depends on:
Blocks:
 
Reported: 2007-06-30 04:19 UTC by Ken Frank
Modified: 2008-02-18 18:06 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
image (1.46 MB, image/gif)
2007-06-30 04:31 UTC, Ken Frank
Details
image (16.88 KB, image/gif)
2007-06-30 04:31 UTC, Ken Frank
Details
image (9.18 KB, image/gif)
2007-10-11 18:38 UTC, Ken Frank
Details
image (32.52 KB, image/gif)
2007-10-11 18:38 UTC, Ken Frank
Details
image (19.65 KB, image/gif)
2007-10-11 18:39 UTC, Ken Frank
Details
image (20.92 KB, image/gif)
2007-10-11 18:39 UTC, Ken Frank
Details
file (798 bytes, text/plain)
2007-10-11 18:40 UTC, Ken Frank
Details
file (1.49 KB, text/plain)
2007-10-11 18:41 UTC, Ken Frank
Details
image (6.53 KB, image/gif)
2007-10-11 18:42 UTC, Ken Frank
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ken Frank 2007-06-30 04:19:13 UTC
seen using cvs but it might happen for subversion also

running in solaris ja locale with pseudo localized nb, although this is not about messages to user.

this happens when project is using default project encoding of utf-8 or changed to the encoding
of the ja solaris locale - euc-jp

it might be that its not the applying of the patch that corrupts the multibyte but that
the patch file is not written in correct encoding.

steps:

A.  follow instructions from test plan, creating a j2se project,  and change encoding to euc-jp and create project with
the name of a class 
with multibyte in it and also then add multibyte variable name.


#  Creation of patch

   1. Modify "Main.java" file and save it.
   2. Modify "NewClass.java" file and save it.
   3. Select project root and invoke Versioning | Export Diff Patch... from main menu. Select desired output file for
patch and confirm the dialog. 

    * EXPECTED RESULT: Patch should be created in your chosen file. If this action is invoked on context that has no
modified files, information dialog should be displayed only. 

# Applying patch

   1. Switch to "Files" view and select project root.
   2. Invoke Versioning | Revert Modifactions from main menu or CVS | Revert Modifactions from popup menu.
   3. Make sure that all modified files are "uptodated" now.
   4. Select project root and invoke Tools | Apply Diff Patch... popup menu action.
   5. Locate and select previously exported patch and press the "Patch" button.
   6. Answer "Yes" in question dialog to confirm watching differences. 

    * EXPECTED RESULT: Verify that new tabs were opened into the editor for all patched files. Verify results generated
by "Command-line Diff" and "Built-in Diff" engines. Verify results displayed by "Graphical Diff Viewer" and "Textual
Diff Viewer" visualizers. 

B. when the patch is applied, and the file opened in editor, the multibyte is not correct - either shown with incorrect
looking multibyte
or question marks - see attachement

C.  some diff viewers show incorrect multibyte and some show correct - this might be related to separate diffs i18n issue.

D. the patchfile itself does not look correct - so it might be that its not written using correct encoding handling

see patchfile attachment - which is from editor running in ja locale.

1. the words near the message about Locally Modified is from a bundle message - and the  multibyte
as part of that pseudo localized message is not correct - but this part is probably ok since patch file says its using utf8
and am viewing it in euc jp solaris ja locale - so this might be ok

2. the multibyte for the integer defined in NewClass, is not correct also and this is a problem since it does not show ok
in the editor.


E. change project encoding to utf-8, which is default for a new project
and follow same steps as above.

1. results are slightly different but not correct -

a. the patch file has correct multibyte of the messages from the bundle file,
but not of the variable with multibyte

b. after patch is made,  you can see in attachment that original multibyte is correct in package name
and in original variable, but that the patched/added variable with ??? is not correct - so even though
patch file says its using utf-8, its does not seem like its being handled that way in all steps.

this attachment is utf8patch.


F. could all cvs/subversion actions from menu be reviewed to see if encoding handling is ok - since there is new 
project encoding property for some projects, but not all projects will implement it for nb6, so assumptions
of encoding sometimes will be based on global encoding and other times use that of the system locale
(previous nb used system locale as project encoding and that of java file unless it was changed via a property,
which is not present anymore.
Comment 1 Ken Frank 2007-06-30 04:31:09 UTC
Created attachment 44590 [details]
image
Comment 2 Ken Frank 2007-06-30 04:31:41 UTC
Created attachment 44591 [details]
image
Comment 3 Maros Sandor 2007-07-31 10:03:29 UTC
I removed our own encoding handler and rewritten everything to FileEncodingQuery API. This should resolve diff/patch
encoding issues.

/shared/data/ccvs/repository/subversion/main/src/org/netbeans/modules/subversion/ui/diff/DiffStreamSource.java,v  <-- 
DiffStreamSource.java
new revision: 1.18; previous revision: 1.17
/shared/data/ccvs/repository/diff/src/org/netbeans/modules/diff/EncodedReaderFactory.java,v  <--  EncodedReaderFactory.java
new revision: delete; previous revision: 1.14
/shared/data/ccvs/repository/diff/src/org/netbeans/modules/diff/DiffAction.java,v  <--  DiffAction.java
new revision: 1.36; previous revision: 1.35
/shared/data/ccvs/repository/versioncontrol/mercurial/src/org/netbeans/modules/mercurial/ui/update/Attic/ResolveConflictsExecutor.java,v
 <--  ResolveConflictsExecutor.java
new revision: 1.1.2.3; previous revision: 1.1.2.2
/shared/data/ccvs/repository/versioncontrol/util/src/org/netbeans/modules/versioning/util/Utils.java,v  <--  Utils.java
new revision: 1.7; previous revision: 1.6
/shared/data/ccvs/repository/javacvs/cvsmodule/src/org/netbeans/modules/versioning/system/cvss/ui/actions/diff/DiffStreamSource.java,v
 <--  DiffStreamSource.java
new revision: 1.32; previous revision: 1.31
/shared/data/ccvs/repository/javacvs/cvsmodule/src/org/netbeans/modules/versioning/system/cvss/ui/actions/diff/ResolveConflictsExecutor.java,v
 <--  ResolveConflictsExecutor.java
new revision: 1.9; previous revision: 1.8
/shared/data/ccvs/repository/subversion/main/src/org/netbeans/modules/subversion/ui/update/ResolveConflictsExecutor.java,v
 <--  ResolveConflictsExecutor.java
new revision: 1.13; previous revision: 1.12
/shared/data/ccvs/repository/versioncontrol/mercurial/src/org/netbeans/modules/mercurial/ui/diff/Attic/DiffStreamSource.java,v
 <--  DiffStreamSource.java
new revision: 1.1.2.3; previous revision: 1.1.2.2
/shared/data/ccvs/repository/diff/src/org/netbeans/api/diff/StreamSource.java,v  <--  StreamSource.java
new revision: 1.9; previous revision: 1.8
Comment 4 Ken Frank 2007-10-11 04:30:09 UTC
am reopening; most of the problems are still present;
I will be clarifying details and adding new gifs tomorrow

but problems are mostly the same as  in original description below.

ken.frank@sun.com
Comment 5 Ken Frank 2007-10-11 18:38:07 UTC
Created attachment 50712 [details]
image
Comment 6 Ken Frank 2007-10-11 18:38:46 UTC
Created attachment 50713 [details]
image
Comment 7 Ken Frank 2007-10-11 18:39:13 UTC
Created attachment 50714 [details]
image
Comment 8 Ken Frank 2007-10-11 18:39:39 UTC
Created attachment 50715 [details]
image
Comment 9 Ken Frank 2007-10-11 18:40:50 UTC
Created attachment 50716 [details]
file
Comment 10 Ken Frank 2007-10-11 18:41:32 UTC
Created attachment 50717 [details]
file
Comment 11 Ken Frank 2007-10-11 18:42:51 UTC
Created attachment 50718 [details]
image
Comment 12 Ken Frank 2007-10-11 18:52:41 UTC
as mentioned before, most problems originally described are still present

here is info on each new gif or file:

1. eucmain.gif - the correct view of multibyte in the java file in project

2. eucpatch.gif - incorect multibyte as shown by this module in editor, both as name of variable in class file a
and in the path and name of the project.

3. eucpatchdiff.ok.gif - the actual diff, in this view, looks ok - ie built in diff engine and graphical diff viewer
and other diff engine/viewer combos ok except the textual - see next gif

4. eucpatchdiff.textual.notok.gif - this shows incorrect multibyte, similar to item 2. above

5. eucpatchfile.notcorrect.gif- this is a view of the patchfile itself from a terminal, taken with a snapshot tool,
from when project was with euc encoding prop of project, in case the attached actual patch file cannot be seen
ok. the multibyte is not correct.

6. eucpatchfile.patch and utfpatchfile.patch - the actual patch file - generated each when project encoding
prop was euc-jp or utf8 ( 2 sepraate projects).  User was running in ja solaris locale for each.

when viewing the utf-8 one from commandline terminal in ja utf-8 locale, the mbyte looks ok
but when viewing the euc one from terminal in ja locale (not same as ja utf8 locale) the mbyte
is not correct.

ken.frank@sun.com
Comment 13 Maros Sandor 2007-10-12 09:34:47 UTC
Does the patch apply correctly in NetBeans?
Comment 14 Ken Frank 2007-10-12 17:39:37 UTC
please tell me steps on how to see if patch applies ok in netbeans, and perhaps some
code to use to see if just having some commented text is not enough (as I have done in this case)

ken.frank@sun.com
Comment 15 Maros Sandor 2007-10-15 09:34:47 UTC
Tools | Apply Diff Patch...
Comment 16 Ken Frank 2007-10-15 16:08:33 UTC
Maros,

In asking question to see if patch applies ok, I was not asking about how to invoke it;
(apply diff patch) but what was the expected results, and if there is some simple
sample that can help us understand what is expected ?

ken.frank@sun.com
Comment 17 Maros Sandor 2007-10-16 08:16:04 UTC
Modify a file, Export Diff Patch for it and back up the modified file. Now roll back your local changes (restore
original file) and try to apply generated patch to the original file. If the current (patched) and backed up file are
equal then the patch applied correctly.
Comment 18 Maros Sandor 2007-10-17 17:53:12 UTC
The merge problem has been fixed and other described issues do not have direct impact on the diff/patch functionality.
Comment 19 Masaki Katakai 2007-12-12 02:16:54 UTC
I tried the simple scenario - it almost works on NetBeans 6.0 FCS.
There is one problem that opening diff file on NetBeans editor becomes
garbled when it contains multibytes. 

http://www.netbeans.org/nonav/issues/showattachment.cgi/50713/eucpatch.gif

Project encoding is UTF-8, so the patch is generated in UTF-8. It's correct.
But the patch file is existing outside project so it seems that it's opened
by using native (EUC) encoding.

I think it's better to file new issue.
Comment 20 Maros Sandor 2008-02-18 18:06:05 UTC
Added FEQ implementation that takes care of diff and patch files generated by the IDE. All such files should now open
correctly in the editor.
ca94f0cc2cb3