This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 52801 - I18N - multibyte from html not displayed properly in new project/file wizard description area in ja locale
Summary: I18N - multibyte from html not displayed properly in new project/file wizard ...
Status: VERIFIED FIXED
Alias: None
Product: projects
Classification: Unclassified
Component: Generic Projects UI (show other bugs)
Version: 4.x
Hardware: All All
: P1 blocker (vote)
Assignee: Jiri Rechtacek
URL:
Keywords: I18N
Depends on:
Blocks:
 
Reported: 2004-12-24 08:37 UTC by hatakyon
Modified: 2007-05-29 18:47 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
screenshot on japanese solaris (54.30 KB, image/png)
2004-12-24 09:05 UTC, hatakyon
Details
binary patch to <nb-home>/ide4/modules (526.48 KB, application/octet-stream)
2005-01-06 11:11 UTC, Jiri Rechtacek
Details
screenshot with patch (61.91 KB, image/png)
2005-01-07 01:34 UTC, hatakyon
Details
patch #2 (526.74 KB, application/octet-stream)
2005-01-07 15:47 UTC, Jiri Rechtacek
Details
patch #3 (520.50 KB, application/octet-stream)
2005-01-09 16:47 UTC, Jiri Rechtacek
Details
pre-read charset from file (527.54 KB, application/octet-stream)
2005-01-10 15:39 UTC, Jiri Rechtacek
Details
works for ISO-2022-JP and other (not with EUC-JP) (526.58 KB, application/octet-stream)
2005-01-10 15:45 UTC, Jiri Rechtacek
Details
proposed fix candidate (525.28 KB, application/octet-stream)
2005-01-11 12:50 UTC, Jiri Rechtacek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description hatakyon 2004-12-24 08:37:58 UTC
Descriptions for New Project Wizard is i18ned 
by .html format.
e.g. description for "NewProject -> Web ->
WebApplication" is i18ned as:

web/project/src/org/netbeans/modules/web/project/ui/resources/emptyProject.html

When I translated it to Japanese and placed
<japanese>.jar file, IDE can pick up 
localized version correctly.
But, IDE cannot dipslay it correctly, 
if I specified encoding tag to it.

IDE does not identify <head> and 
encoding tag correctly.

If localized .html contains <head> and
charset tag, IDE cannot display description
correctly. Description field is
"No Descripions" in Japanese. 
(see attached solaris_with_tag.png)
 
---------------------------------------------------------------------------------
  <HTML><head><META HTTP-EQUIV="Content-Type"
CONTENT="text/html;charset=euc-jp"></head>
    <BODY>
       ... Japanese description in euc-jp
encoding.....
     </BODY>
  </HTML>
 
---------------------------------------------------------------------------------
Comment 1 hatakyon 2004-12-24 09:05:36 UTC
Created attachment 19409 [details]
screenshot on japanese solaris
Comment 2 Ken Frank 2004-12-24 16:09:58 UTC
Has any code in the area that shows html for descriptions
of items in first panel of new project or new file been
changed in nb4.0 ? (or in swing browser if related or in parent
objects of these areas ? Or has code that gets the data
from the <filename>.html file and passes it to this ui area
that displays the html in the new project/file description
area ?

This issue may actually be a p1, since it means that, for at
least windows and at least ja, that no template descriptor
messages can be seen properly.



NOTE - this is not about just that if head tag is used that the
multibyte is seen ok; its really about that the meta encoding 
tag is not read properly. The head tag situation may be a separate
one, but the main thing is, even using just <html> and meta tag,
the multibyte does not display ok, at least on windows ja,
using the full translations.


It seems to be an encoding issue unless its a font setting one.

For many releases, this area where html is shown has been stable
and has shown multibyte correctly if, for example, the meta
encoding tag was used.

In this case, it is being used as usual but now, at least for
ja, some multibyte is not being shown ok, which seems that the
html encoding tag not being read ok or that the encoding handling
of the data is not being done ok, either by the description area
or other parts that pass this data to it.

l10n had tried other meta encoding tag for ja as well as other
manipulation of the data


NOTE - even if head tag is not present, just <html> and meta tag,
the problem shows.

This did not show in i18n testing using a smaller char set in ja
and does not appear to show in zh locale on windows.

In most of the html description files, there is not a head tag
and that hasn;t been used in i18n testing; but in other releases,
the head tag was used in l10n files, and there was no issue,
so I don't think head tag is the main issue here, its that
the multibyte does not display ok.


There needs to be just one localized html file per en file per locale;
since it would mean different suffixes and code for base product
to find one; in past, using correct data manipulation and encoding
tag, ja and zh have managed to have just one html file that shows
multibyte ok on all platforms.

PS am changing to core since I think that is the area for it vs
i18n, but please change to correct subcat.


ken.frank@sun.com
Comment 3 Ken Frank 2004-12-24 18:39:11 UTC
Changing to p1 since, in context of localized release for ja,
it could be viewed as a p1.

Modifying the summary to clarify that the main issue is that the
multibyte is not being displayed properly, regardless of using
meta encoding tags as has been the practice in the past, and that
any issue with head tag is secondary.

ken.frank@sun.com
12-24-2004
Comment 4 David Simonek 2005-01-03 15:38:48 UTC
Passing to Jiri...
Comment 5 Jiri Rechtacek 2005-01-03 20:43:56 UTC
Started investigating what's wrong, work on set-up a simple test case
to figure out the heart of problem. Hint to suspected code:
projects/ui/TemplatesPanelGUI.java, ignored IOException from
description.read(descURL.openStream(),descURL).
Comment 6 Ken Frank 2005-01-05 21:06:49 UTC
can we get a brief update on this, since l10n
returns from break today, and they are in
final ml RC work ?

also,what is full path to projects/ui in browsable
cvs as mentioned below for java file where the
cause might be ?

ken.frank@sun.com
01/05/2005
Comment 7 Jiri Rechtacek 2005-01-06 11:11:19 UTC
Created attachment 19513 [details]
binary patch to <nb-home>/ide4/modules
Comment 8 Jiri Rechtacek 2005-01-06 11:19:32 UTC
I attached the binary patch to ensure if I have found out the problem
and have a fix. Please, copy the attachment to your netbeans
installation, to dir ide4/modules and run IDE. Send back pls also its
messages.log. Thanks.
The patch should work iff the localized html file contains the
'charset' tag in its meta info.
Comment 9 Ken Frank 2005-01-06 21:33:37 UTC
The fix does not seem to work, using the same template html
files/method I usually use ie

emptyProject_ja.html
::::::::::::::
<!-- 
                   Sun Public License Notice
   
   The contents of this file are subject to the Sun Public License
   Version 1.0 (the "License"). You may not use this file except in
   compliance with the License. A copy of the License is available at
   http://www.sun.com/
   
   The Original Code is NetBeans. The Initial Developer of the Original
   Code is Sun Microsystems, Inc. Portions Copyright 1997-2004 Sun
   Microsystems, Inc. All Rights Reserved.

-->

<HTML>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html;charset=x-euc-jp"> 
  
<BODY> otJ¤Èä¥Þ¥ó¥É¤¬¸«¤Ä¤«¤ê¤Þ
    Select Web Application to create a standard project containing an
empty web application. 
    A standard project uses the IDE-generated build script to build,
run, and debug your project.
  </BODY>
</HTML>

l10n does it a bit differently, modifying the encoding of the
html file and using iso-2022-jp tag

but the issue had been seen before using both these methods,
and the file as above has been the kind of meta tag I've used for
many releases.

the html file does still appear ok in ie, but not in descriptor
window

let's see what l10n says about this also.

ken.frank@sun.com
Comment 10 hatakyon 2005-01-07 01:33:37 UTC
Patch doesn't work completely
in my Japanese environment too..

Now this patch recognize "<head><meta>" tags
of template.html as "tag" correctly.
I mean, IDE can display contents of 
template.html with <head><meta> at description area.
But, this patch does not recognize "charset" 
specification of <meta> tag correctly...

When I placed "ISO-2022-JP" encoded .html
to _ja.jar file, IDE can display contents
of this .html,
but, Japanese characters seem garbage. 

( see attached new screenshot please ) 
Comment 11 hatakyon 2005-01-07 01:34:54 UTC
Created attachment 19539 [details]
screenshot with patch
Comment 12 Ken Frank 2005-01-07 01:44:14 UTC
yes, I agree - I think the key issue for now is the
recognition of the meta charset tags, as had been
done in the past, and not anything regarding
head tag - that may be separate or related
issue but use of just head tag could be worked
around i think, but the recoginition of the
meta charset tag is the key item for now.

ken.frank@sun.com
Comment 13 Jiri Rechtacek 2005-01-07 15:47:04 UTC
Created attachment 19554 [details]
patch #2
Comment 14 Jiri Rechtacek 2005-01-07 15:54:27 UTC
Please, try it again with new patch #2. It handles meta tag
differently and logs more information to log
(<user_dir>/var/log/messages.log). Please attach this log to issue. Thanks
Comment 15 Ken Frank 2005-01-07 19:04:35 UTC
The current fix does not work either.

since the issue shows in the swing html browser
also, could it be in some common code or code
handling approach of that browser and the template
descriptor html area ?

the applicable part of messages.log is now

Url nbresloc:/org/netbeans/modules/ant/freeform/resources/webfreeform.
html has contentType: text/html
Url nbresloc:/org/netbeans/modules/ant/freeform/resources/webfreeform.
html has charset: null
Url nbresloc:/org/netbeans/modules/web/project/ui/resources/emptyProje
ct.html has contentType: text/html
Url nbresloc:/org/netbeans/modules/web/project/ui/resources/emptyProje
ct.html has charset: null
Url nbresloc:/org/netbeans/modules/web/project/ui/resources/emptyProje
ct.html has contentType: text/html
Url nbresloc:/org/netbeans/modules/web/project/ui/resources/emptyProje
ct.html has charset: null

--> there is a meta charset entry in
the emptyProject_ja.html file and it is trying
to show that one. The file is as in entry in other
part of this bug.

ken.frank@sun.com
Comment 16 Ken Frank 2005-01-08 05:57:11 UTC
once fix is done, can it be ported soon to 4.1,
or should a separate issue be filed ?

ken.frank@sun.com
Comment 17 Jiri Rechtacek 2005-01-09 16:47:06 UTC
Created attachment 19571 [details]
patch #3
Comment 18 Jiri Rechtacek 2005-01-09 17:04:47 UTC
I have attached the next patch. I don't think it'll works correctly
but I want to rule out this way, it uses the different component to
show a description then HtmlEditorKit. Thanks for help and testing
next patch. I'm sorry for time-consuming fixing.
Frank, the template description has been handled (since NB4.0) totally
differently then in Swing browser. If SwingBrowser doesn't work for
you, file other one issue, maybe it helps also to fix this. Thanks
Comment 19 Ken Frank 2005-01-09 20:36:27 UTC
It still doesn't work - I dont see any debug msgs in messages.log now.

BTW, does this need separate bug filed for 4.1 fixing ?

ken.frank@sun.com

Comment 20 Jiri Rechtacek 2005-01-10 15:39:10 UTC
Created attachment 19581 [details]
pre-read charset from file
Comment 21 Jiri Rechtacek 2005-01-10 15:45:54 UTC
Created attachment 19582 [details]
works for ISO-2022-JP and other (not with EUC-JP)
Comment 22 Ken Frank 2005-01-10 19:17:11 UTC
1. the fix for 2022 does not seem to work
using pseudlocalized with the 2022 tag 

2. the fix for euc does seem to work for 2022 
and euc
for template descriptor area, not browser

3. swing browser does not show html with 2022 tag
ok.

--> you mentioned that both of these worked for you -
were you comparing the mutlibyte as seen in ie with
what you see in templ descrip area as per the gifs
I'd sent ? 

--> I don't think that comparisons to specific
meta charset values should need to be so specific -
was that done in prev nb code ?

I suggest to l10n to check fixes using both ja and zh html
files/jars, to make sure any fixes for ja also still work for zh.

ken.frank@sun.som

Comment 23 hatakyon 2005-01-11 06:02:05 UTC
Hi all,
I checked patch, (id=19582) in my test environment.
It seems work fine for me :-)
It can display "euc-jp", "Shift_JIS" and
"iso-2022-jp" encoded Japanese html files correctly.
I have verified it on Solaris/SPARC, Windows 2K 
and JDS system.

Comment 24 Jan Chalupa 2005-01-11 08:20:51 UTC
Re Ken's questions from 01/07: as soon as a reliable fix is found, it
will be integrated into 4.1. The question is whether it still should
be put into the 4.0 ML release. It's being tracked as P1. Is it
considered to be a showstopper for the ML release?
Comment 25 hatakyon 2005-01-11 08:59:51 UTC
Yes. It is a showstopper for 4.0 ML release.
Could you plan to integrate this fix
to 4.0 ML release please ?
Comment 26 Jiri Rechtacek 2005-01-11 12:47:54 UTC
patch (pre-reads charset tag from html file) integrated in main trunk
(future NB4.1)

Checking in src/org/netbeans/modules/project/ui/TemplatesPanelGUI.java;
/cvs/projects/projectui/src/org/netbeans/modules/project/ui/TemplatesPanelGUI.java,v
 <--  TemplatesPanelGUI.java
new revision: 1.33; previous revision: 1.32
done
Comment 27 Jiri Rechtacek 2005-01-11 12:50:05 UTC
Created attachment 19607 [details]
proposed fix candidate
Comment 28 Jiri Rechtacek 2005-01-11 12:59:57 UTC
Attached the proposed candidate to backport to release40 branch.
The proposed patch is based on patch (id=19582), can read charset
euc-jp and iso-2022-jp as well, works also with czech localized files.
This patch is ready to be integrated in release40 branch if it'll work
for you. Please assure that before merge. Thanks
A similar patch will be prepared for SwingBrowser (issue 53207).
Comment 29 Ken Frank 2005-01-11 20:08:58 UTC
the iso-2022 patch does not work for me but the euc one
(pre-read charset one)
does work - but I think we should go with what l10n has found.

I think its impt that Chinese files be verified also even though
they did not seem to be issue in this - and I asked them if they
can verify the fix.

I will try again with the proposed fix candidate, since it is different
in size from both the other patches.

ken.frank@sun.com
Comment 30 Ken Frank 2005-01-11 20:20:26 UTC
I just ran using the proposed fix candidate and things
seem ok, with euc-jp and iso-2022-jp.

it could be that one of the two earlier fixes were switched in
the testing, so that what I saw and what l10n saw was the same -
or it could be that this final fix has some additional things.

So I think we need to wait for l10n, both ja and zh, to verify
the proposed fix.

Jiri,
for the swing browser one, can you attach fix for that jar
to that bug (I can't remember bug id) and I can check that out
also.

ken.frank@sun.com
Comment 31 hatakyon 2005-01-13 00:50:34 UTC
verified fix candidate (id=19607) in 
Japanese envronment.
it works fine !
Comment 32 Jiri Rechtacek 2005-01-13 13:52:55 UTC
The patch has been backported to release40.
Checking in src/org/netbeans/modules/project/ui/TemplatesPanelGUI.java;
/cvs/projects/projectui/src/org/netbeans/modules/project/ui/TemplatesPanelGUI.java,v
 <--  TemplatesPanelGUI.java
new revision: 1.32.8.1; previous revision: 1.32
done
Comment 33 Ken Frank 2007-05-29 18:47:07 UTC
verified.