This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 131833 - I18N - multibyte characters are shown with square boxes on tabs for xml configuration files
Summary: I18N - multibyte characters are shown with square boxes on tabs for xml confi...
Status: VERIFIED FIXED
Alias: None
Product: xml
Classification: Unclassified
Component: XML Multiview (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P1 blocker (vote)
Assignee: Erno Mononen
URL:
Keywords: I18N
Depends on:
Blocks:
 
Reported: 2008-04-02 14:49 UTC by kaa
Modified: 2008-04-08 17:58 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:

Attachments
sun-web.xml in red circled text with bad mbyte chars (89.63 KB, image/jpeg)
2008-04-02 14:51 UTC, kaa 		Details
web.xml (101.33 KB, image/jpeg)
2008-04-02 14:53 UTC, kaa 		Details
zip of bundle files (15.66 KB, application/x-compressed)
2008-04-03 16:27 UTC, Ken Frank 		Details
bundle file from ddui (17.61 KB, text/plain)
2008-04-03 16:27 UTC, Ken Frank 		Details
filters tab (59.01 KB, image/jpeg)
2008-04-07 18:09 UTC, kaa 		Details
pages tab (109.97 KB, image/jpeg)
2008-04-07 18:10 UTC, kaa 		Details
security tab (59.44 KB, image/jpeg)
2008-04-07 18:11 UTC, kaa 		Details
servlets (56.11 KB, image/jpeg)
2008-04-07 18:13 UTC, kaa 		Details
pseudo bundle I used for this verification (79.64 KB, application/x-compressed)
2008-04-07 18:15 UTC, kaa 		Details
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description kaa 2008-04-02 14:49:36 UTC 
Product Version: NetBeans IDE Dev (Build 200803301102)
Java: 1.6.0_05; Java HotSpot(TM) Client VM 10.0-b19
System: Windows XP version 5.1 running on x86; MS932; ja_JP
I'm running in the Japan locale, using a pseudo localized Netbeans.

Steps:
1. Create Web Application using wizard defaults
2. Open Projects tab and select the project node
3. Expand Configuration Files node
4. Double click web.xml (or sun-web.xml)

The file will be shown in NetBeans using tabs. Each of them has underlined text with square boxes instead of multibyte
characters (see images). The problem exists for win-31j and utf-8 project encodings and doesn't depend on --fontsize
option. Source code of the file looks ok. There is no any affected text with square boxes there. Encoding value in the
code is UTF-8 and doesn't depend on the project one.


Also the issue is reproducible with Sun Deployment Descriptor (sun-ejb-jar.xml)
Looks like it is a general problem.
Comment 1 kaa 2008-04-02 14:51:22 UTC 
Created attachment 59552 [details]
sun-web.xml in red circled text with bad mbyte chars
Comment 2 kaa 2008-04-02 14:53:16 UTC 
Created attachment 59553 [details]
web.xml
Comment 3 Erno Mononen 2008-04-03 09:15:38 UTC 
Not sure on how to reproduce, unicode escaped Japanese chars in .properties don't cause problems for me. Could you 
please attach your Bundle.properties file for the web.xml editor 
(org.netbeans.modules.j2ee.ddloaders.Bundle.properties)? Thanks.
Comment 4 Erno Mononen 2008-04-03 10:08:15 UTC 
I found a way to reproduce. In fact, instead of square boxes I was seeing other characters which I mistook for Japanese.
Comment 5 Erno Mononen 2008-04-03 10:38:55 UTC 
Take that back. Turns out that the problem I saw was caused by invalid unicode escape codes that I used in the bundle 
file, so please attach your bundle file so that I can try with it.
Comment 6 Ken Frank 2008-04-03 16:26:24 UTC 
am attaching a zip of all pseudo localized bundle files from ddloaders ja jar
and one bundle file from ddui jar.  these files have had the native2ascii run on them
as is the usual way of processing them.

ken.frank@sun.com
Comment 7 Ken Frank 2008-04-03 16:27:00 UTC 
Created attachment 59629 [details]
zip of bundle files
Comment 8 Ken Frank 2008-04-03 16:27:44 UTC 
Created attachment 59630 [details]
bundle file from ddui
Comment 9 Erno Mononen 2008-04-03 16:56:25 UTC 
Thanks for the bundles. However, I still can't reproduce this on my configuration - the Japanese chars are displayed 
fine for me. I'll need to try this on Windows.
Comment 10 Ken Frank 2008-04-03 17:20:00 UTC 
yes this is on windows only. choice of platform wasnt made but it mentions it in
description.

I can also send you the pseudo lcoalzied jars that you just install in your
nb and set ja regional settings/reboot and run netbeans; let me know.

ken.frank@sun.com
Comment 11 Erno Mononen 2008-04-04 11:41:47 UTC 
I've been trying to reproduce this on Windows with the attached bundle files and can see square boxes pretty much 
everywhere now, which leads me to think that there is something wrong with how I build/run the ide rather than in the 
ddloaders module itself (in the attached images square boxes appear only in titles). What I'm doing is that I build the 
IDE with the attached bundle files and run it using --locale ja:JP. Could you please post the steps for building and 
running the IDE that lead to the state of the attached screenshots? The localized JAR files would be useful too, but I 
also need to be able to debug this.
Comment 12 Erno Mononen 2008-04-04 15:43:31 UTC 
Reinstalling the Japanese language pack on my Windows helped, I can now reproduce this.
Comment 13 Erno Mononen 2008-04-04 16:57:54 UTC 
Fixed in trunk: http://hg.netbeans.org/main/rev/84ef92226ce3, please verify. Setting the TM to Dev until the fix has 
been reviewed and ported to 6.1.
Comment 14 Erno Mononen 2008-04-07 08:23:27 UTC 
Could you please verify this ASAP so that I can send the fix for the review and then port this to 6.1? Thanks.
Comment 15 kaa 2008-04-07 18:08:52 UTC 
The fix looks incomplete:
Product Version: NetBeans IDE Dev (Build 200804070002)
Java: 1.6.0_05; Java HotSpot(TM) Client VM 10.0-b19
System: Windows XP version 5.1 running on x86; MS932; ja_JP (nb)

Checked with WebApp:

1. web.xml:
Some messages look ok but some of them not.
There are the following tabs on xml designer:
OK: General, References
Not OK: Servlets, Filters, Pages (see images in attach)

2. sun-web.xml
All looks ok.

Also checked with Sun deployment descriptor (sun-application.xml) for EntApp. It looks ok 

This fix impacts the font used for parts of these areas:
org.netbeans.modules.mobility.end2end
   org.netbeans.modules.mobility.jsr172
   org.netbeans.modules.hibernate
   org.netbeans.modules.identity.profile.ui
   org.netbeans.modules.j2ee.ddloaders
   org.netbeans.modules.j2ee.persistence
   org.netbeans.modules.j2ee.sun.ddui
   org.netbeans.modules.mobility.svgcore
   org.netbeans.modules.visualweb.project.navigationloader
   org.netbeans.modules.websvc.wsitconf
   org.netbeans.modules.worklist.editor

Whould you please send to me on which editor of these areas might show the issue,
it can help to see if its ok on those parts also.
Comment 16 kaa 2008-04-07 18:09:50 UTC 
Created attachment 59784 [details]
filters tab
Comment 17 kaa 2008-04-07 18:10:45 UTC 
Created attachment 59785 [details]
pages tab
Comment 18 kaa 2008-04-07 18:11:46 UTC 
Created attachment 59786 [details]
security  tab
Comment 19 kaa 2008-04-07 18:13:37 UTC 
Created attachment 59787 [details]
servlets
Comment 20 kaa 2008-04-07 18:15:51 UTC 
Created attachment 59788 [details]
pseudo bundle I used for this verification
Comment 22 Erno Mononen 2008-04-07 18:58:35 UTC 
I'm afraid I don't know off hand what are all the editors that use xml/multiview, I'll list here the editors I know of:

web.xml 
ejb-jar.xml
persistence.xml
sun-web.xml
sun-ejb-jar.xml
Editor for hibernate.cfg.xml (you need the Hibernate module for this)

In addition I'm aware of the following, but don't know how to invoke them:
- some sort of wsit editor (web services)
- BPEL workflow editor
- in mobility there is at least one editor, possible more (see the jsr172 and svgcore modules in the list)

And then there are some editors in identity and visualweb. 

I'm looking at the remaining issues, seems that their root cause is the same. I was looking for similar usages in the 
module, but for some reason didn't spot these ones. Once they are fixed, I don't think that it is necessary to go 
through all editors, or not at least as part of this issue - if they have similar I18N issues, they should be filed 
against their respective components. I'll provide a fix for the remaining issues soon, it would be great if you could 
still verify it again during today.
Comment 23 Ken Frank 2008-04-07 20:01:23 UTC 
as to additional fix, let us know which specific  smetiste trunk build has the fix 
or which trunk build # on deadlock has it (and specific url to that deadlock location) - since to be sure, we do need to
unzip the build
again, then re pseudo localize the few jars files we are using and retest.
and we want to make sure to keep using trunk, since for other testing we are using
61.

ken.frank@sun.com
Comment 24 Erno Mononen 2008-04-07 20:10:16 UTC 
Fixed in trunk: http://hg.netbeans.org/main/rev/d9df6a7702d1

I hope I have now catched all problematic usages in xml/multiview, please verify it once again. Of course, I can't 
guarantee that the clients of xml/multiview don't override the places where the fix was implemented, so there is a 
possibility that they might suffer from the same issue. I did make a naive search across the listed modules and 
couldn't find suspicious usages.
Comment 25 Erno Mononen 2008-04-07 22:49:05 UTC 
Build #1514 contains the fix, available here: 
http://deadlock.netbeans.org/hudson/job/trunk/1514/artifact/nbbuild/dist/zip/
Comment 26 Erno Mononen 2008-04-08 14:19:49 UTC 
Ported the fixes to 6.1 as agreed with QE.

http://hg.netbeans.org/release61/rev/cb083135283f
http://hg.netbeans.org/release61/rev/238374c6856b
Comment 27 Ken Frank 2008-04-08 17:58:54 UTC 
verifying from trunk build of 0407 on windows, running using pseudo localized
messages for the areas that show messages in web.xml and sunweb.xml - 
have not verified in other parts that might be impacted since not topic of this
issue but based on Erno's comments and research, and additional fix part of this,
that its solved for those areas - we can ask dev of those areas to see if
they overwrite any of this code.

verified using ide with and without --fontsize option.

ken.frank@sun.com