This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 128033 - I18N - NetBeans generates incorrect javadoc URL
Summary: I18N - NetBeans generates incorrect javadoc URL
Status: RESOLVED WORKSFORME
Alias: None
Product: ide
Classification: Unclassified
Component: Extbrowser (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P3 blocker (vote)
Assignee: Jesse Glick
URL:
Keywords: I18N
Depends on:
Blocks:
 
Reported: 2008-02-21 15:27 UTC by kaa
Modified: 2008-08-28 01:25 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
IE screen shot (15.95 KB, text/plain)
2008-07-23 22:26 UTC, _ krystyna
Details
correcting mime type for first attachment (15.95 KB, image/png)
2008-07-23 22:30 UTC, _ krystyna
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kaa 2008-02-21 15:27:57 UTC
Product Version: NetBeans IDE Dev (Build 200802120006)
Java: 1.6.0_03; Java HotSpot(TM) Client VM 1.6.0_03-b05
System: Windows XP version 5.1 running on x86; MS932; ja_JP

Steps:
1. Create WebApp using mbyte chars in its name and path.
2. Add Java Class
3. Generate javadoc

The browser couldn't to open javadoc page with the follwomg message: File not found
Firefox can't find the file at /C:/譁ー隕上ヵ繧ゥ繝ォ繝€/縺ィ邊ョJot邊ョ WebApplication204邊、繧・dist/javadoc/index.html.

URL in browser was:
file:///C:/%E8%AD%81%EF%BD%B0%E9%9A%95%E4%B8%8A%E3%83%B5%E7%B9%A7%EF%BD%A9%E7%B9%9D%EF%BD%AB%E7%B9%9D%C2%80/%E7%B8%BA%EF%BD%A8%E9%82%8A%EF%BD%AEJot%E9%82%8A%EF%BD%AEWebApplication204%E9%82%8A%EF%BD%A4%E7%B9%A7%E3%83%BBdist/javadoc/index.html

Next, I opened javadoc directly. It was shown ok. Its URL in browser was:
file:///C:/%E6%96%B0%E8%A6%8F%E3%83%95%E3%82%A9%E3%83%AB%E3%83%80/%E3%81%A8%E7%B2%AEJot%E7%B2%AEWebApplication204%E7%B2%A4%E3%82%8D/dist/javadoc/index.html

There is a difference in URLs. Probably, NetBeans launches the browser with incorrect one.
The same with VW App, Ejb module. In eng locale without mbyte chars works ok.
Comment 1 kaa 2008-02-21 15:32:48 UTC
I was able to open javadoc directly.
Is it Firefox or NetBeans issue? There is a difference in URLs in their symbols.
The question is: who generates it with incorrectly?
Comment 2 Petr Jiricka 2008-02-22 12:39:14 UTC
> Next, I opened javadoc directly.

What exactly do you mean by "directly"?

Next, what happens when you use the "View" action from the HTML file's popup menu?

I came across a similar issue using different steps:
1. Create a HTML file with a national character in the name
2. Select View in the file's popup menu
=> Browser was not opened.

Looks like the issue of the external browser module, reassigning.
Comment 3 Ch Nguyen 2008-02-26 16:24:34 UTC
Looking into this.
Comment 4 kaa 2008-02-26 17:36:43 UTC
Directly means the following:
1. open file browser
2. navigate to the index.html of the javadocs
3. select this file
4. press enter
Comment 5 Ch Nguyen 2008-02-27 23:03:17 UTC
I'm able to reproduce and still investigating this. This is not a stopper for Beta though.
Comment 6 Ch Nguyen 2008-03-21 02:34:08 UTC
changeset:   74028:0444f3e57183
user:        Chau Nguyen <cnguyencasj@netbeans.org>
date:        Thu Mar 20 17:27:53 2008 -0700
summary:     Fix bug 128033: I18N - NetBeans generates incorrect javadoc URL

To verify, you need to pick a build that has this changeset in it or newer.
Comment 7 kaa 2008-03-26 22:40:15 UTC
Product Version: NetBeans IDE Dev (Build 200803231202)
System: Windows XP version 5.1 running on x86; MS932; ja_JP (nb)
Project encoding UTF-8
Comment 8 Ken Frank 2008-03-26 23:09:30 UTC
please try to verify using win31j project encoding; thats the default encoding
of windows in ja locale and since windows not have a utf-8 locale, users would
still use that encoding at times.

Also, I don't see in issue summary or other places if this is about
firefox or IE or both - please try both cases of project encoding with
each of these browsers.

ken.frank@sun.com
Comment 9 kaa 2008-07-21 15:44:18 UTC
Product Version: NetBeans IDE Dev (Build 20080715190616)
Java: 1.6.0_06; Java HotSpot(TM) Client VM 10.0-b22
System: Windows XP version 5.1 running on x86; MS932; ja_JP (nb)

I was able to reproduce the issue in IE and Mozilla for UTF-8 and win-31j project encoding.
Comment 10 _ krystyna 2008-07-23 22:25:15 UTC
Cannot duplicate with Firefox but I do see a problem with Internet Explorer
NetBeans IDE Dev (Build 200807221016)
Java: 1.6.0_10-rc; Java HotSpot(TM) Client VM 11.0-b13
System: Windows XP version 5.1 running on x86; MS932; ja_JP (nb)  
Project is set at UTF-8


Firefox 2.0.0.16
1. browser opens this correctly upon generation of javadoc:
-file:///C:/Documents%20and%20Settings/Krystyna%20Polomski/My%20Documents/NetBeansProjects/apachejalocalejuly23%E6%97%A5%E6%9C%AC%E5%9B%BD/dist/javadoc/index.html

2. manually loading what the javadoc output panel lists
Browsing: file:/C:/Documents%20and%20Settings/Krystyna%20Polomski/My%20Documents/NetBeansProjects/apachejalocalejuly23日
本国/dist/javadoc/index.html

-When I manually load #2, adjust to correct for the file protocol slashes , the page correctly loads.
3. I can navigate via the FireFox Browser's file to the dist directory to load the page as well.

InternetExplorer
1. browser does not open upon generation of javadoc: I get a cannot find file. See attachment .PNG
2. manually loading the string from the javadoc output panel does not work.
3. Navigating via the IE browser file to the dist directory *does* successfully work.
Comment 11 _ krystyna 2008-07-23 22:26:06 UTC
Created attachment 65434 [details]
IE screen shot
Comment 12 _ krystyna 2008-07-23 22:30:38 UTC
Created attachment 65436 [details]
correcting mime type for first attachment
Comment 13 kaa 2008-07-25 15:05:39 UTC
I found another place where this issue exists:

Product Version: NetBeans IDE Dev (Build 20080721233246)
Java: 1.6.0_06; Java HotSpot(TM) Client VM 10.0-b22
System: Windows XP version 5.1 running on x86; MS932; ja_JP (nb)

1. Create WebApp using multibyte chars in the project path
2. Create New RESTful Web Services from Patterns (use Singleton)
3. Right click on the project node and select“Test RESTful Web Services“

A browser started with a URL for the web service. In case of mbyte chars in path the URL was generated incorrectly. This
is true for IE and Mozilla.

In case of Latin chars everything looks ok.


Comment 14 Peter Zavadsky 2008-07-28 14:17:33 UTC
Krys, how the address in the IE browser looks after navigating according the step #3. Thanks.
Comment 15 _ krystyna 2008-07-29 19:47:07 UTC
Step #3, the address looks like this in IE 
C:\Documents and Settings\Krystyna Polomski\My Documents\
NetBeansProjects\new0729???xxx\dist\javadoc\index.html


Also, another wierd thing that I noticed is that Firefox only fails 
right after IE fails for the same project. I switch my default
browser, create a new Java class, FF will fail. But if
I create a new project, then FF is successful in bringing up the
correct javadoc url. This is step #1 I'm talking about and explains 
why some have seen it fail but I did not.
Comment 16 Peter Zavadsky 2008-07-30 11:52:00 UTC
Krys, I am confused a bit.
The address you provided contains ??? chars, how could that one work? 
Also what do you mean by that FF and IE usage before and after.
Please, put here some steps how to get the problem, then I hope I'll be able to reproduce it, thanks.
Comment 17 kaa 2008-07-30 17:35:27 UTC
Multi byte chars were replaced by question marks in the url.
Thats why it generated incorrect. Probably they need to be encoded but not replaced by question marks.
Comment 18 _ krystyna 2008-07-31 04:45:13 UTC
Peter, my step 3 was just showing my IE browser address bar when I walked manually using file:/// etc.. I walked the 
file path to the project which did have multibyte chars. Perhaps it is browser settings (which I did not change). I 
only changed the WinXP regional language to Japanese. You may have to check with Ken.

RE before and after: I am seeing more Firefox failures as others have reported. Once I am in a project with IE as NB 
default browser which fails to generate correct javadoc url, switching to FF default NB browser *while in the same 
project* will then fail.  I have also seen on a restart, new project, newly generated javadoc does fail for Firefox, if 
I regenerate javadoc a second time it passes!

url 1:  file:///C:/Documents%20and%20Settings/Krystyna%20Polomski/My%20Documents/NetBeansProjects/WebApplication132???
777/dist/javadoc/index.html

url 2:  file:///C:/Documents%20and%20Settings/Krystyna%20Polomski/My%20Documents/NetBeansProjects/WebApplication132%E6%
97%A5%E6%9C%AC%E8%AA%9E777/dist/javadoc/index.html



Comment 19 Peter Zavadsky 2008-07-31 11:12:04 UTC
Yes, Krys, that's my point the url1 (with the ???) means that is can't recognize such chars.
Only the url2 is correct one.

So I need to get on that, why is it so? Whether the IE needs also some special setting or localization to be able to
show it correctly.
Comment 20 Peter Zavadsky 2008-07-31 12:51:57 UTC
Marking this incomplete until we find out whether it is IE localization issue.
Comment 21 Ken Frank 2008-07-31 17:48:05 UTC
several comments

1. I'm assuming browser settings would not need to be changed since
if its just encoding value that impacts the situation,
that nb code/operations would communicate to browser
about it, ie in html meta charset tags

2. I suggest, if not being done yet, separation of investigation of
IE vs firefox, since there might be some other settings of firefox, at least
on solaris, that impact this if using english firefox (by english i just mean
not translated, inside they are all the same except that some language settings might
be different and altho it should not impact about this issue, it might.

but on other hand, am not sure if this firefox situation applies to windows vs solaris.

a. most users might install a localized browser, but not all, but still seems things should
work in any browser since encoding communicated to browser is impt.

b. but also I know it can be how the url sent to browser is encoded or communicated
from nb.

3. also, suggest running a nb project where the project encodings are 

a. default utf-8 

b. win31j, which is the default encoding of the windows ja locale/regional setting.

4. where the multibyte is used might show different symptoms, especially combined with 3. above combos.

 a. use of multibyte in project name but not in path to it
b. vs mbyte in path to it but not name
c. and use of mbyte in both

ken.frank@sun.com

(i know that project name becomes a path but it can make a difference in the errors.
Comment 22 Peter Zavadsky 2008-08-01 10:29:22 UTC
I found issue with the generate javadoc, about how the url is constructed. It should be always up the the client code to
contruct valid URL, passing to the owner.
Also, here I provide suggested fix (however my settings didn't allow me to try it out properly, anyway I am pretty sure
this one should help):

diff -r 3b475c3a0a23 ant.browsetask/antsrc/org/netbeans/modules/ant/browsetask/NbBrowse.java
--- a/ant.browsetask/antsrc/org/netbeans/modules/ant/browsetask/NbBrowse.java	Fri Aug 01 12:32:35 2008 +0530
+++ b/ant.browsetask/antsrc/org/netbeans/modules/ant/browsetask/NbBrowse.java	Fri Aug 01 02:24:17 2008 -0700
@@ -68,7 +68,7 @@ public class NbBrowse extends Task {
     public void execute() throws BuildException {
         if (url != null ^ file == null) throw new BuildException("You must define the url or file attributes",
getLocation());
         if (url == null) {
-            url = file.toURI().toString();
+            url = file.toURI().toASCIIString();
         }
         log("Browsing: " + url);
         try {
Comment 23 Peter Zavadsky 2008-08-01 10:30:13 UTC
I meant to pass it to ant.browsetask module.
Comment 24 Jesse Glick 2008-08-01 19:17:38 UTC
I cannot really comment on the proposed patch. It looks harmless but I don't know why it should be necessary. It seems
to me that if some HtmlBrowser implementation cannot correctly pass Unicode characters in a URL on to an external
browser, that implementation should do its own recoding. The URL created by ant.browsetask is valid as far as I know (if
not then I wonder why the java.net.URL constructor even allows it).

There have been so many comments in here about various scenarios on different OS flavors and browsers which did or did
not work that I do not know what is being discussed any more. If there is a concrete test case that fails reproducibly
for everyone who tries it, that should be described precisely.

Localization of NB or the browser are irrelevant, as are the project name and project encoding. The sole issue is the
characters used in the disk path to the project (and perhaps those used in Java classes within the project).

As is usual for I18N bugs, I am unable to reproduce any problem on Ubuntu 8.04 in a dev build. I make a Java app project
"v češtině" with a class "Třída.java" also with some Czech Javadoc comments inside. Run > Generate Javadoc works fine,
printing

Browsing: file:/tmp/v%20češtině/dist/javadoc/index.html

and Firefox (3.0) displays the Javadoc without issue. The Czech characters also appear naturally in the location bar;
even if I click on "NO FRAMES" and then the constructor, the link works and I see in the Firefox location bar the
natural class name. If I copy the link and paste it here, it is ASCII-coded automatically:

file:///tmp/v%20%C4%8De%C5%A1tin%C4%9B/dist/javadoc/T%C5%99%C3%ADda.html#T%C5%99%C3%ADda()

The only visible difference made by Peter's patch is that the Ant output log shows the characters encoded rather than
natural; Firefox still shows the URL as natural.

As is also usual for I18N bugs, I am unable to even get started reproducing the problem on XP (vanilla US/English SP2
installation) because nothing works well. The same project can be created and compiled, but Javadoc cannot even be
built, much less displayed. I would not recommend Windows users even try to create projects with non-ASCII names.

I do not have installations of Solaris or Mac OS X handy to test with. The last time I checked, Solaris was unable to
properly handle non-ASCII filenames from Java code at least (even with LC_ALL explicitly set to UTF-8, as is the default
for every modern Linux distribution). I did not check how well Solaris Firefox did.
Comment 25 Peter Zavadsky 2008-08-25 18:05:54 UTC
Jessie. As I understand the javadoc, the URL you provide is valid URL, but not valid URI, according RFC 2396 standard
(see URL javadoc: http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.html). I.e. such URL would be host dependend
(which might work on your machine but not at others). It seems to me that to be machine independent it needs to be
encoded according RFC 2396, i.e. to be also valid URI.

The implementation of the URL displayer just takes the provided URL, it is not creator of the URL. And according the
javadoc, the encoding is the responsibility of the creator of the URL. Decoding is up to the 'consumer' of the URL.

I pass it back, to reconsider, or suggest other solution.
Comment 26 Jesse Glick 2008-08-27 21:12:35 UTC
ant.browsetask is not providing a URI or a String etc., it is providing a URL object. It is providing exactly what the
JRE's recommended File.toURI().toURL() would produce, which a great deal of code throughout NetBeans generates all the
time. (In fact the comments in this bug also talk about web service test generation, which has nothing to do with
ant.browsetask.) The URL is already correctly encoding metacharacters (e.g. ' ' -> '%20').

cnguyencasj's changeset looks reasonable at least for SimpleExtBrowserImpl, and puts the fix where it belongs - in
extbrowser, which does the final translation from the internal Unicode form to an octet sequence that can be passed to
an external process. There are only a handful of such places, as compared to arbitrarily many places where URLs are
constructed from files.

As previously mentioned, I was unable to reproduce any issue on Linux, and was unable to even start to reproduce on
Windows because other things are more broken; I am not about to make any change unless I can verify its effect. Anyway
cnguyencasj claims to have done a fix already; if the fix does not work, I cannot help with this.

The best advice for users (as always) is to either (1) use Linux, or (2) do not try to put non-ASCII characters in file
paths to begin with.
Comment 27 Ken Frank 2008-08-27 21:59:52 UTC
I had thought windows is a very  used platform for netbeans - thus don't know why you are closing
this (and other issues in the past) that were seen on windows or other non linux platform,  just because it works for
you on linux.

I don't know if the windows problems mentioned as to why you can't run in windows is related to needing to know how to
run in other locales
or that its other things like broken machine or windows os needing fixing or reinstallation -
but IMO just because you are not able to run on windows does not seem reason to close issues
that do mention windows as a platform on which the issue was seen.

or if netbeans team feels that only ascii can be in url/uri, then think that needs to be clarified
and stated and noted in general and about areas where the name of a project and its path
can become part of the url/uri (since that would mean nb does not allow such characters in any path or project name)
(or workaround by doing what web apps do for context root and replace non ascii with underscore characters)

ken.frank@sun.com
Comment 28 Jesse Glick 2008-08-27 22:06:30 UTC
To repeat - I did test on Windows. Javadoc generation (the main task) failed before I could get to checking whether the
browser was correctly opened on the result (a much less important feature). The javadoc command claimed it could not
find the source files in question; various characters were corrupted in the error message.

I doubt there is anything wrong with my OS installation; I keep a rather clean XP installation in VirtualBox exactly for
this kind of test.

cnguyencasj claimed to have been able to reproduce some problem at some point and claimed to fix it. That is more than I
can say.
Comment 29 Ken Frank 2008-08-27 22:32:35 UTC
ok thanks for clarifying about it.

cnguyen is not on nb team anymore; krystyna - I think you saw the problem in IE and sometimes in firefox
and kaa saw it in firefox and perhaps IE.

I suggest lets leave this closed  and I will continue discussion with team  about if non ascii should not be
allowed at all in places where it might end up in a url or uri - since this kind of situation can apply
to more than one part of nb and think getting one decision/guideline will help.  We have situations in other parts of nb
where it is not allowed and we assume that and have had no problem seeing issues closed when filed about those
situations thus its time to have this situation decided IMO.

ken.frank@sun.com
Comment 30 Jesse Glick 2008-08-28 01:25:13 UTC
Easily and consistently reproducible problems are usually fixable. Charset-related problems suffer from the fact that
potential fixes need to be tested on an array of different systems.