Bug 133174

I will investigate what is wrong.

I am guessing that previous fixes might have been for ja only or that
if it did work before, something in encoding handling/determination
might not be working now due to some new changes, or perhaps
about locale suffix matching.

I copied the zips for ja and for zh to a local dir.

in ja locale, with ja zip added, it works ok to show ja docs

but running in zh locale,on solaris and windows, it does not show zh javadoc,
whether my project is in zh or other project encoding
(project encoding I don't think should matter for this, seems like it would
be based on locale user is running ide in)

also, if both zh and ja javadoc are added to the java platforms, while in zh locale,
it showed the ja javadoc even though was running in zh locale -
and that zh one was first in the list - 
this does not seem correct either.
(or if only ja javadoc was there)

is the zh javadoc found on java.sun.com - I couldn't find current ones and am wondering
if its been available there before.

ken.frank@sun.com

Wait - I copied the ja javadoc from that location and named it as the name
for the zh javadoc

then added that to the java platforms (ie has name with zh_CN but the file
is really the ja one)

and it works ok to show the zh javadoc when in zh locale

---> thus could the zh zip be corrupted or not laid out ok, and that
is what the problem was ?

or perhaps file is ok but not recognized by nb as to layout >

I did copy the files from stated location to my local dir but sizes look the same.

am marking as incomplete so we can get some additional information.

ken.frank@sun.com

Now am not sure if it is layout of zh javadoc or that nb is not
reading things ok, perhaps related to encoding -
the reason is, I found some earlier zh javadoc on download.java site
and same problem happened with using it - but I don't think that 2 separate
zips would have same problem unless there is some fundamental diff
between layout of zh javadoc and ja one and the zh way is not recognized,
vs some encoding handling situation within reading the javadoc.

ken.frank@sun.com

Here is some additional information:
1. I am now in zh_CN locale. But I think this issue is related to that.
2. Javadoc displayed in the pop-up window depends on which zip is the first one in this list. But zh_CN Javadoc will not
appear in any case. For example, I test it in zh locale and put the javadoc zip in this order zh/ja/en, then Japanese
Javadoc will be shown; if the order is zh/en/ja, then English Javadoc will be shown.
3. This issue also exists in NetBeans 5.5, with jdk1.6 installed and javase-6-doc-api-zh_CN.zip added. So it is not just
started from NetBeans 6.
4. The zips are provided by Docweb team. I think they are same with those from java.sun.com.
5. The format or layout of zh_CN zip should be correct, since our team here in Beijing is developing a NB plugin for
Javadoc API. With that plugin installed, the Chinese Javadoc could be shown.

I am also not sure where the problem is, but I guess it may be about how the zh_CN Javadoc zip is loaded in NetBeans IDE.

The current Javadoc zip has such file structure: 
docs/
docs/ja
docs/ja/api
docs/ja/api/index-files;docs/ja/api/java
...So does zh_CN structure. 

I have new findings:
1) In NetBeans 5.5, in order to show the zh_CN Javadoc, the added zip/folder has to be started from docs/zh_CN/api/. Ja
zip could be any format or structure. But for zh_CN, the starting directory has to be ".../api/" . Then both pop-up
window and "Show Javadoc" in browser can display Chinese Javadoc well in NB5.5.
The URL sent to browser:
Zip added: "jar:file:///C:/api/api_zh_CN.zip!/api/java/lang/String.html"
Folder added: "file:///C:/api/docs/zh_CN/api/java/lang/String.html"

2) In NetBeans 6.1 RC2, I did the same setting as 1). But the result is a little weird. With either zip or folder added,
the pop-up window displays English Javadoc (I didn't add any English Javadoc in the list), but if you choose "Show
Javadoc", the Chinese Javadoc can be shown correctly in the external browser.
The URL are same with 1).

removing incomplete keyword; thanks Rebecca for the information.

I did additional experiments but first one comment - a long time ago
there was special coding to handle ja javadoc of some earlier jdk
and perhaps that code might still be used so that only ja javadoc
is found during code completion, even though in some cases the
zh javadoc is shown on help menu if path in zip is different as per
Rebecca comments ?

Here are some further experiments:

I zipped up the files starting with api and also saw same as Rebecca reported.

I changed name of zh dir to zh_CN and zipped as per original and started at
docs path, as in ja file, but it still did not show the zh javadoc.

I had thought that if when basing the zh file at api, then the zh was not being
viewed as a locale name and it was being at least added to the help menu item for javadoc,
then maybe the detection of it as per locale was not happening since it might
have thought the zh_TW locale was needed, since zh can be viewed as zh_TW locale
whereas zh_CN is viewed for PRC.

I ran also in zh_TW locale also with original zh jdoc or renamed to ..zh.jar,
but it did not help.


I also changed all charset values in zh html files to utf-8, as in the ja javadoc html
files, but it did not help.

Also found a pt_BR javadoc zip and added that - it too was not added to help menu
or shown in java code completion popup.

ken.frank@sun.com

ken.frank@sun.com

more info

1. I unzipped the original zh javadoc zip and changed docs/zh dir name to 
docs/ja
and then made a new zip.

2. added only this zip to the java platforms - suffix of the jar file name
is ...zh_CN.zip, and it was added to the help menu

3. and the localized popup appeared in the editor for code completion.

4. Thus seems somehow something is looking for ja dirname/path only, even though I'm
not in ja locale (which should not matter anyway) or that the zh dirname
is not being recoginized as what is needed ?

5. changing original zh jdoc to have path docs/fr, and then zip a new file,
same problem, does not use that jdoc nor put it on help menu.
again, why is ja recognized and not other locale names ?

6. but its strange then that if zip starts at api dir, and no path docs/zh, that
it is recognized and added/used, since in this case there is no locale name as part
of path.

ken.frank@sun.com

ken.frank@sun.com

another part of last comments -

I took ja javadoc and replaced ja dir with zh dir name.
then rezipped and it was not added to help ->javadoc menu
nor used in project. (in ja locale)

I think that layout requirements should be same for any localized
or other added javadoc - if it works with docs/ja/api...
then it needs to work with docs/zh/api
(or docs/zh_CN/api since this javadoc is for zh_CN, not zh.

ken.frank@sun.com

I am downloading the zip file but it is really slow (~1.0KB/s). On java.sun.com I have found only browseable Chinese
version, no zip file.

Anyway it seems that the java platform manager (JavadocForBinaryQueryPlatformImpl) recognizes only Japanese bundle of
javadoc.

Another story is the javadoc search. We would need someone who knows Chinese to write the search engine.

Reassigning to java/project to fix javase-6-doc-api-zh_CN.zip recognition in the platform manager.

1. is it that only ja jar file is found or is the problem in that
unless there is a dir named ja in the path of the javadoc zip
that its not found ?
(and when path begins with api, and no locale dir name, its found also)

2. for the search, I thought the custom search was done just for
earlier 1.2/1.3 jdk and was not needed anymore, and those special
messages are not in bundles anymore as to special translations.

Thus why would there need to be new search written specifically for Chinese ?
There are also other javadoc now being translated or in progress; does it mean
that separate search is really needed for each language ?

3. can this be fixed for first patch even though its just few days away
that fix is needed in trunk ?  since this new uc module is being done
it depends on having this work as to finding the zh or other non ja
javadocs and being able to search them.

ken.frank@sun.com

ad 1) yes the java platform recogizes defalt or Japanese now.
ad 2) in case the javadoc contains translated class, interface, ... keywords or changes html layout the engine is necessary.
ad 3) 1. is fixable, 2. it is rather an enhancement for someone who can speak Chinese. Otherwise we would need to
replace architecture of searching to run not over html javadoc but over the java model index. But it is definitely not
task for 6.1 patches.

thanks for clarifying about fix possibilities -
since this is about just the jdoc being found, shown in help
menu and used when user chooses code completion - is that the part
discussed about in my below comment #1, which was mentionend as
fixable for the patch ?

if so, then suggest we try to do that fix for the patch, and a separate
issue can be filed about the search, since it seems that when the zh 
javadoc is recognized (by using ja dirname in path in its zip)
and on help menu and shown at code completion, 
and even if jdk/src.zip is removed from the platform manager,
that the localized pages are not found and shown, just the en ones.

ken.frank@sun.com

the javadoc search does work if ja javadoc is used, BTW.

ken.frank@sun.com

filed 133560 on the search.

Fixed the j2seplatform part.
ficed in: http://hg.netbeans.org/main/rev/6aeb1fa493fb

does fix of j2se platform part mean the issue described here is fixed
(not the search part) or is there still more to do with this issue ?

am asking since want to verify it as soon as its complete so it can
be in patch1.

ken.frank@sun.com

I used trunk build of after this fix (just javase, not web or j2ee; dont know if that
matters)
and added the original zh javadoc to the platform javadoc.

Now it shows on help menu, its shown during code completion and
help-> browse javadoc shows it in browser with ok encoding chosen.

am in zh locale but am guessing it does not matter what locale user is in
or what project encoding they are using ?

Will wait for other parts of the fix - please let me know what scenarios
are for the other parts.

ken.frank@sun.com

It is true that in the trunk build(2008-04-23_02-01-05) the Chinese Javadoc could be shown in Code Completion Window
correctly with the original javase-6-doc-api-zh_CN.zip added. The file structure is ~/docs/zh/api/...

But it doesn't work if the file structure changes to: such as html_zh_CN.zip (~/html/zh_CN/api/..., this zip is released
on SDN China and most of China users use this one), or api_zh_CN.zip(~/api/..., this structure works in NB5.5). Both
cases work for "Show Javadoc" in browser but don't work in code completion window.

However, if I change ~/html/zh_CN/api to ~/html/zh/api then it works. So it seems that only zh is supported. But we want
both zh and zh_CN are supported and recognized as Simplified Chinese locale. 

~/html/ja/api/... works for Japanese Javadoc;
~/api/...(contains ja Javadoc) doesn't work in code completion window.

The bug is about recognizing javadoc by the java platform manager. Thus java/j2seproject.
In case it is fixed then Code Completion, Show Javadoc, Window->Other->Javadoc View and Help->Javadoc References should
work as expected.

Help->Javadoc Index Search is another issue that depends on the fix but requires also an implementation of language
aware search engine (issue #133560). It belongs to javadoc module.

rebeccaliu: Is some specification of language folder naming available? There is just 'zh' folder in
javase-6-doc-api-zh_CN.zip you have provided.

kfrank: Neither locale nor project encoding does not matter in this case.

I have tried to reproduce with
http://download.java.net/jdk/jdk-api-localizations/jdk-api-zh-cn/builds/latest/html_zh_CN.zip. The reason seems to be
generated file structure. There is renamed document root from 'docs' to 'html'.

I think it is not fully fixed. And the reason why html_zh_CN.zip could not be recognized is not changing docs/ to html/.
It is due to zh_CN. Currently it can only recognize ja, zh, pt. But actually we will use zh_CN, pt_BR, zh_TW and so on.

I tested many possible cases. The only problem is "localized Javadoc cannot be shown in Code Completion window". "Show
Javadoc" is fine in any of the cases. You can see the results below. "+" here means OK; "-" means NOT OK.

+ /docs/ja/api/...
+ /html/ja/api/...
- /docs/ja_JP/api/...
+ /docs/zh/api/...
+ /html/zh/api/...
- /docs/zh_CN/api/...
+ /docs/pt/api/...
- /docs/pt_BR/api/...

So I think the code here:
http://hg.netbeans.org/main/file/6aeb1fa493fb/java.j2seplatform/src/org/netbeans/modules/java/j2seplatform/platformdefinition/JavadocForBinaryQueryPlatformImpl.java
doesn't get all the available locales. Those language_Country locales like zh_CN, ja_JP, pt_BR are ignored. These
locales are commonly used in NetBeans. 

Besides, I don't know if there is a rule for the naming convention of Javadoc or not. But I think at least
docs/zh_CN/api/... docs/pt_BR/api/... should be recognized. javase-6-doc-api-zh_CN.zip uses zh sub-directory since zh_CN
doesn't work in NetBeans IDE. Normally we should use zh_CN for Simplified Chinese, since there are also zh_TW, zh_HK. 

Rebecca,

I assumed that the layout in the original file you mentioned, which had zh, not zh_CN,
was the official one for the official javadoc, but you mention others below that has
zh_CN as path; my verification of the fix so far was using original doc.

But I agree that fix should recognize zh_CN also, or any locale name,
but it would help to know the standard/spec here for localized jdoc - or is there one, ie
docs/localename/api or html/localename/api and are there others ?

knowing that info would help in making this fix.


ken.frank@sun.com

Hi, Ken

Sorry I didn't clarify it very well. The original zip (javase-6-doc-api-zh_CN.zip) was provided by our team here who is
developing the Javadoc API plugin for NetBeans. Its original structure is docs/zh_CN/api/... But it doesn't work in
NetBeans 6.0 during their code development, so they changed it to docs/zh/api/..., then the plugin works well. But
officially we should use docs/zh_CN/api/... for Simplified Chinese Javadoc, not zh. So supporting "language_COUNTRY"
style locales is very important, since we also have pt_BR, zh_TW, etc. 

html/zh_CN/api/... is only used on developers.sun.com.cn (SDN China). This is a special case, I don't know why they
changed docs/ to html/, but that's almost two years ago. Anyway, I will tell them to use unified Javadoc naming
convention and file structure. 

From now on, the file structure for localized Javadoc will always be docs/LOCALE/api/...; the naming convention will be
javase-6-doc-api-LOCALE.zip.

>Its original structure is docs/zh_CN/api/... But it doesn't work in
>NetBeans 6.0 during their code development, so they changed it to docs/zh/api/..., then the plugin works well

Neither docs/zh_CN/api/ nor docs/zh/api/ works in NB 6.0.
After the fix (in dev) both of them works fine, it's important that there is "docs" folder not "html"
Basically the regular grammar is:
S->A
S->B
A->"docs"B
B->"api"D
B-><any>C
C->"api"D
D->"index-files"

So the "docs","api", "index-files" terminals are important.

Please do not change component+subcomponent of the issue. As I wrote it was an issue in the java platform manager, thus
java/j2seplatform.

The zipped Javadoc API layout has been always ./docs/[LOCALE/]api/... In case you use something else in your .zip file
like ./html/... it will not work and it has never worked. Tomas's fix of JavadocForBinaryQueryPlatformImpl.java permits
to use whatever name of the LOCALE folder instead of just 'ja' locale.

Another case is the unzipped javadoc. If you add ./docs, ./html or./whatever that contains locale folder or directly api
folder then JavadocForBinaryQueryPlatformImpl will be able to recognize it as javadoc. In case of 'ja' locale it has
most likely worked also in previous NB versions. But if you unzip ./html folder into JAVA_HOME then
JavadocForBinaryQueryPlatformImpl will not be able to recognize javadoc by default and you will have to add it manually.
For ./docs folder this works.

IMO we can close this as FIXED. If you want to use other path for localized javadocs please file an RFE and include some
specification.

tzezula: 
It is so interesting. I tried so many times, but docs/zh_CN/api/... never worked. I mean the Chinese Javadoc cannot show
in "Code Completion Window". Could you tell me how you make it work? I used the latest trunk build
(http://smetiste.czech.sun.com/builds/netbeans/trunk/daily/2008-04-24_02-01-05/) and added the jdk-6-doc-api-zh_CN.zip
(docs/zh_CN/api/...) which you could get it here: /net/vigor.prc.sun.com/download/rebecca/jdk-6-doc-api-zh_CN.zip

jpokorsky: 
I also checked the code fixes of that java file. Yes, it seems that "Tomas's fix of
JavadocForBinaryQueryPlatformImpl.java permits to use whatever name of the LOCALE folder". But the real testing doesn't
show me expected results. ja, zh or pt is ok. But ja_JP, zh_CN or pt_BR does NOT work. Do you really try it before you
close this issue?

I want to clarify again. The current problem is that localized Javadocs with such standard paths docs/zh_CN/api/... or
docs/pt_BR/api/... cannot be shown in Code Completion Window. I reopen this issue and change the summary. Please do
verify if the fixes indeed fix the problem. Thanks.

Of course I did verify the fix. The java platform supports ./docs/<whatever>/api/ as expected now. You can verify it by
looking to Help->Java References or editor popup->Show Javadoc.

The java code completion uses own code to detect javadoc localization due to performance reasons that does not relate to
the fix and if possible it takes javadoc from src.zip. It relates rather to issue #99487. You can work around it by
using ./docs/zh/ instead of ./docs/zh_CN/ for now. I will fix it soon.

We usually use one issue per problem so it was the reason I closed it since the platform manager works as expected.

code completion fixed as http://hg.netbeans.org/main/rev/0d45a9b12082

I see same as reported by Rebecca, am using the original zh javadoc provided
by her, but I renamed the zh dir to zh_CN and zipped into another zip, then
added to java platform mgr as only javadoc, using new userdir.

-  the javadoc is shown on the help menu and does appear in browser when clicked.

- but the zh javadoc is not shown in java editor when doing code completion

whereas if the zip with the docs/zh/.. path is added, the translated javadoc does appear
in code completion.

WAIT - I see the misunderstanding here that I think I am seeing - its that this fix
was just for having the jdoc be shown on help menu but original issue was about 
both cases of 

a. it not being shown on help menu
b. localized jdoc not being shown in java editor popup

Now I am more clear based on Jan's comments below:
The java code completion uses own code to detect javadoc localization due to performance reasons that does not relate to
the fix and if possible it takes javadoc from src.zip. It relates rather to issue #99487. You can work around it by
using ./docs/zh/ instead of ./docs/zh_CN/ for now. I will fix it soon.

We usually use one issue per problem so it was the reason I closed it since the platform manager works as expected.

---> I agree that one issue per problem should be filed but it would have helped to have
had it noted about this earlier and that another issue should be filed or to refer to 99487 IMO
sometimes we might not know that its different areas that apply to the showing of the popup
compared to the showing on help menu and that sep issues are needed.

also, if its problem of 99487, then wouldnt the en javadoc show in editor with using 
docs/zh vs docs/zh_CN, but it does not.

in any case, should I reopen 99487 or file another issue ?
I think the javadoc showing in editor is as important as this fix, and should be in first
patch also.

please advise.

The bug was about not working detection of localized javadocs. It is handled by the
JavadocForBinaryQueryPlatformImpl.java and all javadoc features depend on this. I have not been aware of the performance
optimization for code completion.

Anyway the code completion is fixed as well. See my comment (Thu Apr 24 13:44:17 +0000 2008) above. I included the fix
here and you can verify it. So please do not reopen other issues.

using trunk 20080424184948, can verify it.

in zh_CN locale solaris; added the jdoc with zh_CN path, ie docs/zh_CN/api
to java platforms javadoc

its on help menu

in java editor, code completion shows the localized jdoc popup.

Rebecca, can you try this with your current/final jdoc files that will
be the official release of these jdocs from sun so we can be sure with
the actual ones our users will be using ?
then if its ok, please comment or reopen if not.

if there are other locale jdoc files besides zh_CN and ja, let me know
official location and I can try them too.

ken.frank@sun.com

Issue '133174' Integrated in NB_Trunk_Production #156 : http://hg.netbeans.org/main/rev/0d45a9b12082,
 with comment: #133174: use all possible locales

Yes, Ken, I confirmed it works very well now. Thank Jan for fixing the problem. It is really helpful to Javadoc API
localization and its use in NB IDE.

By the way, I will ask SDN China replace the html_zh_CN.zip(html/zh_CN/api/...) with the standard zip.

To rebeccaliu:
It's important to the Java Platform that the structure of the zipped javadoc is:
docs/<locale>/api/ or docs/api
where <locale> is any locale, may be even non ISO like zh_CN.

Rebecca,

just to clarify, that all officially released javadoc translated will follow the 
structure you mention ? (I don't know if javadoc is released by more than one team)

if there is some localized javadoc as part of some other opensource project,
can we provide this info to them also so those too will work ok in nb ?

ken.frank@sun.com

switching issue to state : verified/fixed

switching issue to state : verified/fixed

switching issue to state : verified/fixed

to sustaining, can you confirm that this fix will be
in first patch, since it was verified in trunk on time ?

ken.frank@sun.com

The fix has been ported into the release61_fixes branch.

http://hg.netbeans.org/release61_fixes/rev/44e5fbf1a532

One more change during porting of this fix into release61_fixes repository:

http://hg.netbeans.org/release61_fixes/rev/6c83e34d0c3a