This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 41443 - Support HTTP URLs for Javadoc
Summary: Support HTTP URLs for Javadoc
Status: RESOLVED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Javadoc (show other bugs)
Version: 4.x
Hardware: All All
: P2 blocker with 5 votes (vote)
Assignee: Jesse Glick
URL:
Keywords:
: 51605 71148 71773 103200 103859 134538 172341 185273 (view as bug list)
Depends on: 187919 194017 194653 194969
Blocks: 41448 75117
  Show dependency tree
 
Reported: 2004-03-28 17:57 UTC by David Konecny
Modified: 2012-04-10 18:45 UTC (History)
8 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Patch being tested (76.17 KB, patch)
2010-04-21 22:52 UTC, Jesse Glick
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Konecny 2004-03-28 17:57:58 UTC
Consider removing httpfilesystem at all and
directly parse the URL returned by
JavadocForBinaryQuery in case it cannot be
converted to a FileObject. httpfilesystem is not
real http filesystem anyway - it just parses
index.html.
Comment 1 David Konecny 2004-04-29 09:08:01 UTC
Any progress on this?
Comment 2 Jan Pokorsky 2004-04-29 12:22:08 UTC
No, the refactoring stuff has higher priority now.
Comment 3 Jesse Glick 2004-09-28 15:48:21 UTC
Should be considered a candidate for E, I think.
Comment 4 Jan Pokorsky 2004-09-29 15:01:51 UTC
The httpfilesystem code is still present but not used any more in nb
4.0. AFAIK, Tomas already has FO -> URL rewrite in his schedule.
Reassigning
Comment 5 Jan Pokorsky 2004-11-23 14:33:38 UTC
*** Issue 51605 has been marked as a duplicate of this issue. ***
Comment 6 Tomas Zezula 2005-01-11 07:59:36 UTC
*** Issue 45321 has been marked as a duplicate of this issue. ***
Comment 7 Jesse Glick 2005-04-29 23:00:52 UTC
Any plans to do this?
Comment 8 Tomas Zezula 2005-05-02 09:54:28 UTC
Yes, it would be nice to have it. But probably not for 4.2
Comment 9 Tomas Zezula 2006-01-25 10:11:28 UTC
*** Issue 71773 has been marked as a duplicate of this issue. ***
Comment 10 Jesse Glick 2006-02-08 18:45:57 UTC
*** Issue 71148 has been marked as a duplicate of this issue. ***
Comment 11 rptmaestro 2006-07-21 14:09:10 UTC
Updating this RFE for the current version. I changed priority from P1 to P3 
because it's not absolutely essential, but it would be very nice to have.
Comment 12 Jesse Glick 2006-07-21 14:18:46 UTC
Original reported version was 4.0 so Version field should stay there. "Current"
is not used in standard modules at all.

P3 is too low for something with four duplicates IMHO.
Comment 13 Petr Jiricka 2007-02-12 13:26:33 UTC
What is the current outlook for this issue? Has anything changed with Retouche? 

BTW, I must say the subject/description of this issue is rather cryptic - the
real issue is that I can not specify a URL in library manager/Java platform
manager etc., right? A number of issues with a better description are marked as
a duplicate of this issue. Can someone please explain what exactly is needed and
why, beyond the actual UI change in the library manager? Why is it necessary to
parse index.html? Thanks.
Comment 14 Jesse Glick 2007-02-12 22:25:31 UTC
As I understand it, supporting remote (HTTP) URLs for Javadoc in the Library
Manager means rewriting how Javadoc is displayed. Currently the javadoc module
tries to use FileObject's to index and display Javadoc HTML pages. This only
works for local (folder or ZIP) Javadoc. Displaying remote content is pretty
straightforward, but indexing it is a little trickier because you cannot ask
conventional HTTP servers for a list of files in a URL folder; you need to parse
some HTML files with well-known locations to see what URLs can be expected to
exist. This is in contrast to the current FileObject-based impl which would just
look for children of a folder.
Comment 15 Jan Pokorsky 2007-05-03 12:32:23 UTC
*** Issue 103200 has been marked as a duplicate of this issue. ***
Comment 16 Jan Pokorsky 2007-05-14 10:12:31 UTC
*** Issue 103859 has been marked as a duplicate of this issue. ***
Comment 17 Petr Jiricka 2008-09-11 18:53:39 UTC
> indexing it is a little trickier

Is it really necessary to index remote Javadocs? I understand that some features may depend on the indexing, but the
most important aspect of this issue is to simply display the Javadoc; I think users would be happy even if we could just
*display* the remote Javadoc in the completion doc window, without any other features - these would only work on local
Javadoc.
Comment 18 Jesse Glick 2008-09-11 22:20:55 UTC
True, indexing is a secondary priority, though I think it could be implemented without much extra effort - and in a way
that would work uniformly for both local and remote Javadoc, so not bloating the code.
Comment 19 Jan Pokorsky 2009-10-12 11:07:32 UTC
*** Issue 172341 has been marked as a duplicate of this issue. ***
Comment 20 Jesse Glick 2010-04-20 20:29:41 UTC
*** Bug 134538 has been marked as a duplicate of this bug. ***
Comment 21 Jesse Glick 2010-04-20 23:03:31 UTC
I'm looking at a patch for this.
Comment 22 Jesse Glick 2010-04-21 22:52:10 UTC
Created attachment 97825 [details]
Patch being tested
Comment 23 Jesse Glick 2010-04-21 22:55:36 UTC
I think I have it working, including indexing and code completion popup. Review especially from tzezula would be appreciated. I think it's too late for 6.9 (new feature, technically), but could be committed whenever the trunk is released for the next version.
Comment 24 Quality Engineering 2010-04-22 04:33:18 UTC
Integrated into 'main-golden', will be available in build *201004220200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/f2b85b04bb9d
User: Jesse Glick <jglick@netbeans.org>
Log: Removing dead code that would anyway be rewritten for #41443.
Comment 25 Tomas Zezula 2010-04-22 11:30:19 UTC
The patch seems fine, thanks Jesse. The only possible problem may be a performance degradation caused by network IO (url.openStream().close()) in the SourceUtil.getJavadoc() used by code completion, navigator, etc. When the classpath contains a http url attached to jar in the beginning of  classpath it's always checked. Maybe I should add some kind of caching into SourceUtil.getJavadoc(), in the navigator case the same page is looked up several times. Also for normal zip files isn't fo.getInputStream faster then url.openStream()? I am not sure about this and it probably does not matter.
Comment 26 Jesse Glick 2010-04-22 17:45:50 UTC
(In reply to comment #25)
> The only possible problem may be a performance degradation

I thought of this. I did not notice any problem in interactive testing, though I did not measure it. Obviously the code completion Javadoc popup takes longer to appear for e.g. http://java.sun.com/whatever/api/ but I still found it quite fast enough to be usable; results may be less satisfactory on slower sites such as dev.java.net.

> network IO (url.openStream().close()) in SourceUtil.getJavadoc()

I did think of returning InputStream rather than URL so callers just wanting the stream could use it immediately. But some callers really just want the URL, or (for reasons of character encoding) sometimes reopen the stream.

Might be feasible to return a pair of <URL,InputStream> where the caller is obliged to close the IS but could reopen it. Either would be an incompatible API change, I am afraid, unless a new variant method were introduced and the former @Deprecated.

> Maybe I should add some kind of caching into
> SourceUtil.getJavadoc(), in the navigator case the same page is looked up
> several times.

Might be useful. I did not check the Navigator because I don't know what that has to do with Javadoc. I checked the popup in code completion, and the Javadoc window.

> for normal zip files isn't fo.getInputStream faster than url.openStream()?

I'm not sure. The NB module system does install its own handler for 'jar' protocol, though for now this only performs special handling for NB module JARs or their Class-Path extensions. It could probably be extended to delegate to ArchiveURLMapper's cache, providing a transparent speedup for any code using this URL protocol on user JARs.

Or the URLUtil methods could check for 'jar' URLs and treat them specially (URLMapper.findFileObject(url).getInputStream() plus null check); most of the calls to openStream should already occur there, and the others from the javadoc module could probably be factored out there. Would not help in java.source which would need to copy that logic.
Comment 27 Tomas Zezula 2010-04-26 16:46:01 UTC
OK, anyway if there will be some performance problems they will be reported soon after the integration :-)
I am for integrating this patch and if needed we can do caching and special handling of jars if needed. The thing I was afraid is long network timeout when network is unreachable. The Javadoc is calculated in RequestProcessor (at least I hope so) but the throughput is 1. This may cause that the feature will not work during the timeout. Again if it will be problem we can try to interrupt the thread and catch the InterruptedIOException, but I hope this will not be needed.
Comment 28 Jesse Glick 2010-06-16 22:54:16 UTC
I have done various sorts of caching to minimize unnecessary HTTP transactions; use the platform cache for jar protocol; and automatically set the official Javadoc location for newly created (or default) Java platforms between 1.3 and 1.7. I'm sure there are various bugs to be shaken out, and ElementJavadoc could perhaps be refactored to never even ask for remote Javadoc if it has sources available, but everything seems to basically work. core-main #e17b99277530
Comment 29 Quality Engineering 2010-06-18 03:19:15 UTC
Integrated into 'main-golden', will be available in build *201006180001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/e17b99277530
User: Jesse Glick <jglick@netbeans.org>
Log: Issue #41443: support HTTP URLs for Javadoc.
Comment 30 Quality Engineering 2010-06-22 03:19:18 UTC
Integrated into 'main-golden', will be available in build *201006220001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/a17b98a2ace7
User: Jesse Glick <jglick@netbeans.org>
Log: Follow-up to #41443: look up official online Javadoc for 6.x NB platforms automatically.
Comment 31 Petr Jiricka 2010-09-02 13:46:01 UTC
> OK, anyway if there will be some performance problems they will be reported
> soon after the integration :-)

Ok, here they are: bug 187919.
Comment 32 Jesse Glick 2010-10-15 15:17:21 UTC
*** Bug 185273 has been marked as a duplicate of this bug. ***