This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 202500 - Retriever doesn't work with proxy which is set via "Use System Proxy Settings" option
Summary: Retriever doesn't work with proxy which is set via "Use System Proxy Settings...
Status: VERIFIED DUPLICATE of bug 209095
Alias: None
Product: platform
Classification: Unclassified
Component: -- Other -- (show other bugs)
Version: 7.1
Hardware: PC Windows XP
: P3 normal (vote)
Assignee: Jiri Rechtacek
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-09-22 13:35 UTC by Jiri Skrivanek
Modified: 2012-11-14 12:21 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Screenshot - debugger showing configured URLConnection (18.96 KB, image/jpeg)
2011-11-10 15:01 UTC, Svata Dedic
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Skrivanek 2011-09-22 13:35:49 UTC
Google AdSenseForContent Service and AdSenseForSearch Service seem unreachable from the IDE but URL in browser works. To reproduce:

- open Services view
- expand Web Services|Google
- right-click AdSenseForContent Service and choose Refresh. It print this message to the console:

Sep 22, 2011 3:31:26 PM : Retrieving Location: https://sandbox.google.com/api/adsense/v2/AdSenseForSearchService?wsdl 

Error: An I/O error occured. Software caused connection abort: socket write error

Product Version: NetBeans IDE 7.1 Beta (Build 201109212202)
Java: 1.7.0; Java HotSpot(TM) Client VM 21.0-b17
System: Windows XP version 5.1 running on x86; Cp1250; en_US (nb)
Comment 1 Denis Anisimov 2011-09-22 14:12:15 UTC
I'm using 1.6.0_24 Java.
All works fine.
Try to start NB with JDK 1.6.
It seems this is the problem.
Comment 2 Jiri Skrivanek 2011-09-23 08:39:14 UTC
JDK doesn't matter. But proxy does. If I use no proxy, it works. If I am behind the proxy, it doesn't work. Probably proxy parameter is not set while connecting to the provider.
Comment 3 Denis Anisimov 2011-09-23 09:14:20 UTC
(In reply to comment #2)
> JDK doesn't matter. But proxy does. If I use no proxy, it works. If I am behind
> the proxy, it doesn't work. Probably proxy parameter is not set while
> connecting to the provider.

So this is VERY important information and it should be in the summary of the 
bug from very beginning.
Because this is exactly the issue.
Comment 4 Denis Anisimov 2011-09-23 09:26:35 UTC
I cannot reproduce the issue even with proxy server set.
Please provide information how do you set your proxy server.
Please make sure that your proxy works. 
>If I am behind the proxy, it doesn't work.
What does it mean ?
Did you set the proxy in IDE if you are behind the proxy ?
NB is not able to automatically detect if you are behind of proxy.
Comment 5 Jiri Skrivanek 2011-09-27 13:07:18 UTC
In Firefox I have proxy set to http://wpad/wpad.dat. If I start IDE and check Tools>Options, it uses "Use System Proxy Settings". And it works correctly because the start page in IDE is refreshed by remote content. Now if I try to refresh the service it fails as described.
If I am at home, I set No proxy in Firefox and I am able to refresh the service in IDE.
Comment 6 Denis Anisimov 2011-09-28 04:54:11 UTC
(In reply to comment #5)
> In Firefox I have proxy set to http://wpad/wpad.dat. If I start IDE and check
> Tools>Options, it uses "Use System Proxy Settings". And it works correctly
> because the start page in IDE is refreshed by remote content. Now if I try to
> refresh the service it fails as described.
> If I am at home, I set No proxy in Firefox and I am able to refresh the service
> in IDE.

Could you please do additional localization of the problem : please set the proxy 
manually in the settings and describe the result.

By the way I'm not sure that you Firefox settings is your SYSTEM settings.
>And it works correctly
>because the start page in IDE is refreshed by remote content.
There could be a number of reasons why it is shown ( it could be cached ).

As I understand you are trying the proxy with absence of the direct Internet connection. May be it could be issue in  my usecase.
I will check it . But still please localize the problem.
Comment 7 Denis Anisimov 2011-09-28 17:49:59 UTC
OK, I did localization instead of reporter.
All works fine if 
- there is no proxy setting and there is a direct connection via http(s).
- the proxy is set explicitly manually . I know this because I saw a http request in proxy server log file.

There COULD be a problem with "Use System Proxy Settings" option. 
Here is why "COULD" is used : I don't know what mean "Use System Proxy Settings".
I'm pretty sure that Firefox setting is not "System Proxy Settings".
But I've tried to set proxy via IE which should be considered as "System Proxy Settings" in Win XP.
In my case this setting was just ignored. 

But once again : I can't affirm that I've set the "System Proxy Settings" and retriever doesn't work in the result. There is no any help information what 
"Use System Proxy Settings" means. If original reporter has this information
 then please provide it.

About reassignment : WSDL service node which is mentioned in the original bug description delegate content retrieval to org.netbeans.modules.xml.retriever.impl.RetrieverImpl class which is default
Retriever implementation. The three argument method retrieveResource() is used .
So resource access is done inside Retriever. It's responsible for URI access. 
It should care about proxy setting.

To the reporter : please investigate a reason of the issue more deeply each 
time when you file it. The issue should be localized. The description of the issue should be at least about exact problem but not about the consequence.
It also should contain exact setting which is the reason of the bug if any.
Comment 8 Jiri Skrivanek 2011-10-03 09:24:27 UTC
To summarize steps to reproduce:

- set proxy in Internet Explorer and Firefox to script http://wpad/wpad.dat
- start or restart IDE
- open Tools|Options and select "Use System Proxy Settings"
- open Services view
- expand Web Services|Google
- right-click AdSenseForContent Service and choose Refresh. It fails to retrieve https://.... If you try to refresh "StrikeIron|EmailVerify30" it works (maybe because it is http and not https)
- open Tools|Options and provide "Manual Proxy Settings"
- everything works
- with No Proxy (when directly connected to internet) it also works
Comment 9 Svata Dedic 2011-11-07 16:55:08 UTC
Verified that XML retriever actually use proxy, if configured by the system. What "System Proxy" means is defined by NB launcher. On UNIXes, it uses Gnome configuration, KDE configuration, MacOS X configuration and falls back to http_proxy env variable.

On Windows, Registry key Software\\Microsoft\\Windows\\CurrentVersion\\Internet settings under HKEY_CURRENT_USER is read.
Comment 10 Jiri Rechtacek 2011-11-07 18:07:07 UTC
You should use Network Settings API for read correct proxy host/port for given URI and programmatically set Retriever to use them.

http://bits.netbeans.org/dev/javadoc/org-openide-util/org/openide/util/NetworkSettings.html
Comment 11 Svata Dedic 2011-11-08 06:39:49 UTC
I need a piece of advice:

the Retriever currently uses standard java.net.ProxySelector API to select Proxy object, actual implementation used at runtime is org.netbeans.core.NbProxySelector.
ProxySelector.getDefault().select(uri) is used, then 1st proxy from the list is used.

Should I rewrite this code to use Network Settings API directly ?
Comment 12 Jiri Rechtacek 2011-11-08 07:50:21 UTC
(In reply to comment #11)
> I need a piece of advice:
> 
> the Retriever currently uses standard java.net.ProxySelector API to select
> Proxy object, actual implementation used at runtime is
> org.netbeans.core.NbProxySelector.
> ProxySelector.getDefault().select(uri) is used, then 1st proxy from the list is
> used.
That's correct.

> Should I rewrite this code to use Network Settings API directly ?
Not need if you can use ProxySelector. We need to find out cases if any where it does work properly and debug it.
Comment 13 Svata Dedic 2011-11-08 08:10:01 UTC
Good, then if NbProxySelector.select() may work incorrectly for some OS/configuration, then I guess the issue does not belong to xml/retriever component ?
Comment 14 Jiri Rechtacek 2011-11-08 08:14:16 UTC
(In reply to comment #13)
> Good, then if NbProxySelector.select() may work incorrectly for some
> OS/configuration, then I guess the issue does not belong to xml/retriever
> component ?
Depends on where is a problem, I don't think NbProxySelector.select() may work incorrectly for some OS/configuration. Needs to show problematic case and investigate it.
Comment 15 Svata Dedic 2011-11-10 13:16:05 UTC
Mozilla's settings are not shared with the rest of the system, information about http://wpad/wpad.dat is stored in Mozilla's profile preferences (preds.js). Intenet Explorer, for example, is not affected by setting proxy in Mozilla. Neither are Netbeans.

Incidentally, http://www.netbeans.org is accessible even without using the proxy (I assume we are using the same one); so the feeds are loaded - verified using Wireshark spying my local machine: IDE connects directly to netbeans.org (192.9.164.101)

Please verify how your feeds were loaded and whether the system (not just Mozilla browser) knows about the proxy - thanks.
Comment 16 Jiri Skrivanek 2011-11-10 13:41:48 UTC
I have set http://wpad/wpad.dat both in Internet Explorer and Firefox. If I look at IDE log, it contains the following

-Dnetbeans.system_http_proxy=PAC http://wpad/wpad.dat

I don't know what to try with feeds (which feeds?).
Comment 17 Svata Dedic 2011-11-10 15:00:46 UTC
Verified that the system knows about the wpad file. The SecureURLResourceRetriever gets a Proxy object for SOCKS proxy at 138.3.237.7:80 (in my case).

Then urlCon = downloadURL.openConnection(proxy); and then urlCon.connect() is executed. The URLConnection actually holds a properly (?) configured Proxy reference - see the attached picture.


Jirko R., please let me know what should I verify next at the XML retriever side.
Comment 18 Svata Dedic 2011-11-10 15:01:29 UTC
Created attachment 113094 [details]
Screenshot - debugger showing configured URLConnection
Comment 19 Svata Dedic 2011-11-10 15:27:21 UTC
I debugged the setup little further and:

if URL is "https://sandbox.google.com/api/adsense/v2/AdSenseForContentService?wsdl", the NbProxySelector calls org.netbeans.core.NbProxySelector.findProxyForUrl(url) and receives:

SOCKS @ emea-proxy.uk.oracle.com/138.3.237.7:80
SOCKS @  PROXY adc-proxy.oracle.com:80
DIRECT

During startup, news @netbeans.org are accessed, at URL "http://netbeans.org/news.xml", the findProxyForUrl provides these proxies:

HTTP @ emea-proxy.uk.oracle.com/138.3.237.7:80
HTTP @  PROXY adc-proxy.oracle.com:80
DIRECT

I don't think there's a SOCKS proxy and cannot find SOCKS instruction in wpad.dat which is actually used.

The issue seems to be in org.netbeans.core.ProxyAutoConfig, analyzeResult, line 237, where anything != http is taken as SOCKS proxy. In this case, the URI is "https://sandbox.google.com/api/adsense/v2/AdSenseForContentService?wsdl" and scheme == "https".
Comment 20 Ondrej Vrabec 2012-11-14 12:12:08 UTC

*** This bug has been marked as a duplicate of bug 209095 ***