This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 235202 - IDE should support all functions in PAC files
Summary: IDE should support all functions in PAC files
Status: REOPENED
Alias: None
Product: platform
Classification: Unclassified
Component: Proxy (show other bugs)
Version: 7.4
Hardware: PC Windows 7
: P3 normal with 2 votes (vote)
Assignee: Libor Fischmeistr
URL:
Keywords: PATCH_AVAILABLE
: 240225 251729 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-08-29 06:50 UTC by terje7601
Modified: 2015-07-07 09:18 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
View -> IDE Log (36.45 KB, application/octet-stream)
2013-08-29 06:50 UTC, terje7601
Details
View -> IDE Log (Reload proxy) (34.91 KB, text/plain)
2013-08-29 09:20 UTC, terje7601
Details
View -> IDE Log (with -J-Djava.net.preferIPv4Stack=true) (37.21 KB, application/octet-stream)
2013-08-29 12:24 UTC, terje7601
Details
Log file (42.68 KB, text/plain)
2013-09-13 08:08 UTC, MackSix
Details
The sanitized PAC file (2.29 KB, text/plain)
2013-11-12 08:58 UTC, terje7601
Details

Note You need to log in before you can comment on or make changes to this bug.
Description terje7601 2013-08-29 06:50:59 UTC
Created attachment 139400 [details]
View -> IDE Log

On a fresh install of NetBeans, when I go straight to Options & click "Test connection" with "Use System Proxy Settings" selected, I get "Cannot assign requested address: connect".


Product Version: NetBeans IDE Dev (Build 201308280001)
Updates: Updates available
Java: 1.8.0-ea; Java HotSpot(TM) Client VM 25.0-b45
Runtime: Java(TM) SE Runtime Environment 1.8.0-ea-lambda-nightly-h5451-20130819-b104-b00
System: Windows 7 version 6.1 running on x86; Cp1252; en_US (nb)
Comment 1 Libor Fischmeistr 2013-08-29 08:27:00 UTC
On the first view of the log I see you are running on JDK 8. There is the most likely the problem. On JDK 1.8 there is probably still the problem with JavaScript engine and then the PAC file cannot be processed... Please set the proxy manually meantime. Anyway, this should be fixed, I will try to evaluate it more in detail.

If you will run on JDK 7 this issue should not appear.

Thanks for the report
Comment 2 terje7601 2013-08-29 09:20:21 UTC
Created attachment 139414 [details]
View -> IDE Log (Reload proxy)

On a fresh install, after I did "Reload" of "Use System Proxy Settings"
Comment 3 terje7601 2013-08-29 09:22:27 UTC
(In reply to Libor Fischmeistr from comment #1)
> On the first view of the log I see you are running on JDK 8. There is the
> most likely the problem. On JDK 1.8 there is probably still the problem with
> JavaScript engine and then the PAC file cannot be processed... Please set
> the proxy manually meantime. Anyway, this should be fixed, I will try to
> evaluate it more in detail.
> 
> If you will run on JDK 7 this issue should not appear.
> 
> Thanks for the report

Are you referring to bug 230257 ? If so, I verified this is fixed & no longer get this exception with recent NetBeans builds.

The processing of the PAC file seems to be going correct as well: when I do "Reload", the log says "System network proxy reloading succeeded." (see the 2nd attachment)

Thanks for the quick response
Comment 4 Libor Fischmeistr 2013-08-29 10:57:03 UTC
(In reply to terje7601 from comment #3)
> (In reply to Libor Fischmeistr from comment #1)
> > On the first view of the log I see you are running on JDK 8. There is the
> > most likely the problem. On JDK 1.8 there is probably still the problem with
> > JavaScript engine and then the PAC file cannot be processed... Please set
> > the proxy manually meantime. Anyway, this should be fixed, I will try to
> > evaluate it more in detail.
> > 
> > If you will run on JDK 7 this issue should not appear.
> > 
> > Thanks for the report
> 
> Are you referring to bug 230257 ? If so, I verified this is fixed & no
> longer get this exception with recent NetBeans builds.
> 
> The processing of the PAC file seems to be going correct as well: when I do
> "Reload", the log says "System network proxy reloading succeeded." (see the
> 2nd attachment)
> 
> Thanks for the quick response

Yes, I taught it might be related to it. A few moments ago I tried to run IDE on JDK 7 and JDK 8 and on both proxy setting were correctly loaded (PAC). So unfortunately this issue is not reproducible for me.

May I ask you to run on JDK 7? It would be great to know if there is the problem also valid.

Thank you very much
Comment 5 terje7601 2013-08-29 12:20:49 UTC
(In reply to Libor Fischmeistr from comment #4)
> Yes, I taught it might be related to it. A few moments ago I tried to run
> IDE on JDK 7 and JDK 8 and on both proxy setting were correctly loaded
> (PAC). So unfortunately this issue is not reproducible for me.
> 
> May I ask you to run on JDK 7? It would be great to know if there is the
> problem also valid.
> 
> Thank you very much

The problem also exists on JDK 7 for me, giving the exact same error.

Something else I tried, is adding "-J-Djava.net.preferIPv4Stack=true" to "netbeans_default_options" in the file "etc\netbeans.conf". With this I get a different error message, namely:

connect: Address is invalid on local machine, or port is not valid on remote machine

Again, I get the exact same behavior with JDK 7 & JDK 8.

Something else I noted in the logs, are the 2 lines:
INFO [org.netbeans.core.network.proxy.NetworkProxyReloader]: System network proxy TEST - http host: 
INFO [org.netbeans.core.network.proxy.NetworkProxyReloader]: System network proxy TEST - http port: 0

Is it expected that host is empty and port is 0?


Product Version: NetBeans IDE Dev (Build 201308280001)
Updates: Updates available
Java: 1.7.0_25; Java HotSpot(TM) Client VM 23.25-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_25-b17
System: Windows 7 version 6.1 running on x86; Cp1252; en_US (nb)
Comment 6 terje7601 2013-08-29 12:24:47 UTC
Created attachment 139429 [details]
View -> IDE Log (with -J-Djava.net.preferIPv4Stack=true)

start NetBeans with JDK8, do "Reload", then do "Test Connection"
JDK7 gives the same log (except for some insignificant differences in the stack trace, which i pasted below)
Comment 7 terje7601 2013-08-29 12:25:42 UTC
java.net.ConnectException: connect: Address is invalid on local machine, or port is not valid on remote machine
	at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
	at java.net.Socket.connect(Socket.java:579)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:378)
	at sun.net.www.http.HttpClient$1.run(HttpClient.java:430)
	at sun.net.www.http.HttpClient$1.run(HttpClient.java:428)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.net.www.http.HttpClient.privilegedOpenServer(HttpClient.java:427)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:468)
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:203)
	at sun.net.www.http.HttpClient.New(HttpClient.java:290)
	at sun.net.www.http.HttpClient.New(HttpClient.java:306)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:995)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:974)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:849)
	at org.netbeans.core.ui.options.general.GeneralOptionsModel.testHttpConnection(GeneralOptionsModel.java:330)
[catch] at org.netbeans.core.ui.options.general.GeneralOptionsModel.testProxy(GeneralOptionsModel.java:315)
	at org.netbeans.core.ui.options.general.GeneralOptionsModel.access$000(GeneralOptionsModel.java:62)
	at org.netbeans.core.ui.options.general.GeneralOptionsModel$1.run(GeneralOptionsModel.java:244)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1432)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2042)
Comment 8 Libor Fischmeistr 2013-08-29 12:29:46 UTC
No, it isn't expected. There should be correct values - proxy which is returned for netbens.org server.

The weird is, that for the first time, the PAC returns null and when reloaded the PAC starts to response correctly.

I would like to help you, but if I am not able to reproduce, I cannot. And as more as I look at this issue, I thing that the problem will be somewhere in network :/
Comment 9 terje7601 2013-08-30 08:27:46 UTC
(In reply to Libor Fischmeistr from comment #8)
> No, it isn't expected. There should be correct values - proxy which is
> returned for netbens.org server.
> 
> The weird is, that for the first time, the PAC returns null and when
> reloaded the PAC starts to response correctly.

Is this a separate bug for which I should file a new issue?

 
> I would like to help you, but if I am not able to reproduce, I cannot. And
> as more as I look at this issue, I thing that the problem will be somewhere
> in network :/

I've found something that might be causing this: our PCs are running "Kaspersky Endpoint Security 8", which is probably not setup to give Java programs the required permissions (see e.g. here http://forum.kaspersky.com/index.php?s=839138bf56d0f91e8dab1fac661a0c7b&showtopic=255283 where they get the exact same error as I did)
To further investigate this issue, I would like to find out the arguments passed to testHttpConnection ( http://hg.netbeans.org/main-golden/file/7142577c6b27/core.ui/src/org/netbeans/core/ui/options/general/GeneralOptionsModel.java#l330 ). If I would know the value of the URL and Proxy arguments, I'd be able to write a simple standalone test case to provide to our system engineers. I guess the easiest would be to follow the instructions here ( http://wiki.netbeans.org/WorkingWithNetBeansSources ) & modify the code to dump the values?
Comment 10 Libor Fischmeistr 2013-09-02 08:02:48 UTC
(In reply to terje7601 from comment #9)
> (In reply to Libor Fischmeistr from comment #8)
> > No, it isn't expected. There should be correct values - proxy which is
> > returned for netbens.org server.
> > 
> > The weird is, that for the first time, the PAC returns null and when
> > reloaded the PAC starts to response correctly.
> 
> Is this a separate bug for which I should file a new issue?

I would not say, yet.

> > I would like to help you, but if I am not able to reproduce, I cannot. And
> > as more as I look at this issue, I thing that the problem will be somewhere
> > in network :/
> 
> I've found something that might be causing this: our PCs are running
> "Kaspersky Endpoint Security 8", which is probably not setup to give Java
> programs the required permissions (see e.g. here
> http://forum.kaspersky.com/index.
> php?s=839138bf56d0f91e8dab1fac661a0c7b&showtopic=255283 where they get the
> exact same error as I did)
> To further investigate this issue, I would like to find out the arguments
> passed to testHttpConnection (
> http://hg.netbeans.org/main-golden/file/7142577c6b27/core.ui/src/org/
> netbeans/core/ui/options/general/GeneralOptionsModel.java#l330 ). If I would
> know the value of the URL and Proxy arguments, I'd be able to write a simple
> standalone test case to provide to our system engineers. I guess the easiest
> would be to follow the instructions here (
> http://wiki.netbeans.org/WorkingWithNetBeansSources ) & modify the code to
> dump the values?

Good point with the Kaspersky. I've installed it and now I am able to reproduce. This error is shown for me when NB IDE is in "High restricted" category. If it is moved to "Low restricted" the proxy settings are properly loaded.

Due to this I don't think there is something to fix in NetBeans.

Please try to change restrictions in Kaspersky:
In Kaspersky:
- Click on Settings
- Click on Application control
- Click on Applications
- Find NetBeans IDE, change restrictions to Low or Trusted.

Also please report the result.

Thanks
Comment 11 terje7601 2013-09-02 17:50:12 UTC
(In reply to Libor Fischmeistr from comment #10)
> Good point with the Kaspersky. I've installed it and now I am able to
> reproduce. This error is shown for me when NB IDE is in "High restricted"
> category. If it is moved to "Low restricted" the proxy settings are properly
> loaded.
> 
> Due to this I don't think there is something to fix in NetBeans.
> 
> Please try to change restrictions in Kaspersky:
> In Kaspersky:
> - Click on Settings
> - Click on Application control
> - Click on Applications
> - Find NetBeans IDE, change restrictions to Low or Trusted.
> 
> Also please report the result.
> 
> Thanks

I'm in holidays till September 16th, but I'll make sure to report the result once I'm back at work.
Comment 12 MackSix 2013-09-13 08:08:38 UTC
Hi, this does not work for me and never has worked. I have Windows Firewall shut off, AVG Free shut off and Malwarebytes turned off and it give me the error message. I never have trouble with my connection, although it fails to test when System Proxy is selected. 

I have different sorts of errors in my log. See attached.
Comment 13 MackSix 2013-09-13 08:08:56 UTC
Created attachment 140022 [details]
Log file
Comment 14 terje7601 2013-10-04 13:08:39 UTC
I'm pretty sure I found the issue now :) The last NetBeans code in all stack traces is org.netbeans.core.ui.options.general.GeneralOptionsModel.testHttpConnection(GeneralOptionsModel.java:330), which takes 2 parameters URL & Proxy.

So I copied this method & tried reproducing this issue. The following code triggers the exact same errors I get with NetBeans:

// System.setProperty("java.net.preferIPv4Stack", "true");
URL url = new URL("https://www.google.be");
Proxy p = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("ourproxyhost", 0));
testHttpConnection(url, p);

So I believe the problem is that somehow the port is set to 0 instead of the correct value (8080 in my case). When I change the port to 8080, testHttpConnection returns true.

PS: I just checked & this still occurs in 7.4 RC2
Comment 15 Libor Fischmeistr 2013-11-11 10:42:36 UTC
(In reply to terje7601 from comment #14)
> I'm pretty sure I found the issue now :) The last NetBeans code in all stack
> traces is
> org.netbeans.core.ui.options.general.GeneralOptionsModel.
> testHttpConnection(GeneralOptionsModel.java:330), which takes 2 parameters
> URL & Proxy.
> 
> So I copied this method & tried reproducing this issue. The following code
> triggers the exact same errors I get with NetBeans:
> 
> // System.setProperty("java.net.preferIPv4Stack", "true");
> URL url = new URL("https://www.google.be");
> Proxy p = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("ourproxyhost",
> 0));
> testHttpConnection(url, p);
> 
> So I believe the problem is that somehow the port is set to 0 instead of the
> correct value (8080 in my case). When I change the port to 8080,
> testHttpConnection returns true.
> 
> PS: I just checked & this still occurs in 7.4 RC2

Sorry for very late response.

The problem is in:
INFO [org.netbeans.core.network.proxy.NetworkProxyReloader]: System network proxy - mode: auto
INFO [org.netbeans.core.network.proxy.NetworkProxyReloader]: System network proxy - pac url: http://proxybe.jetair.be:8083/
INFO [org.netbeans.core.network.proxy.NetworkProxyReloader]: System network proxy TEST - http host: 
INFO [org.netbeans.core.network.proxy.NetworkProxyReloader]: System network proxy TEST - http port: 0

You can see that the PAC file does not return any proper value on two last lines of the log.

If it would be possible, I want to ask you for the PAC file. I understand there can be some security issues for you to attach it here, but it can be sanitized.

Thanks in advance
Comment 16 terje7601 2013-11-12 08:58:02 UTC
Created attachment 142080 [details]
The sanitized PAC file

and my current version info:

Product Version: NetBeans IDE 7.4 (Build 201310111528)
Java: 1.8.0-ea; Java HotSpot(TM) Client VM 25.0-b45
Runtime: Java(TM) SE Runtime Environment 1.8.0-ea-lambda-nightly-h109-20130902-b106-b00
System: Windows 7 version 6.1 running on x86; Cp1252; en_US (nb)
Comment 17 Libor Fischmeistr 2013-11-13 10:08:25 UTC
(In reply to terje7601 from comment #16)
> Created attachment 142080 [details]
> The sanitized PAC file
> 
> and my current version info:
> 
> Product Version: NetBeans IDE 7.4 (Build 201310111528)
> Java: 1.8.0-ea; Java HotSpot(TM) Client VM 25.0-b45
> Runtime: Java(TM) SE Runtime Environment
> 1.8.0-ea-lambda-nightly-h109-20130902-b106-b00
> System: Windows 7 version 6.1 running on x86; Cp1252; en_US (nb)

Thank you for the PAC. It helped me a lot.

The problem is in "dnsResolve()" and "myIpAddress()" functions which are not supported in our PAC resolving algorithm, but should be.

Unfortunately there is no plan in near future to fix it because the larger changes would be necessary.
Comment 18 terje7601 2013-11-14 11:41:42 UTC
(In reply to Libor Fischmeistr from comment #17)
> Thank you for the PAC. It helped me a lot.
> 
> The problem is in "dnsResolve()" and "myIpAddress()" functions which are not
> supported in our PAC resolving algorithm, but should be.
> 
> Unfortunately there is no plan in near future to fix it because the larger
> changes would be necessary.

I'm glad you found the problem. It's of course a very convenient feature, but in the meantime i can use manual proxy settings.
Comment 19 phansson 2013-12-04 11:45:54 UTC
I suspect that this report and Bug 239016 are at least somewhat similar. The mentioned Bug Report explains why the new "Test Connection" button in 7.4 doesn't work when there's no proxy at all.
Comment 20 Libor Fischmeistr 2013-12-08 11:00:43 UTC
(In reply to phansson from comment #19)
> I suspect that this report and Bug 239016 are at least somewhat similar. The
> mentioned Bug Report explains why the new "Test Connection" button in 7.4
> doesn't work when there's no proxy at all.

After the closer review of this bug it does not look like related. It's because this issue is caused by PAC resolving algorithm.
Comment 21 dieferman 2013-12-15 08:30:32 UTC
Avast Antivirus is also blocking the connection... I disabled its shields temporarily and Netbeans updated normally. Neither applying exceptions on Avast for http://*.netbeans.org nor http://updates.netbeans.org nor http://plugins.netbeans.org allowed me to connect (and obviously not to update).
Comment 22 Libor Fischmeistr 2014-01-14 11:02:02 UTC
*** Bug 240225 has been marked as a duplicate of this bug. ***
Comment 23 phansson 2014-06-08 18:58:28 UTC
> The problem is in "dnsResolve()" and "myIpAddress()" functions which are not
> supported in our PAC resolving algorithm, but should be.

Java Web Start has their own implementation of all Javascript helper functions needed for use with PAC file evaluation. I'm wondering if it can be leveraged also outside of JWS so that it can be used to fix the issue in this report?

The functions I refer to sit in deploy.jar. Have a look at classes in package com.sun.deploy.net.proxy, in particular have a look at Com.sun.deploy.net.proxy.AutoProxyScript.

The Javascript PAC helper methods implemented in JWS are extremely simple. In fact they don't even do DNS lookups. I believe what JWS does is they make sure a string hostname is never passed to FindProxyForURL(url, host). If we can be assured that only an IP string is ever passed to this function then those Javascript helper functions become trivial. In other words in ProxyAutoConfig.java line 177 make sure to always pass an IP address (not a hostname) as the second parameter to FindProxyForURL().

Obviously the use of a class in deploay.jar is then Java implementation dependent so should only be used when executing on a Sun JVM but I believe the Javascript methods are so simple that one can easily copy them out of the JDK source tree and use as-is.


Libor, I hope I'm not sending you off tracks here and I may be in over my head here. Just some pointers.
Comment 24 phansson 2014-06-08 19:43:04 UTC
Further to the issue of having all the required Javascript helper functions available for PAC file evaluation:

There's also project proxy-vole (https://code.google.com/p/proxy-vole/) which takes a different approach than JWS. The proxy-vole project simply implements the needed Javascript helper functions in pure Java and then binds them to the Javascript engine so they can be executed from within the PAC script. I guess it's a matter of taste if one likes the proxy-vole approach or the JWS approach.
Comment 25 phansson 2014-06-18 09:53:47 UTC
The underlying problem here, the missing dnsResolve() and myIpAddress() js functions, has been fixed in this project: https://bitbucket.org/phansson/netbeansproxy2.

The project is based on NetBeans 8.0 code.

Feel free to use it.
Comment 26 Libor Fischmeistr 2015-07-07 09:18:19 UTC
*** Bug 251729 has been marked as a duplicate of this bug. ***