Bug 195879 - Connect to FTP using HTTP proxy not working
Connect to FTP using HTTP proxy not working
Status: VERIFIED FIXED
Product: php
Classification: Unclassified
Component: FTP Support
7.0
All All
: P3 (vote)
: 7.3
Assigned To: Tomas Mysik
issues@php
: 7.0.1_WAIVER_APPROVED
Depends on: 200278 221339
Blocks:
  Show dependency treegraph
 
Reported: 2011-02-23 17:03 UTC by Vladimir Riha
Modified: 2012-11-21 13:39 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
IDE log (44.51 KB, text/plain)
2011-02-23 17:03 UTC, Vladimir Riha
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Riha 2011-02-23 17:03:30 UTC
Info:
Product Version: NetBeans IDE Dev (Build 201102230001)
Java: 1.7.0-ea; Java HotSpot(TM) Client VM 21.0-b02
System: Windows XP version 5.1 running on x86; Cp1252; en_US (nb)


I want to connect to FTP from behind proxy server. So I tried to set proxy  address and protocol in Tools|Options|General and set Manual Proxy Settings and in Advanced Proxy Settings checked the "Use the same proxy settings for all protocols" but nothing works. 

I tried to uncheck the "Use the same proxy settings for all protocols", then I tried Use System Proxy Settings, but still nothing. FTP credentials and proxy settings are 100% correct.

Every time I test connections, it ends up with some error. See the attached log, there are recorded to attempts with two different error messages.
Comment 1 Vladimir Riha 2011-02-23 17:03:53 UTC
Created attachment 106349 [details]
IDE log
Comment 2 Petr Pisl 2011-02-25 08:31:26 UTC
From the exception looks that your NB tries to connect through SOCKS4 and your proxy accept probably SOCKS5. Can it be this case?
Comment 3 bhall7 2011-03-04 02:24:16 UTC
This is indeed a problem. I posted about it on the Netbeans forums (http://forums.netbeans.org/viewtopic.php?t=36596), but haven't heard back.  I'm using Mac OS X 10.6.6 with Netbeans 6.9.1, and I can confirm that the proxy settings do not work as shown in the options.  I cannot get FTP to work from behind my proxy server.  When I connect via OpenVPN to my home LAN (effectively bypassing the proxy), it works just fine.  Also, other apps (such as FileZilla) work perfectly with the same proxy server/port settings for FTP behind my company's proxy server.  There is definitely something in Netbeans that needs to be looked at in regards to how the proxy server works for FTP.

When I try to connect via FTP, I get an error that says, "Cannot connect to server. Cause: timeout: socket is not established." I have entered my proxy server IP address and port in the Netbeans options under "Manual Proxy Settings," and I even tried enabling the proxy setting to "use the same proxy settings for all protocols," but I still cannot get it to connect.
Comment 4 Vladimir Riha 2011-03-04 05:09:36 UTC
I´m sorry for not responding, I will try to find out the version of proxy as soon as possible
Comment 5 OndrejBrejla 2011-03-14 12:56:12 UTC
I tried to evaluate this bug in a last few days, but what I evaluated...just that it doesn't work for me.

My use case:

I was behind a proxy, which used http protocol type (you see, NO SOCKS4 and NO SOCKS5).

When I used FileZilla to connect to my ftp, it worked without any problems.

But when I tried to use netbeans, it failed.

What I noted, was this line in my NB output: 
INFO [org.netbeans.core.NbProxySelector]: Incomplete SOCKS Server [/] found in ProxySelector[Type: 2] for uri socket://salma.profitux.cz:21.

What the hell is that? I filled ONLY "Proxy Settings -> Http Proxy", only http, I didn't checked "Use the same proxy..." in the "More..." settings window.

So I looked into that class source code http://bit.ly/gmIZs2 and I see, that it's logged on line 169. But how could I get there? It's else branch for SOCKS type only...so it's because condition on line 140 was false. Because protocol, of my uri, didn't start with "http". And what is my uri? You can read it from that logged line...it's socket://salma.profitux.cz:21.

And I really don't know, why there is a socket:// prefix...in my ftp configuration I only set "salma.profitux.cz" as a host and "21" as a port. So it was prefixed by someone and somwhere...but why? And where?

So the conclusion:

These errors are invoked when you are using proxy which uses HTTP proxy type.

It doesn't matter what you set to NetBeans proxy config, because everything fallbacks to SOCKS usage.
- if you set SOCKS proxy, and your proxy doesn't use this type, you get Bad version error...
- if you set HTTP proxy only, and your proxy USES this type, it just fallbacks to SOCKS (which isn't configured) and it ends up with an exception

So what should be a next step? Try to evaluate why a "uri" is prefixed by "socket://"...I think, that it shouldn't be. But where is it set? Maybe somwhere in core...:-/

Or maybe that mistake is entirely somewhere else...:-)

Please, can you confirm, that you are using proxy with HTTP PROXY TYPE?
Comment 6 Petr Pisl 2011-03-14 13:35:31 UTC
Ondro, could you please contact Jirka Rechtacek. He is working on proxy setting for all ide.

Jirko, do you have any notion, where could be a problem?
Comment 7 Vladimir Riha 2011-03-22 13:30:11 UTC
Turns out that my proxy doesn't support other protocol than http, so I close the bug as invalid. Sorry for the problems.
Comment 8 Marian Mirilovic 2011-03-22 16:06:14 UTC
v/c
Comment 9 OndrejBrejla 2011-03-24 10:38:36 UTC
I reopened this issue, because the bug is still there. I'm able to reproduce it.

But it's not so simple to fix it. There should be updated "libs.commons_net" module. It's version is too old and in the version 2.2 there is an experimental FTPHTTPClient class which could solve our problem.

But with that it's connected whole rewriting of the FTP support in the php.project module.

Conclusion: 
We should wait when FTPHTTPClient class will no longer be an experimental, then we can update "libs.commons_net" module and try to rewrite FTP support in php.project module. And for now I'll request a waiver.
Comment 10 Marian Mirilovic 2011-03-24 16:53:06 UTC
I agree with waiver for NB 7.0 .
Comment 11 Petr Pisl 2011-05-19 15:26:34 UTC
Still waiting for FTPHTTPClient class will not be an experimental.
Comment 12 Tomas Mysik 2011-06-02 06:31:04 UTC
Taking over.
Comment 13 Tomas Mysik 2011-06-09 12:44:54 UTC
Justification for waiver:

This issue cannot be fixed on NetBeans' side because NetBeans uses third-party library for FTP (Apache commons-net library [1], wrapped as NetBeans library). This library does not support HTTP proxy (at least in the current version). 

Possible solution for NB 7.1:

- update this FTP library to new major version (3.x)
- increase major version of the wrapped NB library (since the new FTP library is not fully compatible with the previous release currently used in NetBeans)
- minor changes in NetBeans code using this library

However, this is not decided yet since the new version of the FTP library supports HTTP proxy only in _experimental_ mode [2].

[1] http://commons.apache.org/net/
[2] http://commons.apache.org/net/api-3.0.1/org/apache/commons/net/ftp/FTPHTTPClient.html
Comment 14 Tomas Mysik 2011-08-31 14:27:04 UTC
OK, since we upgraded FTP library we use in NetBeans, I tried to use the FTPHTTPClient class as mentioned earlier. But I found some issues that means that we cannot use it (right, the class is experimental); filed as [1].

What is the proper way to continue with this issue? I suggest to downgrade it to P4 since there's likely nothing we can do on NetBeans' side and setting TM to Next. If there is better/another resolution for this issue, just let me know or simply do it.

Thanks.
[1] https://issues.apache.org/jira/browse/NET-422
Comment 15 magnus.johansson 2011-09-30 16:58:08 UTC
(In reply to comment #14)
> OK, since we upgraded FTP library we use in NetBeans, I tried to use the
> FTPHTTPClient class as mentioned earlier. But I found some issues that means
> that we cannot use it (right, the class is experimental); filed as [1].
> 
> What is the proper way to continue with this issue? I suggest to downgrade it
> to P4 since there's likely nothing we can do on NetBeans' side and setting TM
> to Next. If there is better/another resolution for this issue, just let me know
> or simply do it.
> 
> Thanks.
> [1] https://issues.apache.org/jira/browse/NET-422

I submitted a patch to apache for that issue or rather https://issues.apache.org/jira/browse/NET-267 to correct/complete the FTPHTTPClient code in commons-net.

Regards, Magnus
Comment 16 Tomas Mysik 2011-10-03 07:46:45 UTC
(In reply to comment #15)
> I submitted a patch to apache for that issue or rather
> https://issues.apache.org/jira/browse/NET-267 to correct/complete the
> FTPHTTPClient code in commons-net.

Thanks a lot for it! But we definitely need to wait for a new version of Commons-Net library (with this patch applied).

Thanks.
Comment 17 tcamminati 2012-11-02 11:04:41 UTC
Is there any progress in this issue? commons-net has fixed this bug https://issues.apache.org/jira/browse/NET-422, so if you upgrade to 3.1 this should do it. 

It is a pain having to sync projects outside netbeans using filezilla.
Comment 18 Tomas Mysik 2012-11-02 11:16:09 UTC
Sorry, I did not noticed it. I need to check whether the commons-net library upgrade can be done for NB 7.3 or not. Thanks for the reminder.
Comment 19 Tomas Mysik 2012-11-02 11:51:29 UTC
The library update is possible, I will do it first and then will try to fix this issue.

Thanks.
Comment 20 Tomas Mysik 2012-11-20 08:29:43 UTC
Should be fixed I hope. Please notice that HTTP proxy is used only for pure FTP (without any TLS).

Láďo or anyone else, please verify if possible.

Thanks.

http://hg.netbeans.org/web-main/rev/fd536fe005ca
Comment 21 Vladimir Riha 2012-11-20 08:35:29 UTC
I'll try it today, thanks.
Comment 22 Vladimir Riha 2012-11-20 12:05:22 UTC
Indeed it seems to work :) I tried to create project, download, upload on save and it works. It's not really fast but that could be due to our proxy...

One think I notice and wasn't clear to me. In Tools|Options if I check to use the same proxy host:port for all protocols, then it doesn't work and IDE log contains:

INFO [org.netbeans.modules.php.project.connections.ftp.FtpClient]: Exception while connecting
java.net.SocketException: Malformed reply from SOCKS server
	at java.net.SocksSocketImpl.readSocksReply(SocksSocketImpl.java:128)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:459)
	at java.net.Socket.connect(Socket.java:579)
	at java.net.Socket.connect(Socket.java:528)
	at java.net.Socket.<init>(Socket.java:425)
	at java.net.Socket.<init>(Socket.java:208)
	at org.apache.commons.net.ftp.FTPHTTPClient.connect(FTPHTTPClient.java:122)
[catch] at org.netbeans.modules.php.project.connections.ftp.FtpClient.connect(FtpClient.java:187)
	at org.netbeans.modules.php.project.connections.RemoteClient.connect(RemoteClient.java:195)
	at org.netbeans.modules.php.project.connections.ui.RemoteConnectionsPanel.testConnection(RemoteConnectionsPanel.java:216)
	at org.netbeans.modules.php.project.connections.ui.RemoteConnectionsPanel$1.run(RemoteConnectionsPanel.java:134)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1454)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2036)

Does it mean that if set, it prefers SOCKS?




Product Version: NetBeans IDE Dev (Build web-main-9264-on-20121120)
Java: 1.7.0_10-ea; Java HotSpot(TM) Client VM 23.6-b04
Runtime: Java(TM) SE Runtime Environment 1.7.0_10-ea-b16
System: Linux version 3.2.0-33-generic-pae running on i386; UTF-8; en_US (nb)
Comment 23 Tomas Mysik 2012-11-20 14:25:29 UTC
(In reply to comment #22)
> Does it mean that if set, it prefers SOCKS?

I really don't know - JirkoŘ, do you know? Anyway, I will look at it.

Thanks a lot.
Comment 24 Tomas Mysik 2012-11-20 14:38:43 UTC
So, I don't know since the proxy is returned the same (just verified using debugger).

BTW FTPHTTPClient is still experimental and the exception comes from it so perhaps it is a bug?
Comment 25 Quality Engineering 2012-11-21 13:39:57 UTC
Integrated into 'main-golden', will be available in build *201211211016* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/fd536fe005ca
User: Tomas Mysik <tmysik@netbeans.org>
Log: #195879 - Connect to FTP using HTTP proxy not working


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo