This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 225885 - sftp behind a proxy
Summary: sftp behind a proxy
Status: VERIFIED FIXED
Alias: None
Product: php
Classification: Unclassified
Component: FTP Support (show other bugs)
Version: 7.3
Hardware: All All
: P3 normal (vote)
Assignee: Tomas Mysik
URL:
Keywords:
Depends on: 226006
Blocks:
  Show dependency tree
 
Reported: 2013-02-08 11:53 UTC by hanryz
Modified: 2014-04-29 08:59 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
proposed fix (6.17 KB, patch)
2013-02-12 08:01 UTC, Tomas Mysik
Details | Diff
corrected fix (6.12 KB, patch)
2013-02-12 08:04 UTC, Tomas Mysik
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description hanryz 2013-02-08 11:53:45 UTC
Hello,

I am trying to setup a NetBeans PHP 5.3 project on a computer behind a proxy server. I am connecting to my home SFTP server listening on port 21 (not 22).

When connected via FileZila the sftp connection works just fine.

I set the Proxy server name manually in "tools/options/manual proxy settings"

Checking for IDE updates starts to work, once the proxy has been setup correctly.

However, testing the SFTP connection wont work with the following symptoms:

1) The proxy is set to work with all protocols: socket timeout
2) The proxy is set to work with only HTTP: a java exception is thrown.

From what I read in BUG#195879 it seems to me that pure FTP-connections are fixed by using the newest commons-net-3.2.jar. 

HOWEVER, SFTP still does not work. It is the case for 7.1.2, 7.3rc2 and the latest night build of 8 of FEB 2013.

If any further info/logs are needed I will provide.
Comment 1 Vladimir Riha 2013-02-08 12:56:45 UTC
Hi, you can try to change connection to passive mode in FTP settings as mentioned in issue 222527 to see if it helps (and proxy to only HTTP protocol). IDE log with the exception is located in your userdir:

[userdir]/var/log/messages.log

To find userdir, go to Help|About
Comment 2 hanryz 2013-02-08 13:36:53 UTC
Well, I've just tested  FTP (pure, implicit, explicit) with Passive Mode, with the expected result that no connection was possible: Some explanations: for pure: because of unknown server response from my SFTP server, (was clear it is not an FTP, but an SFTP server), for implicit and explicit it resulted in an "unknown host" exception, very funny. For FTP I used HTTP proxy.

I am sure that if I changed my server from FTP to SFTP then it will all work. However, I can not afford neither anonymous login nor plain text login. 

At the moment it really seems like NetBeans cannot handle SFTP servers via a proxy.
Comment 3 hanryz 2013-02-10 00:06:02 UTC
Hi guys,

over the weekend I tested Netbeans to connect directly, without any proxy.

I should say that both FTP and SFTP worked. I will test FTP via proxy on Monday and tell you result

See you
Comment 4 Tomas Mysik 2013-02-11 05:48:44 UTC
Please try to use NB 7.3 (RC 2 available now).

Thanks.
Comment 5 hanryz 2013-02-11 08:04:26 UTC
Hi Tomas,

I've tried these versions sofar: 7.1.2; 7.2, 7.3rc2 and the NB from 20130208.

They all not worked for my SFTP proxied server. Here are some details on 7.3rc2:

Pure FTP:

pure FTP connection works (but it WORKS!) only if I set pure HTTP proxy without any SOCKS server being set. 

When SOCKS server is set, "malformed reply from SOCKS server" exception is thrown.

SFTP:

if no SOCKS server is set java.net.unknownHostException is thrown.
if SOCKS server is set "timeout: socket is not established" error is displayed.

So, SFTP does not work at all:(

As I said before, I would be glad to use FTP, but I am not allowed to. Maybe you could look with the info I submitted at what could be wrong? I (and many others) would appreciate this.

Thanks in advance!
Comment 6 Tomas Mysik 2013-02-11 08:13:29 UTC
> Pure FTP:
> 
> pure FTP connection works (but it WORKS!) only if I set pure HTTP proxy without
> any SOCKS server being set. 

Yes, only this "proxy" scenario is supported by libraries we use in NetBeans [1].

> When SOCKS server is set, "malformed reply from SOCKS server" exception is
> thrown.

Unfortunately, unsupported.

> SFTP:

Unfortunately, no proxy is supported.

I will keep this issue opened and once more verify whether the proxy with SFTP is supported or not.

Thanks.
[1]
For FTP: Apache Commons-Net
For SFTP: JSch
Comment 7 Tomas Mysik 2013-02-11 08:17:30 UTC
So, it seems that it is supported [1][2]. I will look at it, unfortunately it is too late for NB 7.3. Perhaps a patch candidate?

Thanks.
[1] http://www.jcraft.com/jsch/examples/ViaHTTP.java
[1] http://www.jcraft.com/jsch/examples/ViaSOCKS5.java
Comment 8 hanryz 2013-02-11 08:21:02 UTC
Thanks Tomas! In the meanwhile (today in the evening) I will try
EXPLICIT/IMPLICIT SSL login on my FTP server. If at least this worked, it would
be possible for me to accept the login method. Do you think HTTP proxy would do
the job for EXPLICIT/IMPLICIT FTP server?

Regarding the patch, I don't care, I could you use a nightly build version if you mean this. When are you going to release a next RC or final?

Thanks in advance...
Comment 9 Tomas Mysik 2013-02-11 08:30:12 UTC
(In reply to comment #8)
> Thanks Tomas! In the meanwhile (today in the evening) I will try
> EXPLICIT/IMPLICIT SSL login on my FTP server. If at least this worked, it would
> be possible for me to accept the login method. Do you think HTTP proxy would do
> the job for EXPLICIT/IMPLICIT FTP server?

I don't think so; IMHO explicit/implicit is only for SSL/TLS connections.

> Regarding the patch, I don't care, I could you use a nightly build version if
> you mean this. When are you going to release a next RC or final?

Next final? Likely not decided yet, it is too early.

Thanks.
Comment 10 hanryz 2013-02-11 08:38:36 UTC
Ok, don't bother, I will check, if it works, ok, not is also fine. When (or better what nightly build version) do you think I can get as the patch candidate to test?

Greets
Comment 11 Tomas Mysik 2013-02-11 08:55:28 UTC
(In reply to comment #10)
> When (or
> better what nightly build version) do you think I can get as the patch
> candidate to test?

A message will automatically appear here but please notice that I am not sure when I will have a time to look at it.

Thanks.
Comment 12 hanryz 2013-02-11 20:35:20 UTC
Just wanted to tell you that I checked TSL/SSL auth via FTP. None worked even via a direct connection (no proxy)...

I am not sure if ImplicitSSL failed because I haven't set up on port 990 forwarding.
But ExplicitSSL seems to fail because of a missing certificate. However, I cannot find any setting for a certificate anywhere: nor on NB nor on my server.

So we forget FTP via Implicit/Explicit SSL. Please guys, have a look at the issue with the SFTP and the proxy asap. This will solve many problems. 

Thanks in advance,
Andre
Comment 13 Tomas Mysik 2013-02-12 08:00:32 UTC
I have created a patch but unfortunately it will not work correctly - see issue #226006 for more details. I will attach the patch here so it can be applied later.

Thanks.
Comment 14 Tomas Mysik 2013-02-12 08:01:49 UTC
Created attachment 131274 [details]
proposed fix
Comment 15 Tomas Mysik 2013-02-12 08:04:11 UTC
Created attachment 131275 [details]
corrected fix
Comment 16 Vladimir Riha 2013-02-12 08:06:53 UTC
(In reply to comment #7)
> Perhaps a patch candidate?

I'd agree but I don't have access to SFTP behind some proxy to verify this
issue. Marking as patch candidate, we can always remove it in case 226006 won't
get fixed or there will be other issues.
Comment 17 hanryz 2013-02-12 08:20:55 UTC
If I understand right, the fix will be applied with the nightly build of tomorrow? I will test it then and tell the result. If there are still issues with it, I would suggest that I create a test account on my SFTP server that you can test it. Would this suit you?
Comment 18 Tomas Mysik 2013-02-12 09:00:31 UTC
So, it seems that it _could_ work if I try to get SOCKS proxy first; please, test it carefully and let us know.

Láďo, could you please add a patch candidate keyword? The patch is IMHO simple.

Thanks.

http://hg.netbeans.org/web-main/rev/74a30d5c2d54
Comment 19 hanryz 2013-02-12 09:03:58 UTC
Hi Tomas,

I have no resources to compile the NB IDE myself. Can I get a nightly build version tomorrow and test it?

Thanks.
Comment 20 Vladimir Riha 2013-02-12 09:11:46 UTC
Tomas: Sorry, I thought I already have.

Hanryz: Thanks, If you could give me account just to try it, it would be great (you can send it to me to vriha@netbeans.org). Thanks. Once it will be in Dev build, there will be a comment from "Quality Engineering" about it (likely tomorrow morning)
Comment 21 hanryz 2013-02-12 09:18:24 UTC
Hi Láďo,

I will create a test account today evening and let you know. Please let me know when you finished testing, by default it will close in 3 days time.

BTW are the dev builds the same as the night builds I can download? Or what are your definitions?
Comment 22 Vladimir Riha 2013-02-12 09:25:23 UTC
(In reply to comment #21)
> Hi Láďo,
> 
> I will create a test account today evening and let you know. Please let me know
> when you finished testing, by default it will close in 3 days time.

Thanks, I'll get to this on Thursday, so it is just fine. 


> BTW are the dev builds the same as the night builds I can download? Or what are
> your definitions?

Yes, just a different terminology. You can download it from 

http://bits.netbeans.org/dev/nightly/
Comment 23 Tomas Mysik 2013-02-12 10:44:50 UTC
So, I have improved the fix, now it should be correct. BTW it should also fix the SOCKS proxy issue for FTP (SOCKS proxy was used instead of HTTP one for FTP). So, please verify both FTP and SFTP.
Also please notice that for SFTP, the SOCKS proxy is preferred - or should we prefer HTTP proxy instead?

Thanks a lot.
http://hg.netbeans.org/web-main/rev/6f2f1e9dfbf5
Comment 24 Tomas Mysik 2013-02-12 10:46:29 UTC
Just a reminder - for patch, both changesets are needed.

http://hg.netbeans.org/web-main/rev/74a30d5c2d54
http://hg.netbeans.org/web-main/rev/6f2f1e9dfbf5
Comment 25 hanryz 2013-02-12 12:14:10 UTC
Hi Tomas,

FTP has not been an issue, if you select HTTP as the proxy type. I think, for pure FTP HTTP proxy should be used by default. For SFTP the SOCKS is the only right choice. I believe that the selection of proxy type can therefore be made automatic. Without user intervention (I may be wrong). 

However, please note that I detected issues in NB with Implicit/Explicit FTP (FTP over SSL/TSL). However, I never use it nor going to use it, so it's not important to me:)

If you check in all the changes regarding the SFTP issue (if not done already) I will download the nightly build tomorrow at 9:00 AM and check if it works. Another test will be done by your colleague. So we are likely to sort the issue out by Thursday, if no unpredictable things will happen.

Regards,
Andre.
Comment 26 Quality Engineering 2013-02-13 01:53:54 UTC
Integrated into 'main-golden', will be available in build *201302122300* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/74a30d5c2d54
User: Tomas Mysik <tmysik@netbeans.org>
Log: #225885 - sftp behind a proxy
Comment 27 hanryz 2013-02-13 11:06:54 UTC
Well done! I checked the functionality today and SFTP via proxy works like a charm. Great support guys, keep it like this! BUG_CLOSED
Comment 28 Tomas Mysik 2013-02-13 11:15:20 UTC
@hanryz: Please, not all related changesets are propagated to the dev builds yet, this one [1] is still missing. Wait for it (see comment #20) and test it again, please.

Thanks.
[1] http://hg.netbeans.org/web-main/rev/6f2f1e9dfbf5
Comment 29 hanryz 2013-02-14 13:08:03 UTC
Got checked the 20130213 nightly build. Same here, it works. I noted that in the two last nightly builds no selection of SOCKS proxy is required anymore. It works with normal HTTP proxy.

Cheers, Andre
Comment 30 Vladimir Riha 2013-02-14 16:10:50 UTC
I tried it with web-main build and it went OK (download and project creation, I don't have privileges for write but it worked for Andre so I guess no problem there as well). Martin will try simple FTP over proxy. We will test FTP more thoroughly later when 7.3.1 will be getting close.


Thanks a lot Andre for the test account!


Product Version: NetBeans IDE Dev (Build web-main-9901-on-20130214)
Java: 1.7.0_15; Java HotSpot(TM) Client VM 23.7-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_15-b02
System: Linux version 3.2.0-35-generic-pae running on i386; UTF-8; en_US (nb)
Comment 31 Vladimir Riha 2013-02-28 16:23:33 UTC
I tried it with plain FTP and having specified SOCKS is still issue, when I "Test Connection", I got dialog saying:

"Cannot connect to ..."
"(Cause: Malformed reply from SOCKS server)"


Removing proxy SOCKS settings helps and upload/download seems to work fine with plain FTP





Product Version: NetBeans IDE Dev (Build 201302272300)
Java: 1.7.0_13; Java HotSpot(TM) Client VM 23.7-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_13-b20
System: Linux version 3.2.0-38-generic-pae running on i386; UTF-8; en_US (nb)
Comment 32 Tomas Mysik 2013-02-28 17:11:24 UTC
I will check it but it worked for me... Do you specified the SOCKS proxy or do you use one proxy for all protocols?

Thanks.
Comment 33 Vladimir Riha 2013-03-01 07:59:45 UTC
(In reply to comment #32)
> I will check it but it worked for me... Do you specified the SOCKS proxy or do
> you use one proxy for all protocols?
> 
> Thanks.

One proxy for all protocols
Comment 34 Martin Kanak 2013-03-04 15:23:04 UTC
Hi Tomas,

I want to tell you that the warning which you improved here [1] and which is also related to this issue was reverted in dev builds. Could you please improve it again? 

Product Version: NetBeans IDE Dev (Build 201303032300)
Java: 1.7.0_13; Java HotSpot(TM) Client VM 23.7-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_13-b20
System: Linux version 3.5.0-25-generic running on i386; UTF-8; en_US (nb)

[1] http://netbeans.org/bugzilla/show_bug.cgi?id=222481
Comment 35 Tomas Mysik 2013-03-05 08:23:44 UTC
(In reply to comment #34)
> I want to tell you that the warning which you improved here [1] and which is
> also related to this issue was reverted in dev builds. Could you please improve
> it again?

It was removed since it should not be needed anymore. But I will have a look at it and return it back if the issue is still there.

Thanks.
Comment 36 Tomas Mysik 2013-03-05 10:18:05 UTC
I had to return the warning back, the error seems to be in Commons-Net library.

Thanks.

http://hg.netbeans.org/web-main/rev/8e3b6daf0e88
Comment 38 Quality Engineering 2013-03-06 01:54:23 UTC
Integrated into 'main-golden', will be available in build *201303052300* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/8e3b6daf0e88
User: Tomas Mysik <tmysik@netbeans.org>
Log: #225885 cont'd - sftp behind a proxy

Warning about SOCKS proxy is back.
Comment 39 Vladimir Riha 2013-03-11 15:05:00 UTC
Verified, thanks

Product Version: NetBeans IDE Dev (Build 201303102300)
Java: 1.7.0_15; Java HotSpot(TM) Client VM 23.7-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_15-b03
System: Linux version 3.2.0-38-generic-pae running on i386; UTF-8; en_US (nb)
Comment 41 Quality Engineering 2013-03-14 21:17:54 UTC
Integrated into 'releases', will be available in build *201303141828* or newer. Wait for official and publicly available build.
Changeset: http://hg.netbeans.org/releases/rev/aa421d6ef524
User: Tomas Mysik <tmysik@netbeans.org>
Log: #225885 - sftp behind a proxy
(transplanted from 74a30d5c2d541f06b5f6b82765cd11d3e2ffbebc)
Comment 42 Vladimir Riha 2013-03-20 08:57:10 UTC
verified in patch1. I can connect to both FTP and SFTP from behind proxy


Product Version: NetBeans IDE 7.3patched (Build 201302132200)
Java: 1.7.0_17; Java HotSpot(TM) Client VM 23.7-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_17-b02
System: Linux version 3.2.0-38-generic-pae running on i386; UTF-8; en_US (nb)
Comment 43 hanryz 2013-06-12 09:06:11 UTC
Hello guys,

I've just downloaded the new NB 7.3.1 and noticed that the "SFTP behind proxy" bug  is still there (see the thread above). And this although the targeted version was the 7.3.1. Can you please confirm that we have to wait for 7.4 in order this bug to get finally fixed??
Thanks.
Comment 44 hanryz 2013-06-12 09:11:06 UTC
Sorry guys, was a false alarm, it is working fine, Just a wrong setting in the IDE-Tools-Options for all the proxies. The check box "use for all" must stay unchecked.
Comment 45 zeleznypa 2014-04-29 08:19:11 UTC
Our company use HTTP proxy on same address as SOCK one but on different port.

When i set propper configuration and test it in preferences window, everything work well but unfortunatelly when I want to create new remote project via SFTP over authorized proxy server, i took "malformed reply from SOCKS server".

Can you help me please?
Comment 46 Tomas Mysik 2014-04-29 08:51:16 UTC
@zeleznypa: Please, do not reopen verified issues. Simply report a new one with exact steps to reproduce.

Thanks.
Comment 47 zeleznypa 2014-04-29 08:57:25 UTC
Sorry for that. New bug is here: https://netbeans.org/bugzilla/show_bug.cgi?id=244157
Comment 48 Tomas Mysik 2014-04-29 08:59:10 UTC
(In reply to zeleznypa from comment #47)
> Sorry for that. New bug is here:
> https://netbeans.org/bugzilla/show_bug.cgi?id=244157

Thanks a lot!