This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 187836 - Remote FTP/SFTP Connection should stay open with keepalives to speed up upload-on-save
Summary: Remote FTP/SFTP Connection should stay open with keepalives to speed up uploa...
Status: RESOLVED FIXED
Alias: None
Product: php
Classification: Unclassified
Component: FTP Support (show other bugs)
Version: 7.1
Hardware: PC Linux
: P2 normal with 4 votes (vote)
Assignee: Tomas Mysik
URL:
Keywords:
: 181247 194406 199256 (view as bug list)
Depends on:
Blocks: 270200
  Show dependency tree
 
Reported: 2010-06-19 17:48 UTC by bompus
Modified: 2017-03-25 23:07 UTC (History)
13 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
ide log (45.24 KB, text/plain)
2011-08-03 07:41 UTC, Vladimir Riha
Details
messages.log (28.39 KB, application/octet-stream)
2011-08-03 08:42 UTC, Vladimir Riha
Details

Note You need to log in before you can comment on or make changes to this bug.
Description bompus 2010-06-19 17:48:01 UTC
Product Version = NetBeans IDE 6.9 (Build 201006101454)
Operating System = Windows 7 version 6.1 running on x86
Java; VM; Vendor = 1.6.0_20
Runtime = Java HotSpot(TM) Client VM 16.3-b01

It would greatly speed up my rapid remote development if the Project Properties -> Run Configuration -> Remote Connection had an option to remain connected as long as the project was opened, and would send keepalives to the FTP/SFTP server. Right now I save and it has to connect to the SFTP server, perform authentication, change directories, upload file, etc.. When it could bypass some of those steps by remaining connected.

Not a show stopper, but something easy to add and implement to make NetBeans even greater.
Comment 1 Tomas Mysik 2010-06-21 07:13:27 UTC
(In reply to comment #0)
> something easy to add and implement to make NetBeans
> even greater.

I see that you have lokked into sources ;)

No, seriously, you are right, I was thinking about it as well some time ago.

Thanks for reporting.
Comment 2 bompus 2010-07-26 05:07:36 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > something easy to add and implement to make NetBeans
> > even greater.
> 
> I see that you have lokked into sources ;)
> 
> No, seriously, you are right, I was thinking about it as well some time ago.
> 
> Thanks for reporting.

Any progress on this Tomas?
Comment 3 Tomas Mysik 2010-08-01 10:01:59 UTC
(In reply to comment #2)
> Any progress on this Tomas?

Not right now, sorry, too many bugs.
Comment 4 Filip Zamboj 2010-09-15 12:37:43 UTC
batch reassigning
Comment 5 mario-amalfiweb 2010-10-23 16:58:31 UTC
*** Bug 181247 has been marked as a duplicate of this bug. ***
Comment 6 Pekz0r 2010-12-28 10:52:19 UTC
I REALLY agree on this one! 
Please prioritize this for the next release. I would love to have it in 7.0/7.1

I'm working a lot with remote projects and it's a pain when you have to wait the extra seconds for the upload every time you make an update to a file.
Comment 7 kovalski 2011-03-09 13:31:43 UTC
That can be really annoying when someone must work mostly with remote projcects.
Comment 8 arkoak 2011-03-29 09:40:42 UTC
really need this one.. 

is it part of 7.0 beta?
Comment 9 Tomas Mysik 2011-06-17 05:29:09 UTC
*** Bug 194406 has been marked as a duplicate of this bug. ***
Comment 10 Tomas Mysik 2011-07-12 14:51:40 UTC
*** Bug 199256 has been marked as a duplicate of this bug. ***
Comment 11 digimuzik 2011-07-26 01:07:53 UTC
Not in 7 yet. I wonder if this should be updated to a defect, as on my hosting service at least, this causes large amounts of updates to fail because too many connections within a short period of time cause a temp-ban, and I am unable to modify any other files.  Then again, if Tomas already has this planned for 7.1 and is actively seeing duplicates pop up, maybe we don't need to bug him any more about it... :-)
Comment 12 Pekz0r 2011-07-26 06:22:50 UTC
(In reply to comment #11)
> Not in 7 yet. I wonder if this should be updated to a defect, as on my hosting
> service at least, this causes large amounts of updates to fail because too many
> connections within a short period of time cause a temp-ban, and I am unable to
> modify any other files.  Then again, if Tomas already has this planned for 7.1
> and is actively seeing duplicates pop up, maybe we don't need to bug him any
> more about it... :-)

Target milestone is 7.1 so i guess we just need to keep our fingers crossed. The roadmap says it should be released at the end of this year, a beta should probably be out pretty soon. Can't wait!
Comment 13 Tomas Mysik 2011-07-26 07:27:50 UTC
Yes, I would really like to deliver it for 7.1. Thanks for your patience.
Comment 14 t3jkb 2011-07-29 13:07:39 UTC
(In reply to comment #11)
> Not in 7 yet. I wonder if this should be updated to a defect, as on my hosting
> service at least, this causes large amounts of updates to fail because too many
> connections within a short period of time cause a temp-ban, and I am unable to
> modify any other files.  Then again, if Tomas already has this planned for 7.1
> and is actively seeing duplicates pop up, maybe we don't need to bug him any
> more about it... :-)

Hi, i've got the same problem even worse:

When setting up a remote project over SFTP or just downloading its source files over SFTP, my IP gets blacklistet from the provider for at least an hour :(

I would be very glad if there would be an update soon :)
Comment 15 Tomas Mysik 2011-07-29 15:19:40 UTC
Not a defect; TM set back to 7.1.
Comment 16 Tomas Mysik 2011-08-01 09:24:51 UTC
Fixed in web-main, used for upload-on-save.

http://hg.netbeans.org/web-main/rev/dc96b968471d

It would be great if anyone could test this feature, both for FTP and SFTP and report exact steps if something does not work as expected.

Use PHP build #3514 or newer, or wait for a message from the build system that will be added to this issue.
Comment 17 Tomas Mysik 2011-08-01 09:36:07 UTC
(In reply to comment #16)
> Use PHP build #3514 or newer

http://bertram.netbeans.org/hudson/job/PHP-build/?
Comment 18 Quality Engineering 2011-08-02 13:43:38 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/dc96b968471d
User: Tomas Mysik <tmysik@netbeans.org>
Log: #187836 - Remote FTP/SFTP Connection should stay open with keepalives to speed up upload-on-save
Comment 19 Pekz0r 2011-08-02 14:40:14 UTC
(In reply to comment #18)
> Integrated into 'main-golden'
> Changeset: http://hg.netbeans.org/main-golden/rev/dc96b968471d
> User: Tomas Mysik <tmysik@netbeans.org>
> Log: #187836 - Remote FTP/SFTP Connection should stay open with keepalives to
> speed up upload-on-save

Awesome! Thank you so much!
I guess it is not a part of the nightly build at http://bits.netbeans.org/download/trunk/nightly/latest/ yet?
I don't know how to build NetBeans on my own so I guess I have to wait for a development version to released before I can try this out.
Comment 20 Tomas Mysik 2011-08-02 21:10:11 UTC
(In reply to comment #19)
> Awesome! Thank you so much!

You are welcome. It would be great if anyone could give it a try.

> I guess it is not a part of the nightly build at
> http://bits.netbeans.org/download/trunk/nightly/latest/ yet?

I don't think so... IMHO message about it will be added here but not sure about it.

> I don't know how to build NetBeans on my own so I guess I have to wait for a
> development version to released before I can try this out.

Just take the last successful build from our PHP build job [1] (please notice that this is temporary URL, the usual URL is [2] not working now).

[1] http://bertram2.netbeans.org:8080/job/PHP-build/
[2] http://bertram.netbeans.org/hudson/job/PHP-build/
Comment 21 Vladimir Riha 2011-08-03 07:41:32 UTC
I tried to have open connection for some time for both FTP and SFTP (both on local machine). I've encountered 2 problems:
1) This happened once with FTP and several times with SFTP (I was able to reproduce it 3 times of 3 attempts). For SFTP, If I save file let's say 6-10minutes after last save, then dialog is shown, saying: "Upload files on save failed (more information in IDE log). Disable this functionality for PROJECTNAME for this session?" Shorter intervals between savings works fine. I'm attaching IDE log for more details. In the FTP case, it's possible that the interval was longer.

2) I got exception during saving file (one among many) and I'm not sure if it's related (issue #200618). Happened to me few times.

So for me, I'm reopening this since it doesn't work for 100%
Comment 22 Vladimir Riha 2011-08-03 07:41:59 UTC
Created attachment 109759 [details]
ide log
Comment 23 Tomas Mysik 2011-08-03 08:28:17 UTC
In the log, I can only see many "connection refused" messages with "Could not parse response code." as the first exception. This has happen during IDE startup so keep-alive has not been in action at all... or am I wrong?

Could you start NB with

-J-Dorg.netbeans.modules.php.project.connections.ftp.level=FINE

reproduce the problem and attach your IDE log?

Thanks a lot.
Comment 24 Vladimir Riha 2011-08-03 08:42:46 UTC
Created attachment 109762 [details]
messages.log

Sorry for the bogus log, but of course it happened during saving/uploading and not at the IDE startup :)

Here's the new one anyway...
Comment 25 Tomas Mysik 2011-08-03 09:01:41 UTC
(In reply to comment #24)
> Here's the new one anyway...

In the log, I see

> Reason: Cannot change to the remote directory /.

Incorrect remote directory? If so, can you file a bug?
Comment 26 Tomas Mysik 2011-08-03 09:04:21 UTC
I tried to reproduce the bug but keep-alive works for me, using vsftpd on localhost. What I did exactly:
- upload-on-save
- change and save file
- wait for cca 25 minutes (keep-alive interval set to 45 seconds)
- change and save file

This scenario works for me, I'm on Linux.
Comment 27 Vladimir Riha 2011-08-03 09:19:19 UTC
OK, there's the new issue #200623.

Except for this (which happens under some circumstances), this works on windows as well. I'll try some other SFTP server to be sure and then  verify this one (if no one else will be faster)
Comment 28 Tomas Mysik 2011-08-03 09:37:58 UTC
(In reply to comment #26)
> - upload-on-save
> - change and save file
> - wait for cca 25 minutes (keep-alive interval set to 45 seconds)
> - change and save file

Tried with SFTP (ssh to external server with OpenSSH), the same result (no error).
Comment 29 Vladimir Riha 2011-08-03 12:38:43 UTC
I've tried some other SFTP and it works fine, so for me it's verified...
Comment 30 ramzih 2011-09-18 08:14:10 UTC
Hi,

I'm reopening this one as I came across the same issue as described in Comment #21 with the exception that I first have a keep alive transfer timeout error dialog alert and then whenever I try to save a file ( upload on save ) the second dialog inviting to disable the feature shows up.

The issue was reproduced on two different desktop machines with two different server machines.

**Scenario 1**

Desktop machine :
Ubuntu Natty desktop 64 with netbeans Build 201109140600

Server machine :
Fresh Ubuntu Lucid server 64 on vmware virtual machine with vanilla virtualmin install ( LAMP environment with a default virtual server created http://software.virtualmin.com/gpl/scripts/install.sh )
FTP Server installed is proftpd 1.3.2c-1ubuntu0.1
With default config file :

TimeoutNoTransfer               600
TimeoutStalled                  600
TimeoutIdle                     1200

**Scenario 2**

Desktop machine :
Ubuntu Maverick desktop 64 with netbeans Build 201109140600/201109170600

Server machine :
Fresh Ubuntu Lucid server 64 on kvm virtual machine with vanilla virtualmin install ( LAMP environment with a default virtual server created http://software.virtualmin.com/gpl/scripts/install.sh )
FTP Server installed is proftpd 1.3.2c-1ubuntu0.1
With default config file :

TimeoutNoTransfer               600
TimeoutStalled                  600
TimeoutIdle                     1200


In both scenarios, desktop and server machines are in the same lan subnet and no firewall between them.

The issue is still there when :

Using Passive/Active mode
Enabling/Disabling preserve permission option
Enabling/Disabling temporary upload option
Increasing/Decreasing ftp timeouts in netbeans
Increasing/Decreasing ftp timeouts in proftpd config file ( make eventually the keep alive error a bit longer to show up if increasing the timeouts but it's still there )

There's no issue with the stable 7.0.1 netbeans release.

Haven't tested another FTP server or an SFTP one though
Comment 31 Tomas Mysik 2011-09-19 07:45:05 UTC
@ramzih:

Feature is implemented, I have created a bug #202280 for your particular problem. Let's discuss your problem there.

Closing back as FIXED.

Thanks.