This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 248431 - Project with remote sources - ftp upload on save / synchronize sometimes fails.
Summary: Project with remote sources - ftp upload on save / synchronize sometimes fails.
Status: RESOLVED WORKSFORME
Alias: None
Product: php
Classification: Unclassified
Component: FTP Support (show other bugs)
Version: 8.0.1
Hardware: All All
: P3 normal (vote)
Assignee: Tomas Mysik
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-11-05 16:21 UTC by digitaltoast
Modified: 2016-08-12 06:22 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Netbeans IDE log (9.07 KB, text/plain)
2014-11-05 16:21 UTC, digitaltoast
Details
Plugin configuration which appears to resolve this issue (39.34 KB, image/png)
2014-11-06 16:54 UTC, digitaltoast
Details
netbeans_8_0_2_ftp_error_log.txt - FTP Log failing to download some of the files (62.18 KB, text/plain)
2014-12-26 22:29 UTC, chavez317
Details
netbeans_8_0_2_ide_log.txt - IDE Log failing to download some of the remote files (155.75 KB, text/plain)
2014-12-26 22:29 UTC, chavez317
Details

Note You need to log in before you can comment on or make changes to this bug.
Description digitaltoast 2014-11-05 16:21:36 UTC
Created attachment 150301 [details]
Netbeans IDE log

I've got a couple of projects set up as "remote sources" synced via ftp, with "upload on save". 

On the remote side is Ubuntu running vsftpd: version 3.0.2 

The problem: Completely randomly, ftp will stall on "list" or "stor". If I wait a LONG time, eventually it will timeout and say upload failed and ask if I want to disable upload OR it will work, after a long time.

I've tried pure ftp, I've tried tls explicit both with and without "encrypt connection only", I've tried passive and active modes. I've looked on the vsftpd and ubuntu system logs and there's nothing to suggest any error. 

The connection this end is not getting dropped, and I do not have any issues with notepad++'s NPFTP, pspad's ftp client, winscp or filezilla when ftping to the same server. 

It can happen twice an hour or 3 times in a row, but once it happens, it just says "synchronizing", and that's it. 

The netbeans ftp window looks like this:

250 Directory successfully changed. 
PWD 
257 "/var/www/mysite/system" 
CWD /var/www/mysite/system/application/controllers 
250 Directory successfully changed. 
PASV 
227 Entering Passive Mode (1x,x,x,x,174,242). 
STOR admin.php 
150 Ok to send data.

the vsftpd log looks like this:

Tue Nov  4 09:54:13 2014 [pid 11038] CONNECT: Client "x.x.x.x" 
Tue Nov  4 09:54:14 2014 [pid 11037] [mysite] OK LOGIN: Client "x.x.x.x" 
Tue Nov  4 09:59:14 2014 [pid 11039] [mysite] OK UPLOAD: Client "x.x.x.x", "/var/www/mysite/system/application/controllers/admin.php", 3428 bytes, 0.01Kbyte/sec

and the netbeans log is attached.

The other thing it does when saving is it gets stuck on "remote synchronization" - there seems to be no way to stop it once it starts going into its "looping progress bar" routine. 

This time there's no error in the IDE log, but the vsftp log looks like this: 

Code:
Tue Nov  4 10:56:26 2014 [pid 11465] CONNECT: Client "x.x.x.x" 
Tue Nov  4 10:56:27 2014 [pid 11464] [mysite] OK LOGIN: Client "x.x.x.x" 
Tue Nov  4 11:01:27 2014 [pid 11466] [mysite] OK UPLOAD: Client "x.x.x.x", "/var/www/mysite/system/application/controllers/admin.php", 3723 bytes, 0.01Kbyte/sec 
Tue Nov  4 11:01:27 2014 [pid 11488] CONNECT: Client "x.x.x.x" 
Tue Nov  4 11:01:28 2014 [pid 11487] [mysite] OK LOGIN: Client "x.x.x.x" 
Tue Nov  4 11:06:28 2014 [pid 11489] [mysite] OK UPLOAD: Client "x.x.x.x", "/var/www/mysite/system/application/controllers/admin.php", 3725 bytes, 0.01Kbyte/sec

Note that it takes some 5 minutes between connecting and uploading the file. During this time, I can do no other upload, save or sync operations. 

Normally the only way to recover is to close NB, restart, add a space to the (unsaved) file to make it think there's been a change, then CTRL-S again. 

Imagine how frustrating this is doing it 5 times an hour sometimes, late at night...

Product Version: NetBeans IDE 8.0.1 (Build 201408251540) 
Updates: NetBeans IDE is updated to version NetBeans 8.0.1 Patch 1.1 
Java: 1.8.0; Java HotSpot(TM) 64-Bit Server VM 25.0-b70 
Runtime: Java(TM) SE Runtime Environment 1.8.0-b132 
System: Windows 8 version 6.2 running on amd64; Cp1252; en_GB (nb)
Comment 1 Tomas Mysik 2014-11-06 06:47:20 UTC
From the log, I can see:

java.net.SocketException: Software caused connection abort: recv failed

This usually means that there is a problem with your connection, see e.g. [1]. Also, NetBeans uses Apache Commons Net library so there could be some related bug there, you can try to have a look.

Anyway, I will try to have a look at it but likely I will not be able to help you since, as I wrote, this seems to be a problem with your connection (therefore also lowering to P3). But I hope I will be able to improve NetBeans behaviour when this situation happens.

Thanks for reporting.
[1] http://stackoverflow.com/questions/135919/java-net-socketexception-software-caused-connection-abort-recv-failed
Comment 2 digitaltoast 2014-11-06 08:54:50 UTC
(In reply to Tomas Mysik from comment #1)

> This usually means that there is a problem with your connection

OK, thanks for taking a look. I did wonder about the connection; I probably should have mentioned also that I ran "ping -t" and also the TBB Broadband Connection Monitor, and no pings were missed during the time it failed.

Would Wireshark help, if I just sent the relevant part of the log? I've not installed it on that machine, but I could if it would help.

But what would REALLY be hugely helpful is any way of aborting it - if not automatically, then manually - if I see it's logged it, then gets stuck on "stor" (which is 99% of the time the error), then at least I could cancel that operation and try again, rather than having to shutdown the whole IDE and restart.

Incidentally, I also forgot to mention that if I don't kill the instance of Netbeans from task manager which is still running after I quit, when I restart netbeans, it still can't connect, as if it's still trying the previous operation.

Might be relevant...

Thanks!
Comment 3 Tomas Mysik 2014-11-06 09:13:20 UTC
(In reply to digitaltoast from comment #2)
> OK, thanks for taking a look.

You are welcome.

> Would Wireshark help, if I just sent the relevant part of the log? I've not
> installed it on that machine, but I could if it would help.

Well, I am not expert in this area but I can try :)

> But what would REALLY be hugely helpful is any way of aborting it - if not
> automatically, then manually - if I see it's logged it, then gets stuck on
> "stor" (which is 99% of the time the error), then at least I could cancel
> that operation and try again, rather than having to shutdown the whole IDE
> and restart.
> 
> Incidentally, I also forgot to mention that if I don't kill the instance of
> Netbeans from task manager which is still running after I quit, when I
> restart netbeans, it still can't connect, as if it's still trying the
> previous operation.

As I mentioned, we use Apache Commons Net so I need to investigate whether it is possible to cancel the given operation or not.

Thanks for reporting.
Comment 4 digitaltoast 2014-11-06 16:54:57 UTC
Created attachment 150328 [details]
Plugin configuration which appears to resolve this issue

I wasn't sure that netbeans still running when closed was correct, so I had an idea to turn off as many plugins as I could, while retaining basic functionality. The screenshot shows this change.

Two things now:
1: When I close netbeans, it no longer remains as a running task.
2: I have been working the whole day now, and I haven't encountered the issue once.

Of course, there's a slim chance I've got incredibly lucky today and it's coincidence. 

As the problem I encounter is so random, it could take days of enabling one plugin at a time. I might never know.

At this stage, I appear to be working.

However, before closing this ticket, it would still be good to get some kind of "cancel operation" button for synchronizing.
Comment 5 Tomas Mysik 2014-11-07 06:32:58 UTC
IMHO you are just lucky, the configuration change you made should not be related to this issue (FTP support is part of the "PHP" plugin and no other plugin can/should influence it).

Thanks for letting us know.
Comment 6 patricknn 2014-11-11 16:46:15 UTC
I have the same Issue using PHP Netbeans 8.0.1
The issue only happen with FTP. I also use SFTP everything runs fine.
Comment 7 patricknn 2014-11-11 17:09:48 UTC
I tried to disable my plugins like your image and no success. I'm still expecting issues.


STOR busca.js.new
150 Accepted data connection
DELE busca.js.new
500 Unknown command
QUIT
426 Transfer aborted

Summary
====================
Failed:
file      cargas/trip/node/busca.js     Cannot upload file busca.js (unknown reason).
Runtime: 32 s, processed: 0 file(s), 0 KB


Always the same issue
Comment 8 Tomas Mysik 2014-11-12 07:04:15 UTC
@patricknn:

(In reply to patricknn from comment #7)
> DELE busca.js.new
> 500 Unknown command

Your server does not support the DELETE command? More information can be likely found in the IDE log.
Comment 9 Tomas Mysik 2014-11-12 07:05:14 UTC
(In reply to Tomas Mysik from comment #8)
> Your server does not support the DELETE command? More information can be
> likely found in the IDE log.

@patricknn: As an ugly workaround you can use "Upload Files Directly" in your FTP server configuration.
Comment 10 patricknn 2014-11-12 11:07:56 UTC
Yes my server support the DELETE command, this happen randomly and ONLY on NETBEANS, if I use the FileZilla for example this does not happen.
Comment 11 Tomas Mysik 2014-11-12 11:15:36 UTC
(In reply to patricknn from comment #10)
> Yes my server support the DELETE command, this happen randomly and ONLY on
> NETBEANS, if I use the FileZilla for example this does not happen.

Hm, the message is clear - perhaps a bug in a FTP library we use in NetBeans? Could you please attach IDE log once this happens to you again?

Thanks.
Comment 12 patricknn 2014-11-12 11:26:29 UTC
Where can I get this log? I use a windows version.
Comment 13 digitaltoast 2014-11-12 11:54:52 UTC
(In reply to patricknn from comment #12)
> Where can I get this log? I use a windows version.

```VIEW > IDE LOG```

Then at the bottom of the screen you'll see some tabs, you want to click onto "IDE LOG".

Although @patricknn 's error is caused by a different message, I wonder if the ftp-plugin could be "mis-reading" or mis-sending some data or responses?

There do seem to be two different things going on here - in my case, it was totally stalling on "synchronizing", in your case, it's aborting with an error.

I know this seems weird and unlikely, but since making those plugin changes, I've not had the issue one single time. All I can think is that perhaps one of the plugins was interfering, but in have no idea in what way that could be.
Comment 14 Tomas Mysik 2014-11-12 12:24:50 UTC
No idea either. We use Apache Commons Net which is - IMHO - the most used Java library for FTP tranfers. We use it from the very beginning (NetBeans 6.5 if I am not wrong) and there were no such issues AFAIK. It is alwayd a good idea to try a clean userdir [1] but it is unlikely that it could help with this issue IMHO.

In any case, I cannot do much more than try to update the library or look at their issues. If anyone has reliable steps to reproduce any FTP issues, please provide them otherwise it is really difficult to help you.

Thanks.
[1] http://wiki.netbeans.org/FaqAlternateUserdir
Comment 15 chavez317 2014-12-26 22:29:20 UTC
Created attachment 151286 [details]
netbeans_8_0_2_ftp_error_log.txt - FTP Log failing to download some of the files
Comment 16 chavez317 2014-12-26 22:29:49 UTC
Created attachment 151287 [details]
netbeans_8_0_2_ide_log.txt - IDE Log failing to download some of the remote files
Comment 17 chavez317 2014-12-26 22:33:27 UTC
I am having the same issue with 8.0.2.  I am attempting to download / synchronize remote php/web files from my godaddy FTP site and it is only downloading *SOME* of the files.  I have to do continuous downloads to grab all of the files.

(See Log files netbeans_8_0_2_ftp_error_log.txt and netbeans_8_0_2_ide_log.txt)

I do not believe that this is a connection or permissions issue as I was able to connect an download all of the files via FileZilla with no problems.
Comment 18 Tomas Mysik 2015-05-13 07:34:58 UTC
@chavez317: In your IDE log, I can see:

java.net.SocketException: Connection reset
javax.net.ssl.SSLException: Connection has been shutdown

This means that the connection is lost from whatever reason. The only idea that comes to my mind here is to compare log of FileZilla and Remote Log of NetBeans whether they are the same (maybe FileZilla does some more, less or other commands? No idea, sorry).

In any case, we use the latest version of Apache Commons Net library so there is no upgrade available right now; I also tried to search their issues but no luck.

So, I am really sorry guys but I have no idea what to do now and how I could help you :/
Comment 19 Tomas Mysik 2016-08-12 06:22:34 UTC
(In reply to Tomas Mysik from comment #18)
> @chavez317: In your IDE log, I can see:
> 
> java.net.SocketException: Connection reset
> javax.net.ssl.SSLException: Connection has been shutdown
> 
> This means that the connection is lost from whatever reason. The only idea
> that comes to my mind here is to compare log of FileZilla and Remote Log of
> NetBeans whether they are the same (maybe FileZilla does some more, less or
> other commands? No idea, sorry).

Closing for now as WORKSFORME; feel free to reopen this issue and provide FileZilla logs for comparison.

Thanks.