size-related rsync bugs?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

size-related rsync bugs?

Dave Howorth
I use rsync to download a directory from a remote daemon. This has
worked well for years but has recently started to give problems. The
rsync transfer is run by a program called dirvish that is run as a cron
job. The log looks like this:


ACTION: rsync -vrltH --delete --stats -D --numeric-ids
--exclude-from=/nfs/scop5/data/msd/sifts-mirror/2016-06-28/exclude
--link-dest=/nfs/scop5/data/msd/sifts-mirror/2016-06-07/tree
rsync.ebi.ac.uk::pub/databases/msd/sifts/
/nfs/scop5/data/msd/sifts-mirror/2016-06-28/tree

receiving incremental file list
csv/pdb_chain_cath_uniprot.csv
csv/pdb_chain_enzyme.csv
csv/pdb_chain_go.csv
broken pipe
RESULTS: warnings = 0, errors = 1

ACTION: rsync -vrltH --delete --stats -D --numeric-ids
--exclude-from=/nfs/scop5/data/msd/sifts-mirror/2016-06-28/exclude
--link-dest=/nfs/scop5/data/msd/sifts-mirror/2016-06-07/tree
rsync.ebi.ac.uk::pub/databases/msd/sifts/
/nfs/scop5/data/msd/sifts-mirror/2016-06-28/tree

receiving incremental file list
csv/pdb_chain_go.csv
broken pipe
RESULTS: warnings = 0, errors = 2

ACTION: rsync -vrltH --delete --stats -D --numeric-ids
--exclude-from=/nfs/scop5/data/msd/sifts-mirror/2016-06-28/exclude
--link-dest=/nfs/scop5/data/msd/sifts-mirror/2016-06-07/tree
rsync.ebi.ac.uk::pub/databases/msd/sifts/
/nfs/scop5/data/msd/sifts-mirror/2016-06-28/tree

receiving incremental file list
csv/pdb_chain_go.csv
broken pipe
RESULTS: warnings = 0, errors = 3

ACTION: rsync -vrltH --delete --stats -D --numeric-ids
--exclude-from=/nfs/scop5/data/msd/sifts-mirror/2016-06-28/exclude
--link-dest=/nfs/scop5/data/msd/sifts-mirror/2016-06-07/tree
rsync.ebi.ac.uk::pub/databases/msd/sifts/
/nfs/scop5/data/msd/sifts-mirror/2016-06-28/tree

receiving incremental file list
csv/pdb_chain_go.csv
broken pipe
RESULTS: warnings = 0, errors = 4

ACTION: rsync -vrltH --delete --stats -D --numeric-ids
--exclude-from=/nfs/scop5/data/msd/sifts-mirror/2016-06-28/exclude
--link-dest=/nfs/scop5/data/msd/sifts-mirror/2016-06-07/tree
rsync.ebi.ac.uk::pub/databases/msd/sifts/
/nfs/scop5/data/msd/sifts-mirror/2016-06-28/tree

receiving incremental file list
csv/pdb_chain_go.csv
broken pipe
RESULTS: warnings = 0, errors = 5


It seems to me that the file pdb_chain_go.csv is triggering the problem.
I notice that in the last transfer that succeeded on 2016-06-07, it was
1107298708 bytes whilst on 2016-06-22 it had grown a bit to 1111629050
so I suppose that one possibility is some size-related bug in rsync?

I downloaded the particular file using FTP without any problems and it
is a text file that appears to be complete without any unusual features.

I was using rsync 3.0.6 but have just installed and used 3.1.2 with the
same problem. I do not control the remote daemon and have not as yet
been able to persuade its administrators to send me its logs. The
protocol version in use is 30 regardless of which version of rsync I use.

I'd be grateful for any ideas that might get my downloads working again.

Thanks, Dave

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: size-related rsync bugs?

Dave Howorth
On 2016-06-29 13:40, Dave Howorth wrote:
> I use rsync to download a directory from a remote daemon. This has
> worked well for years but has recently started to give problems. The
> rsync transfer is run by a program called dirvish that is run as a cron
> job. The log looks like this:
[snip]

Sorry, I should also have included the rsync error messages that I see:


*** Execution cycle 0 ***

rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
rsync: connection unexpectedly closed (14931 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[generator=3.1.2]


*** Execution cycle 1 ***

rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
rsync: connection unexpectedly closed (17555 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[generator=3.1.2]


*** Execution cycle 2 ***

rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
rsync: connection unexpectedly closed (16371 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[generator=3.1.2]


*** Execution cycle 3 ***

rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
rsync: connection unexpectedly closed (16355 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[generator=3.1.2]


*** Execution cycle 4 ***

rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
rsync: connection unexpectedly closed (16347 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[generator=3.1.2]


Cheers, Dave

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: size-related rsync bugs?

Kevin Korb
"Connection reset by peer" means the other end hung up on you.  If there
is no message before that then it didn't bother to tell you why and the
problem would have to be debugged on the other end.

On 06/29/2016 09:16 AM, Dave Howorth wrote:

> On 2016-06-29 13:40, Dave Howorth wrote:
>> I use rsync to download a directory from a remote daemon. This has
>> worked well for years but has recently started to give problems. The
>> rsync transfer is run by a program called dirvish that is run as a cron
>> job. The log looks like this:
> [snip]
>
> Sorry, I should also have included the rsync error messages that I see:
>
>
> *** Execution cycle 0 ***
>
> rsync: read error: Connection reset by peer (104)
> rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
> rsync: connection unexpectedly closed (14931 bytes received so far)
> [generator]
> rsync error: error in rsync protocol data stream (code 12) at io.c(226)
> [generator=3.1.2]
>
>
> *** Execution cycle 1 ***
>
> rsync: read error: Connection reset by peer (104)
> rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
> rsync: connection unexpectedly closed (17555 bytes received so far)
> [generator]
> rsync error: error in rsync protocol data stream (code 12) at io.c(226)
> [generator=3.1.2]
>
>
> *** Execution cycle 2 ***
>
> rsync: read error: Connection reset by peer (104)
> rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
> rsync: connection unexpectedly closed (16371 bytes received so far)
> [generator]
> rsync error: error in rsync protocol data stream (code 12) at io.c(226)
> [generator=3.1.2]
>
>
> *** Execution cycle 3 ***
>
> rsync: read error: Connection reset by peer (104)
> rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
> rsync: connection unexpectedly closed (16355 bytes received so far)
> [generator]
> rsync error: error in rsync protocol data stream (code 12) at io.c(226)
> [generator=3.1.2]
>
>
> *** Execution cycle 4 ***
>
> rsync: read error: Connection reset by peer (104)
> rsync error: error in socket IO (code 10) at io.c(785) [receiver=3.1.2]
> rsync: connection unexpectedly closed (16347 bytes received so far)
> [generator]
> rsync error: error in rsync protocol data stream (code 12) at io.c(226)
> [generator=3.1.2]
>
>
> Cheers, Dave
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,


--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

signature.asc (188 bytes) Download Attachment