[Bug 10675] New: rsyncing >2GB file onto fat32 partition should fail earlier

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 10675] New: rsyncing >2GB file onto fat32 partition should fail earlier

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10675

           Summary: rsyncing >2GB file onto fat32 partition should fail
                    earlier
           Product: rsync
           Version: 3.1.0
          Platform: x86
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]
         QAContact: [hidden email]


i have noticed that while rsyncing (-av) a whole directory (from linux ext4)
onto a fat32 filesystem on a usb disk there was a large >4GB file among the
files to be copied. while rsync should fail immediately encountering such a job
what really happened was strange: rsync continued to read all the other files
in the job but nothing has been written to the usb disk anymore. i have noticed
this since the i/o on my internal disk went to the max of 120 MB/s and the i/o
on the usb disk stayed at 0. at that point i pressed ctrl-c and, remarkably
late, got the corresponding error:

rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(632)
[sender=3.1.0]
rsync: write failed on "/media/luser/usbdisk/folder/dvd.iso": File too large
(27)
rsync error: error in file IO (code 11) at receiver.c(389) [receiver=3.1.0]



how to reproduce: format a usb disk with mkfs.vfat and try to rsync a directory
containing some files plus a large file over the >2GB fat32 limit.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: [Bug 10675] New: rsyncing >2GB file onto fat32 partition should fail earlier

Brian K. White
On 6/26/2014 5:36 PM, [hidden email] wrote:

> https://bugzilla.samba.org/show_bug.cgi?id=10675
>
>             Summary: rsyncing >2GB file onto fat32 partition should fail
>                      earlier
>             Product: rsync
>             Version: 3.1.0
>            Platform: x86
>          OS/Version: Linux
>              Status: NEW
>            Severity: normal
>            Priority: P5
>           Component: core
>          AssignedTo: [hidden email]
>          ReportedBy: [hidden email]
>           QAContact: [hidden email]
>
>
> i have noticed that while rsyncing (-av) a whole directory (from linux ext4)
> onto a fat32 filesystem on a usb disk there was a large >4GB file among the
> files to be copied. while rsync should fail immediately encountering such a job
> what really happened was strange: rsync continued to read all the other files
> in the job but nothing has been written to the usb disk anymore. i have noticed
> this since the i/o on my internal disk went to the max of 120 MB/s and the i/o
> on the usb disk stayed at 0. at that point i pressed ctrl-c and, remarkably
> late, got the corresponding error:
>
> rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(632)
> [sender=3.1.0]
> rsync: write failed on "/media/luser/usbdisk/folder/dvd.iso": File too large
> (27)
> rsync error: error in file IO (code 11) at receiver.c(389) [receiver=3.1.0]
>
>
>
> how to reproduce: format a usb disk with mkfs.vfat and try to rsync a directory
> containing some files plus a large file over the >2GB fat32 limit.
>


I don't think it's reasonable to expect rsync, or any other such tool,
to include a database of all the quirks and limitations of all the
different filesystems in existence, including all their different
versions and options.

But it might be possible for rsync to fail earlier on this particular
problem by just attempting to allocate the space when it fist goes to
transfer the file, and aborting if that fails. It wouldn't know or care
what kind of filesystem the destination has. It wouldn't fail at the
beginning of the rsync job, just at the beginning of that file.

I don't know how to handle --inplace, just add space to the end of the file?

--
bkw
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10675] rsyncing >2GB file onto fat32 partition should fail earlier

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10675

--- Comment #1 from Daniel <[hidden email]> ---
If the Topic should mean ">4GB":
I can confirm the problem.

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html