[Bug 10170] New: rsync should support reflink similar to cp --reflink

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 10170] New: rsync should support reflink similar to cp --reflink

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10170

           Summary: rsync should support reflink similar to cp --reflink
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]
         QAContact: [hidden email]


new filesystems support the reflink system call...
the standard GNU cp utility now supports this too.
this functionality is invaluable for future rsync.
@BasketCase recommended I write this as a feature.

Here is some background reference:
http://thread.gmane.org/gmane.linux.file-systems/31535

Thanks in advance, happy to answer any questions I can!

James

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10170] rsync should support reflink similar to cp --reflink

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10170

--- Comment #1 from Kevin Korb <[hidden email]> 2013-09-27 00:31:00 UTC ---
*** Bug 9041 has been marked as a duplicate of this bug. ***

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10170] rsync should support reflink similar to cp --reflink

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10170

David Taylor <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |davidt-samba-bugzilla@yadt.
                   |                            |co.uk

--- Comment #2 from David Taylor <[hidden email]> ---
Created attachment 10585
  --> https://bugzilla.samba.org/attachment.cgi?id=10585&action=edit
prototype patch for --reflink support

I have written a rough draft of a patch to enable support for (Btrfs) reflinks
in rsync.  It is very lightly tested.

Only Btrfs is supported (using BTRFS_IOC_CLONE), and this is intended to
provide a better alternative to --inplace for updating a backup which will be
snapshotted repeatedly.  It avoids the problems of --inplace with
partial updates and stale hardlinks.

It adds two new options:

--reflink

When updating an existing file, rather than creating an entirely new temporary
file, rsync will create a 'reflink' copy of the original file and update it in
place.  When creating a backup copy of a file, rsync will also create a
'reflink' copy rather than rewriting the file.  If a 'reflink' copy cannot be
performed it will fall back to the normal, non-reflink behaviour.

--reflink-always behaves like --reflink, but disables the fallback if a reflink
fails.

--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10170] rsync should support reflink similar to cp --reflink

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10170

--- Comment #3 from roland <[hidden email]> ---
fantastic !

did not know about that feature in btrfs, but as i already did inplace backups
with rsync on btrfs, i think i will give that a try.

i just wondered about reflink support in zfs and putting a link here for
reference: https://github.com/zfsonlinux/zfs/issues/405

--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10170] rsync should support reflink similar to cp --reflink

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10170

James <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #4 from James <[hidden email]> ---
(In reply to David Taylor from comment #2)

Sweet... Glad to hear someone is hacking on this.

--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10170] rsync should support reflink similar to cp --reflink

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10170

--- Comment #5 from kdave <[hidden email]> ---
(In reply to David Taylor from comment #2)

A few things:

* thanks for working on reflink support in rsync :)

* btrfs is not the only filesystem to support reflink, ocfs2 does as well, so
you might make the formulations more generic

* for btrfs, there are 2 types of the cloning ioctl:
 1) that does file-to-file clone (same for ocfs2), IOC_CLONE
 2) range cloning, IOC_CLONE_RANGE

I believe for rsync option 2 should be implemented in the similar way --inplace
works.

* the proposed patch does not work in all scenarios:

$ btrfs subvol create subv1
$ btrfs subvol create subv2
$ <create big file subv1/testfile>
$ rsync --reflink subv1/testfile subv2

according to strace, rsync does not call the clone ioctl at all (none of the
forked processes)

$ rsync --reflink-always subv1/testfile subv2
testfile
    240,475,844 100%   43.54MB/s    0:00:05 (xfr#1, to-chk=0/1)
rsync: open "/subv2/testfile": No such file or directory (2)
rsync: reflink of "/subv2/.testfile.8bxNk0" failed: No such file or directory
(2)
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1165) [sender=3.1.2dev]

and the target file is not created, the same command without --reflink works.

* for speed comparison, I did 'cp --reflink subv1/testfile subv2' and it takes
near to no time, compared to 5 seconds for bare copy, I'm expecting that rsync
--reflink would take comparable time to cp+reflink

--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html