[Bug 8990] New: --append-verify doesn't spot file differences, even with --checksum

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8990

           Summary: --append-verify doesn't spot file differences, even
                    with --checksum
           Product: rsync
           Version: 3.0.9
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]
         QAContact: [hidden email]


Example:

$ echo "hello" > test1.txt

$ echo "world" > test2.txt

$ rsync --append test1.txt test2.txt

$ diff test1.txt test2.txt
--- test1.txt   2012-06-08 10:23:31.059215000 +0100
+++ test2.txt   2012-06-08 10:23:36.891225000 +0100
@@ -1 +1 @@
-hello
+world

$ rsync --append-verify test1.txt test2.txt

$ diff test1.txt test2.txt
--- test1.txt   2012-06-08 10:23:31.059215000 +0100
+++ test2.txt   2012-06-08 10:23:36.891225000 +0100
@@ -1 +1 @@
-hello
+world

$ rsync --append-verify --checksum test1.txt test2.txt

$ diff test1.txt test2.txt
--- test1.txt   2012-06-08 10:23:31.059215000 +0100
+++ test2.txt   2012-06-08 10:23:36.891225000 +0100
@@ -1 +1 @@
-hello
+world

$ rsync --append-verify --checksum --ignore-times test1.txt test2.txt

$ diff test1.txt test2.txt
--- test1.txt   2012-06-08 10:23:31.059215000 +0100
+++ test2.txt   2012-06-08 10:23:36.891225000 +0100
@@ -1 +1 @@
-hello
+world

Bug confirmed in rsync-3.0.6 and 3.0.9. This syncs correctly with rsync-2.6.9
(using only --append).

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8990

Wayne Davison <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
            Summary|--append-verify doesn't     |It might be nice to make
                   |spot file differences, even |--append-verify also
                   |with --checksum             |transfer non-appending
                   |                            |files
           Severity|normal                      |enhancement

--- Comment #1 from Wayne Davison <[hidden email]> 2012-06-16 17:03:30 UTC ---
The way append currently works is that it only transfers files that have data
to append.  From the manpage:

"If a file needs to be transferred and its size on the receiver is
the same or longer than the size on the sender, the file is skipped."

Thus, it is assumed that if you are using --append (or --append-verify) that
you have narrowed down your transfer list to just files that need to be
appended and that you know have unchanged data prior to the append bits.

It might be nice to allow rsync to do a non-appending transfer for any files it
finds that have a non-longer length (possibly only with --append-verify).  Such
a feature would require some way to alert the sender what kind of file is
currently being processed (possibly using the sum data fields), and that is not
currently possible.  I'm thinking that we don't want to do that with --append
because it is already dangerous if your transfer list is not limited to just
the files that are known good besides needing some appending.  The
--append-verify option is already telling rsync that you aren't sure about the
files, so it might be nice if that option also mode did more updating work.
However, such a change would be incompatible with older rsyncs and is not
currently planned.

I'll change this into an enhancement request for future consideration.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=8990

--- Comment #2 from Matt A <[hidden email]> 2012-06-26 08:07:39 UTC ---
(In reply to comment #1)
> Thus, it is assumed that if you are using --append (or --append-verify) that
> you have narrowed down your transfer list to just files that need to be
> appended and that you know have unchanged data prior to the append bits.

I can see that that is the intended behaviour for --append, but then what does
--append-verify do differently? Prior to version 3.0.0, --append did check that
the files matched after attempting an append, and the way the documentation is
written gives the impression that this behaviour is now implemented by
--append-verify. To quote:

  --append-verify
              This  works  just  like the --append option, but the existing
data on the receiving side is included in the full-file checksum verification
step, which will cause a file to be resent if the final verification step fails
(rsync uses a normal, non-appending --inplace transfer for the resend).

              Note: prior to rsync 3.0.0, the --append option worked like
--append-verify, so if you are interacting with an older rsync (or the transfer
is using a protocol prior  to  30), specifying either append option will
initiate an --append-verify transfer.

I'm pretty sure the current behaviour is a bug.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html