[Bug 10724] New: rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] New: rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

           Summary: rsync 3.1.1 incorrectly creates extra dirs inside
                    --backup-dir on Mac OSX
           Product: rsync
           Version: 3.1.1
          Platform: x86
        OS/Version: Mac OS X
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]
         QAContact: [hidden email]


On MacOSX 10.9.4, new to rsync 3.1.1, I observe the incorrect creation of empty
dirs inside a --backup-dir.

To reproduce:

1. mkdir foo
2. mkdir bar
3. cd foo; touch fun
4. cd ..
5. rsync -trlvhp foo bar

Everything is as it should be: inside bar there is a clone of foo.  But now
continue (from the same working dir as from setp 5):

6. mkdir foo/newdir
7. cd foo; mv fun newdir; cd ..
8. rsync -trlvhpb --delete --backup-dir=mybackups --suffix=~ foo bar

Expected result: the directory bar should look like this:

bar/
bar/foo/
bar/foo/newdir/fun
bar/mybackups/
bar/mybackups/foo/
bar/mybackups/foo/fun~

Actual result: rsync 3.1.1 has created an empty dir "newdir" inside
mybackups.  That is, this is what bar looks like:

bar/
bar/foo/
bar/foo/newdir/fun
bar/mybackups/
bar/mybackups/foo/
bar/mybackups/foo/fun~
bar/mybackups/foo/newdir/

As you can see there is magically a "newdir" there.  It shouldn't be
there, because there never was a "newdir" inside bar/foo prior to step 8.
Therefore, there was no "newdir" there to be "backed up" into mybackups.

This behavior is new to 3.1.1 as best as I can tell.  I'm positive it never
occurred
on any prior version of rsync that I've ever used (2.6-ish?).

The above was just a toy example designed to illustrate the issue.  In
practice, rsync 3.1.1 left dozens of such "ghost" directories inside my
--backup-dir.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #1 from [hidden email] 2014-07-18 03:28:52 UTC ---
My apologies, this bug is also present on rsync 3.1.0.

I still think it's a bug.  Also, I was almost right; it didn't work like this
on 3.0.9 and earlier.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #2 from [hidden email] 2014-07-18 04:26:08 UTC ---
For fun I cloned the git repo and bisected.  The first commit that exhibits the
aforementioned buggy behavior is: 3696674bc62b0c1250027dbeedafdd7ebafdcf8b

which carries the description "More backup improvements".

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: [Bug 10724] New: rsync 3.1.0+ creates all dirs even with --prune-empty-dir (filling disk)

Linda A. Walsh-2
In reply to this post by samba-bugs
I have a problem similar to this bug, not sure if it is the same bug or
not, the consequences
in my case are killing off a daily snapshot mechanism.

As mentioned in the "Concern rsync failing to find attributes..." thread,
I am doing an rsync from A->B while excluding files
in a --compare-dest base dir.

The problem is on a relatively small snapshot space is tighter.  It's
creating a 680MB fs, for a 550MB snapshot.  Then I copy the 550MB
snapshot to the 680MB fs and run out of space because
rsync created 310 thousand empty directories on the target.

If I get rid of them, I regain about 200M of space.

I am using the --prune-empty-dirs switch, but it seems to be ignored.

Is this likely the same bug, or different?


--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #3 from David <[hidden email]> 2014-08-10 12:25:58 UTC ---

I wonder if that commit was trying to fix another bug that exists in at least
3.1.0, and possibly other versions.  When an empty directory is deleted from
source but exists in target, and the --delete option is enabled, the empty
directory is not placed in --backup-dir.  Going to see if I can figure out what
exactly the commit changed.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #4 from David <[hidden email]> 2014-08-10 12:35:52 UTC ---
(In reply to comment #3)
> I wonder if that commit was trying to fix another bug that exists in at least
> 3.1.0, and possibly other versions.  When an empty directory is deleted from
> source but exists in target, and the --delete option is enabled, the empty
> directory is not placed in --backup-dir.  Going to see if I can figure out what
> exactly the commit changed.

Sorry for the self-reply, but I just verified that the bug I noticed exists in
3.0.9, so the commit 3696674bc62b0c1250027dbeedafdd7ebafdcf8b is not related.
Looks like this needs a new bug report.

--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

Wayne Davison <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #5 from Wayne Davison <[hidden email]> ---
*** Bug 11423 has been marked as a duplicate of this bug. ***

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

Wayne Davison <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #6 from Wayne Davison <[hidden email]> ---
In rsync 3.1.x the backup logic was improved to ensure that older, conflicting
files/symlinks couldn't cause problems in a new backup.  For instance, if a
file changed to a directory (and was backed up) and then later a file needed to
be backed up in the new path, the lingering file that now needs to be a
directory caused rsync to die.  The new logic is to ensure that every dir that
exists in the transfer is now a directory in the backup area.  This logic could
be amended to instead ensure that any conflicts are removed, and the dir
creation delayed until needed.

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

Wayne Davison <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED
                 OS|Mac OS X                    |All

--- Comment #7 from Wayne Davison <[hidden email]> ---
This is now fixed in the git source, and will be in the next release.

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #8 from adsh <[hidden email]> ---
Bug is NOT fixed:

rsync --version (on host and client machines)
rsync  version 3.1.2pre1  protocol version 31
...

rsync -btrhv -e "ssh -i /root/.ssh/id_ecdsa_rsync_server" --delete-after \
--backup-dir=/var/backups/server.ua/pkg/old/`date +%F` \
[hidden email]:/home/user/soft/packages/ /var/backups/server.ua/pkg/data

receiving file list ... done
Created backup_dir /var/backups/server.ua/pkg/old/2015-08-13/

sent 19 bytes  received 7.06K bytes  4.72K bytes/sec
total size is 188.73M  speedup is 26,664.10

ls -l /var/backups/server.ua/pkg/old/2015-08-13/
total 0

---

rsync --version (on host and client machines)
rsync  version 3.0.9  protocol version 30
...

rmdir /var/backups/server.ua/pkg/old/2015-08-13/

rsync -btrhv -e "ssh -i /root/.ssh/id_ecdsa_rsync_server" --delete-after \
--backup-dir=/var/backups/server.ua/pkg/old/`date +%F` \
[hidden email]:/home/user/soft/packages/ /var/backups/server.ua/pkg/data

receiving file list ... done

sent 11 bytes  received 7.06K bytes  14.13K bytes/sec
total size is 188.73M  speedup is 26705.61

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #9 from adsh <[hidden email]> ---
Please read https://bugzilla.samba.org/show_bug.cgi?id=10724 carefully.

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #10 from adsh <[hidden email]> ---
Sorry - read this link: https://bugzilla.samba.org/show_bug.cgi?id=11423

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #11 from Wayne Davison <[hidden email]> ---
Yes, the top-level backup dir is created at the start of the transfer.  I don't
plan to change that.

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

[Bug 10724] rsync 3.1.1 creates extra dirs inside --backup-dir

samba-bugs
In reply to this post by samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10724

--- Comment #12 from [hidden email] ---
I can confirm that on rsync 3.1.2pre, the reproduction example I gave at the
start of this thread no longer replicates the issue.  That is, 3.1.2pre appears
at first blush to have fixed the problem.

Thanks!

--
You are receiving this mail because:
You are the QA Contact for the bug.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html