Fwd: --link-dest does not appear to be linking on Cygwin

[Clint Olsen]

Hi:

I have attempted to following some instructions to use --link-dest in
order to preserve space for multiple backups. I'm using rsync on
Cygwin with a NAS (ext4) which does support hard-links on the
filesystem. I've written a short program that does attempt to create a
hard-link on this NAS from Cygwin and it does look to be working. If I
run ls -li on the NAS the inodes are the same. However, on Cygwin the
inodes returned are not. I assumed that what's important is that the
NAS handles the hard-links correctly.

I checked the archives and it looks like a common mistake is
inconsistent usage of the trailing backslash. My usage looks something
like:

% rsync -aP --delete --delete-excluded
--exclude-from=/home/Nancy/.rsync/exclude
--link-dest=/cygdrive/f/macki/current/ Nancy
/cygdrive/f/macki/incomplete-back-2012-06-05T13:53:51

I'm running this from "/cygwin/c/Documents and Settings/" directory,
and the directory under the symlink "current" is "Nancy".

Is there a way to tell if rsync thinks it's found a match and
attempting to link?

Thanks,

-Clint
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- --itemize-changes would be the first place to start.  Also, see if you
can make a hard link on your NAS.  Just because the underlying ext4
supports it doesn't mean that your network mount does.  Try something
like:

cd /cygdrive/f/macki
touch testfile
ln testfile testfile2
ls -ls testfile testfile2

They should both have the same inode number.

On 06/05/12 17:02, Clint Olsen wrote:

> Hi:
>
> I have attempted to following some instructions to use --link-dest
> in order to preserve space for multiple backups. I'm using rsync
> on Cygwin with a NAS (ext4) which does support hard-links on the
> filesystem. I've written a short program that does attempt to
> create a hard-link on this NAS from Cygwin and it does look to be
> working. If I run ls -li on the NAS the inodes are the same.
> However, on Cygwin the inodes returned are not. I assumed that
> what's important is that the NAS handles the hard-links correctly.
>
> I checked the archives and it looks like a common mistake is
> inconsistent usage of the trailing backslash. My usage looks
> something like:
>
> % rsync -aP --delete --delete-excluded
> --exclude-from=/home/Nancy/.rsync/exclude
> --link-dest=/cygdrive/f/macki/current/ Nancy
> /cygdrive/f/macki/incomplete-back-2012-06-05T13:53:51
>
> I'm running this from "/cygwin/c/Documents and Settings/"
> directory, and the directory under the symlink "current" is
> "Nancy".
>
> Is there a way to tell if rsync thinks it's found a match and
> attempting to link?
>
> Thanks,
>
> -Clint

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    [phone]
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/OdL4ACgkQVKC1jlbQAQe9EgCgqYVuiaHi0MTD6MZjISt539aF
7SMAoOJu8y508ZGMySbqgkch6Mellg3g
=DBkr
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Clint Olsen]

Hi:

I will try the itemize switch as you suggested, thanks...

I have tested this on both the Windows box and the NAS. The inode
numbers come back differently on Cygwin but the NAS sees them as the
same:

Nancy@macki /cygdrive/f/macki/test
$ ls -ali
total 1048576
 3297336581512553257 drwxr-xr-x 1 Nancy None 0 Jun  5 11:37 .
16963795924520120485 drwxr-xr-x 1 Nancy None 0 Jun  5 14:07 ..
 5321094878451953619 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo
 9211568837569774878 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo1
 9211568837569774879 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo2
 3595967447105090765 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foolink

NAS> cd /volume1/Backup/macki/test/
NAS> ls -ali
16678913 drwxrwxrwx    2 nancy    users         4096 Jun  5 11:37 .
8814593 drwxrwxrwx    5 nancy    users         4096 Jun  5 14:07 ..
16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foo
16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foo1
16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foo2
16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foolink

The final file "foolink" was created using a C program ala:

$ cat link.c
#include <unistd.h>
#include <stdio.h>

int main(void)
{
    if (link("/cygdrive/f/macki/test/foo",
"/cygdrive/f/macki/test/foolink") == 0) {
        printf("Link was successful\n");
    } else {
        printf("Link was not successful\n");
    }
    return 0;
}

Thanks,

-Clint

On Tue, Jun 5, 2012 at 2:06 PM, Kevin Korb <[hidden email]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> - --itemize-changes would be the first place to start.  Also, see if you
> can make a hard link on your NAS.  Just because the underlying ext4
> supports it doesn't mean that your network mount does.  Try something
> like:
>
> cd /cygdrive/f/macki
> touch testfile
> ln testfile testfile2
> ls -ls testfile testfile2
>
> They should both have the same inode number.
>
> On 06/05/12 17:02, Clint Olsen wrote:
>> Hi:
>>
>> I have attempted to following some instructions to use --link-dest
>> in order to preserve space for multiple backups. I'm using rsync
>> on Cygwin with a NAS (ext4) which does support hard-links on the
>> filesystem. I've written a short program that does attempt to
>> create a hard-link on this NAS from Cygwin and it does look to be
>> working. If I run ls -li on the NAS the inodes are the same.
>> However, on Cygwin the inodes returned are not. I assumed that
>> what's important is that the NAS handles the hard-links correctly.
>>
>> I checked the archives and it looks like a common mistake is
>> inconsistent usage of the trailing backslash. My usage looks
>> something like:
>>
>> % rsync -aP --delete --delete-excluded
>> --exclude-from=/home/Nancy/.rsync/exclude
>> --link-dest=/cygdrive/f/macki/current/ Nancy
>> /cygdrive/f/macki/incomplete-back-2012-06-05T13:53:51
>>
>> I'm running this from "/cygwin/c/Documents and Settings/"
>> directory, and the directory under the symlink "current" is
>> "Nancy".
>>
>> Is there a way to tell if rsync thinks it's found a match and
>> attempting to link?
>>
>> Thanks,
>>
>> -Clint
>
> - --
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>        Kevin Korb                      Phone:    [phone]
>        Systems Administrator           Internet:
>        FutureQuest, Inc.               [hidden email]  (work)
>        Orlando, Florida                [hidden email] (personal)
>        Web page:                       http://www.sanitarium.net/
>        PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk/OdL4ACgkQVKC1jlbQAQe9EgCgqYVuiaHi0MTD6MZjISt539aF
> 7SMAoOJu8y508ZGMySbqgkch6Mellg3g
> =DBkr
> -----END PGP SIGNATURE-----
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Clint Olsen]

Ok, itemize changes prints out some info for each file, and it looks like:

cf...p..... Nancy/Favorites/Links/desktop.ini
cf...p..... Nancy/Favorites/Links/latin music.url
cd...p..... Nancy/Favorites/Microsoft Websites/

I did a quick search on this switch and someone was bitching that the
fields were not easy to decipher w/o reading the source code :)

'P' would appear to be permissions. I do get a couple of
warnings/errors about not being able to preserve all attributes during
the copy, but the permissions should not be changing...

Thanks,

-Clint

On Tue, Jun 5, 2012 at 2:10 PM, Clint Olsen <[hidden email]> wrote:

> Hi:
>
> I will try the itemize switch as you suggested, thanks...
>
> I have tested this on both the Windows box and the NAS. The inode
> numbers come back differently on Cygwin but the NAS sees them as the
> same:
>
> Nancy@macki /cygdrive/f/macki/test
> $ ls -ali
> total 1048576
>  3297336581512553257 drwxr-xr-x 1 Nancy None 0 Jun  5 11:37 .
> 16963795924520120485 drwxr-xr-x 1 Nancy None 0 Jun  5 14:07 ..
>  5321094878451953619 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo
>  9211568837569774878 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo1
>  9211568837569774879 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo2
>  3595967447105090765 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foolink
>
> NAS> cd /volume1/Backup/macki/test/
> NAS> ls -ali
> 16678913 drwxrwxrwx    2 nancy    users         4096 Jun  5 11:37 .
> 8814593 drwxrwxrwx    5 nancy    users         4096 Jun  5 14:07 ..
> 16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foo
> 16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foo1
> 16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foo2
> 16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5 10:57 foolink
>
> The final file "foolink" was created using a C program ala:
>
> $ cat link.c
> #include <unistd.h>
> #include <stdio.h>
>
> int main(void)
> {
>    if (link("/cygdrive/f/macki/test/foo",
> "/cygdrive/f/macki/test/foolink") == 0) {
>        printf("Link was successful\n");
>    } else {
>        printf("Link was not successful\n");
>    }
>    return 0;
> }
>
> Thanks,
>
> -Clint
>
> On Tue, Jun 5, 2012 at 2:06 PM, Kevin Korb <[hidden email]> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> - --itemize-changes would be the first place to start.  Also, see if you
>> can make a hard link on your NAS.  Just because the underlying ext4
>> supports it doesn't mean that your network mount does.  Try something
>> like:
>>
>> cd /cygdrive/f/macki
>> touch testfile
>> ln testfile testfile2
>> ls -ls testfile testfile2
>>
>> They should both have the same inode number.
>>
>> On 06/05/12 17:02, Clint Olsen wrote:
>>> Hi:
>>>
>>> I have attempted to following some instructions to use --link-dest
>>> in order to preserve space for multiple backups. I'm using rsync
>>> on Cygwin with a NAS (ext4) which does support hard-links on the
>>> filesystem. I've written a short program that does attempt to
>>> create a hard-link on this NAS from Cygwin and it does look to be
>>> working. If I run ls -li on the NAS the inodes are the same.
>>> However, on Cygwin the inodes returned are not. I assumed that
>>> what's important is that the NAS handles the hard-links correctly.
>>>
>>> I checked the archives and it looks like a common mistake is
>>> inconsistent usage of the trailing backslash. My usage looks
>>> something like:
>>>
>>> % rsync -aP --delete --delete-excluded
>>> --exclude-from=/home/Nancy/.rsync/exclude
>>> --link-dest=/cygdrive/f/macki/current/ Nancy
>>> /cygdrive/f/macki/incomplete-back-2012-06-05T13:53:51
>>>
>>> I'm running this from "/cygwin/c/Documents and Settings/"
>>> directory, and the directory under the symlink "current" is
>>> "Nancy".
>>>
>>> Is there a way to tell if rsync thinks it's found a match and
>>> attempting to link?
>>>
>>> Thanks,
>>>
>>> -Clint
>>
>> - --
>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>>        Kevin Korb                      Phone:    [phone]
>>        Systems Administrator           Internet:
>>        FutureQuest, Inc.               [hidden email]  (work)
>>        Orlando, Florida                [hidden email] (personal)
>>        Web page:                       http://www.sanitarium.net/
>>        PGP public key available on web site.
>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2.0.17 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAk/OdL4ACgkQVKC1jlbQAQe9EgCgqYVuiaHi0MTD6MZjISt539aF
>> 7SMAoOJu8y508ZGMySbqgkch6Mellg3g
>> =DBkr
>> -----END PGP SIGNATURE-----
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

If rsync thinks the permissions are different and you told it to
preserve the permissions then it can't hard link.  The permissions are
on the actual file not the links so it can't store a file with 2 sets
of permissions without storing 2 copies of the file.

So, either fix the permission issue (you probably can't) or don't tell
rsync to sync the permissions (--perms which is part of your -a)

On 06/05/12 17:18, Clint Olsen wrote:

> Ok, itemize changes prints out some info for each file, and it
> looks like:
>
> cf...p..... Nancy/Favorites/Links/desktop.ini cf...p.....
> Nancy/Favorites/Links/latin music.url cd...p.....
> Nancy/Favorites/Microsoft Websites/
>
> I did a quick search on this switch and someone was bitching that
> the fields were not easy to decipher w/o reading the source code
> :)
>
> 'P' would appear to be permissions. I do get a couple of
> warnings/errors about not being able to preserve all attributes
> during the copy, but the permissions should not be changing...
>
> Thanks,
>
> -Clint
>
> On Tue, Jun 5, 2012 at 2:10 PM, Clint Olsen <[hidden email]>
> wrote:
>> Hi:
>>
>> I will try the itemize switch as you suggested, thanks...
>>
>> I have tested this on both the Windows box and the NAS. The
>> inode numbers come back differently on Cygwin but the NAS sees
>> them as the same:
>>
>> Nancy@macki /cygdrive/f/macki/test $ ls -ali total 1048576
>> 3297336581512553257 drwxr-xr-x 1 Nancy None 0 Jun  5 11:37 .
>> 16963795924520120485 drwxr-xr-x 1 Nancy None 0 Jun  5 14:07 ..
>> 5321094878451953619 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo
>> 9211568837569774878 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo1
>> 9211568837569774879 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57 foo2
>> 3595967447105090765 -rw-r--r-- 1 Nancy None 4 Jun  5 10:57
>> foolink
>>
>> NAS> cd /volume1/Backup/macki/test/ NAS> ls -ali 16678913
>> drwxrwxrwx    2 nancy    users         4096 Jun  5 11:37 .
>> 8814593 drwxrwxrwx    5 nancy    users         4096 Jun  5 14:07
>> .. 16678915 -rwxrwxrwx    4 nancy    users            4 Jun  5
>> 10:57 foo 16678915 -rwxrwxrwx    4 nancy    users            4
>> Jun  5 10:57 foo1 16678915 -rwxrwxrwx    4 nancy    users
>> 4 Jun  5 10:57 foo2 16678915 -rwxrwxrwx    4 nancy    users
>> 4 Jun  5 10:57 foolink
>>
>> The final file "foolink" was created using a C program ala:
>>
>> $ cat link.c #include <unistd.h> #include <stdio.h>
>>
>> int main(void) { if (link("/cygdrive/f/macki/test/foo",
>> "/cygdrive/f/macki/test/foolink") == 0) { printf("Link was
>> successful\n"); } else { printf("Link was not successful\n"); }
>> return 0; }
>>
>> Thanks,
>>
>> -Clint
>>
>> On Tue, Jun 5, 2012 at 2:06 PM, Kevin Korb <[hidden email]>
>> wrote:
> --itemize-changes would be the first place to start.  Also, see if
> you can make a hard link on your NAS.  Just because the underlying
> ext4 supports it doesn't mean that your network mount does.  Try
> something like:
>
> cd /cygdrive/f/macki touch testfile ln testfile testfile2 ls -ls
> testfile testfile2
>
> They should both have the same inode number.
>
> On 06/05/12 17:02, Clint Olsen wrote:
>>>>> Hi:
>>>>>
>>>>> I have attempted to following some instructions to use
>>>>> --link-dest in order to preserve space for multiple
>>>>> backups. I'm using rsync on Cygwin with a NAS (ext4) which
>>>>> does support hard-links on the filesystem. I've written a
>>>>> short program that does attempt to create a hard-link on
>>>>> this NAS from Cygwin and it does look to be working. If I
>>>>> run ls -li on the NAS the inodes are the same. However, on
>>>>> Cygwin the inodes returned are not. I assumed that what's
>>>>> important is that the NAS handles the hard-links
>>>>> correctly.
>>>>>
>>>>> I checked the archives and it looks like a common mistake
>>>>> is inconsistent usage of the trailing backslash. My usage
>>>>> looks something like:
>>>>>
>>>>> % rsync -aP --delete --delete-excluded
>>>>> --exclude-from=/home/Nancy/.rsync/exclude
>>>>> --link-dest=/cygdrive/f/macki/current/ Nancy
>>>>> /cygdrive/f/macki/incomplete-back-2012-06-05T13:53:51
>>>>>
>>>>> I'm running this from "/cygwin/c/Documents and Settings/"
>>>>> directory, and the directory under the symlink "current"
>>>>> is "Nancy".
>>>>>
>>>>> Is there a way to tell if rsync thinks it's found a match
>>>>> and attempting to link?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Clint
>
>>> -- Please use reply-all for most replies to avoid omitting the
>>> mailing list. To unsubscribe or change options:
>>> https://lists.samba.org/mailman/listinfo/rsync Before posting,
>>> read: http://www.catb.org/~esr/faqs/smart-questions.html

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    [phone]
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iEYEARECAAYFAk/OeDMACgkQVKC1jlbQAQeAxgCfVkKDHmvOShUEp4AXj2WmQ4PD
EFgAoN0Iw26kXwL7njCpYNwzvtSn/oLK
=B4+Q
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Francis.Montagnac-2]

On Tue, 05 Jun 2012 23:18:24 +0200 Clint Olsen wrote:
> Ok, itemize changes prints out some info for each file, and it looks like:
...
> I did a quick search on this switch and someone was bitching that the
> fields were not easy to decipher w/o reading the source code :)

Wrong: the --itemize-changes option is fully described in the rsync
manual (like any other options of rsync).

--
[hidden email], Tel: [phone], Bur: C113
INRIA Sophia, 2004, rte des Lucioles, B.P.93 - 06902 Sophia Antipolis Cedex
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html