Skip based on checksum not worked as expected when using with complex filter rules.

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Skip based on checksum not worked as expected when using with complex filter rules.

Hongyi Zhao
Hi all,

I've a local repository for Debian, and I'm doing some rsync testings on
it.  See the following commands for detail:

werner@debian:~/software/LocalRepo$ rsync -amvHKLPRSB131072 -n \
ftp.uk.debian.org::debian/./dists/wheezy-updates/main/\
debian-installer/binary-amd64/Packages.gz \
repo.debian.local/debian/
[snip]
dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz

sent 46 bytes  received 202 bytes  55.11 bytes/sec
total size is 20  speedup is 0.08 (DRY RUN)

werner@debian:~/software/LocalRepo$ rsync -amvHKLPRSB131072 -nc \
ftp.uk.debian.org::debian/./dists/wheezy-updates/main/\
debian-installer/binary-amd64/Packages.gz \
repo.debian.local/debian/
[snip]
sent 46 bytes  received 218 bytes  58.67 bytes/sec
total size is 20  speedup is 0.08 (DRY RUN)

As you can see, if I use checksum on that package, it shown that it's
just the same to my local copy, so there is not need to be rsynced.

But when I use a complex rsync filter rules script based on checksum and
others to check all of the stuff based on the corresponding rules, it
shows that the above package need to be updated, see the following for
detail:

1- My script is named as su_dists.rsync and the content of it is as
follows, sorry for its complex ;-(

___________ begin su_dists.rsync ____________

rsync -amvHKLPRSB131072 -nc --delete --delete-excluded \
-f +_dists/wheezy/**Translation-en* \
-f +_dists/wheezy/**binary-all/Packages.gz \
-f +_dists/wheezy/Release* \
-f P_dists/wheezy-backports/**Translation-en* \
-f P_dists/wheezy-backports/**binary-all/Packages.gz \
-f P_dists/wheezy-backports/Release* \
-f P_dists/wheezy-proposed-updates/**Translation-en* \
-f P_dists/wheezy-proposed-updates/**binary-all/Packages.gz \
-f P_dists/wheezy-proposed-updates/Release* \
-f +_dists/wheezy-updates/**Translation-en* \
-f +_dists/wheezy-updates/**binary-all/Packages.gz \
-f +_dists/wheezy-updates/Release* \
-f +_dists/wheezy/**binary-amd64/Packages.gz \
-f +_dists/wheezy/**installer-amd64/current/*** \
-f P_dists/wheezy-backports/**binary-amd64/Packages.gz \
-f P_dists/wheezy-backports/**installer-amd64/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-amd64/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-amd64/current/*** \
-f +_dists/wheezy-updates/**binary-amd64/Packages.gz \
-f +_dists/wheezy-updates/**installer-amd64/current/*** \
-f +_dists/wheezy/**binary-i386/Packages.gz \
-f +_dists/wheezy/**installer-i386/current/*** \
-f P_dists/wheezy-backports/**binary-i386/Packages.gz \
-f P_dists/wheezy-backports/**installer-i386/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-i386/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-i386/current/*** \
-f +_dists/wheezy-updates/**binary-i386/Packages.gz \
-f +_dists/wheezy-updates/**installer-i386/current/*** \
-f P_dists/jessie/*** \
-f P_dists/jessie-backports/*** \
-f P_dists/jessie-proposed-updates/*** \
-f P_dists/jessie-updates/*** \
-f P_pool/*** \
-f +_dists/***/ \
-f -_* ftp.uk.debian.org::debian/./ ./repo.debian.local/debian/

___________ end su_dists.rsync ____________

2- Then, I run my above script as follows to obtain all of the files
filtered-out according to my filter rules and checkout the ones need to
be updated based on checksum:  

werner@debian:~/software/LocalRepo$ bash su_dists.rsync|awk -F'/' \
'/^dists/ && $NF'
dists/wheezy-updates/Release
dists/wheezy-updates/Release.gpg
dists/wheezy-updates/contrib/binary-i386/Packages.gz
dists/wheezy-updates/contrib/debian-installer/binary-amd64/Packages.gz
dists/wheezy-updates/contrib/debian-installer/binary-i386/Packages.gz
dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz
dists/wheezy-updates/main/debian-installer/binary-i386/Packages.gz
dists/wheezy-updates/non-free/binary-amd64/Packages.gz
dists/wheezy-updates/non-free/binary-i386/Packages.gz
dists/wheezy-updates/non-free/debian-installer/binary-amd64/Packages.gz
dists/wheezy-updates/non-free/debian-installer/binary-i386/Packages.gz
dists/wheezy-updates/non-free/i18n/Translation-en.bz2
dists/wheezy/contrib/debian-installer/binary-all/Packages.gz
dists/wheezy/contrib/debian-installer/binary-amd64/Packages.gz
dists/wheezy/contrib/debian-installer/binary-i386/Packages.gz
dists/wheezy/non-free/debian-installer/binary-all/Packages.gz
dists/wheezy/non-free/debian-installer/binary-amd64/Packages.gz
dists/wheezy/non-free/debian-installer/binary-i386/Packages.gz

As you can see, this time, the following file is considered to be updated:

dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz

But it has been considered not need to be updated in my first run with
the -c/--checksum option from the terminal other than the bash srcipt.

3- If I change my bash script into the following form, i.e., with the `--
no-t' option added to the args list:

___________ begin su_dists.rsync ____________

rsync -amvHKLPRSB131072 -nc --no-t --delete --delete-excluded \
-f +_dists/wheezy/**Translation-en* \
-f +_dists/wheezy/**binary-all/Packages.gz \
-f +_dists/wheezy/Release* \
-f P_dists/wheezy-backports/**Translation-en* \
-f P_dists/wheezy-backports/**binary-all/Packages.gz \
-f P_dists/wheezy-backports/Release* \
-f P_dists/wheezy-proposed-updates/**Translation-en* \
-f P_dists/wheezy-proposed-updates/**binary-all/Packages.gz \
-f P_dists/wheezy-proposed-updates/Release* \
-f +_dists/wheezy-updates/**Translation-en* \
-f +_dists/wheezy-updates/**binary-all/Packages.gz \
-f +_dists/wheezy-updates/Release* \
-f +_dists/wheezy/**binary-amd64/Packages.gz \
-f +_dists/wheezy/**installer-amd64/current/*** \
-f P_dists/wheezy-backports/**binary-amd64/Packages.gz \
-f P_dists/wheezy-backports/**installer-amd64/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-amd64/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-amd64/current/*** \
-f +_dists/wheezy-updates/**binary-amd64/Packages.gz \
-f +_dists/wheezy-updates/**installer-amd64/current/*** \
-f +_dists/wheezy/**binary-i386/Packages.gz \
-f +_dists/wheezy/**installer-i386/current/*** \
-f P_dists/wheezy-backports/**binary-i386/Packages.gz \
-f P_dists/wheezy-backports/**installer-i386/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-i386/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-i386/current/*** \
-f +_dists/wheezy-updates/**binary-i386/Packages.gz \
-f +_dists/wheezy-updates/**installer-i386/current/*** \
-f P_dists/jessie/*** \
-f P_dists/jessie-backports/*** \
-f P_dists/jessie-proposed-updates/*** \
-f P_dists/jessie-updates/*** \
-f P_pool/*** \
-f +_dists/***/ \
-f -_* ftp.uk.debian.org::debian/./ ./repo.debian.local/debian/

___________ end su_dists.rsync ____________

This time, I will obtain the correct file-lists that should be updated
based on the checksum checking, see following for detail:

werner@debian:~/software/LocalRepo$ bash su_dists.rsync|awk -F'/' \
'/^dists/ && $NF'
dists/wheezy-updates/Release
dists/wheezy-updates/Release.gpg
dists/wheezy-updates/non-free/i18n/Translation-en.bz2

As you can see, for my case, there only have three files which need to be
updated all toghether at the moment.

Any hints on the above issues?

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Kevin Korb
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Any time rsync wants to do something to a file you don't think it
should be see what --itemize-changes says about it.  In fact, -v is
almost entirely useless without --itemize-changes.  In fact, if I was
in charge I would make -v include --itemize-changes.

On 04/17/2015 09:21 PM, Hongyi Zhao wrote:

> Hi all,
>
> I've a local repository for Debian, and I'm doing some rsync
> testings on it.  See the following commands for detail:
>
> werner@debian:~/software/LocalRepo$ rsync -amvHKLPRSB131072 -n \
> ftp.uk.debian.org::debian/./dists/wheezy-updates/main/\
> debian-installer/binary-amd64/Packages.gz \
> repo.debian.local/debian/ [snip]
> dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz
>
>  sent 46 bytes  received 202 bytes  55.11 bytes/sec total size is
> 20  speedup is 0.08 (DRY RUN)
>
> werner@debian:~/software/LocalRepo$ rsync -amvHKLPRSB131072 -nc \
> ftp.uk.debian.org::debian/./dists/wheezy-updates/main/\
> debian-installer/binary-amd64/Packages.gz \
> repo.debian.local/debian/ [snip] sent 46 bytes  received 218 bytes
> 58.67 bytes/sec total size is 20  speedup is 0.08 (DRY RUN)
>
> As you can see, if I use checksum on that package, it shown that
> it's just the same to my local copy, so there is not need to be
> rsynced.
>
> But when I use a complex rsync filter rules script based on
> checksum and others to check all of the stuff based on the
> corresponding rules, it shows that the above package need to be
> updated, see the following for detail:
>
> 1- My script is named as su_dists.rsync and the content of it is as
>  follows, sorry for its complex ;-(
>
> ___________ begin su_dists.rsync ____________
>
> rsync -amvHKLPRSB131072 -nc --delete --delete-excluded \ -f
> +_dists/wheezy/**Translation-en* \ -f
> +_dists/wheezy/**binary-all/Packages.gz \ -f
> +_dists/wheezy/Release* \ -f
> P_dists/wheezy-backports/**Translation-en* \ -f
> P_dists/wheezy-backports/**binary-all/Packages.gz \ -f
> P_dists/wheezy-backports/Release* \ -f
> P_dists/wheezy-proposed-updates/**Translation-en* \ -f
> P_dists/wheezy-proposed-updates/**binary-all/Packages.gz \ -f
> P_dists/wheezy-proposed-updates/Release* \ -f
> +_dists/wheezy-updates/**Translation-en* \ -f
> +_dists/wheezy-updates/**binary-all/Packages.gz \ -f
> +_dists/wheezy-updates/Release* \ -f
> +_dists/wheezy/**binary-amd64/Packages.gz \ -f
> +_dists/wheezy/**installer-amd64/current/*** \ -f
> P_dists/wheezy-backports/**binary-amd64/Packages.gz \ -f
> P_dists/wheezy-backports/**installer-amd64/current/*** \ -f
> P_dists/wheezy-proposed-updates/**binary-amd64/Packages.gz \ -f
> P_dists/wheezy-proposed-updates/**installer-amd64/current/*** \ -f
> +_dists/wheezy-updates/**binary-amd64/Packages.gz \ -f
> +_dists/wheezy-updates/**installer-amd64/current/*** \ -f
> +_dists/wheezy/**binary-i386/Packages.gz \ -f
> +_dists/wheezy/**installer-i386/current/*** \ -f
> P_dists/wheezy-backports/**binary-i386/Packages.gz \ -f
> P_dists/wheezy-backports/**installer-i386/current/*** \ -f
> P_dists/wheezy-proposed-updates/**binary-i386/Packages.gz \ -f
> P_dists/wheezy-proposed-updates/**installer-i386/current/*** \ -f
> +_dists/wheezy-updates/**binary-i386/Packages.gz \ -f
> +_dists/wheezy-updates/**installer-i386/current/*** \ -f
> P_dists/jessie/*** \ -f P_dists/jessie-backports/*** \ -f
> P_dists/jessie-proposed-updates/*** \ -f P_dists/jessie-updates/***
> \ -f P_pool/*** \ -f +_dists/***/ \ -f -_*
> ftp.uk.debian.org::debian/./ ./repo.debian.local/debian/
>
> ___________ end su_dists.rsync ____________
>
> 2- Then, I run my above script as follows to obtain all of the
> files filtered-out according to my filter rules and checkout the
> ones need to be updated based on checksum:
>
> werner@debian:~/software/LocalRepo$ bash su_dists.rsync|awk -F'/'
> \ '/^dists/ && $NF' dists/wheezy-updates/Release
> dists/wheezy-updates/Release.gpg
> dists/wheezy-updates/contrib/binary-i386/Packages.gz
> dists/wheezy-updates/contrib/debian-installer/binary-amd64/Packages.gz
>
>
dists/wheezy-updates/contrib/debian-installer/binary-i386/Packages.gz
> dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz
>
>
dists/wheezy-updates/main/debian-installer/binary-i386/Packages.gz
> dists/wheezy-updates/non-free/binary-amd64/Packages.gz
> dists/wheezy-updates/non-free/binary-i386/Packages.gz
> dists/wheezy-updates/non-free/debian-installer/binary-amd64/Packages.gz
>
>
dists/wheezy-updates/non-free/debian-installer/binary-i386/Packages.gz

> dists/wheezy-updates/non-free/i18n/Translation-en.bz2
> dists/wheezy/contrib/debian-installer/binary-all/Packages.gz
> dists/wheezy/contrib/debian-installer/binary-amd64/Packages.gz
> dists/wheezy/contrib/debian-installer/binary-i386/Packages.gz
> dists/wheezy/non-free/debian-installer/binary-all/Packages.gz
> dists/wheezy/non-free/debian-installer/binary-amd64/Packages.gz
> dists/wheezy/non-free/debian-installer/binary-i386/Packages.gz
>
> As you can see, this time, the following file is considered to be
> updated:
>
> dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz
>
>  But it has been considered not need to be updated in my first run
> with the -c/--checksum option from the terminal other than the bash
> srcipt.
>
> 3- If I change my bash script into the following form, i.e., with
> the `-- no-t' option added to the args list:
>
> ___________ begin su_dists.rsync ____________
>
> rsync -amvHKLPRSB131072 -nc --no-t --delete --delete-excluded \ -f
> +_dists/wheezy/**Translation-en* \ -f
> +_dists/wheezy/**binary-all/Packages.gz \ -f
> +_dists/wheezy/Release* \ -f
> P_dists/wheezy-backports/**Translation-en* \ -f
> P_dists/wheezy-backports/**binary-all/Packages.gz \ -f
> P_dists/wheezy-backports/Release* \ -f
> P_dists/wheezy-proposed-updates/**Translation-en* \ -f
> P_dists/wheezy-proposed-updates/**binary-all/Packages.gz \ -f
> P_dists/wheezy-proposed-updates/Release* \ -f
> +_dists/wheezy-updates/**Translation-en* \ -f
> +_dists/wheezy-updates/**binary-all/Packages.gz \ -f
> +_dists/wheezy-updates/Release* \ -f
> +_dists/wheezy/**binary-amd64/Packages.gz \ -f
> +_dists/wheezy/**installer-amd64/current/*** \ -f
> P_dists/wheezy-backports/**binary-amd64/Packages.gz \ -f
> P_dists/wheezy-backports/**installer-amd64/current/*** \ -f
> P_dists/wheezy-proposed-updates/**binary-amd64/Packages.gz \ -f
> P_dists/wheezy-proposed-updates/**installer-amd64/current/*** \ -f
> +_dists/wheezy-updates/**binary-amd64/Packages.gz \ -f
> +_dists/wheezy-updates/**installer-amd64/current/*** \ -f
> +_dists/wheezy/**binary-i386/Packages.gz \ -f
> +_dists/wheezy/**installer-i386/current/*** \ -f
> P_dists/wheezy-backports/**binary-i386/Packages.gz \ -f
> P_dists/wheezy-backports/**installer-i386/current/*** \ -f
> P_dists/wheezy-proposed-updates/**binary-i386/Packages.gz \ -f
> P_dists/wheezy-proposed-updates/**installer-i386/current/*** \ -f
> +_dists/wheezy-updates/**binary-i386/Packages.gz \ -f
> +_dists/wheezy-updates/**installer-i386/current/*** \ -f
> P_dists/jessie/*** \ -f P_dists/jessie-backports/*** \ -f
> P_dists/jessie-proposed-updates/*** \ -f P_dists/jessie-updates/***
> \ -f P_pool/*** \ -f +_dists/***/ \ -f -_*
> ftp.uk.debian.org::debian/./ ./repo.debian.local/debian/
>
> ___________ end su_dists.rsync ____________
>
> This time, I will obtain the correct file-lists that should be
> updated based on the checksum checking, see following for detail:
>
> werner@debian:~/software/LocalRepo$ bash su_dists.rsync|awk -F'/'
> \ '/^dists/ && $NF' dists/wheezy-updates/Release
> dists/wheezy-updates/Release.gpg
> dists/wheezy-updates/non-free/i18n/Translation-en.bz2
>
> As you can see, for my case, there only have three files which need
> to be updated all toghether at the moment.
>
> Any hints on the above issues?
>
> Regards
>

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlUxsxwACgkQVKC1jlbQAQcTNQCgnjBUW0D3nS0e7y/HL5lQozyw
eGIAnjokKS3M+U0IM2xKUy9xSb/n5rUa
=ZLxY
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Hongyi Zhao
On Fri, 17 Apr 2015 21:27:56 -0400, Kevin Korb wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Any time rsync wants to do something to a file you don't think it should
> be see what --itemize-changes says about it.  In fact, -v is almost
> entirely useless without --itemize-changes.  In fact, if I was in charge
> I would make -v include --itemize-changes.

Thanks a lot for your hints.

Based on your hints, I use the `-cniv' options combinations and do the
extract as follows on the output of rsync:

awk '/^>fc/'

And it does the trick.

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Kevin Korb
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

So, I am assuming that without --itemize-changes you couldn't tell the
difference between "I am transferring a file" and "I am fixing the
timestamp on a file".

Even if I am right, you are probably still mis-using --checksum.  When
you use --checksum you force a checksumming of absolutely everything
on both ends before anything else happens.  Without --checksum a file
with a wrong timestamp will be delta-xferred.  A delta-xfer is at
worst the same as a both side checksum.  If you let it delta-xfer
files with wrong timestamps that is much faster than --checksum since
- --checksum wastes time checksumming files that can't possibly have a
checksum match.

Simply put, if you use --checksum without one of --itemize-changes,
- --link-dest, or --write-batch you are doing something horribly wrong.


On 04/17/2015 11:34 PM, Hongyi Zhao wrote:

> On Fri, 17 Apr 2015 21:27:56 -0400, Kevin Korb wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>
>> Any time rsync wants to do something to a file you don't think it
>> should be see what --itemize-changes says about it.  In fact, -v
>> is almost entirely useless without --itemize-changes.  In fact,
>> if I was in charge I would make -v include --itemize-changes.
>
> Thanks a lot for your hints.
>
> Based on your hints, I use the `-cniv' options combinations and do
> the extract as follows on the output of rsync:
>
> awk '/^>fc/'
>
> And it does the trick.
>
> Regards
>

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlUx02YACgkQVKC1jlbQAQeNKACg5NU2TiogzWzqMii98NExtI+V
ou4AoLNW320LrRCCYp9vZQIN4sLruWKW
=oM7m
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Hongyi Zhao
On Fri, 17 Apr 2015 23:45:42 -0400, Kevin Korb wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> So, I am assuming that without --itemize-changes you couldn't tell the
> difference between "I am transferring a file" and "I am fixing the
> timestamp on a file".
>
> Even if I am right, you are probably still mis-using --checksum.  When
> you use --checksum you force a checksumming of absolutely everything on
> both ends before anything else happens.  Without --checksum a file with
> a wrong timestamp will be delta-xferred.  A delta-xfer is at worst the
> same as a both side checksum.  If you let it delta-xfer files with wrong
> timestamps that is much faster than --checksum since - --checksum wastes
> time checksumming files that can't possibly have a checksum match.

Yes, with `--checksum' is a time-consuming thing ;-(

>
> Simply put, if you use --checksum without one of --itemize-changes,
> - --link-dest, or --write-batch you are doing something horribly wrong.

So, I think that use the `-niv' combinations and then do the extract as
follows:

awk '/^>f/'

Should enuogh for most cases to check the stuff which are need to be
updated.

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Kevin Korb
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I still don't see an output from rsync --itemize-changes with an A/B
difference.

On 04/18/2015 12:15 AM, Hongyi Zhao wrote:

> On Fri, 17 Apr 2015 23:45:42 -0400, Kevin Korb wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>
>> So, I am assuming that without --itemize-changes you couldn't
>> tell the difference between "I am transferring a file" and "I am
>> fixing the timestamp on a file".
>>
>> Even if I am right, you are probably still mis-using --checksum.
>> When you use --checksum you force a checksumming of absolutely
>> everything on both ends before anything else happens.  Without
>> --checksum a file with a wrong timestamp will be delta-xferred.
>> A delta-xfer is at worst the same as a both side checksum.  If
>> you let it delta-xfer files with wrong timestamps that is much
>> faster than --checksum since - --checksum wastes time
>> checksumming files that can't possibly have a checksum match.
>
> Yes, with `--checksum' is a time-consuming thing ;-(
>
>>
>> Simply put, if you use --checksum without one of
>> --itemize-changes, - --link-dest, or --write-batch you are doing
>> something horribly wrong.
>
> So, I think that use the `-niv' combinations and then do the
> extract as follows:
>
> awk '/^>f/'
>
> Should enuogh for most cases to check the stuff which are need to
> be updated.
>
> Regards
>

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlUx4A4ACgkQVKC1jlbQAQeXkwCg3PV/RHRH0YmJexnhT2UipZ2j
NPsAnA1CDtilemMOagzjA5nPV+8/vW+k
=6FVH
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Hongyi Zhao
On Sat, 18 Apr 2015 00:39:42 -0400, Kevin Korb wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I still don't see an output from rsync --itemize-changes with an A/B
> difference.

What do you mean by saying the above things?  

Do you mean only the `--itemize-changes' is enough for my case?

Or

Do you want to see the output from my case of running on the different
rsync options?

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Kevin Korb
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I mean that a listing of a /path/to/filename is utterly useless.
- --itemize-changes specifies why a particular /path/to/filename is
listed.It also describes what will happen to that file.  All in a very
short string of characters.

Simply put, --itemize-changes is so determinative that every second
you resist using it is evidence that you are falsifying a nonsensical
problem with rsync.

On 04/18/2015 12:58 AM, Hongyi Zhao wrote:

> On Sat, 18 Apr 2015 00:39:42 -0400, Kevin Korb wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>
>> I still don't see an output from rsync --itemize-changes with an
>> A/B difference.
>
> What do you mean by saying the above things?
>
> Do you mean only the `--itemize-changes' is enough for my case?
>
> Or
>
> Do you want to see the output from my case of running on the
> different rsync options?
>
> Regards
>

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlUx51EACgkQVKC1jlbQAQcfcwCgpZgRJkt7dMPVVflrLdw89C0b
t+IAn2lEgkE32tOmPWecOSVX1FsDXycg
=kHmo
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: Skip based on checksum not worked as expected when using with complex filter rules.

Hongyi Zhao
On Sat, 18 Apr 2015 01:10:41 -0400, Kevin Korb wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I mean that a listing of a /path/to/filename is utterly useless.
> - --itemize-changes specifies why a particular /path/to/filename is
> listed.It also describes what will happen to that file.  All in a very
> short string of characters.
>
> Simply put, --itemize-changes is so determinative that every second you
> resist using it is evidence that you are falsifying a nonsensical
> problem with rsync.

Got it, thanks a lot.

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html