rsync slow exclude folder

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

rsync slow exclude folder

Дугин Сергей
Hi, Rsync.

rsync  version 3.0.9  protocol version 30

rsync -a --exclude="tmp/*" /home/ /backup/home/

It is necessary that the contents of the folder tmp copy, but the folder tmp in
backup was created.

If the folder /home/tmp/ is many millions of files, rsync
think of this folder can be seen through the lsof -p PID

If you do so: rsync -a --exclude="tmp/" /home/ /backup/home/
then backup is done very quickly, but tmp folder in the backup does not
will be created.

How to make so that rsync long thought over such folders, but
This created a backup of them empty?

======================================================================

Нужно  чтобы содержимое папки tmp не копировалось, но сама папка tmp в
бекапе создавалась.

Если   в  папке  /home/tmp/  находится  много  миллионов файлов, rsync
думает над этой папкой это видно через lsof -p PID

Если сдлать так:  rsync -a --exclude="tmp/" /home/ /backup/home/
то  бекап  делается  очень  быстро,  но при этом папка tmp в бекапе не
появляется.

Как  сделать так чтобы rsync долго не думал над такими папками, но при
этом создавал их пустыми в бекапе?



--
С уважением,
 Дугин Сергей                          mailto:[hidden email]
 QWARTA

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync slow exclude folder

Paul Slootman-5
On Sat 28 Feb 2015, ????? ?????? wrote:

>
> rsync  version 3.0.9  protocol version 30
>
> rsync -a --exclude="tmp/*" /home/ /backup/home/
>
> It is necessary that the contents of the folder tmp copy, but the folder tmp in
> backup was created.
>
> If the folder /home/tmp/ is many millions of files, rsync
> think of this folder can be seen through the lsof -p PID
>
> If you do so: rsync -a --exclude="tmp/" /home/ /backup/home/
> then backup is done very quickly, but tmp folder in the backup does not
> will be created.
>
> How to make so that rsync long thought over such folders, but
> This created a backup of them empty?

I think the problem is that you only excluded files in tmp, not also
directories. So if you have a directory /home/tmp/subdir/ that directory
will still be created on the destination side. That is why rsync has to
check every entry in the tmp directory: to see if it is a directory.

Try --exclude="tmp/**" to exclude everything under tmp, including
directories.


Paul
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync slow exclude folder

Francis.Montagnac-2
In reply to this post by Дугин Сергей

Hi

On Sat, 28 Feb 2015 06:41:59 +0300 Дугин Сергей wrote:

> rsync -a --exclude="tmp/*" /home/ /backup/home/

> If the folder /home/tmp/ is many millions of files, rsync
> think of this folder can be seen through the lsof -p PID

Or strace yes. It seems that rsync does an lstat for any file before
applying the exclude rules.

Possible optimisation?

> If you do so: rsync -a --exclude="tmp/" /home/ /backup/home/
> then backup is done very quickly, but tmp folder in the backup does not
> will be created.

> How to make so that rsync long thought over such folders, but
> This created a backup of them empty?

Since the backup with --exclude="tmp/" is fast it may be acceptable to
make two passes, one for the main stuff, the second to populate the
tmp/ directories.

Example:

  rsync -a --delete --exclude='tmp/' --filter='protect tmp/' /home/ /backup/home
  (cd /home && find . -name tmp -prune -print ) \
      | rsync -a --files-from - /home/ /backup/home

Francis
--
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html