The patch write-devices.diff (rw-devices.diff) fix

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

The patch write-devices.diff (rw-devices.diff) fix

Vaclav Ovsik
Hello,
I faced the problem of synchronization block devices and found the patch
write-devices.diff by Darryl Dixon, that went trough this mailing list
http://lists.samba.org/archive/rsync/2009-October/024037.html

Unfortunately the patch was one flaw for me. It doesn't improve
performance of synchronization. When I tried to test it using something
like:

./rsync --rsync-path\="$PWD/rsync" -v --bwlimit=100 --progress --copy-devices --write-devices /dev/mapper/vg-pokus1 root@localhost:/dev/mapper/vg-pokus2

where pokus1 & pokus2 are two identically initialized block devices
(logical volumes) 32MiB in size. All the data was passed trough
local connection without the benefit. (I set --bwlimit low, so I can see
evidently the rsync performance).

By adding more -v I found, that some stat() call in the generator.c
reports destination device size is zero. I added some code handling
block device size. Option --ignore-times is implied for --rw-devices.
A new patch rw-devices.diff is attached. It is combination of
copy-devices.diff, write-devices.diff and some new code from me.


bobek:/data/soft/rsync/rsync# ./rsync --rsync-path\="$PWD/rsync" -v --bwlimit=100 --progress --rw-devices root@localhost:/dev/mapper/vg-pokus2 /dev/mapper/vg-pokus1
vg-pokus2
     33,554,432 100%   48.78MB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 40,601 bytes  received 23,264 bytes  18,247.14 bytes/sec
total size is 33,554,432  speedup is 525.40
bobek:/data/soft/rsync/rsync# dd if=/dev/urandom bs=1M seek=16 count=1 of=/dev/mapper/vg-pokus2
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198132 s, 5.3 MB/s
bobek:/data/soft/rsync/rsync# ./rsync --rsync-path\="$PWD/rsync" -v --bwlimit=100 --progress --rw-devices root@localhost:/dev/mapper/vg-pokus2 /dev/mapper/vg-pokus1
vg-pokus2
     33,554,432 100%    2.95MB/s    0:00:10 (xfr#1, to-chk=0/1)

sent 40,601 bytes  received 1,076,931 bytes  82,780.15 bytes/sec
total size is 33,554,432  speedup is 30.03

Now I have questions:
    1. Is it worth to have two separate patches copy-devices.diff,
       write-devices.diff? (It is hard to apply them one after other,
       because they overlays.)
    2. Should be there two options `--copy-devices', `--write-devices'?
       I thing one option e.g. `--rw-devices' should be sufficient.
    3. I think the above option should collide with many other options
       like `--archive', `--recursive',...

Any suggestions?
Regards
--
Zito

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

rw-devices.diff (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The patch write-devices.diff (rw-devices.diff) fix

Darryl Dixon
Hi Zito,

> Hello,
> I faced the problem of synchronization block devices and found the patch
> write-devices.diff by Darryl Dixon, that went trough this mailing list
> http://lists.samba.org/archive/rsync/2009-October/024037.html
>
> Unfortunately the patch was one flaw for me. It doesn't improve
> performance of synchronization. When I tried to test it using something
> like:

Yes. I figured it would be something to do with the block device size
calculation, as my initial patch simply punted on that (0 as you saw) - I
didn't have time/motivation to dig through and research the necessary
magic to get good data out. Well done for finding and implementing the
required code :)

>
> ./rsync --rsync-path\="$PWD/rsync" -v --bwlimit=100 --progress
> --copy-devices --write-devices /dev/mapper/vg-pokus1
> root@localhost:/dev/mapper/vg-pokus2
>
> where pokus1 & pokus2 are two identically initialized block devices
> (logical volumes) 32MiB in size. All the data was passed trough
> local connection without the benefit. (I set --bwlimit low, so I can see
> evidently the rsync performance).
>
> By adding more -v I found, that some stat() call in the generator.c
> reports destination device size is zero. I added some code handling
> block device size. Option --ignore-times is implied for --rw-devices.
> A new patch rw-devices.diff is attached. It is combination of
> copy-devices.diff, write-devices.diff and some new code from me.

Excellent stuff. This is what I love about open source software - one
person can get the ball rolling and then the effort and expertise of
others combines to improve matters for everyone.

> Now I have questions:
>     1. Is it worth to have two separate patches copy-devices.diff,
>        write-devices.diff? (It is hard to apply them one after other,
>        because they overlays.)
>     2. Should be there two options `--copy-devices', `--write-devices'?
>        I thing one option e.g. `--rw-devices' should be sufficient.
>     3. I think the above option should collide with many other options
>        like `--archive', `--recursive',...

I agree maintaining the separate patches are hard. Personally, I'd prefer
to see just one merged patch as you suggest.

regards,
Darryl Dixon
Winterhouse Consulting Ltd
http://www.winterhouseconsulting.com
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html