How to set absolute path for rsync?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How to set absolute path for rsync?

Samba - rsync mailing list

Dear experts,

 

I would like get your help for the issue with --partial-dir=absolute path.

 

I’m trying to use rsync to send big file to server and config --partial-dir=absolute path since I want to store temp partial file in /tmp/ directory and keep the destination folder clear.

 

My server is joe28, client is joe27. And here is the config on server.

 

root@joe28:~# cat /etc/rsyncd.conf

# Global settings

address = 10.148.34.28

numeric ids = yes

uid = 0

read only = no

reverse lookup = no

dont compress = *

timeout = 10

[JOE]

        path = /home/joe/rsync

        comment = Joe's rsync test

 

Sending big file from joe27 to joe28:

 

root@joe27:~# ls -l /home/joe/rsync/flash_image

-rw-r--r-- 1 root root 766028824 Aug 15 15:23 /home/joe/rsync/flash_image

 

root@joe27:~# rsync -a --delete --delay-updates --no-relative --partial-dir=/tmp /home/joe/rsync/ rsync://10.148.34.28:873/JOE -P

building file list ...

2 files to consider

./

flash_image

   766028824 100%  108.69MB/s    0:00:06 (xfer#1, to-check=0/2)

 

sent 766122429 bytes  received 30 bytes  61289796.72 bytes/sec

total size is 766028824  speedup is 1.00

 

However, I still see the partial file in the local directory, but not in /tmp which I configured.

 

root@joe28:~# watch --interval=1 ls -al /home/joe/rsync/ /tmp/flash*

Every 1.0s: ls -al /home/joe/rsync/ /tmp/flash*                                         Tue Aug 15 15:35:55 2017

 

ls: cannot access /tmp/flash*: No such file or directory

/home/joe/rsync/:

total 77064

drwxr-xr-x  2 root root        4096 Aug 15 15:35 .

drwxr-xr-x 32 joe  joe         4096 Aug 15 15:01 ..

-rw-------  1 root nogroup 78905344 Aug 15 15:35 .flash_image.xlIBqo

 

And finally I saw the “tmp” folder is created on server. That is not my expected.

 

Every 1.0s: ls -al /home/joe/rsync/ /tmp/flash*                                         Tue Aug 15 15:36:05 2017

 

ls: cannot access /tmp/flash*: No such file or directory

/home/joe/rsync/:

total 748092

drwxr-xr-x  3 root root         4096 Aug 15 15:14 .

drwxr-xr-x 32 joe  joe          4096 Aug 15 15:01 ..

-rw-r--r--  1 root root    766028824 Aug 15 15:23 flash_image

drwx------  2 root nogroup      4096 Aug 15 15:36 tmp

 

Would you please give some suggestions? Appreciate the help!

 

BTW. I tried with --temp-dir but same result. Rsync version is 3.1.2.

 

Regards,

-Joe


--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: How to set absolute path for rsync?

Samba - rsync mailing list
With rsyncd you can't.  You would have to switch to rsync over ssh...

rsync -a --delete --delay-updates --partial-dir=/tmp -P /home/joe/rsync/
10.148.34.28:/home/joe/rsync/

Also, when you use --partial-dir you probably want to also use --temp-dir.


On 08/15/2017 07:19 PM, Joe Qiao via rsync wrote:

> Dear experts,
>
>  
>
> I would like get your help for the issue with --partial-dir=_absolute path._
>
>  
>
> I’m trying to use rsync to send big file to server and config
> --partial-dir=_absolute path_ since I want to store temp partial file in
> /tmp/ directory and keep the destination folder clear.
>
>  
>
> My server is joe28, client is joe27. And here is the config on server.
>
>  
>
> root@joe28:~# cat /etc/rsyncd.conf
>
> # Global settings
>
> address = 10.148.34.28
>
> numeric ids = yes
>
> uid = 0
>
> read only = no
>
> reverse lookup = no
>
> dont compress = *
>
> timeout = 10
>
> [JOE]
>
>         path = /home/joe/rsync
>
>         comment = Joe's rsync test
>
>  
>
> Sending big file from joe27 to joe28:
>
>  
>
> root@joe27:~# ls -l /home/joe/rsync/flash_image
>
> -rw-r--r-- 1 root root 766028824 Aug 15 15:23 /home/joe/rsync/flash_image
>
>  
>
> root@joe27:~# rsync -a --delete --delay-updates
> --no-relative --partial-dir=/tmp /home/joe/rsync/
> rsync://10.148.34.28:873/JOE <http://10.148.34.28:873/JOE> -P
>
> building file list ...
>
> 2 files to consider
>
> ./
>
> flash_image
>
>    766028824 100%  108.69MB/s    0:00:06 (xfer#1, to-check=0/2)
>
>  
>
> sent 766122429 bytes  received 30 bytes  61289796.72 bytes/sec
>
> total size is 766028824  speedup is 1.00
>
>  
>
> However, I still see the partial file in the local directory, but not in
> /tmp which I configured.
>
>  
>
> root@joe28:~# watch --interval=1 ls -al /home/joe/rsync/ /tmp/flash*
>
> Every 1.0s: ls -al /home/joe/rsync/
> /tmp/flash*                                         Tue Aug 15 15:35:55 2017
>
>  
>
> ls: cannot access /tmp/flash*: No such file or directory
>
> /home/joe/rsync/:
>
> total 77064
>
> drwxr-xr-x  2 root root        4096 Aug 15 15:35 .
>
> drwxr-xr-x 32 joe  joe         4096 Aug 15 15:01 ..
>
> -rw-------  1 root nogroup 78905344 Aug 15 15:35 .flash_image.xlIBqo
>
>  
>
> And finally I saw the “tmp” folder is created on server. That is not my
> expected.
>
>  
>
> Every 1.0s: ls -al /home/joe/rsync/
> /tmp/flash*                                         Tue Aug 15 15:36:05 2017
>
>  
>
> ls: cannot access /tmp/flash*: No such file or directory
>
> /home/joe/rsync/:
>
> total 748092
>
> drwxr-xr-x  3 root root         4096 Aug 15 15:14 .
>
> drwxr-xr-x 32 joe  joe          4096 Aug 15 15:01 ..
>
> -rw-r--r--  1 root root    766028824 Aug 15 15:23 flash_image
>
> drwx------  2 root nogroup      4096 Aug 15 15:36 tmp
>
>  
>
> Would you please give some suggestions? Appreciate the help!
>
>  
>
> BTW. I tried with --temp-dir but same result. Rsync version is 3.1.2.
>
>  
>
> Regards,
>
> -Joe
>
>
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,


--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

signature.asc (231 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to set absolute path for rsync?

Samba - rsync mailing list
Thanks so much for the quick reply, Kevin!

I tried with ssh and --partial-dir, it looks the partial file still will be stored in local dir, but not in /tmp.


Every 1.0s: ls -al /home/joe/rsync/ /tmp/                                               Tue Aug 15 17:29:30 2017

/home/joe/rsync/:
total 408840
drwxr-xr-x  2 root root      4096 Aug 15 17:29 .
drwxr-xr-x 32 joe  joe       4096 Aug 15 15:01 ..
-rw-------  1 root root 418643968 Aug 15 17:29 .flash_image.14WoMV   <<<<<<<<<<<<<<<<<<

/tmp/:
total 32
drwxrwxrwt  7 root    root    4096 Aug 15 17:28 .
drwxr-xr-x 24 root    root    4096 Mar 27 15:05 ..
drwxrwxrwt  2 lightdm lightdm 4096 Aug  7 09:23 at-spi2
drwxrwxrwt  2 root    root    4096 Aug  7 09:23 .ICE-unix
drwx------  2 lightdm lightdm 4096 Aug  7 09:24 pulse-bYV3V1IM0vBC
drwx------  2 lightdm lightdm 4096 Aug  7 09:24 pulse-PKdhtXMmr18n
-rw-rw-r--  1 lightdm lightdm    0 Aug  7 09:24 unity_support_test.1
-r--r--r--  1 root    root      11 Aug  7 09:23 .X0-lock
drwxrwxrwt  2 root    root    4096 Aug  7 09:23 .X11-unix


And tried again with ssh and --temp-dir, it works.


Every 1.0s: ls -al /home/joe/rsync/ /tmp/                                               Tue Aug 15 17:28:15 2017

/home/joe/rsync/:
total 8
drwxr-xr-x  2 root root 4096 Aug 15 15:14 .
drwxr-xr-x 32 joe  joe  4096 Aug 15 15:01 ..

/tmp/:
total 581664
drwxrwxrwt  7 root    root         4096 Aug 15 17:28 .
drwxr-xr-x 24 root    root         4096 Mar 27 15:05 ..
drwxrwxrwt  2 lightdm lightdm      4096 Aug  7 09:23 at-spi2
-rw-------  1 root    root    595591168 Aug 15 17:28 .flash_image.AX9F1y    <<<<<<<<<<<<<<<<<<<<<
drwxrwxrwt  2 root    root         4096 Aug  7 09:23 .ICE-unix
drwx------  2 lightdm lightdm      4096 Aug  7 09:24 pulse-bYV3V1IM0vBC
drwx------  2 lightdm lightdm      4096 Aug  7 09:24 pulse-PKdhtXMmr18n
-rw-rw-r--  1 lightdm lightdm         0 Aug  7 09:24 unity_support_test.1
-r--r--r--  1 root    root           11 Aug  7 09:23 .X0-lock
drwxrwxrwt  2 root    root         4096 Aug  7 09:23 .X11-unix

Anyway, at least the ssh + temp-dir works for me. Thank you!

Regards,
-Joe


On Tue, Aug 15, 2017 at 5:14 PM, Kevin Korb via rsync <[hidden email]> wrote:
With rsyncd you can't.  You would have to switch to rsync over ssh...

rsync -a --delete --delay-updates --partial-dir=/tmp -P /home/joe/rsync/
10.148.34.28:/home/joe/rsync/

Also, when you use --partial-dir you probably want to also use --temp-dir.


On 08/15/2017 07:19 PM, Joe Qiao via rsync wrote:
> Dear experts,
>
>
>
> I would like get your help for the issue with --partial-dir=_absolute path._
>
>
>
> I’m trying to use rsync to send big file to server and config
> --partial-dir=_absolute path_ since I want to store temp partial file in
> /tmp/ directory and keep the destination folder clear.
>
>
>
> My server is joe28, client is joe27. And here is the config on server.
>
>
>
> root@joe28:~# cat /etc/rsyncd.conf
>
> # Global settings
>
> address = 10.148.34.28
>
> numeric ids = yes
>
> uid = 0
>
> read only = no
>
> reverse lookup = no
>
> dont compress = *
>
> timeout = 10
>
> [JOE]
>
>         path = /home/joe/rsync
>
>         comment = Joe's rsync test
>
>
>
> Sending big file from joe27 to joe28:
>
>
>
> root@joe27:~# ls -l /home/joe/rsync/flash_image
>
> -rw-r--r-- 1 root root 766028824 Aug 15 15:23 /home/joe/rsync/flash_image
>
>
>
> root@joe27:~# rsync -a --delete --delay-updates
> --no-relative --partial-dir=/tmp /home/joe/rsync/
> rsync://10.148.34.28:873/JOE <http://10.148.34.28:873/JOE> -P
>
> building file list ...
>
> 2 files to consider
>
> ./
>
> flash_image
>
>    766028824 100%  108.69MB/s    0:00:06 (xfer#1, to-check=0/2)
>
>
>
> sent 766122429 bytes  received 30 bytes  61289796.72 bytes/sec
>
> total size is 766028824  speedup is 1.00
>
>
>
> However, I still see the partial file in the local directory, but not in
> /tmp which I configured.
>
>
>
> root@joe28:~# watch --interval=1 ls -al /home/joe/rsync/ /tmp/flash*
>
> Every 1.0s: ls -al /home/joe/rsync/
> /tmp/flash*                                         Tue Aug 15 15:35:55 2017
>
>
>
> ls: cannot access /tmp/flash*: No such file or directory
>
> /home/joe/rsync/:
>
> total 77064
>
> drwxr-xr-x  2 root root        4096 Aug 15 15:35 .
>
> drwxr-xr-x 32 joe  joe         4096 Aug 15 15:01 ..
>
> -rw-------  1 root nogroup 78905344 Aug 15 15:35 .flash_image.xlIBqo
>
>
>
> And finally I saw the “tmp” folder is created on server. That is not my
> expected.
>
>
>
> Every 1.0s: ls -al /home/joe/rsync/
> /tmp/flash*                                         Tue Aug 15 15:36:05 2017
>
>
>
> ls: cannot access /tmp/flash*: No such file or directory
>
> /home/joe/rsync/:
>
> total 748092
>
> drwxr-xr-x  3 root root         4096 Aug 15 15:14 .
>
> drwxr-xr-x 32 joe  joe          4096 Aug 15 15:01 ..
>
> -rw-r--r--  1 root root    766028824 Aug 15 15:23 flash_image
>
> drwx------  2 root nogroup      4096 Aug 15 15:36 tmp
>
>
>
> Would you please give some suggestions? Appreciate the help!
>
>
>
> BTW. I tried with --temp-dir but same result. Rsync version is 3.1.2.
>
>
>
> Regards,
>
> -Joe
>
>
>

--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
        Kevin Korb                      Phone:    <a href="tel:%28407%29%20252-6853" value="+14072526853">(407) 252-6853
        Systems Administrator           Internet:
        FutureQuest, Inc.               [hidden email]  (work)
        Orlando, Florida                [hidden email] (personal)
        Web page:                       http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,


--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: How to set absolute path for rsync?

Samba - rsync mailing list
On Tue 15 Aug 2017, Joe Qiao via rsync wrote:

> Thanks so much for the quick reply, Kevin!
>
> I tried with ssh and --partial-dir, it looks the partial file still will be
> stored in local dir, but not in /tmp.
>
>
> Every 1.0s: ls -al /home/joe/rsync/ /tmp/
>             Tue Aug 15 17:29:30 2017
>
> /home/joe/rsync/:
> total 408840
> drwxr-xr-x  2 root root      4096 Aug 15 17:29 .
> drwxr-xr-x 32 joe  joe       4096 Aug 15 15:01 ..
> *-rw-------  1 root root 418643968 Aug 15 17:29 .flash_image.14WoMV

No, you misunderstand the meaning of "partial" here.

--partial-dir is where the partially transferred file is stored if rsync
is interrupted with --partial (I'm not sure if --partial-dir implies
--partial, it probably does).

It does NOT mean where the temporary files are stored during transfer.


Paul

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html