rsync - Management of UIDs and GIDs when reserved on a Remote system

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

rsync - Management of UIDs and GIDs when reserved on a Remote system

Michael-3
Hi Wayne et al,

I've been using Linux for a little over a year now.

I'm making improvements to the English phrasing used by luckyBackup which
uses rsync. English is my first language and I'm doing this so that I can,
then, more accurately make translations for the Spanish language.

I would like to know how rsync would manage the situation where the UID and
GID(s), of a file being copied to a Remote system, have been reserved on the
Remote system. Does rsync search for and allocate the next available IDs?

If so, how would a Restore operation function? Can rsync create a log,
during a Backup, to record the newly allocated IDs, on the Remote system, so
that they can be mapped back to the Source system?

Thanks, Michael.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Simon Hobson-2
Michael <[hidden email]> wrote:

> I would like to know how rsync would manage the situation where the UID and
> GID(s), of a file being copied to a Remote system, have been reserved on the
> Remote system. Does rsync search for and allocate the next available IDs?

AFAIK, if you don't specify the "numeric-ids" option, then it will match by user/group name - ie it'll do a numeric to name conversion at the source, lookup the name at the destination, get the numeric value for the destination, and convert. So if user fred is UID 1000 on the source and 1010 on the destination, files owned by 1000 on the source will end up as UID 1010 on the destination.
If there sin't a matching name then AFAIK it'll do no conversion - you may end up with files owned by either a different user or no user at all. With the above, if user fred doesn't exist on the destination, then files will end up owned by UID 1010.


What is fun is copying files on a filesystem from a different system (mounting the virtual disks for a VM on the host) and copying them elsewhere while forgetting the numeric-ids option. You end up with some "interesting" file ownerships !

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Kevin Korb
In reply to this post by Michael-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rsync has 4 options for this...

1) Write all files as the owner and group that rsync is running as.
This is the default and the only possibility when not running as root
since only root can create files as a different UID.

2) Copy the user name and group name.  This is what you get when you
are running as root and use --owner --group or --archive.  This means
if the 2 systems have the same user name but with different UIDs rsync
will translate so that the files come out with the same user name and
group name on both systems if possible.  If the user name or group
name doesn't exist on the target system then the file/dir is created
with whatever UID or GID number regardless of whether it is in use by
another user or not.

3) --numeric-ids copies both as the number.  This is what you normally
want for backups.  This way you have the exact UID and GID stored on
the backups regardless of what those numbers mean to the backup
system.  Just remember to use --numeric-ids when you restore.  This is
especially important if you have to restore from a live environment
that has completely different UIDs and GIDs than your normal system.
That environment would have no idea how to translate names from option
#2 during a restore.

4) --fake-super.  This writes files like option #1 but also stores
what the owner and group name or numeric IDs should be in a user
xattr.  The same info about --numeric-ids applies.  Root access not
needed except during restore.

On 05/18/2015 02:52 PM, Michael wrote:

> Hi Wayne et al,
>
> I've been using Linux for a little over a year now.
>
> I'm making improvements to the English phrasing used by luckyBackup
> which uses rsync. English is my first language and I'm doing this
> so that I can, then, more accurately make translations for the
> Spanish language.
>
> I would like to know how rsync would manage the situation where the
> UID and GID(s), of a file being copied to a Remote system, have
> been reserved on the Remote system. Does rsync search for and
> allocate the next available IDs?
>
> If so, how would a Restore operation function? Can rsync create a
> log, during a Backup, to record the newly allocated IDs, on the
> Remote system, so that they can be mapped back to the Source
> system?
>
> Thanks, Michael.
>

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlVaQ30ACgkQVKC1jlbQAQc58wCcDoGUwb6kotWfPYIL0UTeFhyV
BWsAoNPE2ghqQX4xqE8wDlWilYPsShLe
=nZpc
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Michael-3
Kevin Korb <kmk <at> sanitarium.net> writes:
>
> 3) --numeric-ids copies both as the number.  This is what you normally
> want for backups.  This way you have the exact UID and GID stored on
> the backups regardless of what those numbers mean to the backup
> system.  Just remember to use --numeric-ids when you restore.  This is
> especially important if you have to restore from a live environment
> that has completely different UIDs and GIDs than your normal system.
> That environment would have no idea how to translate names from option
> #2 during a restore.

Hi Kevin,

Regarding the option --numeric-ids, how would rsync manage the situation
where the UID and GID(s), of a file being copied to a Remote system, have
been reserved on the Remote system. Does rsync search for and allocate the
next available IDs?

Thanks, Michael.



--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Kevin Korb
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

No, rsync would store the files with the numeric UID and GID as is on
the source without regard to the existence or non-existence of a
matching account.  This could mean that ls would show numbers in those
columns, or incorrect names in those columns, or (perhaps by
coincidence) show the correct names in those columns.  The important
thing is that if you backup and restore with --numeric-ids you will
get back what you started with regardless of what the backup system
thinks about the data.

On 05/18/2015 09:16 PM, Michael wrote:

> Kevin Korb <kmk <at> sanitarium.net> writes:
>>
>> 3) --numeric-ids copies both as the number.  This is what you
>> normally want for backups.  This way you have the exact UID and
>> GID stored on the backups regardless of what those numbers mean
>> to the backup system.  Just remember to use --numeric-ids when
>> you restore.  This is especially important if you have to restore
>> from a live environment that has completely different UIDs and
>> GIDs than your normal system. That environment would have no idea
>> how to translate names from option #2 during a restore.
>
> Hi Kevin,
>
> Regarding the option --numeric-ids, how would rsync manage the
> situation where the UID and GID(s), of a file being copied to a
> Remote system, have been reserved on the Remote system. Does rsync
> search for and allocate the next available IDs?
>
> Thanks, Michael.
>
>
>

- --
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
        Kevin Korb Phone:    (407) 252-6853
        Systems Administrator Internet:
        FutureQuest, Inc. [hidden email]  (work)
        Orlando, Florida [hidden email] (personal)
        Web page: http://www.sanitarium.net/
        PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlVakU4ACgkQVKC1jlbQAQe6AACcDo7ecpCTUgDJkxBlx05OJ57D
pf8AniaeelYUKcJDl+Zl/xjGcYkcmjOW
=nRGQ
-----END PGP SIGNATURE-----
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Simon Hobson-2
Kevin Korb <[hidden email]> wrote:

> No, rsync would store the files with the numeric UID and GID as is on
> the source without regard to the existence or non-existence of a
> matching account.  This could mean that ls would show numbers in those
> columns, or incorrect names in those columns, or (perhaps by
> coincidence) show the correct names in those columns.  The important
> thing is that if you backup and restore with --numeric-ids you will
> get back what you started with regardless of what the backup system
> thinks about the data.

Yes, that's what I do for my backups. Since the backup system doesn't have the same software (and hence S/W specific users) as the various systems I back up, it means a lot of files with numeric owner/group - and of course, some with seemingly strange ownership where the backup system has a different user/group for the same ID !
If you have to restore, then the system you restore to needs to have the same passwd and group files - which is not a problem when you are restoring files to the same machine, or restoring the entire machine (in which case the passwd and group files come back with the rest of the files during the restore.

The problem is if you need to restore to a different machine - in that case you will have problems. There isn't an easy way round that.

So you either use numeric IDs in which case the backup is the same as your source; or you don't, in which case you need to do some careful user-id and group-id management to avoid problems (ie ensure that the destination has a user and group name to match ever name used on every source) and rsync will match names.

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Michael-3
In reply to this post by Kevin Korb
Thanks for the clarification Kevin!!



--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Reply | Threaded
Open this post in threaded view
|

Re: rsync - Management of UIDs and GIDs when reserved on a Remote system

Michael-3
In reply to this post by Simon Hobson-2
Thanks for your input Simon!!

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html