s4: new classicupgrade and uids

[Sergey Urushkin]

Hi!
I've just made a test upgrade from s3 with the new uid/gid migration
feature and I have some questions:

1. Computer accounts have objectclass:posixAccount and uidNumber
attributes. What is it for? As far as I know unix computer accounts are
needed only for s3 dc, am I right? If so, than computer accounts should
be excluded somehow.

2. 'Administrator' hasn't got an uidNumber (while it had it in
openldap), so it makes me map it manually. Is it a bug or feature?

3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
need to add NIS domain to AD. And then add some attributes to
accounts/groups. Why not to add NIS domain (it's a simple ldif) to
config while provisioning (named as workgroup by default and also have
an provision/classicupgrade option to change the name) and then
additionally modify users like this:
changetype: modify
replace: msSFU30NisDomain
msSFU30NisDomain: $NISDOMAIN
-
replace: msSFU30Name
msSFU30Name: $USER

and groups like this:

changetype: modify
replace: msSFU30NisDomain
msSFU30NisDomain: $NISDOMAIN
-
replace: msSFU30Name
msSFU30Name: $GROUP

Thanks.

--
Best regards,
Sergey Urushkin

[steve-2]

On 06/21/2012 02:43 PM, Sergey Urushkin wrote:
> Hi!
> I've just made a test upgrade from s3 with the new uid/gid migration
> feature and I have some questions:
>
> 1. Computer accounts have objectclass:posixAccount and uidNumber
> attributes. What is it for? As far as I know unix computer accounts are
> needed only for s3 dc, that we can join Linux clients to those attributes so am I right? If so, than computer accounts should
> be excluded somehow.
Hi
We use those attributes so that we can join Linux clients to the domain
and users can login with them without having to use winbind.

>
> 2. 'Administrator' hasn't got an uidNumber (while it had it in
> openldap), so it makes me map it manually. Is it a bug or feature?
>
> 3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
> need to add NIS domain to AD. And then add some attributes to
> accounts/groups. Why not to add NIS domain (it's a simple ldif) to
> config while provisioning (named as workgroup by default and also have
> an provision/classicupgrade option to change the name) and then
> additionally modify users like this:
> changetype: modify
> replace: msSFU30NisDomain
> msSFU30NisDomain: $NISDOMAIN
> -
> replace: msSFU30Name
> msSFU30Name: $USER
>
> and groups like this:
>
> changetype: modify
> replace: msSFU30NisDomain
> msSFU30NisDomain: $NISDOMAIN
> -
> replace: msSFU30Name
> msSFU30Name: $GROUP
>
> Thanks.
>

We made a simple script to do this without changing the schema. We
manage most of the domain from the s4DC and rarely touch ADUC. The main
reason we don't use winbind is because our unixHomeDrirectory attributes
do not all point at the same directory.

The script is here. It has many similarities to your sfu example for the
most part.
http://dl.dropbox.com/u/45150875/s4bind.tar.gz
Cheers,
Steve

[Sergey Urushkin]

Hi.

21.06.2012 19:09, steve написал:

> On 06/21/2012 02:43 PM, Sergey Urushkin wrote:
>> Hi!
>> I've just made a test upgrade from s3 with the new uid/gid migration
>> feature and I have some questions:
>>
>> 1. Computer accounts have objectclass:posixAccount and uidNumber
>> attributes. What is it for? As far as I know unix computer accounts are
>> needed only for s3 dc, that we can join Linux clients to those
>> attributes so am I right? If so, than computer accounts should
>> be excluded somehow.
> Hi
> We use those attributes so that we can join Linux clients to the
> domain and users can login with them without having to use winbind.

But how can computer's uidnumber attribute help us with this... Joining
works without computer posix accounts. ldapd, sss, nss_ldap doesn't need
computer accounts. We can also export computer creds into keytab without
posixaccount objectclass. Uids are only needed for owning files and nss
info, and I don't see any reason to support setting computer as an owner
and logging in as a computer (at least by default).
>>
>> 2. 'Administrator' hasn't got an uidNumber (while it had it in
>> openldap), so it makes me map it manually. Is it a bug or feature?
Here, seems that "Guest" account is also affected.

>>
>> 3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
>> need to add NIS domain to AD. And then add some attributes to
>> accounts/groups. Why not to add NIS domain (it's a simple ldif) to
>> config while provisioning (named as workgroup by default and also have
>> an provision/classicupgrade option to change the name) and then
>> additionally modify users like this:
>> changetype: modify
>> replace: msSFU30NisDomain
>> msSFU30NisDomain: $NISDOMAIN
>> -
>> replace: msSFU30Name
>> msSFU30Name: $USER
>>
>> and groups like this:
>>
>> changetype: modify
>> replace: msSFU30NisDomain
>> msSFU30NisDomain: $NISDOMAIN
>> -
>> replace: msSFU30Name
>> msSFU30Name: $GROUP
>>
>> Thanks.
>>
> We made a simple script to do this without changing the schema. We
> manage most of the domain from the s4DC and rarely touch ADUC.

But we still don't have something like 'samba-tool user unix ...' and
'samba-tool domain nis ...' and all managing of unix attribs is made
with external tools (you suggested, I have one too).
Schema modification I made works and replicates well (I've been using it
for tests for more then half of a year) , so I don't see actual reason
why not just to implement it. In summary to begin we need one ldif with
2 params - basedn and nis domain, one option to provision/classicupgrade
(--nis-domain), and user/group provisioning modifications showed above.
I would be glad to make it but my python is not good, anyway I'm ready
to help with testing.
> The main reason we don't use winbind is because our unixHomeDrirectory
> attributes do not all point at the same directory.
>
It's a problem for s4 winbind, but s3 winbind works nicely with rfc2307
schema and respects attribs like uid, unixHomeDirectory.
> The script is here. It has many similarities to your sfu example for
> the most part.
> http://dl.dropbox.com/u/45150875/s4bind.tar.gz
>
Thanks for the script.

Also, another question appeared:

4. gidNumber for users is not set too. Until it's set we can not use s3
winbind rfc2307 mode with s4 dc. Maybe it's also about loginshell,
unixHomeDirectory...

Thanks.

--
Best regards,
Sergey Urushkin

[steve-2]

On 06/22/2012 09:09 AM, Sergey Urushkin wrote:

> Hi.
>
> 21.06.2012 19:09, steve написал:
>> On 06/21/2012 02:43 PM, Sergey Urushkin wrote:
>>> Hi!
>>> I've just made a test upgrade from s3 with the new uid/gid migration
>>> feature and I have some questions:
>>>
>>> 1. Computer accounts have objectclass:posixAccount and uidNumber
>>> attributes. What is it for? As far as I know unix computer accounts are
>>> needed only for s3 dc, that we can join Linux clients to those
>>> attributes so am I right? If so, than computer accounts should
>>> be excluded somehow.
>> Hi
>> We use those attributes so that we can join Linux clients to the
>> domain and users can login with them without having to use winbind.
> But how can computer's uidnumber attribute help us with this... Joining
> works without computer posix accounts. ldapd, sss, nss_ldap doesn't need
> computer accounts.

We need computer accounts for Kerberos to work. It is the user who logs
in, but only if the computer account is present. To be able to login to
Linux, the rfc2307 attributes need to be present. We currently store:
uidNumber
gidNumber
loginShell
uid (map   passwd    uid    samAccountName)
unixHomeDirectory (map    passwd    homeDirectory    unixHomeDirectory)
gid
(nss mappings in brackets) in the directory.

A Linux user logs in and is placed in his home directory. Files he
creates there have the correct attributes. It is just as if he has
logged in as in the old days under NIS.

>   We can also export computer creds into keytab without
> posixaccount objectclass. Uids are only needed for owning files and nss
> info, and I don't see any reason to support setting computer as an owner
> and logging in as a computer (at least by default).
The computer does not log in. The computer account is stored in the
keytab (we use net ads join on the Linux client and then net ads keytab
create) on the client and in the directory. We need the keytab on the
client for kerberized NFS. The user logs in and the user owns files.
>>> 2. 'Administrator' hasn't got an uidNumber (while it had it in
>>> openldap), so it makes me map it manually. Is it a bug or feature?
> Here, seems that "Guest" account is also affected.
You can add rfc2307 attributes to Administrator if you like as you but
any kerberos stuff Administrator has to do (like kerberized nss ldap
binds) can be done via kinit or setting up e.g. a keytab for a user who
can access nss. Also we have root on Linux who can do just about
everything on the directory without kerberos.

>>> 3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
>>> need to add NIS domain to AD. And then add some attributes to
>>> accounts/groups. Why not to add NIS domain (it's a simple ldif) to
>>> config while provisioning (named as workgroup by default and also have
>>> an provision/classicupgrade option to change the name) and then
>>> additionally modify users like this:
>>> changetype: modify
>>> replace: msSFU30NisDomain
>>> msSFU30NisDomain: $NISDOMAIN
>>> -
>>> replace: msSFU30Name
>>> msSFU30Name: $USER
>>>
>>> and groups like this:
>>>
>>> changetype: modify
>>> replace: msSFU30NisDomain
>>> msSFU30NisDomain: $NISDOMAIN
>>> -
>>> replace: msSFU30Name
>>> msSFU30Name: $GROUP
>>>
>>> Thanks.
>>>
>> We made a simple script to do this without changing the schema. We
>> manage most of the domain from the s4DC and rarely touch ADUC.
> But we still don't have something like 'samba-tool user unix ...' and
> 'samba-tool domain nis ...' and all managing of unix attribs is made
> with external tools (you suggested, I have one too).

No, we don't have that in samba-tool. That is what our scripts do. They
give you the equivalent of e.g.
samba-tool user add steve --unix
or
samba-tool group --make-unix

> Schema modification I made works and replicates well (I've been using it
> for tests for more then half of a year) , so I don't see actual reason
> why not just to implement it. In summary to begin we need one ldif with
> 2 params - basedn and nis domain, one option to provision/classicupgrade
> (--nis-domain), and user/group provisioning modifications showed above.
> I would be glad to make it but my python is not good, anyway I'm ready
> to help with testing.
>> The main reason we don't use winbind is because our unixHomeDrirectory
>> attributes do not all point at the same directory.
>>
> It's a problem for s4 winbind, but s3 winbind works nicely with rfc2307
> schema and respects attribs like uid, unixHomeDirectory.
>> The script is here. It has many similarities to your sfu example for
>> the most part.
>> http://dl.dropbox.com/u/45150875/s4bind.tar.gz
>>
> Thanks for the script.
>
> Also, another question appeared:
>
> 4. gidNumber for users is not set too. Until it's set we can not use s3
> winbind rfc2307 mode with s4 dc. Maybe it's also about loginshell,
> unixHomeDirectory...
>
> Thanks.
>

We set:
  objectClass posixGroup
  gidNumber
for groups created with samba-tool so that they appear correctly via nss
(getent and stuff).

The gidNumber for the user is also stored along with his other rfc2307
stuff in the directory.

e.g. We made Domain Users into a posix group.

Cheers,
Steve

[Andrew Bartlett]

On Thu, 2012-06-21 at 16:43 +0400, Sergey Urushkin wrote:
> Hi!
> I've just made a test upgrade from s3 with the new uid/gid migration
> feature and I have some questions:
>
> 1. Computer accounts have objectclass:posixAccount and uidNumber
> attributes. What is it for? As far as I know unix computer accounts are
> needed only for s3 dc, am I right? If so, than computer accounts should
> be excluded somehow.

Computers can log in and own files, and so we need to preserve the
uidNumber that has been assigned, to preserve this file ownership.

> 2. 'Administrator' hasn't got an uidNumber (while it had it in
> openldap), so it makes me map it manually. Is it a bug or feature?

Simply a bug.

> 3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
> need to add NIS domain to AD. And then add some attributes to
> accounts/groups. Why not to add NIS domain (it's a simple ldif) to
> config while provisioning (named as workgroup by default and also have
> an provision/classicupgrade option to change the name) and then
> additionally modify users like this:
> changetype: modify
> replace: msSFU30NisDomain
> msSFU30NisDomain: $NISDOMAIN
> -
> replace: msSFU30Name
> msSFU30Name: $USER
>
> and groups like this:
>
> changetype: modify
> replace: msSFU30NisDomain
> msSFU30NisDomain: $NISDOMAIN
> -
> replace: msSFU30Name
> msSFU30Name: $GROUP

I thought that the whole point of the new rfc2307 support was to avoid
needing to set these SFU attributes?

Andrew Bartlett

--
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

[Sergey Urushkin]

HI.

22.06.2012 12:11, Andrew Bartlett написал:

> On Thu, 2012-06-21 at 16:43 +0400, Sergey Urushkin wrote:
>> Hi!
>> I've just made a test upgrade from s3 with the new uid/gid migration
>> feature and I have some questions:
>>
>> 1. Computer accounts have objectclass:posixAccount and uidNumber
>> attributes. What is it for? As far as I know unix computer accounts are
>> needed only for s3 dc, am I right? If so, than computer accounts should
>> be excluded somehow.
> Computers can log in and own files, and so we need to preserve the
> uidNumber that has been assigned, to preserve this file ownership.

I know they can with these settings, I'm just wondering where such
ownership could be used with nix systems?
>> 2. 'Administrator' hasn't got an uidNumber (while it had it in
>> openldap), so it makes me map it manually. Is it a bug or feature?
> Simply a bug.
As I wrote in another branch - 'guest' is also affected. Should I write
report, or it'll be fixed in place soon?

>> 3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
>> need to add NIS domain to AD. And then add some attributes to
>> accounts/groups. Why not to add NIS domain (it's a simple ldif) to
>> config while provisioning (named as workgroup by default and also have
>> an provision/classicupgrade option to change the name) and then
>> additionally modify users like this:
>> changetype: modify
>> replace: msSFU30NisDomain
>> msSFU30NisDomain: $NISDOMAIN
>> -
>> replace: msSFU30Name
>> msSFU30Name: $USER
>>
>> and groups like this:
>>
>> changetype: modify
>> replace: msSFU30NisDomain
>> msSFU30NisDomain: $NISDOMAIN
>> -
>> replace: msSFU30Name
>> msSFU30Name: $GROUP
> I thought that the whole point of the new rfc2307 support was to avoid
> needing to set these SFU attributes?

Maybe I missed discussion about it, I'm just trying to say that support
for these attributes (as a nis domain schema modification) are needed to
manage unix account via dsa.msc.
'msSFU30NisDomain' (and that mean nis schema too) is needed to be able
to change unix atrributes.
'msSFU30Name' seems to be optional, it is filled with account name by
dsa.msc automatically after pressing apply button on unix tab, and since
it can't be changed via dsa.msc (e.g. after renaming account), it can be
leaved unmanaged by samba too, so resulting minimal ldif for
users/groups is:

changetype: modify
replace: msSFU30NisDomain
msSFU30NisDomain: $NISDOMAIN

I don't understand what disadvantages we'll get after adding NIS support to s4?

Also, another question appeared from another branch of this threat:
4. gidNumber for users is not set too after upgrade. Until it's set we
can not use s3 winbind rfc2307 mode with s4 dc. Maybe it's also about
loginshell, unixHomeDirectory...

# samba --version
Version 4.0.0beta3-GIT-9089d48

Thanks.

--
Best regards,
Sergey Urushkin

[Andrew Bartlett]

On Fri, 2012-06-22 at 13:42 +0400, Sergey Urushkin wrote:

> HI.
>
> 22.06.2012 12:11, Andrew Bartlett написал:
> > On Thu, 2012-06-21 at 16:43 +0400, Sergey Urushkin wrote:
> >> Hi!
> >> I've just made a test upgrade from s3 with the new uid/gid migration
> >> feature and I have some questions:
> >>
> >> 1. Computer accounts have objectclass:posixAccount and uidNumber
> >> attributes. What is it for? As far as I know unix computer accounts are
> >> needed only for s3 dc, am I right? If so, than computer accounts should
> >> be excluded somehow.
> > Computers can log in and own files, and so we need to preserve the
> > uidNumber that has been assigned, to preserve this file ownership.
> I know they can with these settings, I'm just wondering where such
> ownership could be used with nix systems?
> >> 2. 'Administrator' hasn't got an uidNumber (while it had it in
> >> openldap), so it makes me map it manually. Is it a bug or feature?
> > Simply a bug.
> As I wrote in another branch - 'guest' is also affected. Should I write
> report, or it'll be fixed in place soon?
> >> 3. To have an ability to manage user's uid, gid, etc. through dsa.msc we
> >> need to add NIS domain to AD. And then add some attributes to
> >> accounts/groups. Why not to add NIS domain (it's a simple ldif) to
> >> config while provisioning (named as workgroup by default and also have
> >> an provision/classicupgrade option to change the name) and then
> >> additionally modify users like this:
> >> changetype: modify
> >> replace: msSFU30NisDomain
> >> msSFU30NisDomain: $NISDOMAIN
> >> -
> >> replace: msSFU30Name
> >> msSFU30Name: $USER
> >>
> >> and groups like this:
> >>
> >> changetype: modify
> >> replace: msSFU30NisDomain
> >> msSFU30NisDomain: $NISDOMAIN
> >> -
> >> replace: msSFU30Name
> >> msSFU30Name: $GROUP
> > I thought that the whole point of the new rfc2307 support was to avoid
> > needing to set these SFU attributes?
> Maybe I missed discussion about it, I'm just trying to say that support
> for these attributes (as a nis domain schema modification) are needed to
> manage unix account via dsa.msc.
> 'msSFU30NisDomain' (and that mean nis schema too) is needed to be able
> to change unix atrributes.
> 'msSFU30Name' seems to be optional, it is filled with account name by
> dsa.msc automatically after pressing apply button on unix tab, and since
> it can't be changed via dsa.msc (e.g. after renaming account), it can be
> leaved unmanaged by samba too, so resulting minimal ldif for
> users/groups is:
>
> changetype: modify
> replace: msSFU30NisDomain
> msSFU30NisDomain: $NISDOMAIN
>
> I don't understand what disadvantages we'll get after adding NIS support to s4?

I can't think of any.  

> Also, another question appeared from another branch of this threat:
> 4. gidNumber for users is not set too after upgrade. Until it's set we
> can not use s3 winbind rfc2307 mode with s4 dc. Maybe it's also about
> loginshell, unixHomeDirectory...
>
> # samba --version
> Version 4.0.0beta3-GIT-9089d48

The best approach would be for you to prepare a series of patches to
improve the python script that handles this:
source4/scripting/python/samba/upgrade.py

I want this tool to be as useful as possible to administrators, so
rather than me guessing, you know what you need, and can test that it
works just right.  I can then review and hopefully include them!

On primaryGroupID/gidNumber, we should carefully look at how that
interatiction works.  On loginshell etc, the main issue with including
them (which makes sense) is that at the moment, the s4 winbind won't
honour them (the clients can of course) .  I know this is a pain point,
but it's also harder to fix (patches for this more difficult issue also
welcome).

Thanks,

Andrew Bartlett

--
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

[steve-2]

On 22/06/12 12:04, Andrew Bartlett wrote:
> On Fri, 2012-06-22 at 13:42 +0400, Sergey Urushkin wrote:
>> HI.
>>
>> 22.06.2012 12:11, Andrew Bartlett написал:
>>> On Thu, 2012-06-21 at 16:43 +0400, Sergey Urushkin wrote:

>
> On primaryGroupID/gidNumber, we should carefully look at how that
> interatiction works.

We take the primaryGrouopID to be the group that that user creates files
as when there is no acl set. e.g. in his home directory. We add the
posixGroup class to the group and give it a gidNumber arrtibute which is
related to the primaryGroupID. e.g. Domain Users we set to gidNumber
20513 which relates to its primaryGroupID of 513. The low numbers of the
primaryGroupID can collide with local Linux groups so adding say, 20000
to it takes care of that whilst still making it readable (as opposed to
the 3000000+ values for both uidNumber and gidumber from idmap)

   On loginshell etc, the main issue with including
> them (which makes sense) is that at the moment, the s4 winbind won't
> honour them (the clients can of course) .  I know this is a pain point,
> but it's also harder to fix (patches for this more difficult issue also
> welcome).

rfc2307 works well in the directory apart from uidNumber and gidNumber
which still have to come from idmap even though we have them set in the
directory. We have to set them in the directory and then change the xid
values in idmap to correspond, otherwise s3fs sets file ownership
according to idmap, not the values that are stored in the directory.

We are discussing the
idmap config DOMAIN : backend = rfc2307
on another thread. If that works it would be a big step forward.

Cheers and thanks for your time,
Steve

[Sergey Urushkin]

22.06.2012 14:04, Andrew Bartlett пишет:

> On Fri, 2012-06-22 at 13:42 +0400, Sergey Urushkin wrote:
>
>
> 22.06.2012 12:11, Andrew Bartlett написал:
>>> On Thu, 2012-06-21 at 16:43 +0400, Sergey Urushkin wrote:
>>>> 2. 'Administrator' hasn't got an uidNumber (while it had it in
>>>> openldap), so it makes me map it manually. Is it a bug or feature?
>>> Simply a bug.
>> As I wrote in another branch - 'guest' is also affected. Should I write
>> report, or it'll be fixed in place soon?

Hi, here is the patch that fixes this issue:

--- upgrade.py.orig    2012-07-23 09:25:27.000000000 +0400
+++ upgrade.py    2012-07-24 11:40:58.107080685 +0400
@@ -675,8 +675,9 @@
             continue
         username = entry['account_name']
         if entry['rid'] < 1000:
-            logger.info("  Skipping wellknown rid=%d (for
username=%s)", entry['rid'], username)
-            continue
+            if username.lower() != 'administrator' and username.lower()
!= 'guest' and username.lower() != 'krbtgt':
+                logger.info("  Skipping wellknown rid=%d (for
username=%s)", entry['rid'], username.lower())
+                continue
         if entry['rid'] >= next_rid:
             next_rid = entry['rid'] + 1
 
@@ -866,7 +867,11 @@
             else:
                 logger.warn('User root has been kept in the directory,
it should be removed in favour of the Administrator user')
 
-        s4_passdb.add_sam_account(userdata[username])
+        if username.lower() == 'administrator' or username.lower() ==
'guest' or username.lower() == 'krbtgt':
+            logger.warn("  Skipping wellknown provisioned user '%s'",
username)
+        else:
+            s4_passdb.add_sam_account(userdata[username])
+
         if username in uids:
             add_ad_posix_idmap_entry(result.samdb,
userdata[username].user_sid, uids[username], "ID_TYPE_UID", logger)
             if (username in homes) and (homes[username] != None) and \


Also, I have to say that "if entry['rid'] < 1000:" check gives an error
at the "adding users to groups" stage (nonexisting user). Ways to solve it:
 1. Stop provision with error if such accounts exist (think it's the best)
 2. Add some workaround to the function that lists members
 3. Remove this check.

Thanks.

--
Best regards,
Sergey Urushkin

[Sergey Urushkin]

Hi.
Some time ago I sent the patch to the list, but didn't get answer. For
better readability it's attached again.
The problem with it now is that it may set administrator uid to non-zero
value (what will break GP editing, until appropriate posix acls is set.
As a workaround - chown -R administrator path/to/sysvol). It may need
additional warning message.
Also, please, take a look at the end of the message, there is a problem
which will likely appear sooner or later with some installation.
What's your thoughts about all these?
Thanks.


-------- Исходное сообщение --------
Тема: Re: s4: new classicupgrade and uids
Дата: Tue, 24 Jul 2012 11:57:31 +0400
От: Sergey Urushkin <[hidden email]>
Кому: Andrew Bartlett <[hidden email]>
Копия: Samba technical <[hidden email]>



22.06.2012 14:04, Andrew Bartlett пишет:

> On Fri, 2012-06-22 at 13:42 +0400, Sergey Urushkin wrote:
>
>
> 22.06.2012 12:11, Andrew Bartlett написал:
>>> On Thu, 2012-06-21 at 16:43 +0400, Sergey Urushkin wrote:
>>>> 2. 'Administrator' hasn't got an uidNumber (while it had it in
>>>> openldap), so it makes me map it manually. Is it a bug or feature?
>>> Simply a bug.
>> As I wrote in another branch - 'guest' is also affected. Should I write
>> report, or it'll be fixed in place soon?

Hi, here is the patch that fixes this issue:

--- upgrade.py.orig    2012-07-23 09:25:27.000000000 +0400
+++ upgrade.py    2012-07-24 11:40:58.107080685 +0400
@@ -675,8 +675,9 @@
             continue
         username = entry['account_name']
         if entry['rid'] < 1000:
-            logger.info("  Skipping wellknown rid=%d (for
username=%s)", entry['rid'], username)
-            continue
+            if username.lower() != 'administrator' and username.lower()
!= 'guest' and username.lower() != 'krbtgt':
+                logger.info("  Skipping wellknown rid=%d (for
username=%s)", entry['rid'], username.lower())
+                continue
         if entry['rid'] >= next_rid:
             next_rid = entry['rid'] + 1
 
@@ -866,7 +867,11 @@
             else:
                 logger.warn('User root has been kept in the directory,
it should be removed in favour of the Administrator user')
 
-        s4_passdb.add_sam_account(userdata[username])
+        if username.lower() == 'administrator' or username.lower() ==
'guest' or username.lower() == 'krbtgt':
+            logger.warn("  Skipping wellknown provisioned user '%s'",
username)
+        else:
+            s4_passdb.add_sam_account(userdata[username])
+
         if username in uids:
             add_ad_posix_idmap_entry(result.samdb,
userdata[username].user_sid, uids[username], "ID_TYPE_UID", logger)
             if (username in homes) and (homes[username] != None) and \


Also, I have to say that "if entry['rid'] < 1000:" check gives an error
at the "adding users to groups" stage (nonexisting user). Ways to solve it:
 1. Stop provision with error if such accounts exist (think it's the best)
 2. Add some workaround to the function that lists members
 3. Remove this check.

Thanks.

--
Best regards,
Sergey Urushkin

[Andrew Bartlett]

On Wed, 2012-08-08 at 11:01 +0400, Sergey Urushkin wrote:
> Hi.
> Some time ago I sent the patch to the list, but didn't get answer. For
> better readability it's attached again.
> The problem with it now is that it may set administrator uid to non-zero
> value (what will break GP editing, until appropriate posix acls is set.
> As a workaround - chown -R administrator path/to/sysvol). It may need
> additional warning message.

My thoughts are that I don't like the upgrade handling any of the well
known users, except perhaps to assign a UID.  Even then, things like
administrator UID should be connected via the provision() stage.

> Also, I have to say that "if entry['rid'] < 1000:" check gives an error
> at the "adding users to groups" stage (nonexisting user). Ways to solve it:
>  1. Stop provision with error if such accounts exist (think it's the best)
>  2. Add some workaround to the function that lists members
>  3. Remove this check.

I'm not entirely sure what you mean here.  I guess we do need to upgrade
groups of any well known users we find?

Andrew Bartlett

--
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

[Sergey Urushkin]

08.08.2012 12:55, Andrew Bartlett пишет:

> On Wed, 2012-08-08 at 11:01 +0400, Sergey Urushkin wrote:
>> Hi.
>> Some time ago I sent the patch to the list, but didn't get answer. For
>> better readability it's attached again.
>> The problem with it now is that it may set administrator uid to non-zero
>> value (what will break GP editing, until appropriate posix acls is set.
>> As a workaround - chown -R administrator path/to/sysvol). It may need
>> additional warning message.
> My thoughts are that I don't like the upgrade handling any of the well
> known users, except perhaps to assign a UID.  Even then, things like
> administrator UID should be connected via the provision() stage.

Then, may be we should add some default uid/gid attributes to wellknown
users/groups on the provision stage (if --use-rfc2307 is used add them
to the directory too). That would also be a step to provision default
posix acls for sysvol. Here is the list which I feel needed and
suggested values.

name : sid : uid/gid
---------------------------------
Administrators    :    S-1-5-32-544    :    300544                -
should to be synced on all dcs (it's a part of default ms's sysvol
permissions)
Server Operators    :    S-1-5-32-549    :    300549            - the same
Authenticated Users    :    S-1-5-11    :    300011              - the same
Local System    :    S-1-5-18    :    300018/300018             - the same

Administrator    :    S-1-5-21domain-500    :    0/0
Guest    :    S-1-5-21domain-501    :    nobody's uid/nobody's gid
Anonymous    :    S-1-5-7    :    nobody's gid

What do you think?

>> Also, I have to say that "if entry['rid'] < 1000:" check gives an error
>> at the "adding users to groups" stage (nonexisting user). Ways to solve it:
>>  1. Stop provision with error if such accounts exist (think it's the best)
>>  2. Add some workaround to the function that lists members
>>  3. Remove this check.
> I'm not entirely sure what you mean here.  I guess we do need to upgrade
> groups of any well known users we find?
>
If group contains user with rid < 1000 (skipped), then
add_users_to_group ends with error (because this user hasn't been
actually imported), and upgrading fails.
So, if there are 1000 users, we have to wait for a really long time
(while all users are being imported) to just see an error at the end. I
think we should give an error immediately, or test somehow for user
existence in s4 db before trying to add it to some group.  

--
Best regards,
Sergey Urushkin

[Andrew Bartlett]

On Wed, 2012-08-08 at 15:21 +0400, Sergey Urushkin wrote:

>
> 08.08.2012 12:55, Andrew Bartlett пишет:
>
> > On Wed, 2012-08-08 at 11:01 +0400, Sergey Urushkin wrote:
> > > Hi.
> > > Some time ago I sent the patch to the list, but didn't get answer. For
> > > better readability it's attached again.
> > > The problem with it now is that it may set administrator uid to non-zero
> > > value (what will break GP editing, until appropriate posix acls is set.
> > > As a workaround - chown -R administrator path/to/sysvol). It may need
> > > additional warning message.
> > My thoughts are that I don't like the upgrade handling any of the well
> > known users, except perhaps to assign a UID.  Even then, things like
> > administrator UID should be connected via the provision() stage.
> Then, may be we should add some default uid/gid attributes to
> wellknown users/groups on the provision stage (if --use-rfc2307 is
> used add them to the directory too). That would also be a step to
> provision default posix acls for sysvol. Here is the list which I feel
> needed and suggested values.
>
> name : sid : uid/gid
> ---------------------------------
> Administrators    :    S-1-5-32-544    :    300544                -
> should to be synced on all dcs (it's a part of default ms's sysvol
> permissions)
> Server Operators    :    S-1-5-32-549    :    300549            - the
> same
> Authenticated Users    :    S-1-5-11    :    300011              - the
> same
> Local System    :    S-1-5-18    :    300018/300018             - the
> same
>
> Administrator    :    S-1-5-21domain-500    :    0/0
> Guest    :    S-1-5-21domain-501    :    nobody's uid/nobody's gid
> Anonymous    :    S-1-5-7    :    nobody's gid
>
> What do you think?

I don't mind assigning a UID in the directory based on the imported
values to well known users, but I'm just saying we need to be careful
about actually 'importing' the user.  

> > > Also, I have to say that "if entry['rid'] < 1000:" check gives an error
> > > at the "adding users to groups" stage (nonexisting user). Ways to solve it:
> > >  1. Stop provision with error if such accounts exist (think it's the best)
> > >  2. Add some workaround to the function that lists members
> > >  3. Remove this check.
> > I'm not entirely sure what you mean here.  I guess we do need to upgrade
> > groups of any well known users we find?
> >
> If group contains user with rid < 1000 (skipped), then
> add_users_to_group ends with error (because this user hasn't been
> actually imported), and upgrading fails.
> So, if there are 1000 users, we have to wait for a really long time
> (while all users are being imported) to just see an error at the end.
> I think we should give an error immediately, or test somehow for user
> existence in s4 db before trying to add it to some group.  

What users are you importing with a sid below 1000 that Samba4 doesn't
already define?

Andrew Bartlett

--
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

[Sergey Urushkin]

08.08.2012 15:29, Andrew Bartlett пишет:

> On Wed, 2012-08-08 at 15:21 +0400, Sergey Urushkin wrote:
>> 08.08.2012 12:55, Andrew Bartlett пишет:
>>
>>> On Wed, 2012-08-08 at 11:01 +0400, Sergey Urushkin wrote:
>>>> Hi.
>>>> Some time ago I sent the patch to the list, but didn't get answer. For
>>>> better readability it's attached again.
>>>> The problem with it now is that it may set administrator uid to non-zero
>>>> value (what will break GP editing, until appropriate posix acls is set.
>>>> As a workaround - chown -R administrator path/to/sysvol). It may need
>>>> additional warning message.
>>> My thoughts are that I don't like the upgrade handling any of the well
>>> known users, except perhaps to assign a UID.  Even then, things like
>>> administrator UID should be connected via the provision() stage.
>> Then, may be we should add some default uid/gid attributes to
>> wellknown users/groups on the provision stage (if --use-rfc2307 is
>> used add them to the directory too). That would also be a step to
>> provision default posix acls for sysvol. Here is the list which I feel
>> needed and suggested values.
>>
>> name : sid : uid/gid
>> ---------------------------------
>> Administrators    :    S-1-5-32-544    :    300544                -
>> should to be synced on all dcs (it's a part of default ms's sysvol
>> permissions)
>> Server Operators    :    S-1-5-32-549    :    300549            - the
>> same
>> Authenticated Users    :    S-1-5-11    :    300011              - the
>> same
>> Local System    :    S-1-5-18    :    300018/300018             - the
>> same
>>
>> Administrator    :    S-1-5-21domain-500    :    0/0
>> Guest    :    S-1-5-21domain-501    :    nobody's uid/nobody's gid
>> Anonymous    :    S-1-5-7    :    nobody's gid
>>
>> What do you think?
> I don't mind assigning a UID in the directory based on the imported
> values to well known users, but I'm just saying we need to be careful
> about actually 'importing' the user.

If you are suggesting importing only uid and gid for wellknown accounts,
then I'll change the patch such way.
But could you explain why we need to import it via provision()? For
example, "domain admins" is a wellknown account too, but we don't import
it via provision().
Another question is what I wrote previously: for >1 DC's domains we need
to write at least several account's uid/gid to the ldap while
provisioning, to have acls synced. So, some uid/gid initialization is
needed for provision anyway. And in the case of classicupgrade
s3-existed account uids would be overwritten. Doesn't it look reasonable?

>>>> Also, I have to say that "if entry['rid'] < 1000:" check gives an error
>>>> at the "adding users to groups" stage (nonexisting user). Ways to solve it:
>>>>  1. Stop provision with error if such accounts exist (think it's the best)
>>>>  2. Add some workaround to the function that lists members
>>>>  3. Remove this check.
>>> I'm not entirely sure what you mean here.  I guess we do need to upgrade
>>> groups of any well known users we find?
>>>
>> If group contains user with rid < 1000 (skipped), then
>> add_users_to_group ends with error (because this user hasn't been
>> actually imported), and upgrading fails.
>> So, if there are 1000 users, we have to wait for a really long time
>> (while all users are being imported) to just see an error at the end.
>> I think we should give an error immediately, or test somehow for user
>> existence in s4 db before trying to add it to some group.  
> What users are you importing with a sid below 1000 that Samba4 doesn't
> already define?
>

I haven't yet met such users, I've got this issue while testing
suggested patch (I changed the sid manually for one of the users). I'm
just trying to say if someone's s3db contains such user (I guess some
do), we'll need to do something with it, the more so because we already
have this check.

--
Best regards,
Sergey Urushkin

[Andrew Bartlett]

On Thu, 2012-08-09 at 10:25 +0400, Sergey Urushkin wrote:

> 08.08.2012 15:29, Andrew Bartlett пишет:
> > On Wed, 2012-08-08 at 15:21 +0400, Sergey Urushkin wrote:
> >> 08.08.2012 12:55, Andrew Bartlett пишет:
> >>
> >>> On Wed, 2012-08-08 at 11:01 +0400, Sergey Urushkin wrote:
> >>>> Hi.
> >>>> Some time ago I sent the patch to the list, but didn't get answer. For
> >>>> better readability it's attached again.
> >>>> The problem with it now is that it may set administrator uid to non-zero
> >>>> value (what will break GP editing, until appropriate posix acls is set.
> >>>> As a workaround - chown -R administrator path/to/sysvol). It may need
> >>>> additional warning message.
> >>> My thoughts are that I don't like the upgrade handling any of the well
> >>> known users, except perhaps to assign a UID.  Even then, things like
> >>> administrator UID should be connected via the provision() stage.
> >> Then, may be we should add some default uid/gid attributes to
> >> wellknown users/groups on the provision stage (if --use-rfc2307 is
> >> used add them to the directory too). That would also be a step to
> >> provision default posix acls for sysvol. Here is the list which I feel
> >> needed and suggested values.
> >>
> >> name : sid : uid/gid
> >> ---------------------------------
> >> Administrators    :    S-1-5-32-544    :    300544                -
> >> should to be synced on all dcs (it's a part of default ms's sysvol
> >> permissions)
> >> Server Operators    :    S-1-5-32-549    :    300549            - the
> >> same
> >> Authenticated Users    :    S-1-5-11    :    300011              - the
> >> same
> >> Local System    :    S-1-5-18    :    300018/300018             - the
> >> same
> >>
> >> Administrator    :    S-1-5-21domain-500    :    0/0
> >> Guest    :    S-1-5-21domain-501    :    nobody's uid/nobody's gid
> >> Anonymous    :    S-1-5-7    :    nobody's gid
> >>
> >> What do you think?
> > I don't mind assigning a UID in the directory based on the imported
> > values to well known users, but I'm just saying we need to be careful
> > about actually 'importing' the user.
> If you are suggesting importing only uid and gid for wellknown accounts,
> then I'll change the patch such way.
> But could you explain why we need to import it via provision()? For
> example, "domain admins" is a wellknown account too, but we don't import
> it via provision().

We do create "domain admins" during provision.

> Another question is what I wrote previously: for >1 DC's domains we need
> to write at least several account's uid/gid to the ldap while
> provisioning, to have acls synced. So, some uid/gid initialization is
> needed for provision anyway. And in the case of classicupgrade
> s3-existed account uids would be overwritten. Doesn't it look reasonable?

It is reasonable to set these UID values to the s3 accounts, but I don't
want to import the s3 accounts, just modify the provision-provided
accounts.  We also need to take care that some things are done during
the provision, that uses the id map, so we have to set

> >>>> Also, I have to say that "if entry['rid'] < 1000:" check gives an error
> >>>> at the "adding users to groups" stage (nonexisting user). Ways to solve it:
> >>>>  1. Stop provision with error if such accounts exist (think it's the best)
> >>>>  2. Add some workaround to the function that lists members
> >>>>  3. Remove this check.
> >>> I'm not entirely sure what you mean here.  I guess we do need to upgrade
> >>> groups of any well known users we find?
> >>>
> >> If group contains user with rid < 1000 (skipped), then
> >> add_users_to_group ends with error (because this user hasn't been
> >> actually imported), and upgrading fails.
> >> So, if there are 1000 users, we have to wait for a really long time
> >> (while all users are being imported) to just see an error at the end.
> >> I think we should give an error immediately, or test somehow for user
> >> existence in s4 db before trying to add it to some group.  
> > What users are you importing with a sid below 1000 that Samba4 doesn't
> > already define?
> >
> I haven't yet met such users, I've got this issue while testing
> suggested patch (I changed the sid manually for one of the users). I'm
> just trying to say if someone's s3db contains such user (I guess some
> do), we'll need to do something with it, the more so because we already
> have this check.

I guess we should error out.

Andrew Bartlett

--
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

[Sergey Urushkin]

09.08.2012 11:17, Andrew Bartlett пишет:

> On Thu, 2012-08-09 at 10:25 +0400, Sergey Urushkin wrote:
>> 08.08.2012 15:29, Andrew Bartlett пишет:
>>> On Wed, 2012-08-08 at 15:21 +0400, Sergey Urushkin wrote:
>>>> 08.08.2012 12:55, Andrew Bartlett пишет:
>>>>
>>>>> On Wed, 2012-08-08 at 11:01 +0400, Sergey Urushkin wrote:
>>>>>> Hi.
>>>>>> Some time ago I sent the patch to the list, but didn't get answer. For
>>>>>> better readability it's attached again.
>>>>>> The problem with it now is that it may set administrator uid to non-zero
>>>>>> value (what will break GP editing, until appropriate posix acls is set.
>>>>>> As a workaround - chown -R administrator path/to/sysvol). It may need
>>>>>> additional warning message.
>>>>> My thoughts are that I don't like the upgrade handling any of the well
>>>>> known users, except perhaps to assign a UID.  Even then, things like
>>>>> administrator UID should be connected via the provision() stage.
>>>> Then, may be we should add some default uid/gid attributes to
>>>> wellknown users/groups on the provision stage (if --use-rfc2307 is
>>>> used add them to the directory too). That would also be a step to
>>>> provision default posix acls for sysvol. Here is the list which I feel
>>>> needed and suggested values.
>>>>
>>>> name : sid : uid/gid
>>>> ---------------------------------
>>>> Administrators    :    S-1-5-32-544    :    300544                -
>>>> should to be synced on all dcs (it's a part of default ms's sysvol
>>>> permissions)
>>>> Server Operators    :    S-1-5-32-549    :    300549            - the
>>>> same
>>>> Authenticated Users    :    S-1-5-11    :    300011              - the
>>>> same
>>>> Local System    :    S-1-5-18    :    300018/300018             - the
>>>> same
>>>>
>>>> Administrator    :    S-1-5-21domain-500    :    0/0
>>>> Guest    :    S-1-5-21domain-501    :    nobody's uid/nobody's gid
>>>> Anonymous    :    S-1-5-7    :    nobody's gid
>>>>
>>>> What do you think?
>>> I don't mind assigning a UID in the directory based on the imported
>>> values to well known users, but I'm just saying we need to be careful
>>> about actually 'importing' the user.
>> If you are suggesting importing only uid and gid for wellknown accounts,
>> then I'll change the patch such way.
>> But could you explain why we need to import it via provision()? For
>> example, "domain admins" is a wellknown account too, but we don't import
>> it via provision().
> We do create "domain admins" during provision.

Sorry, I meant we don't import "domain admins" gid.
>> Another question is what I wrote previously: for >1 DC's domains we need
>> to write at least several account's uid/gid to the ldap while
>> provisioning, to have acls synced. So, some uid/gid initialization is
>> needed for provision anyway. And in the case of classicupgrade
>> s3-existed account uids would be overwritten. Doesn't it look reasonable?
> It is reasonable to set these UID values to the s3 accounts, but I don't
> want to import the s3 accounts, just modify the provision-provided
> accounts.  We also need to take care that some things are done during
> the provision, that uses the id map, so we have to set
The patch doesn't create any accounts, it just adds posix attributes to
provisioned wellknown accounts (like if they were imported accounts),
see this part of the patch:

-        s4_passdb.add_sam_account(userdata[username])
+        if username.lower() == 'administrator' or username.lower() ==
'guest' or username.lower() == 'krbtgt':
+            logger.warn("  Skipping wellknown provisioned user '%s'",
username)
+        else:
+            s4_passdb.add_sam_account(userdata[username])


And about provisioning uid/gid for some wellknown accounts in ldap. As I
understood you don't mind implementing this, then I'll try to prepare
another patch.

>
>>>>>> Also, I have to say that "if entry['rid'] < 1000:" check gives an error
>>>>>> at the "adding users to groups" stage (nonexisting user). Ways to solve it:
>>>>>>  1. Stop provision with error if such accounts exist (think it's the best)
>>>>>>  2. Add some workaround to the function that lists members
>>>>>>  3. Remove this check.
>>>>> I'm not entirely sure what you mean here.  I guess we do need to upgrade
>>>>> groups of any well known users we find?
>>>>>
>>>> If group contains user with rid < 1000 (skipped), then
>>>> add_users_to_group ends with error (because this user hasn't been
>>>> actually imported), and upgrading fails.
>>>> So, if there are 1000 users, we have to wait for a really long time
>>>> (while all users are being imported) to just see an error at the end.
>>>> I think we should give an error immediately, or test somehow for user
>>>> existence in s4 db before trying to add it to some group.  
>>> What users are you importing with a sid below 1000 that Samba4 doesn't
>>> already define?
>>>
>> I haven't yet met such users, I've got this issue while testing
>> suggested patch (I changed the sid manually for one of the users). I'm
>> just trying to say if someone's s3db contains such user (I guess some
>> do), we'll need to do something with it, the more so because we already
>> have this check.
> I guess we should error out.
>

I'll change the patch that way.

Thanks.

--
Best regards,
Sergey Urushkin