Quantcast

SMB3 Unix extensions

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

SMB3 Unix extensions

Samba - samba-technical mailing list
Hi,

I would like to start a discussion about SMB3 Unix extensions. So far
I have the following questions:

1) Do we have any docs describing the protocol draft?
2) What is the current status? Are there some difficulties with the
current state? What is left to be done?

Best regards,
Pavel Shilovsky

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Tue, Apr 04, 2017 at 12:13:27AM -0700, Pavel Shilovsky via samba-technical wrote:
> Hi,
>
> I would like to start a discussion about SMB3 Unix extensions. So far
> I have the following questions:
>
> 1) Do we have any docs describing the protocol draft?

Do we have a protocol draft at all? Many months ago I just started to
hack something up using our smb2 client library and Samba, but I did
not get very far. At that point nobody could point me at any protocol
ideas. Maybe some SDC slides, but apart from that?

> 2) What is the current status? Are there some difficulties with the
> current state? What is left to be done?

I think the main thing is that someone just needs to do *SOMETHING*. I
know I'm as guilty as everybody else is, but I think here the "code
speaks" mantra is overdue. A big empty space can be filled :-)

After so many years of just talking about it, we should be willing to
accept anything that works at all. It should be versioned, so that we
can fix mistakes later, but that's about it.

So -- Go, go, go! :-)

Just my 2ct.

Volker

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi Pavel,

I'm also interested in working on this.

Pavel Shilovsky via samba-technical <[hidden email]>
writes:
> 1) Do we have any docs describing the protocol draft?
> 2) What is the current status? Are there some difficulties with the
> current state? What is left to be done?

AFAIK this is the closest thing we have to a draft:

https://wiki.samba.org/index.php/SMB3-Linux

(sorry if you get this email twice, I'm having email issues..)

--
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi Pavel,

> I would like to start a discussion about SMB3 Unix extensions. So far
> I have the following questions:
>
> 1) Do we have any docs describing the protocol draft?

We discussed that at some events, Jeremy can you do a brain dump
of what we already discussed, so that we don't start over and over again.

> 2) What is the current status? Are there some difficulties with the
> current state? What is left to be done?

We're basically back to zero...

metze


signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
And we do have the slides from SDC.

We did reach some conclusions -
1) open with posix create context on root of share will allow us to
determined if the server understands posix (so we don't try to send
the posix create context on subsequent opens and have it ignored)
2) the capability flags to be returned were discussed at SDC and were
pretty simple
3) most features can be done without adding info levels (just with the
create context)
4) a new info level (e.g. for fsinfo) was discussed but lower priority
and a number was not reserved for new info levels
5) need some info back from Microsoft on opinions about inferring mode
from the ACL and also about the 'nfs symlink' (assuming that the other
form of symlink reparse point is admin only) vs. simulated symlinks
(ala MF symlinks that Apple uses e.g.)

On Tue, Apr 4, 2017 at 9:11 AM, Stefan Metzmacher via samba-technical
<[hidden email]> wrote:

> Hi Pavel,
>
>> I would like to start a discussion about SMB3 Unix extensions. So far
>> I have the following questions:
>>
>> 1) Do we have any docs describing the protocol draft?
>
> We discussed that at some events, Jeremy can you do a brain dump
> of what we already discussed, so that we don't start over and over again.
>
>> 2) What is the current status? Are there some difficulties with the
>> current state? What is left to be done?
>
> We're basically back to zero...
>
> metze
>



--
Thanks,

Steve

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Tue, Apr 18, 2017 at 10:16:26AM -0500, Steve French wrote:

> And we do have the slides from SDC.
>
> We did reach some conclusions -
> 1) open with posix create context on root of share will allow us to
> determined if the server understands posix (so we don't try to send
> the posix create context on subsequent opens and have it ignored)
> 2) the capability flags to be returned were discussed at SDC and were
> pretty simple
> 3) most features can be done without adding info levels (just with the
> create context)
> 4) a new info level (e.g. for fsinfo) was discussed but lower priority
> and a number was not reserved for new info levels
> 5) need some info back from Microsoft on opinions about inferring mode
> from the ACL and also about the 'nfs symlink' (assuming that the other
> form of symlink reparse point is admin only) vs. simulated symlinks
> (ala MF symlinks that Apple uses e.g.)

I've been doing a lot of thinking about SMB2-UNIX-symlinks
since the recent CVE security patch.

As SambaXP is only 2 weeks away can we get all the stakeholders
in a room together and try and hash out a plan to deal with
the issues with creating UNIX symlinks/reparse points ?

Some of this is Samba specific, which isn't useful to external
implementors, but if possible I'd really like to re-use the
existing SMB2+ reparse mechanisms to implement UNIX extension
sylinks.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
2017-04-19 10:03 GMT-07:00 Jeremy Allison <[hidden email]>:

> On Tue, Apr 18, 2017 at 10:16:26AM -0500, Steve French wrote:
>> And we do have the slides from SDC.
>>
>> We did reach some conclusions -
>> 1) open with posix create context on root of share will allow us to
>> determined if the server understands posix (so we don't try to send
>> the posix create context on subsequent opens and have it ignored)
>> 2) the capability flags to be returned were discussed at SDC and were
>> pretty simple
>> 3) most features can be done without adding info levels (just with the
>> create context)
>> 4) a new info level (e.g. for fsinfo) was discussed but lower priority
>> and a number was not reserved for new info levels
>> 5) need some info back from Microsoft on opinions about inferring mode
>> from the ACL and also about the 'nfs symlink' (assuming that the other
>> form of symlink reparse point is admin only) vs. simulated symlinks
>> (ala MF symlinks that Apple uses e.g.)
>
> I've been doing a lot of thinking about SMB2-UNIX-symlinks
> since the recent CVE security patch.
>
> As SambaXP is only 2 weeks away can we get all the stakeholders
> in a room together and try and hash out a plan to deal with
> the issues with creating UNIX symlinks/reparse points ?
>
> Some of this is Samba specific, which isn't useful to external
> implementors, but if possible I'd really like to re-use the
> existing SMB2+ reparse mechanisms to implement UNIX extension
> sylinks.

Thanks for the answers. Unfortunately I am not going to SambaXP, so
won't be able to participate in the discussion in person.

I don't understand why to we need to rely on posix create context on a
root of a share if we can use Negotiate phase for this and get posix
capability flags from Negotiate response posix context?

Best regards,
Pavel Shilovsky

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Wed, Apr 19, 2017 at 04:41:58PM -0700, Pavel Shilovsky wrote:

> 2017-04-19 10:03 GMT-07:00 Jeremy Allison <[hidden email]>:
> > On Tue, Apr 18, 2017 at 10:16:26AM -0500, Steve French wrote:
> >> And we do have the slides from SDC.
> >>
> >> We did reach some conclusions -
> >> 1) open with posix create context on root of share will allow us to
> >> determined if the server understands posix (so we don't try to send
> >> the posix create context on subsequent opens and have it ignored)
> >> 2) the capability flags to be returned were discussed at SDC and were
> >> pretty simple
> >> 3) most features can be done without adding info levels (just with the
> >> create context)
> >> 4) a new info level (e.g. for fsinfo) was discussed but lower priority
> >> and a number was not reserved for new info levels
> >> 5) need some info back from Microsoft on opinions about inferring mode
> >> from the ACL and also about the 'nfs symlink' (assuming that the other
> >> form of symlink reparse point is admin only) vs. simulated symlinks
> >> (ala MF symlinks that Apple uses e.g.)
> >
> > I've been doing a lot of thinking about SMB2-UNIX-symlinks
> > since the recent CVE security patch.
> >
> > As SambaXP is only 2 weeks away can we get all the stakeholders
> > in a room together and try and hash out a plan to deal with
> > the issues with creating UNIX symlinks/reparse points ?
> >
> > Some of this is Samba specific, which isn't useful to external
> > implementors, but if possible I'd really like to re-use the
> > existing SMB2+ reparse mechanisms to implement UNIX extension
> > sylinks.
>
> Thanks for the answers. Unfortunately I am not going to SambaXP, so
> won't be able to participate in the discussion in person.
>
> I don't understand why to we need to rely on posix create context on a
> root of a share if we can use Negotiate phase for this and get posix
> capability flags from Negotiate response posix context?

We don't. The negotiate phase is the correct place to do this
(find out if the server supports UNIX extentions). Steve is misremembering
the outcome of the discussion in Redmond from last year.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Fri, Apr 28, 2017 at 10:56:28AM +0200, Stefan Metzmacher wrote:

> Am 27.04.2017 um 18:13 schrieb Jeremy Allison:
> > On Thu, Apr 27, 2017 at 04:36:30PM +0200, Stefan Metzmacher wrote:
> >> Am 20.04.2017 um 17:52 schrieb Jeremy Allison:
> >>> On Wed, Apr 19, 2017 at 08:18:01PM -0500, Steve French wrote:
> >>>> If server guys are willing to add a negotiate context ... fine with me
> >>>> - but IIRC that was vetoed because at least one or two servers don't
> >>>> support negotiate context (Azure e.g.?)
> >>>
> >>> Not that I recall. Negotiate context was decided as the
> >>> right place to do it - no bullshit CreateContext on "\"
> >>> requests (which really don't make sense).
> >>
> >> As far as I remember the idea was to just do this by file handle.
> >>
> >> So the client will just try to use the create context and the returned
> >> create context defines the features available just for that file handle.
> >
> > Yes, that's right.
> >
> >> So NO global negotiation anymore!
> >
> > That was my original idea, but no, we need it. The reason is
> > (David Kruse pointed this out) that it allows the client to
> > determine if a UNIX create context that is ignored by the server
> > is not returned because the server can't grant or doesn't want
> > to grant UNIX handles on this part of the file system, vs
> > a server that doesn't grant UNIX handles because it doesn't
> > implement UNIX extensions.
> >
> > The initial Negprot extension tells the client that yes,
> > I can do UNIX handles I just don't want to for this handle.
>
> Why does a client need to care about this?
> Just avoiding a few bytes for the ignored create context?

No, there was a good reason, I just can't remember
it right now whilst I'm writing my second SambaXP
talk :-).

Steve, can you remember the use case ?

> BTW: why is this a private discussion?

Souldn't be. Adding samba-technical back !

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Fri, Apr 28, 2017 at 10:53 AM, Jeremy Allison <[hidden email]> wrote:

> On Fri, Apr 28, 2017 at 10:56:28AM +0200, Stefan Metzmacher wrote:
>> Am 27.04.2017 um 18:13 schrieb Jeremy Allison:
>> > On Thu, Apr 27, 2017 at 04:36:30PM +0200, Stefan Metzmacher wrote:
>> >>
>> >> As far as I remember the idea was to just do this by file handle.
>> >>
>> >> So the client will just try to use the create context and the returned
>> >> create context defines the features available just for that file handle.
>> >
>> > Yes, that's right.
>> >
>> >> So NO global negotiation anymore!
>> >
>> > That was my original idea, but no, we need it. The reason is
>> > (David Kruse pointed this out) that it allows the client to
>> > determine if a UNIX create context that is ignored by the server
>> > is not returned because the server can't grant or doesn't want
>> > to grant UNIX handles on this part of the file system, vs
>> > a server that doesn't grant UNIX handles because it doesn't
>> > implement UNIX extensions.
>> >
>> > The initial Negprot extension tells the client that yes,
>> > I can do UNIX handles I just don't want to for this handle.
>>
>> Why does a client need to care about this?
>> Just avoiding a few bytes for the ignored create context?
>
> No, there was a good reason, I just can't remember
> it right now whilst I'm writing my second SambaXP
> talk :-).
>
> Steve, can you remember the use case ?

The client needs to know if the server supports Unix Extensions before
it sends the SMB2 CREATE request so that it doesn't attempt to do an
unrecoverable
operation (create supersede/overwrite or delete e.g) which behaves
differently based
on whether case sensitive turns out to be supported or not.

--
Thanks,

Steve

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Fri, Apr 28, 2017 at 12:38:03PM -0500, Steve French wrote:

> On Fri, Apr 28, 2017 at 10:53 AM, Jeremy Allison <[hidden email]> wrote:
> > On Fri, Apr 28, 2017 at 10:56:28AM +0200, Stefan Metzmacher wrote:
> >> Am 27.04.2017 um 18:13 schrieb Jeremy Allison:
> >> > On Thu, Apr 27, 2017 at 04:36:30PM +0200, Stefan Metzmacher wrote:
> >> >>
> >> >> As far as I remember the idea was to just do this by file handle.
> >> >>
> >> >> So the client will just try to use the create context and the returned
> >> >> create context defines the features available just for that file handle.
> >> >
> >> > Yes, that's right.
> >> >
> >> >> So NO global negotiation anymore!
> >> >
> >> > That was my original idea, but no, we need it. The reason is
> >> > (David Kruse pointed this out) that it allows the client to
> >> > determine if a UNIX create context that is ignored by the server
> >> > is not returned because the server can't grant or doesn't want
> >> > to grant UNIX handles on this part of the file system, vs
> >> > a server that doesn't grant UNIX handles because it doesn't
> >> > implement UNIX extensions.
> >> >
> >> > The initial Negprot extension tells the client that yes,
> >> > I can do UNIX handles I just don't want to for this handle.
> >>
> >> Why does a client need to care about this?
> >> Just avoiding a few bytes for the ignored create context?
> >
> > No, there was a good reason, I just can't remember
> > it right now whilst I'm writing my second SambaXP
> > talk :-).
> >
> > Steve, can you remember the use case ?
>
> The client needs to know if the server supports Unix Extensions before
> it sends the SMB2 CREATE request so that it doesn't attempt to do an
> unrecoverable
> operation (create supersede/overwrite or delete e.g) which behaves
> differently based
> on whether case sensitive turns out to be supported or not.

Thanks Steve ! I knew there was a unassailable use case here, I just
forgot what it was (deep in writing a VFS presentation right
now :-).

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
For your VFS presentation,   could I request that
* "Add discard API to the VFS layer"

be put on the roadmap ?    :-)

On Fri, Apr 28, 2017 at 10:41 AM, Jeremy Allison via samba-technical
<[hidden email]> wrote:

> On Fri, Apr 28, 2017 at 12:38:03PM -0500, Steve French wrote:
>> On Fri, Apr 28, 2017 at 10:53 AM, Jeremy Allison <[hidden email]> wrote:
>> > On Fri, Apr 28, 2017 at 10:56:28AM +0200, Stefan Metzmacher wrote:
>> >> Am 27.04.2017 um 18:13 schrieb Jeremy Allison:
>> >> > On Thu, Apr 27, 2017 at 04:36:30PM +0200, Stefan Metzmacher wrote:
>> >> >>
>> >> >> As far as I remember the idea was to just do this by file handle.
>> >> >>
>> >> >> So the client will just try to use the create context and the returned
>> >> >> create context defines the features available just for that file handle.
>> >> >
>> >> > Yes, that's right.
>> >> >
>> >> >> So NO global negotiation anymore!
>> >> >
>> >> > That was my original idea, but no, we need it. The reason is
>> >> > (David Kruse pointed this out) that it allows the client to
>> >> > determine if a UNIX create context that is ignored by the server
>> >> > is not returned because the server can't grant or doesn't want
>> >> > to grant UNIX handles on this part of the file system, vs
>> >> > a server that doesn't grant UNIX handles because it doesn't
>> >> > implement UNIX extensions.
>> >> >
>> >> > The initial Negprot extension tells the client that yes,
>> >> > I can do UNIX handles I just don't want to for this handle.
>> >>
>> >> Why does a client need to care about this?
>> >> Just avoiding a few bytes for the ignored create context?
>> >
>> > No, there was a good reason, I just can't remember
>> > it right now whilst I'm writing my second SambaXP
>> > talk :-).
>> >
>> > Steve, can you remember the use case ?
>>
>> The client needs to know if the server supports Unix Extensions before
>> it sends the SMB2 CREATE request so that it doesn't attempt to do an
>> unrecoverable
>> operation (create supersede/overwrite or delete e.g) which behaves
>> differently based
>> on whether case sensitive turns out to be supported or not.
>
> Thanks Steve ! I knew there was a unassailable use case here, I just
> forgot what it was (deep in writing a VFS presentation right
> now :-).
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Fri, Apr 28, 2017 at 12:20:02PM -0700, ronnie sahlberg wrote:
> For your VFS presentation,   could I request that
> * "Add discard API to the VFS layer"
>
> be put on the roadmap ?    :-)

Checking - Looks like it's already done ! See:

VFS_FALLOCATE_FL_PUNCH_HOLE

and this is plumbed down into source3/lib/system.c:sys_fallocate().

You get to this via SMB2 ioctl FSCTL_SET_ZERO_DATA.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: SMB3 Unix extensions

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
> -----Original Message-----
> From: samba-technical [mailto:[hidden email]] On
> Behalf Of Jeremy Allison via samba-technical
> Sent: Wednesday, April 19, 2017 1:03 PM
> To: Steve French <[hidden email]>
> Cc: Stefan Metzmacher <[hidden email]>; Pavel Shilovsky
> <[hidden email]>; samba-technical <samba-
> [hidden email]>
> Subject: Re: SMB3 Unix extensions
>
> On Tue, Apr 18, 2017 at 10:16:26AM -0500, Steve French wrote:
> > And we do have the slides from SDC.
> >
> > We did reach some conclusions -
> > 1) open with posix create context on root of share will allow us to
> > determined if the server understands posix (so we don't try to send
> > the posix create context on subsequent opens and have it ignored)
> > 2) the capability flags to be returned were discussed at SDC and were
> > pretty simple
> > 3) most features can be done without adding info levels (just with the
> > create context)
> > 4) a new info level (e.g. for fsinfo) was discussed but lower priority
> > and a number was not reserved for new info levels
> > 5) need some info back from Microsoft on opinions about inferring mode
> > from the ACL and also about the 'nfs symlink' (assuming that the other
> > form of symlink reparse point is admin only) vs. simulated symlinks
> > (ala MF symlinks that Apple uses e.g.)
>
> I've been doing a lot of thinking about SMB2-UNIX-symlinks since the recent
> CVE security patch.
>
> As SambaXP is only 2 weeks away can we get all the stakeholders in a room
> together and try and hash out a plan to deal with the issues with creating UNIX
> symlinks/reparse points ?

Gladly, and my slides say so too! I'm covering in-progress and open items as part of my SambaXP talk. 😊

We've got some notes from conversations at the last Plugfest, and some other material that I'll try to pull together. We'll have time at SambaXP to organize, and hopefully some of you are coming to the Redmond event we're hosting in June? That will give some other Microsoft folks a chance to join in person, since not everyone will be in Göttingen.

Tom.


>
> Some of this is Samba specific, which isn't useful to external implementors, but
> if possible I'd really like to re-use the existing SMB2+ reparse mechanisms to
> implement UNIX extension sylinks.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: SMB3 Unix extensions

Samba - samba-technical mailing list
On Fri, Apr 28, 2017 at 07:34:47PM +0000, Tom Talpey wrote:

> > -----Original Message-----
> > From: samba-technical [mailto:[hidden email]] On
> > Behalf Of Jeremy Allison via samba-technical
> > Sent: Wednesday, April 19, 2017 1:03 PM
> > To: Steve French <[hidden email]>
> > Cc: Stefan Metzmacher <[hidden email]>; Pavel Shilovsky
> > <[hidden email]>; samba-technical <samba-
> > [hidden email]>
> > Subject: Re: SMB3 Unix extensions
> >
> > On Tue, Apr 18, 2017 at 10:16:26AM -0500, Steve French wrote:
> > > And we do have the slides from SDC.
> > >
> > > We did reach some conclusions -
> > > 1) open with posix create context on root of share will allow us to
> > > determined if the server understands posix (so we don't try to send
> > > the posix create context on subsequent opens and have it ignored)
> > > 2) the capability flags to be returned were discussed at SDC and were
> > > pretty simple
> > > 3) most features can be done without adding info levels (just with the
> > > create context)
> > > 4) a new info level (e.g. for fsinfo) was discussed but lower priority
> > > and a number was not reserved for new info levels
> > > 5) need some info back from Microsoft on opinions about inferring mode
> > > from the ACL and also about the 'nfs symlink' (assuming that the other
> > > form of symlink reparse point is admin only) vs. simulated symlinks
> > > (ala MF symlinks that Apple uses e.g.)
> >
> > I've been doing a lot of thinking about SMB2-UNIX-symlinks since the recent
> > CVE security patch.
> >
> > As SambaXP is only 2 weeks away can we get all the stakeholders in a room
> > together and try and hash out a plan to deal with the issues with creating UNIX
> > symlinks/reparse points ?
>
> Gladly, and my slides say so too! I'm covering in-progress and open items as part of my SambaXP talk. 😊
>
> We've got some notes from conversations at the last Plugfest, and some other material that I'll try to pull together. We'll have time at SambaXP to organize, and hopefully some of you are coming to the Redmond event we're hosting in June? That will give some other Microsoft folks a chance to join in person, since not everyone will be in Göttingen.

Yes I'm planning to be there (that reminds me to book my
flights/hotel :-).

Loading...