Work on SMB3 persistent handles

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Work on SMB3 persistent handles

Samba - samba-technical mailing list
Hi all,

recently, the topic of implementing SMB3 persistent handles
in Samba has been taking on some traction again. Mainly
driven by Chris Hertel. We need to make sure that we have
a general purpose implementation that works on any file system,
but we should also make sure to offer hooks that allow
special file systems in the back-end to expose some features
of making handle info persistent. It was originally Chris who
raiesed this requirement.

Now I have recently talked to the nice folks from vmware
(copied in this mail) who are very interested in implementing
persistent handles in Samba with a similar file-system support,
and would like to get involved in the development process.

Hence we though I'd start this thread in the open so that we
can align our efforts and hopefully come up with a good set
of changes that are generic enough to satisfy the various
backend storage solutions. So I copied samba-technical
as well as explicitly the Samba people who have recently been
looking into persistent handles most actively. (Apologies
if I forgot anyone..)
So Chris, Ralph, Metze, samba folks, meet Rick, Wenguang,
Albert, and Satish.

Maybe, if appropriate, someone could summarize the latest
state of the discussions/design/prototyping...

Cheers - Michael

signature.asc (169 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
Michael:  Thanks for calling my attention to this thread.

You wrote:
> Maybe, if appropriate, someone could summarize the latest
> state of the discussions/design/prototyping...

I can summarize my own status:

* I have done the work necessary to negotiate Persistent Handle support in
the Session Setup.  Similarly, I have added per-share configuration to
support Continuous Availability in the Tree Connect.  Both of these
additions work nicely.

* Internally to Samba, I have added fields and flags that identify a share
as being CA, and a file handle as being Persistent.

Those are basic steps needed to negotiate Continuous Availability on the
wire, and to signal internally that CA is in use.

In theory, given the way that Durable Handle data is stored, we *should* be
able to pass Persistent Handle tests in this configuration.  As long as the
Samba node that owns the handle does not actually go down (that is, as long
as the test is performed by dropping the network connection), the data
should be maintained and we should be able to re-establish the Persistent
Handle.

I'm not quite there yet.  I am not yet sending back the correct blob
following the recovery.  This *should* be an easy fix.

Once that all works, the next step is to actually make the handle data
persistent.  My first attempt at doing this was to modify the Open calls on
four [C]TDB databases that store the handle state.  Doing so *should* have
made the data persistent, but instead it caused Samba to crash with the
stack trace showing errors generated in libtevent.

As Michael mentioned, my hope is that we can expose the calls that are
needed to store and retrieve the Durable/Persistent handle data in such a
way that individual vendors (like VMware) can leverage their own product's
features.

So... That's an introduction.  Let me know if there are questions I can answer.

Chris -)-----

On 10/20/2017 07:46 AM, Michael Adam wrote:

> Hi all,
>
> recently, the topic of implementing SMB3 persistent handles
> in Samba has been taking on some traction again. Mainly
> driven by Chris Hertel. We need to make sure that we have
> a general purpose implementation that works on any file system,
> but we should also make sure to offer hooks that allow
> special file systems in the back-end to expose some features
> of making handle info persistent. It was originally Chris who
> raised this requirement.
>
> Now I have recently talked to the nice folks from vmware
> (copied in this mail) who are very interested in implementing
> persistent handles in Samba with a similar file-system support,
> and would like to get involved in the development process.
>
> Hence we though I'd start this thread in the open so that we
> can align our efforts and hopefully come up with a good set
> of changes that are generic enough to satisfy the various
> backend storage solutions. So I copied samba-technical
> as well as explicitly the Samba people who have recently been
> looking into persistent handles most actively. (Apologies
> if I forgot anyone..)
> So Chris, Ralph, Metze, samba folks, meet Rick, Wenguang,
> Albert, and Satish.
>
> Maybe, if appropriate, someone could summarize the latest
> state of the discussions/design/prototyping...
>
> Cheers - Michael


Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
On 2017-10-23 at 13:17 -0500, Christopher R. Hertel wrote:

> Michael:  Thanks for calling my attention to this thread.
>
> You wrote:
> > Maybe, if appropriate, someone could summarize the latest
> > state of the discussions/design/prototyping...
>
> I can summarize my own status:
>
> * I have done the work necessary to negotiate Persistent Handle support in
> the Session Setup.  Similarly, I have added per-share configuration to
> support Continuous Availability in the Tree Connect.  Both of these
> additions work nicely.
>
> * Internally to Samba, I have added fields and flags that identify a share
> as being CA, and a file handle as being Persistent.
>
> Those are basic steps needed to negotiate Continuous Availability on the
> wire, and to signal internally that CA is in use.
>
> In theory, given the way that Durable Handle data is stored, we *should* be
> able to pass Persistent Handle tests in this configuration.  As long as the
> Samba node that owns the handle does not actually go down (that is, as long
> as the test is performed by dropping the network connection), the data
> should be maintained and we should be able to re-establish the Persistent
> Handle.
>
> I'm not quite there yet.  I am not yet sending back the correct blob
> following the recovery.  This *should* be an easy fix.
Since we've been talking about this a bit now, I've revived the
patches that Metze and I had in our WIP branches since 2012, have
rebased them on top of current master and made them work again.
It's mainly the protocol boilerplate code. I.e. there is no
persistence of the DBs or records, also the subtleties of how
persistent handles are different from durable handles in semantic
are not all covered yet, in particular no calling of
SMB_VFS_DURABLE_DISCONNECT() under other circumstances than
before, but for a start it does survive our (smbtorture's)
durable-v2-open test (against nt4_dc) including the persistent
open tests. Nothing fancy yet but just for comparing.

https://git.samba.org/?p=obnox/samba/samba-obnox.git;a=shortlog;h=refs/heads/persistent-wip

Also attached for convenience.

Cheers - Michael




> Once that all works, the next step is to actually make the handle data
> persistent.  My first attempt at doing this was to modify the Open calls on
> four [C]TDB databases that store the handle state.  Doing so *should* have
> made the data persistent, but instead it caused Samba to crash with the
> stack trace showing errors generated in libtevent.
>
> As Michael mentioned, my hope is that we can expose the calls that are
> needed to store and retrieve the Durable/Persistent handle data in such a
> way that individual vendors (like VMware) can leverage their own product's
> features.
>
> So... That's an introduction.  Let me know if there are questions I can answer.
>
> Chris -)-----
>
> On 10/20/2017 07:46 AM, Michael Adam wrote:
> > Hi all,
> >
> > recently, the topic of implementing SMB3 persistent handles
> > in Samba has been taking on some traction again. Mainly
> > driven by Chris Hertel. We need to make sure that we have
> > a general purpose implementation that works on any file system,
> > but we should also make sure to offer hooks that allow
> > special file systems in the back-end to expose some features
> > of making handle info persistent. It was originally Chris who
> > raised this requirement.
> >
> > Now I have recently talked to the nice folks from vmware
> > (copied in this mail) who are very interested in implementing
> > persistent handles in Samba with a similar file-system support,
> > and would like to get involved in the development process.
> >
> > Hence we though I'd start this thread in the open so that we
> > can align our efforts and hopefully come up with a good set
> > of changes that are generic enough to satisfy the various
> > backend storage solutions. So I copied samba-technical
> > as well as explicitly the Samba people who have recently been
> > looking into persistent handles most actively. (Apologies
> > if I forgot anyone..)
> > So Chris, Ralph, Metze, samba folks, meet Rick, Wenguang,
> > Albert, and Satish.
> >
> > Maybe, if appropriate, someone could summarize the latest
> > state of the discussions/design/prototyping...
> >
> > Cheers - Michael
>

persistent-wip (16K) Download Attachment
signature.asc (169 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
On 2017-10-24 at 00:58 +0200, Michael Adam via samba-technical wrote:

> On 2017-10-23 at 13:17 -0500, Christopher R. Hertel wrote:
> > Michael:  Thanks for calling my attention to this thread.
> >
> > You wrote:
> > > Maybe, if appropriate, someone could summarize the latest
> > > state of the discussions/design/prototyping...
> >
> > I can summarize my own status:
> >
> > * I have done the work necessary to negotiate Persistent Handle support in
> > the Session Setup.  Similarly, I have added per-share configuration to
> > support Continuous Availability in the Tree Connect.  Both of these
> > additions work nicely.
> >
> > * Internally to Samba, I have added fields and flags that identify a share
> > as being CA, and a file handle as being Persistent.
> >
> > Those are basic steps needed to negotiate Continuous Availability on the
> > wire, and to signal internally that CA is in use.
> >
> > In theory, given the way that Durable Handle data is stored, we *should* be
> > able to pass Persistent Handle tests in this configuration.  As long as the
> > Samba node that owns the handle does not actually go down (that is, as long
> > as the test is performed by dropping the network connection), the data
> > should be maintained and we should be able to re-establish the Persistent
> > Handle.
> >
> > I'm not quite there yet.  I am not yet sending back the correct blob
> > following the recovery.  This *should* be an easy fix.
>
> Since we've been talking about this a bit now, I've revived the
> patches that Metze and I had in our WIP branches since 2012, have
> rebased them on top of current master and made them work again.
> It's mainly the protocol boilerplate code. I.e. there is no
> persistence of the DBs or records, also the subtleties of how
> persistent handles are different from durable handles in semantic
> are not all covered yet, in particular no calling of
> SMB_VFS_DURABLE_DISCONNECT() under other circumstances than
> before, but for a start it does survive our (smbtorture's)
> durable-v2-open test (against nt4_dc) including the persistent
> open tests. Nothing fancy yet but just for comparing.
>
> https://git.samba.org/?p=obnox/samba/samba-obnox.git;a=shortlog;h=refs/heads/persistent-wip
>
> Also attached for convenience.
Warning... I just noticed that this was not rebased to
the latest state of master but a few days back.
So it does (of course) not cleanly rebase on top of
Ralph's refactoring of smbd_smb2_create_send(). ;-)

Will look into it...

Cheers - Michael

signature.asc (169 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
Note (stating the hopefully obvious):

I did not share that patchset to compete with Chris' work
but in the hope to be helpful in his progress!

Chris, feel free to use / merge / discard as you fee fit.
(I'd guess Metze would agree).

Also I'd be interested to see any WIP patches of yours,
no matter how incomplete.

Cheers - Michael

On 2017-10-24 at 01:11 +0200, Michael Adam via samba-technical wrote:

> On 2017-10-24 at 00:58 +0200, Michael Adam via samba-technical wrote:
> > On 2017-10-23 at 13:17 -0500, Christopher R. Hertel wrote:
> > > Michael:  Thanks for calling my attention to this thread.
> > >
> > > You wrote:
> > > > Maybe, if appropriate, someone could summarize the latest
> > > > state of the discussions/design/prototyping...
> > >
> > > I can summarize my own status:
> > >
> > > * I have done the work necessary to negotiate Persistent Handle support in
> > > the Session Setup.  Similarly, I have added per-share configuration to
> > > support Continuous Availability in the Tree Connect.  Both of these
> > > additions work nicely.
> > >
> > > * Internally to Samba, I have added fields and flags that identify a share
> > > as being CA, and a file handle as being Persistent.
> > >
> > > Those are basic steps needed to negotiate Continuous Availability on the
> > > wire, and to signal internally that CA is in use.
> > >
> > > In theory, given the way that Durable Handle data is stored, we *should* be
> > > able to pass Persistent Handle tests in this configuration.  As long as the
> > > Samba node that owns the handle does not actually go down (that is, as long
> > > as the test is performed by dropping the network connection), the data
> > > should be maintained and we should be able to re-establish the Persistent
> > > Handle.
> > >
> > > I'm not quite there yet.  I am not yet sending back the correct blob
> > > following the recovery.  This *should* be an easy fix.
> >
> > Since we've been talking about this a bit now, I've revived the
> > patches that Metze and I had in our WIP branches since 2012, have
> > rebased them on top of current master and made them work again.
> > It's mainly the protocol boilerplate code. I.e. there is no
> > persistence of the DBs or records, also the subtleties of how
> > persistent handles are different from durable handles in semantic
> > are not all covered yet, in particular no calling of
> > SMB_VFS_DURABLE_DISCONNECT() under other circumstances than
> > before, but for a start it does survive our (smbtorture's)
> > durable-v2-open test (against nt4_dc) including the persistent
> > open tests. Nothing fancy yet but just for comparing.
> >
> > https://git.samba.org/?p=obnox/samba/samba-obnox.git;a=shortlog;h=refs/heads/persistent-wip
> >
> > Also attached for convenience.
>
> Warning... I just noticed that this was not rebased to
> the latest state of master but a few days back.
> So it does (of course) not cleanly rebase on top of
> Ralph's refactoring of smbd_smb2_create_send(). ;-)
>
> Will look into it...
>
> Cheers - Michael


signature.asc (169 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
On 10/24/2017 10:29 AM, Michael Adam wrote:
:
:
> Chris, feel free to use / merge / discard as you see fit.
> (I'd guess Metze would agree).

I will do my best to merge the solutions so that they can be applied against
master.

> Also I'd be interested to see any WIP patches of yours,
> no matter how incomplete.

git://git.samba.org/crh/continuousavailability.git

That code is testable as-is.  It has no negative impact on the rest of
Samba, and succeeds at negotiating Persistent Handles (per Session) and
Continuous Availability (per share).  It does not yet have the changes
needed to actually store the Persistent Handle data in a recoverable way,
but that's not necessary for initial testing.

Persistent Handles should be recoverable even without persistent backing
store.  In a network outage, a Persistent Handle would work the same way as
a Durable Handle, so a test that simply drops the TCP connection and then
reconnects should get correct Persistent Handle behavior.

I think that Ralph had some smbtorture code that could do some of this testing.

What I am working on now is making sure that the code in my repository
provides a correct response to a Persistent Handle reconnect.  If I can get
that part working, then the only part missing will be the actual persistent
data storage/recovery.

Chris -)-----

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
Hi Chris,
I’ve already checked out your code from the CA branch and played with it.

I made a POC based on your git repository with some hack.  This is test I did:
1. Copy a large file to a CA enabled share from Windows 2012 R2 client
2. Restart Samba daemons
The persistent file handle is reconnected successfully, and the file copy continues without any error.
The purpose of the POC is to identify the gaps we have for supporting persistent file handles.

I attached my patch in the attachment. The patch is very simple:
1. Remove TDB_CLEAR_IF_FIRST from open path of locking.tdb and smbXsrv_open_global.tdb
2. Disable the code for checking whether server_id is disconnected.

I assume you’re trying to store server_id when Samba deamon is shutdown so that persistent file handle can be reconnected successfully?

--
Thanks!
Albert
 

On 10/25/17, 2:17 AM, "Christopher R. Hertel" <[hidden email]> wrote:

    On 10/24/2017 10:29 AM, Michael Adam wrote:
    :
    :
    > Chris, feel free to use / merge / discard as you see fit.
    > (I'd guess Metze would agree).
   
    I will do my best to merge the solutions so that they can be applied against
    master.
   
    > Also I'd be interested to see any WIP patches of yours,
    > no matter how incomplete.
   
    git://git.samba.org/crh/continuousavailability.git
   
    That code is testable as-is.  It has no negative impact on the rest of
    Samba, and succeeds at negotiating Persistent Handles (per Session) and
    Continuous Availability (per share).  It does not yet have the changes
    needed to actually store the Persistent Handle data in a recoverable way,
    but that's not necessary for initial testing.
   
    Persistent Handles should be recoverable even without persistent backing
    store.  In a network outage, a Persistent Handle would work the same way as
    a Durable Handle, so a test that simply drops the TCP connection and then
    reconnects should get correct Persistent Handle behavior.
   
    I think that Ralph had some smbtorture code that could do some of this testing.
   
    What I am working on now is making sure that the code in my repository
    provides a correct response to a Persistent Handle reconnect.  If I can get
    that part working, then the only part missing will be the actual persistent
    data storage/recovery.
   
    Chris -)-----
   


samba_restart_POC.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
Hi Chris,

Are you going to implement persistent file handles based on some interfaces provided by cluster file system or based on CTDB?

VMware only concerns about the former one.

If the cluster file system provides a small fast POSIX file system which is used to store the TDBs, would that be sufficient for supporting persistent file handles?

As regard to the fast POSIX file system, it could be a memory based file system which is replicated to at least 2 nodes, or it can be on NVMe or SSD etc.



In our architecture, different nodes serve different shares, so the same TDB won’t be accessed/competed by different hosts simultaneously. Hopefully the performance won’t be very bad.



In order to ensure any SMB request are processed transactionally. i.e. Samba can recover itself to a consistent state whenever one node crashes. I think we’ll need the following guarantee for entries related to persistent file handles:

·          Update to a single TDB is transactional.  Looks like current TDB only support a single transaction, making it transactional might downgrade its performance dramatically.

·          It’s possible to update multiple TDBs when process a single SMB request like open. We need to ensure the update to those multiple TDBs are transactional as well. Probably we may need to introduce an assistant TDB for ensuring the transactional semantic of processing a single SMB request.



Thoughts?



--

Thanks!

Albert



On 10/26/17, 12:09 AM, "Albert Guo" <[hidden email]> wrote:



    Hi Chris,

    I’ve already checked out your code from the CA branch and played with it.



    I made a POC based on your git repository with some hack.  This is test I did:

    1.                Copy a large file to a CA enabled share from Windows 2012 R2 client

    2.                Restart Samba daemons

    The persistent file handle is reconnected successfully, and the file copy continues without any error.

    The purpose of the POC is to identify the gaps we have for supporting persistent file handles.



    I attached my patch in the attachment. The patch is very simple:

    1.                Remove TDB_CLEAR_IF_FIRST from open path of locking.tdb and smbXsrv_open_global.tdb

    2.                Disable the code for checking whether server_id is disconnected.



    I assume you’re trying to store server_id when Samba deamon is shutdown so that persistent file handle can be reconnected successfully?



    --

    Thanks!

    Albert





    On 10/25/17, 2:17 AM, "Christopher R. Hertel" <[hidden email]> wrote:



        On 10/24/2017 10:29 AM, Michael Adam wrote:

        :

        :

        > Chris, feel free to use / merge / discard as you see fit.

        > (I'd guess Metze would agree).



        I will do my best to merge the solutions so that they can be applied against

        master.



        > Also I'd be interested to see any WIP patches of yours,

        > no matter how incomplete.



        git://git.samba.org/crh/continuousavailability.git



        That code is testable as-is.  It has no negative impact on the rest of

        Samba, and succeeds at negotiating Persistent Handles (per Session) and

        Continuous Availability (per share).  It does not yet have the changes

        needed to actually store the Persistent Handle data in a recoverable way,

        but that's not necessary for initial testing.



        Persistent Handles should be recoverable even without persistent backing

        store.  In a network outage, a Persistent Handle would work the same way as

        a Durable Handle, so a test that simply drops the TCP connection and then

        reconnects should get correct Persistent Handle behavior.



        I think that Ralph had some smbtorture code that could do some of this testing.



        What I am working on now is making sure that the code in my repository

        provides a correct response to a Persistent Handle reconnect.  If I can get

        that part working, then the only part missing will be the actual persistent

        data storage/recovery.



        Chris -)-----






Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi folks,

On Wed, Oct 25, 2017 at 04:09:21PM +0000, Albert Guo via samba-technical wrote:
> I made a POC based on your git repository with some hack.  This is test I did:
> 1. Copy a large file to a CA enabled share from Windows 2012 R2 client
> 2. Restart Samba daemons
> The persistent file handle is reconnected successfully, and the file copy
> continues without any error.

hooray! :)

Fwiw I've been exploring the design space for fur in my spare time and also took
some notes on possible designs for the persistent storage layer and APIs. You
can find my current design doc here:

git branch: <https://git.samba.org/?p=slow/samba.git;a=shortlog;h=refs/heads/persistent-handles>
git commit: <https://git.samba.org/?p=slow/samba.git;a=commitdiff;h=164ae1364fbe250f2da87a53aa7e571d6548d93f>
as txt: <https://git.samba.org/?p=slow/samba.git;a=blob_plain;f=STEPS.txt;h=cee0f64c69811698679b0bf131db526549f3dd53;hb=164ae1364fbe250f2da87a53aa7e571d6548d93f>
as html: <https://www.samba.org/~slow/STEPS.html>

The links are a bit broken in the text version.

The branch also has a bunch of torture tests for various aspects of persistent
handles as well as some wip patches for the SMB2 frontend and the FSA
subsystems. But most of that stuff is more akind of taking notes. Some stuff is
from going through the Simplivity patchset (see below) and splitting it up into
microcommits [1].

For the storage backend I'm currently leaning towards the "2.1.2 New database
for persistent handles" design and will start prototyping in my spare time. I'm
only making slow progress as there's just too much other stuff that wants my
attention. :)

> The purpose of the POC is to identify the gaps we have for supporting persistent file handles.

For upstream we mostly needed a proper storage layer that also works in the
clustered case.

One important thing that may not be obvious is that you have to protect
disconnected persistent handles from contenting opens, cf MS-SMB2 3.3.5.9 and
3.3.4.7. Though the latter is currently incomplete and will be updated my MS in
the future explaining how to deal with lease breaks and disconnected PHs. I can
share the details if you like, I haven't yet added this to my design doc
neither.

> I attached my patch in the attachment. The patch is very simple:
>
> 1. Remove TDB_CLEAR_IF_FIRST from open path of locking.tdb and
>    smbXsrv_open_global.tdb

Without adding transactions this has the risk of corrupting the databases in the
event of a crash. Adding transcations has performance implications. Iirc patches
for persistent handles were submitted to the mailing list, directly or
indirectly, at least once. I have a munged version of the persistent handle
patchset from Simplivity here:

<https://git.samba.org/?p=slow/samba.git;a=shortlog;h=refs/heads/persistent-simplivity>

Hth!

-slow

[1] <https://git.samba.org/?p=slow/samba.git;a=shortlog;h=refs/heads/persistent-simplivity-steps>

--
Ralph Boehme, Samba Team       https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
On 10/25/2017 11:48 AM, Albert Guo wrote:
> Hi Chris,

Hi, Albert,

> Are you going to implement persistent file handles based on some interfaces
> provided by cluster file system or based on CTDB?

As Michael mentioned, my own goal is to expose the interfaces so that you,
or I, or some other developer can adapt the solution to their platform with
ease.

> VMware only concerns about the former one.

Unfortunately, much of the work that others have put into this has been
quite focused on CTDB.  That's one reason that it is useful to you and me
and others to separate out the protocol changes from the changes to the
backing store.  If we can get the protocol bits right, then we can tackle
the backing store separately.

To that point, it is worth noting that Persistent Handles *can be tested*
without having a working persistent backing store.  Persistent Handles (as
you have already discovered) can be tested by breaking the network
connection.  The client will attempt to re-establish the connection and will
request the restoration of the Persistent Handle.

> If the cluster file system provides a small fast POSIX file system which is
> used to store the TDBs, would that be sufficient for supporting persistent
> file handles?

That depends entirely on how you want to implement the persistent backing
store.  I am aware of one solution (which, alas, was never released) that
worked like this:

For each server IP address (aka "public address" in CTDB-speak), there was a
TDB file stored on cluster-shared storage.  Only the node serving on the
given IP address had the database open so no cross-cluster locking was needed.

In the event of a node failure, the IP address(es) would move to active
server nodes, and so would access to the TDB files.

> As regard to the fast POSIX file system, it could be a memory based file
> system which is replicated to at least 2 nodes, or it can be on NVMe or SSD etc.

These are implementation details so I'll put them aside, except to say that
they illustrate exactly why it is that we believe it is really important to
be able to expose the backing store.

> In our architecture, different nodes serve different shares, so the same TDB
> won’t be accessed/competed by different hosts simultaneously. Hopefully the
> performance won’t be very bad.

:-)

> In order to ensure any SMB request are processed transactionally. i.e. Samba
> can recover itself to a consistent state whenever one node crashes. I think
> we’ll need the following guarantee for entries related to persistent file
> handles:
>
> ·          Update to a single TDB is transactional.  Looks like current TDB
> only support a single transaction, making it transactional might downgrade
> its performance dramatically.
>
> ·          It’s possible to update multiple TDBs when process a single SMB
> request like open. We need to ensure the update to those multiple TDBs are
> transactional as well. Probably we may need to introduce an assistant TDB
> for ensuring the transactional semantic of processing a single SMB request.
>
> Thoughts?

Several.  Hopefully they're helpful.

- Below you mention that you removed TDB_CLEAR_IF_FIRST from the open
  calls to locking.tdb and smbXsrv_open_global.tdb.  I did the same as
  a test, but I also removed TDB_CLEAR_IF_FIRST from the
  byte-range-locking and leases databases.  These two additional
  databases are also used to maintain handle state.

  Unfortunately, the change resulted in random core dumps related to
  libtevent.

- My current goal is to merge my changes with the changes Michael
  posted and update the merged set with master.  Given the fixes that
  Michael has provided, we should be able to get full protocol support
  for Persistent Handles.  Then we only need to work on the backing
  store.

- Regarding the backing store...  All four of the databases we've
  identified are opened and accessed via the DBwrap API.  DBwrap hides
  the differences between CTDB and TDB, and also provides for
  "transactions".  The problem is that DBwrap is somewhat specific to
  [C]TDB.  We may be able to work around that, though.

- My hope had been that we could expose the relevant calls via
  Samba's VFS layer.  I still think that's a worthy goal but we may
  need to get there incrementally.

Chris -)-----

                                                                                                                                                                                                                                                                                                                                                                

> On 10/26/17, 12:09 AM, "Albert Guo" <[hidden email]> wrote:
>
>  
>
>     Hi Chris,
>
>     I’ve already checked out your code from the CA branch and played with it.
>
>    
>
>     I made a POC based on your git repository with some hack.  This is test
> I did:
>
>     1.                Copy a large file to a CA enabled share from Windows
> 2012 R2 client
>
>     2.                Restart Samba daemons
>
>     The persistent file handle is reconnected successfully, and the file
> copy continues without any error.
>
>     The purpose of the POC is to identify the gaps we have for supporting
> persistent file handles.
>
>    
>
>     I attached my patch in the attachment. The patch is very simple:
>
>     1.                Remove TDB_CLEAR_IF_FIRST from open path of
> locking.tdb and smbXsrv_open_global.tdb
>
>     2.                Disable the code for checking whether server_id is
> disconnected.
>
>     
>
>     I assume you’re trying to store server_id when Samba deamon is shutdown
> so that persistent file handle can be reconnected successfully?
>
>    
>
>     --
>
>     Thanks!
>
>     Albert
>
>     
>
>     
>
>     On 10/25/17, 2:17 AM, "Christopher R. Hertel" <[hidden email]> wrote:
>
>    
>
>         On 10/24/2017 10:29 AM, Michael Adam wrote:
>
>         :
>
>         :
>
>         > Chris, feel free to use / merge / discard as you see fit.
>
>         > (I'd guess Metze would agree).
>
>        
>
>         I will do my best to merge the solutions so that they can be applied
> against
>
>         master.
>
>        
>
>         > Also I'd be interested to see any WIP patches of yours,
>
>         > no matter how incomplete.
>
>        
>
>         git://git.samba.org/crh/continuousavailability.git
>
>        
>
>         That code is testable as-is.  It has no negative impact on the rest of
>
>         Samba, and succeeds at negotiating Persistent Handles (per Session) and
>
>         Continuous Availability (per share).  It does not yet have the changes
>
>         needed to actually store the Persistent Handle data in a recoverable
> way,
>
>         but that's not necessary for initial testing.
>
>        
>
>         Persistent Handles should be recoverable even without persistent backing
>
>         store.  In a network outage, a Persistent Handle would work the same
> way as
>
>         a Durable Handle, so a test that simply drops the TCP connection and
> then
>
>         reconnects should get correct Persistent Handle behavior.
>
>        
>
>         I think that Ralph had some smbtorture code that could do some of
> this testing.
>
>        
>
>         What I am working on now is making sure that the code in my repository
>
>         provides a correct response to a Persistent Handle reconnect.  If I
> can get
>
>         that part working, then the only part missing will be the actual
> persistent
>
>         data storage/recovery.
>
>        
>
>         Chris -)-----
>
>        
>
>     
>
>     
>

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
On Wed, Oct 25, 2017 at 04:48:22PM +0000, Albert Guo via samba-technical wrote:
> Are you going to implement persistent file handles based on some
> interfaces provided by cluster file system or based on CTDB?

The main interface Samba internally uses for the databases is the
dbwrap interface, found in lib/dbwrap/dbwrap.h. The #1 central routine
for the backend (dbwrap_private.h) is fetch_locked. We have several
optimizations in place like parse_record and do_locked, but if you
give us a fast fetch_locked() in whatever implementation, we're happy.
If that persists a node failure, this would be a good basis for
implementing the necessary databases.

I'm trying to move more and more internal routines away from
fetch_locked() to do_locked(), but because do_locked can trivially be
implemented on top of fetch_locked, this can be seen as just an
optimization minimizing use to malloc.

Volker

--
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
On Thu, Oct 26, 2017 at 01:37:21PM -0500, Christopher R. Hertel via samba-technical wrote:

> Several.  Hopefully they're helpful.
>
> - Below you mention that you removed TDB_CLEAR_IF_FIRST from the open
>   calls to locking.tdb and smbXsrv_open_global.tdb.  I did the same as
>   a test, but I also removed TDB_CLEAR_IF_FIRST from the
>   byte-range-locking and leases databases.  These two additional
>   databases are also used to maintain handle state.
>
>   Unfortunately, the change resulted in random core dumps related to
>   libtevent.

That's very surprising. Can we follow up here? libtevent should be
completely unrelated to TDB_CLEAR_IF_FIRST, and as long as the tdb is
not corrupted, we should have routines to live well with stale
records. Your crashes might point at issues where we could become
better in surviving corrupt database contents.

> - Regarding the backing store...  All four of the databases we've
>   identified are opened and accessed via the DBwrap API.  DBwrap hides
>   the differences between CTDB and TDB, and also provides for
>   "transactions".  The problem is that DBwrap is somewhat specific to
>   [C]TDB.  We may be able to work around that, though.

No, I don't see dbwrap as specific to [c]tdb. In fact, a very early
prototype that demostrated a clustered version of Samba was
dbwrap_file. This used a file per record, to be stored on a central
clustered file system. It turned out to be surprisingly fast, but
still more expensive than raw tdb. This has in the meantime been
removed, because ctdb turned out to be our way to cluster tdbs. But it
should be a pretty simple excercise to revive this. As I wrote in my
previous mail, as long as you give us fetch_locked, we're 99% there.

Volker

--
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi Ralph,
Your design doc is really very helpful!
I got server questions:
1. The first approach, i.e. “2.1.1 New dbwrap backend with support for per record persistency”
a. Is this approach coupled with CTDB?
b. Do we still need to update multiple DBs with this approach? How to guarantee atomicity if updating multiple TDBs? Looks like your idea is to set a flag when the update for persistent state completes.
“To maintain atomicity wrt to storing the FSA state, as the final step after successfully processing a persistent handle request, set the SHARE_MODE_FLAG_PERSISTENT_OPEN flag int struct share_mode_entry.flags.”
        Does that mean if this flag isn’t set for a “persistent file handle” during recovery, we should throw away all the records that’s associated with this file handle?
2. The second approach, i.e. “2.1.2 New database for persistent handles”
My understanding is we just create a single DB for storing all states related to persistent handles. Correct?
One question came to my mind is that what’s the correct behavior if updating to this persistent database fails? Should we failback to durable file handles?
With this approach, looks like we can maintain a very simple interface between the protocol and the backing store of persistent handles.

3. I’m confused when you say below fields are not needed for persistent handles:
“Looking up MS-SMB2 for uses to these elements show that none of them is required for persistent handles, because:

Open.DesiredAccess: not used in MS-SMB2, not needed
Open.ShareMode: not used in MS-SMB2, not needed
Open.FileAttributes: not used in MS-SMB2, not needed
Open.CreateDisposition: not used in MS-SMB2, not needed
…”

It’s clearly stated in MS-SMB2 that all of the above fields are included in SMB2 Create request, why are they not needed?

Looking forward for you reply and thank you very much!

--
Thanks!
Albert
 

On 10/27/17, 1:27 AM, "Ralph Böhme" <[hidden email]> wrote:

    Hi folks,
   
    On Wed, Oct 25, 2017 at 04:09:21PM +0000, Albert Guo via samba-technical wrote:
    > I made a POC based on your git repository with some hack.  This is test I did:
    > 1. Copy a large file to a CA enabled share from Windows 2012 R2 client
    > 2. Restart Samba daemons
    > The persistent file handle is reconnected successfully, and the file copy
    > continues without any error.
   
    hooray! :)
   
    Fwiw I've been exploring the design space for fur in my spare time and also took
    some notes on possible designs for the persistent storage layer and APIs. You
    can find my current design doc here:
   
    git branch: <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.samba.org_-3Fp-3Dslow_samba.git-3Ba-3Dshortlog-3Bh-3Drefs_heads_persistent-2Dhandles&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=Bk5iUseabyaWg90GPDEUCaM-L-4qcX-jcPXouyrwtGU&e=>
    git commit: <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.samba.org_-3Fp-3Dslow_samba.git-3Ba-3Dcommitdiff-3Bh-3D164ae1364fbe250f2da87a53aa7e571d6548d93f&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=kB3tKArCLk-GlIZG-Rmd8L7y4ce8afcaB-Bh7SWBkIw&e=>
    as txt: <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.samba.org_-3Fp-3Dslow_samba.git-3Ba-3Dblob-5Fplain-3Bf-3DSTEPS.txt-3Bh-3Dcee0f64c69811698679b0bf131db526549f3dd53-3Bhb-3D164ae1364fbe250f2da87a53aa7e571d6548d93f&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=6OEPmZZ_3JGlBudHinytiMJcpggGco3awWEpi8Hs2_M&e=>
    as html: <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.samba.org_-7Eslow_STEPS.html&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=C81WY4gWo8TyW18w0APU7pIRvemltsHq5c37nrMrRs8&e=>
   
    The links are a bit broken in the text version.
   
    The branch also has a bunch of torture tests for various aspects of persistent
    handles as well as some wip patches for the SMB2 frontend and the FSA
    subsystems. But most of that stuff is more akind of taking notes. Some stuff is
    from going through the Simplivity patchset (see below) and splitting it up into
    microcommits [1].
   
    For the storage backend I'm currently leaning towards the "2.1.2 New database
    for persistent handles" design and will start prototyping in my spare time. I'm
    only making slow progress as there's just too much other stuff that wants my
    attention. :)
   
    > The purpose of the POC is to identify the gaps we have for supporting persistent file handles.
   
    For upstream we mostly needed a proper storage layer that also works in the
    clustered case.
   
    One important thing that may not be obvious is that you have to protect
    disconnected persistent handles from contenting opens, cf MS-SMB2 3.3.5.9 and
    3.3.4.7. Though the latter is currently incomplete and will be updated my MS in
    the future explaining how to deal with lease breaks and disconnected PHs. I can
    share the details if you like, I haven't yet added this to my design doc
    neither.
   
    > I attached my patch in the attachment. The patch is very simple:
    >
    > 1. Remove TDB_CLEAR_IF_FIRST from open path of locking.tdb and
    >    smbXsrv_open_global.tdb
   
    Without adding transactions this has the risk of corrupting the databases in the
    event of a crash. Adding transcations has performance implications. Iirc patches
    for persistent handles were submitted to the mailing list, directly or
    indirectly, at least once. I have a munged version of the persistent handle
    patchset from Simplivity here:
   
    <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.samba.org_-3Fp-3Dslow_samba.git-3Ba-3Dshortlog-3Bh-3Drefs_heads_persistent-2Dsimplivity&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=SWr75Nnn-4Bqlfa7rNICav4K1dOflUuU2osvRzHKwLo&e=>
   
    Hth!
   
    -slow
   
    [1] <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.samba.org_-3Fp-3Dslow_samba.git-3Ba-3Dshortlog-3Bh-3Drefs_heads_persistent-2Dsimplivity-2Dsteps&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=767MQnbxEF31tYabyaVZo-SF6Yj-AN2ZwRFYAi50SyY&e=>
   
    --
    Ralph Boehme, Samba Team       https://urldefense.proofpoint.com/v2/url?u=https-3A__samba.org_&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=WKbGGPFCZ6ChDd4wI3CmxHPvVYLra1wf3KqaYgh2_68&e=
    Samba Developer, SerNet GmbH   https://urldefense.proofpoint.com/v2/url?u=https-3A__sernet.de_en_samba_&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oBV_s4chh_toxw_syDSqBdc9MbFHdMf73pR7m-8ru6A&s=1Dj8hdoPplrgGYnU99BiGNQktaKNtHPZNmW2kk8X9uI&e=
   

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
On 10/30/2017 11:31 AM, Albert Guo wrote:
> Hi Ralph,
> Your design doc is really very helpful!
> I got server questions:
> 1. The first approach, i.e. “2.1.1 New dbwrap backend with support for per record persistency”
> a. Is this approach coupled with CTDB?

The model that Ralph and I (and others) discussed last September at the
Storage Developers Conference (SDC) was, indeed, coupled on CTDB.  I am sure
Ralph will correct me if I misstate, but as I understand it there was a
strong desire within the Samba Team to ensure that Samba itself--without any
FS-specific modifications--could provide Persistent Handle support.

As you know, I was advocating for a method that would allow vendor-specific
plug-ins.  It was agreed that we could set that as a longer-term goal.

This is, however, why I am keeping the protocol work separate from the
backing store modifications.  I believe that we can get the protocol piece
working nicely, add CTDB-based backing store, and then work on pluggable
modules.

> b. Do we still need to update multiple DBs with this approach? How to guarantee atomicity if updating multiple TDBs? Looks like your idea is to set a flag when the update for persistent state completes.

The design proposed at the SDC was to to augment the existing setup.  The
existing [C]TDB databases for locks, leases, and handles would still be
used.  Whenever a record associated with a Persistent Handle would be
changed, an *additional* database (a persistent [C]TDB database) would be
updated as well.  Only records related to a Persistent Handle would be
updated this way.

Whenever Persistent Handle data is retrieved, the persistent backing store
will be checked first.  If the entire cluster goes down, then when the
databases were re-opened the first step will be to clear the volatile
databases and then reload them from the persistent backing store, thus
allowing the handles to be re-opened as Persistent Handles and recovered.

I look forward to comments from others on the rest of your questions.

Chris -)-----

> “To maintain atomicity wrt to storing the FSA state, as the final step after successfully processing a persistent handle request, set the SHARE_MODE_FLAG_PERSISTENT_OPEN flag int struct share_mode_entry.flags.”
> Does that mean if this flag isn’t set for a “persistent file handle” during recovery, we should throw away all the records that’s associated with this file handle?
> 2. The second approach, i.e. “2.1.2 New database for persistent handles”
> My understanding is we just create a single DB for storing all states related to persistent handles. Correct?
> One question came to my mind is that what’s the correct behavior if updating to this persistent database fails? Should we failback to durable file handles?
> With this approach, looks like we can maintain a very simple interface between the protocol and the backing store of persistent handles.
>
> 3. I’m confused when you say below fields are not needed for persistent handles:
> “Looking up MS-SMB2 for uses to these elements show that none of them is required for persistent handles, because:
>
> Open.DesiredAccess: not used in MS-SMB2, not needed
> Open.ShareMode: not used in MS-SMB2, not needed
> Open.FileAttributes: not used in MS-SMB2, not needed
> Open.CreateDisposition: not used in MS-SMB2, not needed
> …”
>
> It’s clearly stated in MS-SMB2 that all of the above fields are included in SMB2 Create request, why are they not needed?
>
> Looking forward for you reply and thank you very much!
>

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
On Mon, Oct 30, 2017 at 12:04:29PM -0500, Christopher R. Hertel via samba-technical wrote:
> The model that Ralph and I (and others) discussed last September at the
> Storage Developers Conference (SDC) was, indeed, coupled on CTDB.  I am sure
> Ralph will correct me if I misstate, but as I understand it there was a
> strong desire within the Samba Team to ensure that Samba itself--without any
> FS-specific modifications--could provide Persistent Handle support.

The main reason for this is end-to-end testability without any special
file system support. We might of course compromise on efficiency, but
the functionality must (IMHO) be testable on a pretty plain posix
system. This is also in the interest of any OEM, because otherwise we
(the Samba Team) will definitely break the infrastructure around
persistent file handles sooner or later, and OEMs usually notice only
months later when they pull in a fresh upstream release.

Volker

--
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
On 10/30/2017 03:48 PM, Volker Lendecke wrote:

> On Mon, Oct 30, 2017 at 12:04:29PM -0500, Christopher R. Hertel via samba-technical wrote:
>> The model that Ralph and I (and others) discussed last September at the
>> Storage Developers Conference (SDC) was, indeed, coupled on CTDB.  I am sure
>> Ralph will correct me if I misstate, but as I understand it there was a
>> strong desire within the Samba Team to ensure that Samba itself--without any
>> FS-specific modifications--could provide Persistent Handle support.
>
> The main reason for this is end-to-end testability without any special
> file system support. We might of course compromise on efficiency, but
> the functionality must (IMHO) be testable on a pretty plain posix
> system. This is also in the interest of any OEM, because otherwise we
> (the Samba Team) will definitely break the infrastructure around
> persistent file handles sooner or later, and OEMs usually notice only
> months later when they pull in a fresh upstream release.

...and that makes perfect sense.

Note also that with such an implementation you could even create Persistent
Handles on non-clustered file systems, such as XFS or EXT4.  This would be
useful, for example, on systems that can reboot quickly after a catastrophic
failure, or in those rare cases in which an smbd instance crashes.

However, several vendors (including the one that employs me) have expressed
the desire to be able to integrate Persistent Handle support into their
existing cluster file system infrastructure.

Chris -)-----

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
On 2017-10-30 at 12:04 -0500, Christopher R. Hertel wrote:

> On 10/30/2017 11:31 AM, Albert Guo wrote:
> > Hi Ralph,
> > Your design doc is really very helpful!
> > I got server questions:
> > 1. The first approach, i.e. “2.1.1 New dbwrap backend with support for per record persistency”
> > a. Is this approach coupled with CTDB?
>
> The model that Ralph and I (and others) discussed last September at the
> Storage Developers Conference (SDC) was, indeed, coupled on CTDB.  I am sure
> Ralph will correct me if I misstate, but as I understand it there was a
> strong desire within the Samba Team to ensure that Samba itself--without any
> FS-specific modifications--could provide Persistent Handle support.
This is true, but to my understanding as a new dbwrap-backend it
will not be exclusively coupled with CTDB. In a clustered case
(samba+ctdb) it would use CTDB, in a non-clustered case, plain TDB.

Cheers - Michael


> As you know, I was advocating for a method that would allow vendor-specific
> plug-ins.  It was agreed that we could set that as a longer-term goal.
>
> This is, however, why I am keeping the protocol work separate from the
> backing store modifications.  I believe that we can get the protocol piece
> working nicely, add CTDB-based backing store, and then work on pluggable
> modules.
>
> > b. Do we still need to update multiple DBs with this approach? How to guarantee atomicity if updating multiple TDBs? Looks like your idea is to set a flag when the update for persistent state completes.
>
> The design proposed at the SDC was to to augment the existing setup.  The
> existing [C]TDB databases for locks, leases, and handles would still be
> used.  Whenever a record associated with a Persistent Handle would be
> changed, an *additional* database (a persistent [C]TDB database) would be
> updated as well.  Only records related to a Persistent Handle would be
> updated this way.
>
> Whenever Persistent Handle data is retrieved, the persistent backing store
> will be checked first.  If the entire cluster goes down, then when the
> databases were re-opened the first step will be to clear the volatile
> databases and then reload them from the persistent backing store, thus
> allowing the handles to be re-opened as Persistent Handles and recovered.
>
> I look forward to comments from others on the rest of your questions.
>
> Chris -)-----
>
> > “To maintain atomicity wrt to storing the FSA state, as the final step after successfully processing a persistent handle request, set the SHARE_MODE_FLAG_PERSISTENT_OPEN flag int struct share_mode_entry.flags.”
> > Does that mean if this flag isn’t set for a “persistent file handle” during recovery, we should throw away all the records that’s associated with this file handle?
> > 2. The second approach, i.e. “2.1.2 New database for persistent handles”
> > My understanding is we just create a single DB for storing all states related to persistent handles. Correct?
> > One question came to my mind is that what’s the correct behavior if updating to this persistent database fails? Should we failback to durable file handles?
> > With this approach, looks like we can maintain a very simple interface between the protocol and the backing store of persistent handles.
> >
> > 3. I’m confused when you say below fields are not needed for persistent handles:
> > “Looking up MS-SMB2 for uses to these elements show that none of them is required for persistent handles, because:
> >
> > Open.DesiredAccess: not used in MS-SMB2, not needed
> > Open.ShareMode: not used in MS-SMB2, not needed
> > Open.FileAttributes: not used in MS-SMB2, not needed
> > Open.CreateDisposition: not used in MS-SMB2, not needed
> > …”
> >
> > It’s clearly stated in MS-SMB2 that all of the above fields are included in SMB2 Create request, why are they not needed?
> >
> > Looking forward for you reply and thank you very much!
> >

signature.asc (169 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi Albert,

On Mon, Oct 30, 2017 at 04:31:40PM +0000, Albert Guo wrote:
> Your design doc is really very helpful!
> I got server questions:
> 1. The first approach, i.e. “2.1.1 New dbwrap backend with support for per record persistency”
> a. Is this approach coupled with CTDB?

No. It works with and without ctdb.

> b. Do we still need to update multiple DBs with this approach?

Sure.

> How to guarantee atomicity if updating multiple TDBs? Looks like your idea is
> to set a flag when the update for persistent state completes.

To get something managable it would be helpful to merge locking.tdb and
brlock.tdb, cf

https://lists.samba.org/archive/samba-technical/2017-October/123583.html
https://lists.samba.org/archive/samba-technical/2017-November/123628.html

Both tdbs merged we just have one store of the persistent FSA state in the
locking.tdb.

> Does that mean if this flag isn’t set for a “persistent file handle” during
> recovery, we should throw away all the records that’s associated with this
> file handle?

Not quite. We need a reliable cleanup like the scavenger for durable opens. I
haven't yet really turned to this problem.

> 2. The second approach, i.e. “2.1.2 New database for persistent handles”

After some more tinkering I came to the conclustion that this approach doesn't
work. I've removed it from the design doc.

> My understanding is we just create a single DB for storing all states related
> to persistent handles. Correct?  One question came to my mind is that what’s
> the correct behavior if updating to this persistent database fails? Should we
> failback to durable file handles?  With this approach, looks like we can
> maintain a very simple interface between the protocol and the backing store of
> persistent handles.
>
> 3. I’m confused when you say below fields are not needed for persistent
> handles: “Looking up MS-SMB2 for uses to these elements show that none of them
> is required for persistent handles, because:
>
> Open.DesiredAccess: not used in MS-SMB2, not needed
> Open.ShareMode: not used in MS-SMB2, not needed
> Open.FileAttributes: not used in MS-SMB2, not needed
> Open.CreateDisposition: not used in MS-SMB2, not needed
> …”
>
> It’s clearly stated in MS-SMB2 that all of the above fields are included in
> SMB2 Create request, why are they not needed?

It's not needed at the file-handle layer in smbXsrv_open_global.tdb.

-slow

--
Ralph Boehme, Samba Team       https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
Hi Ralph
    > 2. The second approach, i.e. “2.1.2 New database for persistent handles”
   
    After some more tinkering I came to the conclustion that this approach doesn't
    work. I've removed it from the design doc.
 Could you give me a bit more explanation on why you think this approach doesn’t work?  This was attractive to me as well on an initial thought because it really keeps the interface between Samba and backend storage radically simple.

--
Thanks!
Albert
 

On 11/8/17, 6:36 AM, "Ralph Böhme" <[hidden email]> wrote:

    Hi Albert,
   
    On Mon, Oct 30, 2017 at 04:31:40PM +0000, Albert Guo wrote:
    > Your design doc is really very helpful!
    > I got server questions:
    > 1. The first approach, i.e. “2.1.1 New dbwrap backend with support for per record persistency”
    > a. Is this approach coupled with CTDB?
   
    No. It works with and without ctdb.
   
    > b. Do we still need to update multiple DBs with this approach?
   
    Sure.
   
    > How to guarantee atomicity if updating multiple TDBs? Looks like your idea is
    > to set a flag when the update for persistent state completes.
   
    To get something managable it would be helpful to merge locking.tdb and
    brlock.tdb, cf
   
    https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.samba.org_archive_samba-2Dtechnical_2017-2DOctober_123583.html&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oajnaRk4e-b4sNByrFCveWF2ie9FyKHILo1_kyPHy74&s=Yb-idzKhZOml-27g-X4X4qOUQPlNkqJT9UhuCi3HrIA&e=
    https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.samba.org_archive_samba-2Dtechnical_2017-2DNovember_123628.html&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oajnaRk4e-b4sNByrFCveWF2ie9FyKHILo1_kyPHy74&s=MnP4MvaJqpSuPZjbmL-75U3fo1MiswZcf_DDTSO4pjU&e=
   
    Both tdbs merged we just have one store of the persistent FSA state in the
    locking.tdb.
   
    > Does that mean if this flag isn’t set for a “persistent file handle” during
    > recovery, we should throw away all the records that’s associated with this
    > file handle?
   
    Not quite. We need a reliable cleanup like the scavenger for durable opens. I
    haven't yet really turned to this problem.
   
    > 2. The second approach, i.e. “2.1.2 New database for persistent handles”
   
    After some more tinkering I came to the conclustion that this approach doesn't
    work. I've removed it from the design doc.
   
    > My understanding is we just create a single DB for storing all states related
    > to persistent handles. Correct?  One question came to my mind is that what’s
    > the correct behavior if updating to this persistent database fails? Should we
    > failback to durable file handles?  With this approach, looks like we can
    > maintain a very simple interface between the protocol and the backing store of
    > persistent handles.
    >
    > 3. I’m confused when you say below fields are not needed for persistent
    > handles: “Looking up MS-SMB2 for uses to these elements show that none of them
    > is required for persistent handles, because:
    >
    > Open.DesiredAccess: not used in MS-SMB2, not needed
    > Open.ShareMode: not used in MS-SMB2, not needed
    > Open.FileAttributes: not used in MS-SMB2, not needed
    > Open.CreateDisposition: not used in MS-SMB2, not needed
    > …”
    >
    > It’s clearly stated in MS-SMB2 that all of the above fields are included in
    > SMB2 Create request, why are they not needed?
   
    It's not needed at the file-handle layer in smbXsrv_open_global.tdb.
   
    -slow
   
    --
    Ralph Boehme, Samba Team       https://urldefense.proofpoint.com/v2/url?u=https-3A__samba.org_&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oajnaRk4e-b4sNByrFCveWF2ie9FyKHILo1_kyPHy74&s=akBJMlzvHkZwTTwIcNwnIScz-4zzk-oFyYbDUEqjniw&e=
    Samba Developer, SerNet GmbH   https://urldefense.proofpoint.com/v2/url?u=https-3A__sernet.de_en_samba_&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=wpZz2HyaI5SCm3Td3B2EHA&m=oajnaRk4e-b4sNByrFCveWF2ie9FyKHILo1_kyPHy74&s=yWWgBRyPzURqzdEylitMPE0VLe4GFxjyW81So67jslY&e=
   

Reply | Threaded
Open this post in threaded view
|

Re: Work on SMB3 persistent handles

Samba - samba-technical mailing list
Hi Albert,

On Thu, Nov 09, 2017 at 07:46:29AM +0000, Albert Guo wrote:
> > 2. The second approach, i.e. “2.1.2 New database for persistent handles”
> >
> > After some more tinkering I came to the conclustion that this approach doesn't
> > work. I've removed it from the design doc.

> Could you give me a bit more explanation on why you think this approach
> doesn’t work?

Because storing the (persistent) handle state is only one part, the other one is
the FSA state.

Without extensive filesystem support with a lower level OS API for persistent
handles and a complete redesign of our durable and future persistent handle
support on top of that, we have to implement the required FSA state guarantees
in Samba and in clustered Samba.

-slow

--
Ralph Boehme, Samba Team       https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/

12