The number of stat calls caused by a single CREATE request

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

The number of stat calls caused by a single CREATE request

Samba - samba-technical mailing list
Hi folks,

I have been looking at the performance of CREATE and am seeing as many
as 15 stat-like requests for a single create.

Now, some of those are the result of our VFS module trying to get
CREATE TIME, but I have the stack-trace info for 11 of them and it is
not pretty:

1. Four of them result from calls to smbd_check_access_rights (a
couple it seems are for calculating the MxAc Extra Create Parameter.)
2. Two of them at least result from vfs_GetWd
3. The majority of them are for "." while two are for the actual path
that is ".".

It is amazing to me that we do not use VALID_STAT to handle that, and
I have a VALID_STAT use in the VFS module  but it seems Samba is
creating fresh struct smb_filenames all over the place for ".".

--
Regards,
Richard Sharpe
(何以解憂?唯有杜康。--曹操)

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The number of stat calls caused by a single CREATE request

Samba - samba-technical mailing list
On Thu, 2017-07-27 at 17:37 -0700, Richard Sharpe via samba-technical wrote:

> Hi folks,
>
> I have been looking at the performance of CREATE and am seeing as many
> as 15 stat-like requests for a single create.
>
> Now, some of those are the result of our VFS module trying to get
> CREATE TIME, but I have the stack-trace info for 11 of them and it is
> not pretty:
>
> 1. Four of them result from calls to smbd_check_access_rights (a
> couple it seems are for calculating the MxAc Extra Create Parameter.)
> 2. Two of them at least result from vfs_GetWd
> 3. The majority of them are for "." while two are for the actual path
> that is ".".

Can it be somehow related to the series of patches that went in for fixing CVE-2017-2619? Yes,
vfs_GetWd() results in stat call and introduction of non_widelink_open() and process_symlink_open()
via those CVE fixes adds to the number of invocations of vfs_GetWd(). Moreover open_dir_safely() now
does an extra check using check_name() on "."

Please correct me if I am wrong.

> It is amazing to me that we do not use VALID_STAT to handle that, and
> I have a VALID_STAT use in the VFS module  but it seems Samba is
> creating fresh struct smb_filenames all over the place for ".".
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The number of stat calls caused by a single CREATE request

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Howdy Richard,

On Thu, Jul 27, 2017 at 05:37:45PM -0700, Richard Sharpe via samba-technical wrote:

> I have been looking at the performance of CREATE and am seeing as many
> as 15 stat-like requests for a single create.
>
> Now, some of those are the result of our VFS module trying to get
> CREATE TIME, but I have the stack-trace info for 11 of them and it is
> not pretty:
>
> 1. Four of them result from calls to smbd_check_access_rights (a
> couple it seems are for calculating the MxAc Extra Create Parameter.)
> 2. Two of them at least result from vfs_GetWd
> 3. The majority of them are for "." while two are for the actual path
> that is ".".
>
> It is amazing to me that we do not use VALID_STAT to handle that, and
> I have a VALID_STAT use in the VFS module  but it seems Samba is
> creating fresh struct smb_filenames all over the place for ".".

yes, but can you check with perf or similar tools whether this still shows up in
profiles? Once the metadata is cached after the first access subsequent stat
calls shouldn't hurt too badly. When I worked on this last year I was able to
get rid of a few redundant stat calls, but I left some for you. ;)

Cheerio!
-slow

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The number of stat calls caused by a single CREATE request

Samba - samba-technical mailing list
On Wed, Aug 09, 2017 at 03:15:16PM +0200, Ralph Böhme via samba-technical wrote:

> Howdy Richard,
>
> On Thu, Jul 27, 2017 at 05:37:45PM -0700, Richard Sharpe via samba-technical wrote:
> > I have been looking at the performance of CREATE and am seeing as many
> > as 15 stat-like requests for a single create.
> >
> > Now, some of those are the result of our VFS module trying to get
> > CREATE TIME, but I have the stack-trace info for 11 of them and it is
> > not pretty:
> >
> > 1. Four of them result from calls to smbd_check_access_rights (a
> > couple it seems are for calculating the MxAc Extra Create Parameter.)
> > 2. Two of them at least result from vfs_GetWd
> > 3. The majority of them are for "." while two are for the actual path
> > that is ".".
> >
> > It is amazing to me that we do not use VALID_STAT to handle that, and
> > I have a VALID_STAT use in the VFS module  but it seems Samba is
> > creating fresh struct smb_filenames all over the place for ".".
>
> yes, but can you check with perf or similar tools whether this still shows up in
> profiles? Once the metadata is cached after the first access subsequent stat
> calls shouldn't hurt too badly. When I worked on this last year I was able to
> get rid of a few redundant stat calls, but I left some for you. ;)

My guess is that this hurts VFS vendors who have custom
VFS modules that don't cache the stat values, but do some
RPC-calls on each one. We should certainly tidy up these
in our code if we can.

Loading...