ls internal API works faster than smb_stat

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ls internal API works faster than smb_stat

shravan a
Hi,


LibSmbClient exposes an api  *int **smbc_opendir*(*const* *char* *durl) which
accepts the parent folder path and fetches metadata about all its immediate
children and returns the directory handle to the parent folder. The api
*struct* smbc_dirent **smbc_readdir*(*unsigned* *int* dh) accepts this
directory handle(dh) and returns metadata as a structure (smbc_dirent) that
contains limited info such as smbc_type, dirlen, commentlen,comment,
namelen, name. Repetitive calls to this API fetches the next child metadata
in sequence and returns the structure. Unfortunately, it doesn’t give any
information regarding the resource size(important in case of files),
create/modified date time. To get these info, this library exposes two
API’s,

*int* *smbc_stat*(*const* *char* *url, *struct* stat *st) - that accepts
the url and

*int **smbc_fstat*(*int* fd, *struct* stat *st) that accepts the file
descriptor.

     Both returns the required metadata as a structure ‘stat’ which
contains all the time stamps(create, modified, access) and the size.  Both
needs to be invoked once for every child resource within the parent
resource. smbc_stat() is found to be an expensive task since it makes an
extra SMB call to the resource , while smbc_fstat() is found to be faster
since it accepts the file descriptor. But this still requires the file
descriptor which can be obtained only via another smbc_opendir(resource
url) or smbc_open(resource url).

  Another API I tried out is –

*Int **smbc_getxattr*(*const* *char* *fname,

              *const* *char* *name,

              *const* *void* *value,

              size_t size)

     This accepts the resourceurl, and the attribute name, returns the
attribute value. But this API also needs to be invoked once per child
resource within the parent folder which again is expensive.



This extra API call for every child resource still makes the whole process
expensive.


However, when I open a smb session from the shell "ls" command performs
very fast. The function that "ls" internally calls in "list_dir".

I would like to write a public API that would call the internal
implementation of "ls". Thoughts?


Thanks!

Shravan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ls internal API works faster than smb_stat

Jeremy Allison
On Tue, Feb 14, 2017 at 11:11:10PM +0530, shravan a wrote:

> Hi,
>
>
> LibSmbClient exposes an api  *int **smbc_opendir*(*const* *char* *durl) which
> accepts the parent folder path and fetches metadata about all its immediate
> children and returns the directory handle to the parent folder. The api
> *struct* smbc_dirent **smbc_readdir*(*unsigned* *int* dh) accepts this
> directory handle(dh) and returns metadata as a structure (smbc_dirent) that
> contains limited info such as smbc_type, dirlen, commentlen,comment,
> namelen, name. Repetitive calls to this API fetches the next child metadata
> in sequence and returns the structure. Unfortunately, it doesn’t give any
> information regarding the resource size(important in case of files),
> create/modified date time. To get these info, this library exposes two
> API’s,
>
> *int* *smbc_stat*(*const* *char* *url, *struct* stat *st) - that accepts
> the url and
>
> *int **smbc_fstat*(*int* fd, *struct* stat *st) that accepts the file
> descriptor.
>
>      Both returns the required metadata as a structure ‘stat’ which
> contains all the time stamps(create, modified, access) and the size.  Both
> needs to be invoked once for every child resource within the parent
> resource. smbc_stat() is found to be an expensive task since it makes an
> extra SMB call to the resource , while smbc_fstat() is found to be faster
> since it accepts the file descriptor. But this still requires the file
> descriptor which can be obtained only via another smbc_opendir(resource
> url) or smbc_open(resource url).
>
>   Another API I tried out is –
>
> *Int **smbc_getxattr*(*const* *char* *fname,
>
>               *const* *char* *name,
>
>               *const* *void* *value,
>
>               size_t size)
>
>      This accepts the resourceurl, and the attribute name, returns the
> attribute value. But this API also needs to be invoked once per child
> resource within the parent folder which again is expensive.
>
>
>
> This extra API call for every child resource still makes the whole process
> expensive.
>
>
> However, when I open a smb session from the shell "ls" command performs
> very fast. The function that "ls" internally calls in "list_dir".
>
> I would like to write a public API that would call the internal
> implementation of "ls". Thoughts?

What you need is readdirplus - a readdir that also returns
stat info.

Add a new call to libsmbclient called smbc_readdirplus().

The internal call to cli_list gets most of the info you
want (except for the EA values) so you will need to change
the SMBC_opendir_ctx() function to use an internal callback
that does more with the info returned by cli_list.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ls internal API works faster than smb_stat

Samba - samba-technical mailing list
Hi all,

Raised a PR which addresses this.
https://github.com/samba-team/samba/pull/94

P.S notification script is broken hence sending the mail

On Wed, Feb 15, 2017 at 12:17 AM, Jeremy Allison <[hidden email]> wrote:

> On Tue, Feb 14, 2017 at 11:11:10PM +0530, shravan a wrote:
> > Hi,
> >
> >
> > LibSmbClient exposes an api  *int **smbc_opendir*(*const* *char* *durl)
> which
> > accepts the parent folder path and fetches metadata about all its
> immediate
> > children and returns the directory handle to the parent folder. The api
> > *struct* smbc_dirent **smbc_readdir*(*unsigned* *int* dh) accepts this
> > directory handle(dh) and returns metadata as a structure (smbc_dirent)
> that
> > contains limited info such as smbc_type, dirlen, commentlen,comment,
> > namelen, name. Repetitive calls to this API fetches the next child
> metadata
> > in sequence and returns the structure. Unfortunately, it doesn’t give any
> > information regarding the resource size(important in case of files),
> > create/modified date time. To get these info, this library exposes two
> > API’s,
> >
> > *int* *smbc_stat*(*const* *char* *url, *struct* stat *st) - that accepts
> > the url and
> >
> > *int **smbc_fstat*(*int* fd, *struct* stat *st) that accepts the file
> > descriptor.
> >
> >      Both returns the required metadata as a structure ‘stat’ which
> > contains all the time stamps(create, modified, access) and the size.
> Both
> > needs to be invoked once for every child resource within the parent
> > resource. smbc_stat() is found to be an expensive task since it makes an
> > extra SMB call to the resource , while smbc_fstat() is found to be faster
> > since it accepts the file descriptor. But this still requires the file
> > descriptor which can be obtained only via another smbc_opendir(resource
> > url) or smbc_open(resource url).
> >
> >   Another API I tried out is –
> >
> > *Int **smbc_getxattr*(*const* *char* *fname,
> >
> >               *const* *char* *name,
> >
> >               *const* *void* *value,
> >
> >               size_t size)
> >
> >      This accepts the resourceurl, and the attribute name, returns the
> > attribute value. But this API also needs to be invoked once per child
> > resource within the parent folder which again is expensive.
> >
> >
> >
> > This extra API call for every child resource still makes the whole
> process
> > expensive.
> >
> >
> > However, when I open a smb session from the shell "ls" command performs
> > very fast. The function that "ls" internally calls in "list_dir".
> >
> > I would like to write a public API that would call the internal
> > implementation of "ls". Thoughts?
>
> What you need is readdirplus - a readdir that also returns
> stat info.
>
> Add a new call to libsmbclient called smbc_readdirplus().
>
> The internal call to cli_list gets most of the info you
> want (except for the EA values) so you will need to change
> the SMBC_opendir_ctx() function to use an internal callback
> that does more with the info returned by cli_list.
>
>


--

*RegardsPuran Chand*
*Member of Technical Staff*
Loading...