Issues accessing ZFS-shares on Linux

[Samba - General mailing list]

Hi,

I am having a really strange problem with my Samba shares on Debian
Buster. None of the users can access any shares, which reside on
ZFS-filesystem. Any other share works just fine. For example, if I
create a normal folder to /home with same permissions and replace a
ZFS-share with that, it works fine.

When accessing any ZFS-shares the following error is recorded:
Jan 08 22:39:56 punishedkorppu smbd[27893]: [2018/01/08 22:39:56.075343,
 0] ../source3/smbd/service.c:774(make_connection_snum)
Jan 08 22:39:56 punishedkorppu smbd[27893]:   canonicalize_connect_path
failed for service rex, path /tank/rex

The system in use is Debian Buster with Samba 2:4.7.3+dfsg-1 and ZoL
0.7.4-1 from Debian repo. Selinux is disabled and apparmor too.

Any ideas what might be wrong with this setup?


--- information ---

The mountpoint really is /tank/rex according to zfs list. It can also be
accessed by any user locally.

NAME                        USED  AVAIL  REFER  MOUNTPOINT
tank                       7.74T  7.24T   222K  /tank
tank/homedir               96.8G  7.24T   222K  /tank/homedir
tank/mei                   5.98G  7.24T  5.98G  /tank/mei
tank/rex                   7.64T  7.24T  11.1G  /tank/rex

Folder permissions ls -hal /tank
drwxr-xr-x  7 root  root    7 Jan  8 19:04 ./
drwxr-xr-x  1 root  root  284 Jan  8 21:07 ../
drwx------  2 rex   root    2 Jul 30 06:22 davtmp/
drwx---r-x 13 root  root   13 Sep  6 23:13 homedir/
drwxrwx--- 11 root  mei    11 Dec 24 18:35 mei/
drwxrwxrwx 12 rex   rex    20 Dec 24 19:09 rex/

smb.conf from testparm -s:

# Global parameters
[global]
        async smb echo handler = Yes
        dns proxy = No
        log file = /var/log/samba/log.%m
        map to guest = Bad User
        max log size = 1000
        obey pam restrictions = Yes
        pam password change = Yes
        panic action = /usr/share/samba/panic-action %d
        passwd chat = *Enter\snew\s*\spassword:* %n\n
*Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
        passwd program = /usr/bin/passwd %u
        server role = standalone server
        unix password sync = Yes
        idmap config * : backend = tdb
        aio read size = 1
        aio write size = 1
        use sendfile = Yes


[rex]
        comment = REX
        create mask = 0660
        force create mode = 0660
        force directory mode = 0770
        force group = rex
        path = /tank/rex
        read only = No
        valid users = @rex


[mei]
        comment = MEI
        create mask = 0660
        force create mode = 0660
        force directory mode = 0770
        force group = mei
        path = /tank/mei
        read only = No
        valid users = @mei


[homedir]
        comment = Homedir
        create mask = 0600
        force directory mode = 0700
        path = /tank/homedir
        read only = No

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

[Samba - General mailing list]

On Mon, Jan 08, 2018 at 11:06:37PM +0200, John Doe via samba wrote:

> Hi,
>
> I am having a really strange problem with my Samba shares on Debian
> Buster. None of the users can access any shares, which reside on
> ZFS-filesystem. Any other share works just fine. For example, if I
> create a normal folder to /home with same permissions and replace a
> ZFS-share with that, it works fine.
>
> When accessing any ZFS-shares the following error is recorded:
> Jan 08 22:39:56 punishedkorppu smbd[27893]: [2018/01/08 22:39:56.075343,
>  0] ../source3/smbd/service.c:774(make_connection_snum)
> Jan 08 22:39:56 punishedkorppu smbd[27893]:   canonicalize_connect_path
> failed for service rex, path /tank/rex
>
> The system in use is Debian Buster with Samba 2:4.7.3+dfsg-1 and ZoL
> 0.7.4-1 from Debian repo. Selinux is disabled and apparmor too.
>
> Any ideas what might be wrong with this setup?

Do an strace. canonicalize_connect_path() is a very simple
function, but my guess is the call to SMB_VFS_REALPATH()
is failing for some reason on your ZFS filesystem.

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

[Samba - General mailing list]

I added one testshare /home/testijako and connected to it with the same
credentials as I would connect to ZFS-shares. Then I did the strace to
that particular PID and tried connecting to one ZFS-share. There was
indeed an error which might have something to do with this issue:

Line 2001: lstat("/tank/rex", 0x7fff1f6fb2c0)      = -1 ENOENT (No such
file or directory)

Im sure that folder exists (files have been omitted):
aurinko@punishedkorppu:~$ cd /tank/rex
aurinko@punishedkorppu:/tank/rex$ ls -hal
total 32M
drwxrwxrwx   12 rex     rex       20 Dec 24 19:09  .

Full stracefile can be found here:
https://pastebin.com/raw/i5BpVPuY


Jeremy Allison via samba kirjoitti 9.1.2018 klo 0:57:

> On Mon, Jan 08, 2018 at 11:06:37PM +0200, John Doe via samba wrote:
>> Hi,
>>
>> I am having a really strange problem with my Samba shares on Debian
>> Buster. None of the users can access any shares, which reside on
>> ZFS-filesystem. Any other share works just fine. For example, if I
>> create a normal folder to /home with same permissions and replace a
>> ZFS-share with that, it works fine.
>>
>> When accessing any ZFS-shares the following error is recorded:
>> Jan 08 22:39:56 punishedkorppu smbd[27893]: [2018/01/08 22:39:56.075343,
>>  0] ../source3/smbd/service.c:774(make_connection_snum)
>> Jan 08 22:39:56 punishedkorppu smbd[27893]:   canonicalize_connect_path
>> failed for service rex, path /tank/rex
>>
>> The system in use is Debian Buster with Samba 2:4.7.3+dfsg-1 and ZoL
>> 0.7.4-1 from Debian repo. Selinux is disabled and apparmor too.
>>
>> Any ideas what might be wrong with this setup?
>
> Do an strace. canonicalize_connect_path() is a very simple
> function, but my guess is the call to SMB_VFS_REALPATH()
> is failing for some reason on your ZFS filesystem.
>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

[Samba - General mailing list]

On Tue, Jan 09, 2018 at 07:57:41AM +0200, John Doe via samba wrote:

> I added one testshare /home/testijako and connected to it with the same
> credentials as I would connect to ZFS-shares. Then I did the strace to
> that particular PID and tried connecting to one ZFS-share. There was
> indeed an error which might have something to do with this issue:
>
> Line 2001: lstat("/tank/rex", 0x7fff1f6fb2c0)      = -1 ENOENT (No such
> file or directory)
>
> Im sure that folder exists (files have been omitted):
> aurinko@punishedkorppu:~$ cd /tank/rex
> aurinko@punishedkorppu:/tank/rex$ ls -hal
> total 32M
> drwxrwxrwx   12 rex     rex       20 Dec 24 19:09  .

Well that looks like your issue. You need to figure
out why  lstat("/tank/rex") is failing on an existing
directory.

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

[Samba - General mailing list]

I just noticed that by running by commands /usr/sbin/smbd -D or
/usr/sbin/smbd -i without systemd's unit, all shares work perfectly so
the problem must then be somehow related to systemd.. Let the testing
continue..

I also tested what happens if I comment out everything and just use
ExecStart=/usr/sbin/smbd -D as that command worked on the console. That
did not help.

For the record, this is the default unit-file:
[Unit]
Description=Samba SMB Daemon
Documentation=man:smbd(8) man:samba(7) man:smb.conf(5)
After=network.target nmbd.service winbind.service

[Service]
Type=notify
NotifyAccess=all
PIDFile=/var/run/samba/smbd.pid
LimitNOFILE=16384
EnvironmentFile=-/etc/default/samba
ExecStart=/usr/sbin/smbd $SMBDOPTIONS
ExecReload=/bin/kill -HUP $MAINPID
LimitCORE=infinity

[Install]
WantedBy=multi-user.target

Jeremy Allison via samba kirjoitti 9.1.2018 klo 21:47:

> On Tue, Jan 09, 2018 at 07:57:41AM +0200, John Doe via samba wrote:
>> I added one testshare /home/testijako and connected to it with the same
>> credentials as I would connect to ZFS-shares. Then I did the strace to
>> that particular PID and tried connecting to one ZFS-share. There was
>> indeed an error which might have something to do with this issue:
>>
>> Line 2001: lstat("/tank/rex", 0x7fff1f6fb2c0)      = -1 ENOENT (No such
>> file or directory)
>>
>> Im sure that folder exists (files have been omitted):
>> aurinko@punishedkorppu:~$ cd /tank/rex
>> aurinko@punishedkorppu:/tank/rex$ ls -hal
>> total 32M
>> drwxrwxrwx   12 rex     rex       20 Dec 24 19:09  .
>
> Well that looks like your issue. You need to figure
> out why  lstat("/tank/rex") is failing on an existing
> directory.
>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

[Samba - General mailing list]

On Wed, Jan 10, 2018 at 04:51:23AM +0200, John Doe via samba wrote:

> I just noticed that by running by commands /usr/sbin/smbd -D or
> /usr/sbin/smbd -i without systemd's unit, all shares work perfectly so
> the problem must then be somehow related to systemd.. Let the testing
> continue..
>
> I also tested what happens if I comment out everything and just use
> ExecStart=/usr/sbin/smbd -D as that command worked on the console. That
> did not help.
>
> For the record, this is the default unit-file:
> [Unit]
> Description=Samba SMB Daemon
> Documentation=man:smbd(8) man:samba(7) man:smb.conf(5)
> After=network.target nmbd.service winbind.service
>
> [Service]
> Type=notify
> NotifyAccess=all
> PIDFile=/var/run/samba/smbd.pid
> LimitNOFILE=16384
> EnvironmentFile=-/etc/default/samba
> ExecStart=/usr/sbin/smbd $SMBDOPTIONS
> ExecReload=/bin/kill -HUP $MAINPID
> LimitCORE=infinity
>
> [Install]
> WantedBy=multi-user.target

Oh how strange. Must be something to do with systemd's mount
facility I'd guess.

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

[Samba - General mailing list]

I think it may have something to do with my disks being encrypted. This
issue happened after updating systemd to version 236, ZoL to 0.7.4 and
kernel to 4.14.

I have always mounted the pool manually by first opening LUKS-encrypted
disks and after that issuing zpool import tank.

Is there any way to still use systemd to manage smbd or do I have to
just always start it manually?

This is how the pool has been set up:

root@punishedkorppu /# lsblk -a
NAME       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda          8:0    0   2.7T  0 disk
└─luks-sda 254:0    0   2.7T  0 crypt
sdb          8:16   0   2.7T  0 disk
└─luks-sdb 254:1    0   2.7T  0 crypt
sdc          8:32   0   2.7T  0 disk
└─luks-sdc 254:2    0   2.7T  0 crypt
sdd          8:48   0   2.7T  0 disk
└─luks-sdd 254:3    0   2.7T  0 crypt
sde          8:64   0   2.7T  0 disk
└─luks-sde 254:4    0   2.7T  0 crypt
sdf          8:80   0   2.7T  0 disk
└─luks-sdf 254:5    0   2.7T  0 crypt
sdg          8:96   0   2.7T  0 disk
└─luks-sdg 254:6    0   2.7T  0 crypt
sdh          8:112  0   2.7T  0 disk
└─luks-sdh 254:7    0   2.7T  0 crypt
sdi          8:128  1 119.2G  0 disk
├─sdi1       8:129  1   512M  0 part  /boot/efi
└─sdi2       8:130  1   118G  0 part  /
root@punishedkorppu /# zpool status
  pool: tank
 state: ONLINE
  scan: scrub repaired 0B in 18h41m with 0 errors on Mon Dec 25 17:18:44
2017
config:

        NAME          STATE     READ WRITE CKSUM
        tank          ONLINE       0     0     0
          raidz2-0    ONLINE       0     0     0
            luks-sda  ONLINE       0     0     0
            luks-sdb  ONLINE       0     0     0
            luks-sdc  ONLINE       0     0     0
            luks-sdd  ONLINE       0     0     0
            luks-sdg  ONLINE       0     0     0
            luks-sdh  ONLINE       0     0     0
            luks-sdf  ONLINE       0     0     0
            luks-sde  ONLINE       0     0     0

errors: No known data errors


Jeremy Allison via samba kirjoitti 10.1.2018 klo 5:19:

> On Wed, Jan 10, 2018 at 04:51:23AM +0200, John Doe via samba wrote:
>> I just noticed that by running by commands /usr/sbin/smbd -D or
>> /usr/sbin/smbd -i without systemd's unit, all shares work perfectly so
>> the problem must then be somehow related to systemd.. Let the testing
>> continue..
>>
>> I also tested what happens if I comment out everything and just use
>> ExecStart=/usr/sbin/smbd -D as that command worked on the console. That
>> did not help.
>>
>> For the record, this is the default unit-file:
>> [Unit]
>> Description=Samba SMB Daemon
>> Documentation=man:smbd(8) man:samba(7) man:smb.conf(5)
>> After=network.target nmbd.service winbind.service
>>
>> [Service]
>> Type=notify
>> NotifyAccess=all
>> PIDFile=/var/run/samba/smbd.pid
>> LimitNOFILE=16384
>> EnvironmentFile=-/etc/default/samba
>> ExecStart=/usr/sbin/smbd $SMBDOPTIONS
>> ExecReload=/bin/kill -HUP $MAINPID
>> LimitCORE=infinity
>>
>> [Install]
>> WantedBy=multi-user.target
>
> Oh how strange. Must be something to do with systemd's mount
> facility I'd guess.
>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba