Memory leak in samba-ad-dc on 4.5.x not related to aio?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory leak in samba-ad-dc on 4.5.x not related to aio?

Samba - General mailing list
On somewhat long-running samba AD DC instances (4.5.8-Debian, Stretch),
we're seeming massive RAM utilization even with little/no clients connected:

> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root      9937  0.0  0.7 532004  7364 ?        Ss   Apr26   0:00 /usr/sbin/samba
> root      9980  0.0  0.4 532004  4304 ?        S    Apr26   0:00 /usr/sbin/samba
> root      9981  0.5  2.0 538720 20708 ?        S    Apr26  66:45 /usr/sbin/samba
> root      9983  0.0  0.8 536156  8444 ?        S    Apr26   0:02 /usr/sbin/samba
> root      9984  0.0  0.4 532004  4348 ?        S    Apr26   0:00 /usr/sbin/samba
> root      9985  0.1  2.3 534544 23740 ?        S    Apr26  19:29 /usr/sbin/samba
> root      9986  0.0  0.7 532004  7744 ?        S    Apr26   0:23 /usr/sbin/samba
> root      9987  0.0  1.3 538276 13716 ?        S    Apr26   2:02 /usr/sbin/samba
> root      9988  0.8 59.4 1571936 606488 ?      S    Apr26 114:41 /usr/sbin/samba
> root      9991  0.0  0.4 532004  4316 ?        S    Apr26   0:00 /usr/sbin/samba
> root      9992  0.0  0.4 532004  4352 ?        S    Apr26   0:00 /usr/sbin/samba
> root      9993  0.0  1.2 536156 13120 ?        S    Apr26   1:19 /usr/sbin/samba
> root      9994  0.0  0.8 532004  8996 ?        S    Apr26   0:27 /usr/sbin/samba
> root      9995  0.0  1.0 534544 11116 ?        S    Apr26  11:26 /usr/sbin/samba

As far as I can see (testparm), aio is on its defaults, and should be
disabled:

> # Global parameters
> [global]
> netbios name = VILLACH-DC-SEM
> realm = AD.TAO.AT
> workgroup = AD
> dns forwarder = 8.8.8.8
> ldap ssl = start tls
> ldap ssl ads = Yes
> kerberos method = system keytab
> server role = active directory domain controller
> tls cafile = /usr/local/share/ca-certificates/tao-ad-ca.crt
> tls certfile = /etc/ssl/certs/villach-dc-sem.ad.tao.at.crt
> tls enabled = Yes
> tls keyfile = /etc/ssl/private/villach-dc-sem.ad.tao.at.key
> template homedir = /home/%U
> template shell = /bin/zsh
> dsdb:schema update allowed = true
> idmap_ldb:use rfc2307 = yes
> include = /etc/samba/site.conf
>
> [homes]
> msdfs proxy = \\graz-file\homes
> msdfs root = Yes
>
> [netlogon]
> path = /var/lib/samba/sysvol/ad.tao.at/scripts
> read only = No
>
> [sysvol]
> path = /var/lib/samba/sysvol
> read only = No


The output of smbd pool-usage is a whopping 4MB, uploaded here:

https://up.tao.at/-13984af5bf5d36c3da17685a7f76bc8b/pool-usage.txt

Is this related to the aio bug, or something else entirely?

--
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP [hidden email] | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167
https://pave.software – PAVE Password Manager

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in samba-ad-dc on 4.5.x not related to aio?

Samba - General mailing list
On Fri, May 05, 2017 at 09:42:47AM +0200, Sven Schwedas via samba wrote:
> > root      9988  0.8 59.4 1571936 606488 ?      S    Apr26 114:41 /usr/sbin/samba

Can you post /proc/9988/smaps somewhere?

Volker

--
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:[hidden email]

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in samba-ad-dc on 4.5.x not related to aio?

Samba - General mailing list
On 2017-05-05 10:09, Volker Lendecke wrote:
> On Fri, May 05, 2017 at 09:42:47AM +0200, Sven Schwedas via samba wrote:
>>> root      9988  0.8 59.4 1571936 606488 ?      S    Apr26 114:41 /usr/sbin/samba
>
> Can you post /proc/9988/smaps somewhere?

Sure, http://up.tao.at/-0242609d03e69298bb27155078663d51/smaps.9988.txt


--
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP [hidden email] | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167
https://pave.software – PAVE Password Manager

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in samba-ad-dc on 4.5.x not related to aio?

Samba - General mailing list
On Fri, May 05, 2017 at 10:21:05AM +0200, Sven Schwedas wrote:
> On 2017-05-05 10:09, Volker Lendecke wrote:
> > On Fri, May 05, 2017 at 09:42:47AM +0200, Sven Schwedas via samba wrote:
> >>> root      9988  0.8 59.4 1571936 606488 ?      S    Apr26 114:41 /usr/sbin/samba
> >
> > Can you post /proc/9988/smaps somewhere?
>
> Sure, http://up.tao.at/-0242609d03e69298bb27155078663d51/smaps.9988.txt

Ok, it's actually the heap. I was hoping that it's a mmap'ed ldb. So
we do have something fishy going on that. Sorry, I con't have an easy
recipe.

Volker

--
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:[hidden email]

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in samba-ad-dc on 4.5.x not related to aio?

Samba - General mailing list
On 2017-05-05 11:34, Volker Lendecke wrote:

> On Fri, May 05, 2017 at 10:21:05AM +0200, Sven Schwedas wrote:
>> On 2017-05-05 10:09, Volker Lendecke wrote:
>>> On Fri, May 05, 2017 at 09:42:47AM +0200, Sven Schwedas via samba wrote:
>>>>> root      9988  0.8 59.4 1571936 606488 ?      S    Apr26 114:41 /usr/sbin/samba
>>>
>>> Can you post /proc/9988/smaps somewhere?
>>
>> Sure, http://up.tao.at/-0242609d03e69298bb27155078663d51/smaps.9988.txt
>
> Ok, it's actually the heap. I was hoping that it's a mmap'ed ldb. So
> we do have something fishy going on that. Sorry, I con't have an easy
> recipe.

I guess I'll open a bug report for it then. Any other information you
need from me for it?

--
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP [hidden email] | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in samba-ad-dc on 4.5.x not related to aio?

Samba - General mailing list
On 2017-05-05 11:40, Sven Schwedas via samba wrote:

> On 2017-05-05 11:34, Volker Lendecke wrote:
>> On Fri, May 05, 2017 at 10:21:05AM +0200, Sven Schwedas wrote:
>>> On 2017-05-05 10:09, Volker Lendecke wrote:
>>>> On Fri, May 05, 2017 at 09:42:47AM +0200, Sven Schwedas via samba wrote:
>>>>>> root      9988  0.8 59.4 1571936 606488 ?      S    Apr26 114:41 /usr/sbin/samba
>>>>
>>>> Can you post /proc/9988/smaps somewhere?
>>>
>>> Sure, http://up.tao.at/-0242609d03e69298bb27155078663d51/smaps.9988.txt
>>
>> Ok, it's actually the heap. I was hoping that it's a mmap'ed ldb. So
>> we do have something fishy going on that. Sorry, I con't have an easy
>> recipe.
>
> I guess I'll open a bug report for it then. Any other information you
> need from me for it?

We managed to "reproduce" the issue on another DC (i.e., it popped up
without us doing anything), so I filed a bug report with the data for
both machines/runaway processes:

https://bugzilla.samba.org/show_bug.cgi?id=12778

--
Mit freundlichen Grüßen, / Best Regards,
Sven Schwedas, Systemadministrator
Mail/XMPP [hidden email] | Skype sven.schwedas
TAO Digital | Lendplatz 45 | A8020 Graz
https://www.tao-digital.at | Tel +43 680 301 7167

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba