Log rotation issue

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Log rotation issue

Samba - General mailing list
Hi,

We recently migrated the samba-ad-dc from 4.6.5 to 4.7.1. Over a period
of last 15 days or so we have been observing that although the samba log
rotates, upon rotation, samba will start writing the logs to both the
main log file and the <main log>.old file simultaneously. While, the
main log file rotates properly, instead of deleting old file and create
a new one on rotation, samba continues to write into the <main log>.old
file and file size keeps increasing.

Any suggestion to fix it? Also, we need some suggestions to use wild
card such as %h, %m etc for samba log.

Our smb.conf is shown below.

# Global parameters
[global]
         netbios name = DC1
         realm = ***********.COM
         server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbindd, ntp_signd, kcc, dnsupdate
         workgroup = ********
         server role = active directory domain controller
         idmap_ldb:use rfc2307 = yes
         ldap server require strong auth = no

#eventlog list = Application System Security SyslogLinux
         eventlog list = Security
         #Log Level
         log level = 3
         log file = /var/log/samba/samba.log
         max log size = 1000000

[netlogon]
         path = /usr/local/samba/var/locks/sysvol/***********.com/scripts
         read only = No

[sysvol]
         path = /usr/local/samba/var/locks/sysvol
         read only = No

[shares]
         comment = For all Users
         path = /home/shares
         read only = No


--

Thanks & Regards,


Anantha Raghava

Do not print this e-mail unless required. Save Paper & trees.

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list


Am 30.11.2017 um 13:17 schrieb Anantha Raghava via samba:

> Hi,
>
> We recently migrated the samba-ad-dc from 4.6.5 to 4.7.1. Over a
> period of last 15 days or so we have been observing that although the
> samba log rotates, upon rotation, samba will start writing the logs to
> both the main log file and the <main log>.old file simultaneously.
> While, the main log file rotates properly, instead of deleting old
> file and create a new one on rotation, samba continues to write into
> the <main log>.old file and file size keeps increasing.
>
> Any suggestion to fix it? Also, we need some suggestions to use wild
> card such as %h, %m etc for samba log.
>
> Our smb.conf is shown below.
>
> # Global parameters
> [global]
>         netbios name = DC1
>         realm = ***********.COM
>         server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc,
> drepl, winbindd, ntp_signd, kcc, dnsupdate
>         workgroup = ********
>         server role = active directory domain controller
>         idmap_ldb:use rfc2307 = yes
>         ldap server require strong auth = no
>
> #eventlog list = Application System Security SyslogLinux
>         eventlog list = Security
>         #Log Level
>         log level = 3
>         log file = /var/log/samba/samba.log
>         max log size = 1000000
>
> [netlogon]
>         path = /usr/local/samba/var/locks/sysvol/***********.com/scripts
>         read only = No
>
> [sysvol]
>         path = /usr/local/samba/var/locks/sysvol
>         read only = No
>
> [shares]
>         comment = For all Users
>         path = /home/shares
>         read only = No
>
>
You can try and add the "copytruncate" option to your samba logrotate
config file.
This is what i have to use on debian wheezy:
/etc/logrotate.d/samba
-----------------------------

...

/var/log/samba/log.samba {
     weekly
     missingok
     rotate 7
     copytruncate
     compress
     delaycompress
     notifempty
}
----------------------------

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list
In reply to this post by Samba - General mailing list
Hi,

Continuing on the same subject, the VARIABLE SUBSTITUTIONS are not
working. for example, I want to have the date & time attached to the
samba log file. Hence I modified the smb.conf

log file = /var/log/samba/samba.%T.log

The file with name "samba.%T.log" gets created in /var/log/samba folder
and %T will not get substituted with current date & time.

I request for your guidance to correct this part. Also the samba
continues to write into .old file as well and not finding any means to
correct it.


Many thanks for your guidance in advance.

--

Thanks & Regards,


Anantha Raghava

Do not print this e-mail unless required. Save Paper & trees.

On 30/11/17 5:47 PM, Anantha Raghava wrote:

>
> Hi,
>
> We recently migrated the samba-ad-dc from 4.6.5 to 4.7.1. Over a
> period of last 15 days or so we have been observing that although the
> samba log rotates, upon rotation, samba will start writing the logs to
> both the main log file and the <main log>.old file simultaneously.
> While, the main log file rotates properly, instead of deleting old
> file and create a new one on rotation, samba continues to write into
> the <main log>.old file and file size keeps increasing.
>
> Any suggestion to fix it? Also, we need some suggestions to use wild
> card such as %h, %m etc for samba log.
>
> Our smb.conf is shown below.
>
> # Global parameters
> [global]
>          netbios name = DC1
>          realm = ***********.COM
>          server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbindd, ntp_signd, kcc, dnsupdate
>          workgroup = ********
>          server role = active directory domain controller
>          idmap_ldb:use rfc2307 = yes
>          ldap server require strong auth = no
>
> #eventlog list = Application System Security SyslogLinux
>          eventlog list = Security
>          #Log Level
>          log level = 3
>          log file = /var/log/samba/samba.log
>          max log size = 1000000
>
> [netlogon]
>          path = /usr/local/samba/var/locks/sysvol/***********.com/scripts
>          read only = No
>
> [sysvol]
>          path = /usr/local/samba/var/locks/sysvol
>          read only = No
>
> [shares]
>          comment = For all Users
>          path = /home/shares
>          read only = No
>
> --
>
> Thanks & Regards,
>
>
> Anantha Raghava
>
> Do not print this e-mail unless required. Save Paper & trees.
>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list


Am 01.12.2017 um 11:36 schrieb Anantha Raghava via samba:

> Hi,
>
> Continuing on the same subject, the VARIABLE SUBSTITUTIONS are not
> working. for example, I want to have the date & time attached to the
> samba log file. Hence I modified the smb.conf
>
> log file = /var/log/samba/samba.%T.log
>
> The file with name "samba.%T.log" gets created in /var/log/samba
> folder and %T will not get substituted with current date & time.
>
> I request for your guidance to correct this part. Also the samba
> continues to write into .old file as well and not finding any means to
> correct it.
>
>
> Many thanks for your guidance in advance.
>
I'll give it another try.

See here about your logfile issue
https://bugzilla.samba.org/show_bug.cgi?id=12935. As I laready wrote,
using copytruncate for logrotate fixed that issue for me. I also assume
your system does not use systemd.
I do not use variables so someone else may jump in here.

Sincere,
Achim~

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list
In reply to this post by Samba - General mailing list
On Fri, 2017-12-01 at 16:06 +0530, Anantha Raghava via samba wrote:

> Hi,
>
> Continuing on the same subject, the VARIABLE SUBSTITUTIONS are not
> working. for example, I want to have the date & time attached to the
> samba log file. Hence I modified the smb.conf
>
> log file = /var/log/samba/samba.%T.log
>
> The file with name "samba.%T.log" gets created in /var/log/samba folder
> and %T will not get substituted with current date & time.

Yes, currently the 'samba' side of things (compared with smbd) doesn't
know how to substitute the macros.  Now part of this is due to global
variables and multiple clients per connection, but it doesn't really
excuse us also skipping things like %T.

> I request for your guidance to correct this part. Also the samba
> continues to write into .old file as well and not finding any means to
> correct it.

I think this is because each process only checks if the file should be
rotated every few lines, so they can hold the log open for quite some
time.  Ideally a global signal would be sent saying which file (or even
just that a file) just got rotated, so everyone can re-open.  

Patches are most welcome!

Andrew Bartlett
--
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba


--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list
Hello Andrew,

Thanks for proper explanation.

To overcome this,  also as we need to store logs for long,  now we have
written a shell script,  executed as a crop every three minutes that checks
the file size.  If the file size is 1 GB or above,  it moves the log file
to <logfile>.timestamp.log.old and clears the main log file.

We have commented out the max log size in sub.conf.

If need be I can share the script here

Best regards,
Anantha Raghava

On Dec 2, 2017 10:57 AM, "Andrew Bartlett" <[hidden email]> wrote:

> On Fri, 2017-12-01 at 16:06 +0530, Anantha Raghava via samba wrote:
> > Hi,
> >
> > Continuing on the same subject, the VARIABLE SUBSTITUTIONS are not
> > working. for example, I want to have the date & time attached to the
> > samba log file. Hence I modified the smb.conf
> >
> > log file = /var/log/samba/samba.%T.log
> >
> > The file with name "samba.%T.log" gets created in /var/log/samba folder
> > and %T will not get substituted with current date & time.
>
> Yes, currently the 'samba' side of things (compared with smbd) doesn't
> know how to substitute the macros.  Now part of this is due to global
> variables and multiple clients per connection, but it doesn't really
> excuse us also skipping things like %T.
>
> > I request for your guidance to correct this part. Also the samba
> > continues to write into .old file as well and not finding any means to
> > correct it.
>
> I think this is because each process only checks if the file should be
> rotated every few lines, so they can hold the log open for quite some
> time.  Ideally a global signal would be sent saying which file (or even
> just that a file) just got rotated, so everyone can re-open.
>
> Patches are most welcome!
>
> Andrew Bartlett
> --
> Andrew Bartlett                       http://samba.org/~abartlet/
> Authentication Developer, Samba Team  http://samba.org
> Samba Developer, Catalyst IT          http://catalyst.net.nz/
> services/samba
>
>
--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list
On Sat, 2017-12-02 at 11:58 +0530, Anantha Raghava wrote:

> Hello Andrew,
>
> Thanks for proper explanation. 
>
> To overcome this,  also as we need to store logs for long,  now we
> have written a shell script,  executed as a crop every three minutes
> that checks the file size.  If the file size is 1 GB or above,  it
> moves the log file to <logfile>.timestamp.log.old and clears the main
> log file. 
>
> We have commented out the max log size in sub.conf.
>
> If need be I can share the script here

I'm not sure that will help, that is pretty much what was happening
before.  The key is to send a -HUP to the whole process tree to make
them re-open the logs, otherwise the other child samba processes will
still just write to the .old

I know this sucks.  I think the solution is to have a new reopen-logs
message created so this can be used in logroate, and to have it used
internally as I mentioned.

Andrew Bartlett
--
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba


--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list
Hello Andrew,

I am now experimenting with my test setup on the log rotation part.

While testing different scripts, I observed a very peculiar behavior
with log writing process.

If the log file name has a variable to be substituted, say %T, it
continues to write into it without creating .old file. If it is a simple
name like samba.log or dc1.current.log etc., after about 10 MB, the log
rotates,with or without my script and appends .old to actual log file
and continues to write into .old file. Could you please provide pointers
where to look for this log writing routine and why this peculiar
behavior? My smb.conf and the script I am using is shown below.

Samba version is 4.7.1

/*smb.conf*/

# Global parameters
[global]
         netbios name = DC1
         realm = EXZA.COM
         server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc,
drepl, winbindd, ntp_signd, kcc, dnsupdate
         workgroup = EXZA
         server role = active directory domain controller
         ldap server require strong auth = No
         log level = 10
         log file = /var/log/samba/dc1.current.log
         #"With this file name, the log gets rotated when it reaches 10
MB or above in size, One cannot exactly say at what size it rotates. The
process appends .old                 to the file name and continues to
write into .old file."

[netlogon]
         path = /usr/local/samba/var/locks/sysvol/exza.com/scripts
         read only = No

[sysvol]
         path = /usr/local/samba/var/locks/sysvol
         read only = No/*
*/

/*Script used to rotate log*/

File size is set to 10 MB in this script. This script runs in background
(not as a cron).
**/**/

#!/bin/bash
MaxFileSize=10000000
while true
do
     file_size=`du -b /var/log/samba/dc1.current.log | tr -s '\t' ' ' |
cut -d' ' -f1`
     if [ $file_size -gt $MaxFileSize ];then
         timestamp=`date +%Y%m%d%H%M%S`
         cp -pr "/var/log/samba/dc1.current.log"
"/var/log/samba/dc1.$timestamp.log.old"
         echo > /var/log/samba/dc1.current.log
     fi

done/*
*/

--

Thanks & Regards,


Anantha Raghava


Do not print this e-mail unless required. Save Paper & trees.

On 02/12/17 12:07 PM, Andrew Bartlett wrote:

> On Sat, 2017-12-02 at 11:58 +0530, Anantha Raghava wrote:
>> Hello Andrew,
>>
>> Thanks for proper explanation.
>>
>> To overcome this,  also as we need to store logs for long,  now we
>> have written a shell script,  executed as a crop every three minutes
>> that checks the file size.  If the file size is 1 GB or above,  it
>> moves the log file to <logfile>.timestamp.log.old and clears the main
>> log file.
>>
>> We have commented out the max log size in sub.conf.
>>
>> If need be I can share the script here
> I'm not sure that will help, that is pretty much what was happening
> before.  The key is to send a -HUP to the whole process tree to make
> them re-open the logs, otherwise the other child samba processes will
> still just write to the .old
>
> I know this sucks.  I think the solution is to have a new reopen-logs
> message created so this can be used in logroate, and to have it used
> internally as I mentioned.
>
> Andrew Bartlett

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba
Reply | Threaded
Open this post in threaded view
|

Re: Log rotation issue

Samba - General mailing list
Hi,

Samba is replacing the %T variable with actual timestamp. But timestamp
format is different as shown in this log entry.

Unable to open new log file '/var/log/samba/dc1.2017/12/04 00:05:54.log': No such file or directory

"Unable to open new log file '/var/log/samba/dc1.2017*/12/04
00:05:54*.log': No such file or directory"

2017/12/04 00:05:54 part of the file name is the actual culprit. Because
of the format, file name is getting interpreted as folder
".../dc.2017/12/" and file as "04 00:05:54.log". This file will never
get created. Because of this probably we see that a log file as
"dc1.%T.log" created instead.

If the timestamp format is corrected, %T variable can be handled.

--

Thanks & Regards,


Anantha Raghava


Do not print this e-mail unless required. Save Paper & trees.
On 03/12/17 8:48 PM, Anantha Raghava wrote:

>
> Hello Andrew,
>
> I am now experimenting with my test setup on the log rotation part.
>
> While testing different scripts, I observed a very peculiar behavior
> with log writing process.
>
> If the log file name has a variable to be substituted, say %T, it
> continues to write into it without creating .old file. If it is a
> simple name like samba.log or dc1.current.log etc., after about 10 MB,
> the log rotates,with or without my script and appends .old to actual
> log file and continues to write into .old file. Could you please
> provide pointers where to look for this log writing routine and why
> this peculiar behavior? My smb.conf and the script I am using is shown
> below.
>
> Samba version is 4.7.1
>
> /*smb.conf*/
>
> # Global parameters
> [global]
>         netbios name = DC1
>         realm = EXZA.COM
>         server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc,
> drepl, winbindd, ntp_signd, kcc, dnsupdate
>         workgroup = EXZA
>         server role = active directory domain controller
>         ldap server require strong auth = No
>         log level = 10
>         log file = /var/log/samba/dc1.current.log
>         #"With this file name, the log gets rotated when it reaches 10
> MB or above in size, One cannot exactly say at what size it rotates.
> The process appends .old                 to the file name and
> continues to write into .old file."
>
> [netlogon]
>         path = /usr/local/samba/var/locks/sysvol/exza.com/scripts
>         read only = No
>
> [sysvol]
>         path = /usr/local/samba/var/locks/sysvol
>         read only = No/*
> */
>
> /*Script used to rotate log*/
>
> File size is set to 10 MB in this script. This script runs in
> background (not as a cron).
>
> #!/bin/bash
> MaxFileSize=10000000
> while true
> do
>     file_size=`du -b /var/log/samba/dc1.current.log | tr -s '\t' ' ' |
> cut -d' ' -f1`
>     if [ $file_size -gt $MaxFileSize ];then
>         timestamp=`date +%Y%m%d%H%M%S`
>         cp -pr "/var/log/samba/dc1.current.log"
> "/var/log/samba/dc1.$timestamp.log.old"
>         echo > /var/log/samba/dc1.current.log
>     fi
>
> done/*
> */
>
> --
>
> Thanks & Regards,
>
>
> Anantha Raghava
>
>
> Do not print this e-mail unless required. Save Paper & trees.
>
> On 02/12/17 12:07 PM, Andrew Bartlett wrote:
>> On Sat, 2017-12-02 at 11:58 +0530, Anantha Raghava wrote:
>>> Hello Andrew,
>>>
>>> Thanks for proper explanation.
>>>
>>> To overcome this,  also as we need to store logs for long,  now we
>>> have written a shell script,  executed as a crop every three minutes
>>> that checks the file size.  If the file size is 1 GB or above,  it
>>> moves the log file to <logfile>.timestamp.log.old and clears the main
>>> log file.
>>>
>>> We have commented out the max log size in sub.conf.
>>>
>>> If need be I can share the script here
>> I'm not sure that will help, that is pretty much what was happening
>> before.  The key is to send a -HUP to the whole process tree to make
>> them re-open the logs, otherwise the other child samba processes will
>> still just write to the .old
>>
>> I know this sucks.  I think the solution is to have a new reopen-logs
>> message created so this can be used in logroate, and to have it used
>> internally as I mentioned.
>>
>> Andrew Bartlett
>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba