On Tuesday, March 01, 2016 02:57:45 PM Maupertuis Philippe wrote:
The kernel is : 2.6.32-573.12.1.el6.x86_64
And the whole audit.rules file is :
<snip>
During the hour preceding the fence we got these events from the
passive
node Key Summary Report
===========================
total key
===========================
891 system_commands (ping)
And on the active node :
Key Summary Report
===========================
total key
===========================
1330 system_commands
286 deletion
I am going to follow your advice and to open a call with redhat.
Anyway, I am interested in knowing if auditd has been reported to cause
trouble without generating many events.
Those numbers work out to 27 events per minute. That's not really a lot of
events. To see if its the rules or auditd causing the iowait, you might set
the logging format to NOLOG. This will discard events rather than log them. If
you still have iowait, its something to do with the rules. If that cleared it
up, then auditd might be the source. Either way, put the format back to raw.
I did some benchmarking of auditd over the holidays and posted some results
here:
https://www.redhat.com/archives/linux-audit/2015-December/msg00061.html
I'd recommend:
flush = incremental
freq = 100
for a modest performance improvement.
-Steve
-----Message d'origine-----
De : Paul Moore [mailto:paul@paul-moore.com]
Envoyé : mardi 1 mars 2016 14:25
À : Maupertuis Philippe
Cc : linux-audit(a)redhat.com
Objet : Re: auditd and redhat cluster
On Mon, Feb 29, 2016 at 7:45 AM, Maupertuis Philippe
<philippe.maupertuis(a)worldline.com> wrote:
> Hi list,
>
> One clusters fenced the passive node around two hours after auditd
> was started.
>
> We have found that iowait has increased since auditd was started and
> was unusually high.
>
> Auditd wasn’t generating many messages and there were no noticeable
> added activity on the disk were the audit and syslog files were written.
>
> Besides watches, the only general rules were :
>
> # creation
> -a exit,always -F arch=b32 -S creat -S mkdir -S mknod -S link -S
> symlink -S mkdirat -S mknodat -S linkat -S symlinkat -F uid=root -F
> success=1 -k creation -a exit,always -F arch=b64 -S creat -S mkdir -S
> mknod -S link -S symlink -S mkdirat -S mknodat -S linkat -S symlinkat
> -F uid=root -F success=1 -k creation
>
> # deletion
> -a exit,always -F arch=b32 -S rmdir -S unlink -S unlinkat -F uid=root
> -F
> success=1 -k deletion
> -a exit,always -F arch=b64 -S rmdir -S unlink -S unlinkat -F uid=root
> -F
> success=1 -k deletion
>
> After the rebot we deleted all rules and didn’t notice extra iowait
> anymore.
>
> Could these rules be the cause of additional iowait even if not
> generating many events (around 20 in two hours) ?
>
> Is there any other auditd mechanism that could explain this phenomenon ?
>
> I would appreciate any hints.
Hi Philippe,
First, as this is a RH cluster product, I would suggest contacting RH
support with your question if you haven't already; this list is primarily
for upstream development and support.
If you are able to experiment with the system, or have a test environment, I
would suggest trying to narrow down the list of audit rules/watches to see
which rules/watches have the most affect on the iowait times. You've
listed four rules, but you didn't list the watches you have configured.
Also, what kernel version are you using?
--
paul moore
www.paul-moore.com
!!!*************************************************************************
************ "Ce message et les pièces jointes sont confidentiels et
réservés à l'usage exclusif de ses destinataires. Il peut également être
protégé par le secret professionnel. Si vous recevez ce message par erreur,
merci d'en avertir immédiatement l'expéditeur et de le détruire.
L'intégrité du message ne pouvant être assurée sur Internet, la
responsabilité de Worldline ne pourra être recherchée quant au contenu de
ce message. Bien que les meilleurs efforts soient faits pour maintenir
cette transmission exempte de tout virus, l'expéditeur ne donne aucune
garantie à cet égard et sa responsabilité ne saurait être recherchée pour
tout dommage résultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely
for the addressee; it may also be privileged. If you receive this e-mail in
error, please notify the sender immediately and destroy it. As its
integrity cannot be secured on the Internet, the Worldline liability cannot
be triggered for the message content. Although the sender endeavours to
maintain a computer virus-free network, the sender does not warrant that
this transmission is virus-free and will not be liable for any damages
resulting from any virus transmitted.!!!"
--
Linux-audit mailing list
Linux-audit(a)redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit