report audit wait metric in audit status reply
On 7/1/20 3:32 PM, Max Englander wrote:
In environments where the preservation of audit events and
predictable
usage of system memory are prioritized, admins may use a combination of
--backlog_wait_time and -b options at the risk of degraded performance
resulting from backlog waiting. In some cases, this risk may be
preferred to lost events or unbounded memory usage. Ideally, this risk
can be mitigated by making adjustments when backlog waiting is detected.
However, detection can be diffult using the currently available metrics.
For example, an admin attempting to debug degraded performance may
falsely believe a full backlog indicates backlog waiting. It may turn
out the backlog frequently fills up but drains quickly.
To make it easier to reliably track degraded performance to backlog
waiting, this patch makes the following changes:
Add a new field backlog_wait_sum to the audit status reply. Initialize
this field to zero. Add to this field the total time spent by the
current task on scheduled timeouts while the backlog limit is exceeded.
Tested on Ubuntu 18.04 using complementary changes to the audit
userspace:https://github.com/linux-audit/audit-userspace/pull/134.
Max,
Along those lines, the current failure actions (silent, printk, panic)
are kind of restrictive. I guess one can filter on the printk messages
and redirect those to a userspace handler which might do something
specific to the operating environment? Is that how you would handle it?
Or would your admin just look for it and report? If you have any
shareable info I'd appreciate seeing it. Almost no one I can think of
would want a panic to happen, but only almost. No one who needs some
level of assurance would want "silent".
FWIW I looked at the kernel printk calls, and although I maybe looked at
the wrong one, even though on boot I'm seeing drops from the "auditctl
-s", I do not see any output in the dmesg buffer that appears to be
indicative of this. My guess there is that on bootup, the auditd
userspace config has not yet been activated and it's likely using a
"silent" default...but here I realize was not the focus of your effort.
Just musing.
I applaud your efforts in this area, and if you are able to share any
practices about handling the backlogs I'd appreciate seeing that.
V/R,
LCB
--
Lenny Bruzenak
MagitekLTD