netlink ACK handling in audit_set_pid()
by Chris Riches
We are experiencing strange failures where the audit daemon fails to
start on boot, hitting an ENOBUFS error on its audit_set_pid() call.
This can be reproduced by repeatedly restarting the audit daemon while
the system is under heavy audit load. This also seems to be dependent
on the number of CPUs - we can reproduce this with 2 CPUs but not with
48.
Tracing showed a race between the kernel enabling audit messages to be
sent to the daemon and actually sending the ACK, wherein the socket
buffer could get filled by audit messages before the ACK could be sent,
leading to the ACK being dropped and ENOBUFS set on the socket by
netlink code. A patch to mitigate this race from the kernel side is
separately under discussion on the audit subsystem mailing list:
https://lore.kernel.org/audit/20230922152749.244197-1-chris.riches@nutani...
It's worth noting that this is almost certainly the same issue observed
in this thread from last month (participants CCed):
https://listman.redhat.com/archives/linux-audit/2023-September/020087.html
Here, I am hoping to discuss ACK handling from the userspace side. The
current implementation is a little odd - check_ack() will happily
return success without seeing an ACK if a non-ACK message is top of the
socket queue, but will fail if no message arrives within the timeout.
It also of course fails if ENOBUFS is set on the socket, but this
failure only seems to matter when doing audit_set_pid() - similar
failures during main-loop message processing are logged but otherwise
ignored, as far as I can tell.
I'm not sure I quite understand the intentions of the code, but it
seems odd to let ENOBUFS be a fatal error here, given that it likely
means the socket buffer got flooded with audit messages, and thus
audit_set_pid() succeeded. Perhaps we should just ignore ENOBUFS or
even set NETLINK_NO_ENOBUFS?
It may also be worth increasing the netlink socket buffer size, though
this could only make the issue less likely and would not be sufficient
under arbitrarily heavy audit loads.
Finally, there is another oddity in audit_set_pid() that is tangential
to this discussion but worth highlighting: if the wmode parameter is
WAIT_YES, then there is some additional ACK-handling which waits for
100 milliseconds and eats the top message of the socket queue if one
arrives, without inspecting it. This seems completely wrong as the ACK
will have already been consumed by check_ack() if there was one, and so
the best this code can do is nothing, and at worst (quite likely) it
will swallow a genuine audit message without ever recording it.
- Chris
6 months
Couldn't get audit messages for 'listen' on kernel 4.19.0-6-686-pae
by Rinat Gadelshin
Hello there!
I'm facing a strange problem.
I have not been able to get audit reports for any "network" syscall
on one of the computers from my test bench.
I mean 'connect', 'accept4', 'listen', 'bind', 'socket'.
The following example shows that auditd couldn't get them too ('listen'
at least).
But I've received a report about 'execve' called by the same process.
Could you tell me what can I do in order to receive audit messages for
the syscalls.
from this version of the kernel?
Any help will be will be appreciated.
root@deb101-x86-0009:~# netcat -v -l -p 4242 &
[2] 13481
root@deb101-x86-0009:~# listening on [any] 4242 ...
root@deb101-x86-0009:~# echo "Test" | nc -q 0 127.0.0.1 4242
connect to [127.0.0.1] from localhost [127.0.0.1] 36650
Test
root@deb101-x86-0009:~# skill -p 13481
[2]+ Done netcat -v -l -p 4242
root@deb101-x86-0009:~# ausearch -p 13481
----
time->Fri Oct 20 22:00:42 2023
type=PROCTITLE msg=audit(1697828442.603:2697):
proctitle=6E6574636174002D76002D6C002D700034323432
type=PATH msg=audit(1697828442.603:2697): item=1
name="/lib/ld-linux.so.2" inode=655382 dev=fe:00 mode=0100755 ouid=0
ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0000000000000000
cap_fi=0000000000000000 cap_fe=0 cap_fver=0
type=PATH msg=audit(1697828442.603:2697): item=0 name="/usr/bin/netcat"
inode=664887 dev=fe:00 mode=0100755 ouid=0 ogid=0 rdev=00:00
nametype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0
cap_fver=0
type=CWD msg=audit(1697828442.603:2697): cwd="/root"
type=EXECVE msg=audit(1697828442.603:2697): argc=5 a0="netcat" a1="-v"
a2="-l" a3="-p" a4="4242"
type=SYSCALL msg=audit(1697828442.603:2697): arch=40000003 syscall=11
success=yes exit=0 a0=e36400 a1=d9d9e0 a2=e3a310 a3=584988 items=2
ppid=12968 pid=13481 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0
sgid=0 fsgid=0 tty=pts1 ses=4 comm="netcat"
exe="/usr/bin/nc.traditional" subj==unconfined key=(null)
root@deb101-x86-0009:~# auditctl -l
-a always,exit -F arch=b32 -S fork,execve,clone,vfork,execveat
-a always,exit -F arch=b32 -S bind,connect,listen,accept4
root@deb101-x86-0009:~# auditctl -s
enabled 1
failure 1
pid 13393
rate_limit 0
backlog_limit 8192
lost 0
backlog 0
backlog_wait_time 0
loginuid_immutable 0 unlocked
root@deb101-x86-0009:~# uname -a
Linux deb101-x86-0009.avp.ru.local 4.19.0-6-686-pae #1 SMP Debian
4.19.67-2+deb10u2 (2019-11-11) i686 GNU/Linux
root@deb101-x86-0009:~# cat /etc/debian_version
10.1
root@deb101-x86-0009:~#
Regards
Rinat
6 months, 3 weeks
Re: Need help with af_unix audisd plugin
by Rinat Gadelshin
Steve, thank you so much =)
I suppose you meant `ncat -U --recv-only` due to `nc` doesn't have
`--recv-only` option.
ncat works as expected (shows incoming audit messages).
Regards
Rinat
On 14.10.2023 00:42, Steve Grubb wrote:
> Hello,
>
> On Tuesday, October 10, 2023 11:53:06 AM EDT Rinat Gadelshin wrote:
>> Could I ask your help with the plugin?
> The mail list might get a faster response. I sometimes get busy.
>
>> I try to check it by the following way on my Ubuntu 20.04:
>>
>> - `systemctl stop auditd`
>> - set 'active' parameter to 'yes' (file /etc/audisp/plugins.d/af_unix.conf)
>> - `systecmtl start auditd`
>> - `systemctl status auditd` shows that the service is running.
>> - `auditctl -w /tmp/delme`
>> - `auditctl -l` shows that the rule has been successfully added.
>> - `ls -l /var/run/audispd_events` prints "srwxr-xr-x 1 root root 0 okt
>> 10 18:38 /var/run/audispd_events"
>> - launch `nc -Ul /var/run/audispd_events` in another terminal
>> - `echo 1 > /tmp/delme`
>>
>> Expected result: `nc` has received some audit events for the file.
>> Actual result: `nc` has received nothing.
> nc -U --recv-only /var/run/audispd_events
>
>> Can you tell me what I did wrong?
> See above.
>
> -Steve
>
>
7 months
Re: [linux-audit/audit-userspace] Aureport on stream of data (Issue #324)
by Burn Alting
Yes please, 2 questions :
1) Is there a way to run aureport on updating auditd logs ? That is, not
running aureport on all logs, just updating the last aureport with the
recent addition of logs ?
2) Could aureport run on combined auditd logs from more than one computor
and produce multiple outputs ?
Thank you
To answer the above
For 1. use the -checkpoint option of ausearch to generate the events.
For 2. assuming you disseminate the source hosts on the aggregating host, again
multiple invocations of ausearch will work with the -checkpoint option.
Rgds
7 months, 1 week
Sycall Rules vs Watch Rules
by Amjad Gabbar
Hi,
I have done some analysis and digging into how both the watch rules and
syscall rules are translated.
>From my understanding, in terms of logging, both the below rules are
similar. There is no difference in either of the rules.
1. -w /etc -p wa -k ETC_WATCH
2. -a always,exit -F arch=b64 -S <all syscalls part of the write and attr
classes> -F dir=/etc -F perm=wa -k ETC_WATCH
The write and attr classes consist of syscalls in
“include/asm-generic/audit_*.h“.
The perm flag is needed in the second case for including open/openat
syscalls which are not a part of the write and attr syscall list.
I'd like to verify if what I mentioned earlier is accurate, and I have an
additional point but depends on whether this is accurate.
Ali
7 months, 1 week
