10:18 a.m.
Hello all,
I have just run into the problem that many of you have: trying to parse
the audit logs.
Yesterday I read through the linux-audit mail archive. Here are the
related topics I have found:
https://www.redhat.com/archives/linux-audit/2006-March/msg00093.html
https://www.redhat.com/archives/linux-audit/2006-March/msg00158.html
https://www.redhat.com/archives/linux-audit/2007-November/msg00036.html
https://www.redhat.com/archives/linux-audit/2008-January/msg00082.html
https://www.redhat.com/archives/linux-audit/2008-March/msg00024.html
https://www.redhat.com/archives/linux-audit/2008-May/msg00029.html
https://www.redhat.com/archives/linux-audit/2008-June/msg00005.html
https://www.redhat.com/archives/linux-audit/2008-August/msg00078.html
https://www.redhat.com/archives/linux-audit/2009-March/msg00018.html
From these I see these requirements (correct me if I am wrong):
- must be backwards-compatible (doesn't break user-space on FC2, etc)
- kernel does no verifying of incoming user-space strings
- kernel must output strings in a "simple" format (e.g. no XML :-)
- able to write a parser that guarantees all (relevant) input ends up in
output
- use disk space efficiently
- handle UTF-8
Based on things other people have proposed, how does this sound:
- radix prefixes for any non-base10 number (I think audit mostly does
this already?)
- hex-encode strings (and do not quote) if:
-- contains non-ASCII or non-printable characters
- quote strings if:
-- contains whitespace or '=' or '"' (in which case you have to
output
something like '\"'
-- entirely {hex,octal,base10} characters
Or we could just save a little more headache at the cost of
space/readability and hex-encode on '=' and '"' too. Looking at
auparse, we may have to hexencode with embedded '"'.
Check if you need to encode first, then check for quoting. Something
like...
// somewhere in kernel/audit.c ?
char *audit_log_sane_string(char *str, size_t slen) {
int quoteme = 0;
size_t i, numhex = 0;
for(i = 0; i < slen; i++) {
if (!isprint(str[i])) return(hexencode(str));
if (isspace(str[i]) || str[i] == '=' || str[i] == '"') quoteme =
1;
if (isxdigit(str[i])) numhex++; // xdigit covers base8,10,16
}
if (quoteme || numhex == slen) return(quote(str));
return(strdup(str)); // kstrdup...?
}
Oh, and if anyone has ideas for making shadow-utils play nicer with
audit, I possibly have that kind of time on my hands. Also, getting rid
of the extra punctuation [:(,)] would be great.
What do you all think?
Joshua Roys
11:41 a.m.
Joshua Roys wrote:
Hello all,
I have just run into the problem that many of you have: trying to
parse the audit logs.
Based on things other people have proposed, how does this sound:
- hex-encode strings (and do not quote) if:
-- contains non-ASCII or non-printable characters
- quote strings if:
-- contains whitespace or '=' or '"' (in which case you have to
output
something like '\"'
-- entirely {hex,octal,base10} characters
What do you all think?
Your suggestion requires a kernel change. NAK on any change
which
preserves hex-encoded strings, it was a bad idea to begin with, it
remains a bad idea.
The reason why kernel audit output has not changed is fear of breaking
current user space code. However it's been often stated the only code
which is supposed to directly parse audit output is code from the audit
package (e.g. auparse, etc.).
Strings should be formatted as strings which means enclosed in double
quotes with standard C99 escaping.
As it stands now the audit libraries have hard coded lists of every
field the kernel can emit in an audit message. The test for decoding hex
strings is based on whether the field is known to be a string. Field
values *never* currently begin with a quote. If the kernel audit code
was modified to format strings such that they are always enclosed in
quotes the following positive things would occur:
* No need for hard coded list of which fields are string values.
* As long as the audit libraries are used for parsing it's fully
backwards compatible (because during parsing you first look for a quote,
if it's there you know its a string value, otherwise fall back to the
legacy logic).
* Strings are always human readable and it's obvious what is a string.
* C99 string encoding is trivial and extremely efficient so there is no
burden on the kernel.
All in all a win/win situation.
--
John Dennis <jdennis(a)redhat.com>
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
2:55 p.m.
On 03/27/2009 12:41 PM, John Dennis wrote:
Strings should be formatted as strings which means enclosed in
double
quotes with standard C99 escaping.
As it stands now the audit libraries have hard coded lists of every
field the kernel can emit in an audit message. The test for decoding hex
strings is based on whether the field is known to be a string. Field
values *never* currently begin with a quote. If the kernel audit code
was modified to format strings such that they are always enclosed in
quotes the following positive things would occur:
* No need for hard coded list of which fields are string values.
* As long as the audit libraries are used for parsing it's fully
backwards compatible (because during parsing you first look for a quote,
if it's there you know its a string value, otherwise fall back to the
legacy logic).
* Strings are always human readable and it's obvious what is a string.
* C99 string encoding is trivial and extremely efficient so there is no
burden on the kernel.
All in all a win/win situation.
Hello all,
The following is to provoke discussion; this is an issue I would like to
see fixed, and I have the time to work on it at present.
Attached is a C program that has 2 output formats depending on a global
flag (you can change the flag by incrementing argc). I use it to
demonstrate 2 options, but if one were picked, it could be used to
preserve backwards compatibility via a proc file. The binary format and
API are inspired from both an email by Paul Moore:
https://www.redhat.com/archives/linux-audit/2008-January/msg00087.html
extprot, a self-describing binary encoding:
http://eigenclass.org/R2/writings/extprot-extensible-protocols-intro
and some SSH buffer management code I wrote. Oh, and it should look
suspiciously similar to some current audit code.
The text output is basically what John Dennis was looking for, I think.
Although I don't quote numbers or do any radix prefixing right now.
Anyway. Comments? Anything I should change? Is this worth pursuing at
all (does something like this have any hope of being integrated)?
$ gcc testformat.c
$ ./a.out
(binary display via xxd)
$ ./a.out q
(text output)
One last thing: as you can see, the 3rd key/value pair msg='...' has
key/value pairs inside it, kind of like the current setup where audit
gets stuff from other code. It has a k/v user_said=\'...\'...\' with a
quote in the middle. Basically, I would also like to being some sanity
to users of audit, even though that may be insanely difficult, as has
been mentioned. Perhaps only exposing the k/v interface to them.
Something. Anything.
Thanks,
Joshua Roys
6230
days inactive
6243
days old
linux-audit@lists.linux-audit.osci.io
2 comments
2 participants
participants (2)
-
John Dennis -
Joshua Roys