Convert audit log to JSON/XML with aushape

Hi everyone, I would like to introduce a tool I have been working on for a while with Steven Grubb's guidance. It's an audit log converter called "aushape". It is based on auparse and can convert raw audit log to JSON and XML. https://github.com/Scribery/aushape Aushape can be used standalone, or as an audispd plugin for on-the-fly conversion. The output schema is being designed to correspond closely to original log structure, but to be event-, rather than record-oriented. I.e. the log consists of a series of events, with each containing one or more records. Ultimately, the schema and part of the conversion code will be generated from the official record and field dictionaries, and the intent is to have aushape a part of auditd distribution. I would like to build something that will have more uses than just within Red Hat projects I'm working on, so I ask you to please take a look at aushape, its interface and output schemas, tell me what you think about it, if you can use it, and what you would like changed. I will be glad to answer any questions you might have. One of the aims for me is to be able to store the converted audit log in ElasticSearch and query it from there easily with the help of Kibana. That puts some limits on the output structure. One of them is that an output event can only contain unique record types, e.g. it can't contain repeated "PATH" or "EXECVE" records. Instead, those repeated records are aggregated and stored as an array under the correponding record type container. "EXECVE" records are decoded into a simple argument list and "PATH" records are sorted by their item ID, in particular. This is done to simplify matching on specific record fields in ElasticSearch and to avoid using "nested" type, which Kibana has difficulties with. At the moment aushape output includes raw log messages it was generated from to help with debugging, but eventually that will be optional. Features coming soon are limiting event and record sizes (they can get very big theoretically), and reporting any conversion errors in-band, as special kind of events, preserving the original log lines, so they could be re-processed. Nick