1:22 p.m.
Hello,
I would like to point out 2 new standards that have been posted to the linux
audit web page. The first establishes the events around system start up and
shutdown. This is important because it sets the session boundaries for when a
system is up or down or crashed.
http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
The second standard is more of a forward looking standard. It explains how the
audit daemon and utilities will perform event enrichment before being stored
long term in an aggregator. The target for implementation is the 2.5 release
of the audit daemon.
http://people.redhat.com/sgrubb/audit/event-enrichment
Let me know if anyone has feedback on these standards, especially the second
one.
-Steve
1:58 p.m.
On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
Hello,
I would like to point out 2 new standards that have been posted to the linux
audit web page. The first establishes the events around system start up and
shutdown. This is important because it sets the session boundaries for when a
system is up or down or crashed.
http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
The second standard is more of a forward looking standard. It explains how the
audit daemon and utilities will perform event enrichment before being stored
long term in an aggregator. The target for implementation is the 2.5 release
of the audit daemon.
http://people.redhat.com/sgrubb/audit/event-enrichment
Let me know if anyone has feedback on these standards, especially the second
one.
Were these two specification documents created based on published
standards from an established standards body, e.g. NIST, IETF, etc?
If so, I think it would be helpful for you to reference the published
standard in your documents. If these specifications are an early
draft standard intended to be submitted to a standards body then I
would recommend mentioning the intended group in the document.
--
paul moore
www.paul-moore.com
2:25 p.m.
On Tuesday, December 08, 2015 02:58:18 PM Paul Moore wrote:
On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb(a)redhat.com>
wrote:
> Hello,
>
> I would like to point out 2 new standards that have been posted to the
> linux audit web page. The first establishes the events around system
> start up and shutdown. This is important because it sets the session
> boundaries for when a system is up or down or crashed.
>
> http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
>
> The second standard is more of a forward looking standard. It explains how
> the audit daemon and utilities will perform event enrichment before being
> stored long term in an aggregator. The target for implementation is the
> 2.5 release of the audit daemon.
>
> http://people.redhat.com/sgrubb/audit/event-enrichment
>
> Let me know if anyone has feedback on these standards, especially the
> second one.
Were these two specification documents created based on published
standards from an established standards body, e.g. NIST, IETF, etc?
No. All of the standards published to date is documenting what exists and why.
The needs are typically driven by common criteria and the need to detect
certain kinds of events for intrusion detection or anomalous conditions.
If so, I think it would be helpful for you to reference the
published
standard in your documents. If these specifications are an early
draft standard intended to be submitted to a standards body then I
would recommend mentioning the intended group in the document.
No intention of that at this point. The main issue is that we have put a lot
of patches into various utilities. We need other "like" utilities to follow
the same rules. But when you say "follow the same rules", you need some rules
published for them to follow.
The side effect is that third parties can better write analysis programs
without having to reverse engineer what the see in the event stream. They can
go straight to the source and write a program to look for certain things.
-Steve
6:28 p.m.
On Tuesday, December 08, 2015 03:25:22 PM Steve Grubb wrote:
On Tuesday, December 08, 2015 02:58:18 PM Paul Moore wrote:
> On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
> > Hello,
> >
> > I would like to point out 2 new standards that have been posted to the
> > linux audit web page. The first establishes the events around system
> > start up and shutdown. This is important because it sets the session
> > boundaries for when a system is up or down or crashed.
> >
> > http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> >
> > The second standard is more of a forward looking standard. It explains
> > how
> > the audit daemon and utilities will perform event enrichment before
> > being
> > stored long term in an aggregator. The target for implementation is the
> > 2.5 release of the audit daemon.
> >
> > http://people.redhat.com/sgrubb/audit/event-enrichment
> >
> > Let me know if anyone has feedback on these standards, especially the
> > second one.
>
> Were these two specification documents created based on published
> standards from an established standards body, e.g. NIST, IETF, etc?
No. All of the standards published to date is documenting what exists and
why. The needs are typically driven by common criteria and the need to
detect certain kinds of events for intrusion detection or anomalous
conditions.
Okay, let's not call these "standards" and just stick with
"specifications".
The term standards has all sorts of connotations associated with it, both good
and bad, and I think we should be clear when we start talking with other
developers. I think it would also be *very* helpful if you could explain the
motivation behind these specs so we understand what problems you are trying to
solve and what requirements you are trying to meet; you talk about this a bit
in the conclusion, but more background would be nice.
Another nit-picky comment, in the future I would suggest sending the specs
inline in your mail; having to go to my browser to read your document and then
cut-and-paste it into my email to comment on it means your request for
feedback goes to the bottom of my todo list. Lower the bar for collaboration
as much as possible, if you inline the text all we have to do is hit "reply"
to comment on the specs.
Anyway, on to your docs ...
* https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
System Lifecycle Auditing
=========================
This document will go over the events that are associated with starting up
a system and shutting it down. Knowing what events make up these actions
allows an analytical application to know the boundaries of all sessions and
actions a user may perform. It also allows identification of crashed systems
or malfunctioning services. The following table lists the events that make
up the system lifecycle in the audit trail:
AUDIT_SYSTEM_BOOT - System boot
AUDIT_SYSTEM_RUNLEVEL - System runlevel change
AUDIT_DAEMON_START - Audit daemon startup record
AUDIT_DAEMON_ABORT - Audit daemon error stop record
AUDIT_SERVICE_START - Service (daemon) start
AUDIT_SERVICE_STOP - Service (daemon) stop
AUDIT_SYSTEM_SHUTDOWN - System shutdown
AUDIT_DAEMON_END - Audit daemon normal stop record
Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just an
AUDIT_DAEMON_STOP with a field to indicate if it was a normal shutdown or a
failure as outlined in the spec? This would be more consistent with the other
daemons and the shutdown result field could potentially be reused by the init
systems for other daemons (assuming the information was conveyed via return
values or some other mechanism).
Lifecycle of the system
=======================
When the system is powered on, control is eventually turned over to an
init daemon. This daemon is responsible for starting up all other system
services and performing an order system shutdown when asked. This init
daemon should send a AUDIT_SYSTEM_BOOT event after it has done its own
initialization. Most init systems have different targets or modes of
operation that the system is turned over for interactive sessions. Examples
are multi-user console, multi-user graphical, etc.
You mention it later, but it might be a good idea to mention in this paragraph
that the audit daemon should be started as early as possible by the init
daemon.
Init will determine what runlevel the system is ultimately going to
try to
achieve. When gets there or it fails to get there, it shall issue an
AUDIT_SYSTEM_RUNLEVEL event to denote which mode of operation it was going
to be in. If an admin requests that the system switch to another runlevel,
then it should issue another AUDIT_SYSTEM_RUNLEVEL event.
I think it would be good to have a discussion about runlevels that don't
follow the traditional integer numbering, e.g. string based runlevels.
* https://people.redhat.com/sgrubb/audit/event-enrichment
Audit Event Enrichment
======================
There are times when the audit events are stored in another machine and need
to be searched at a later date. Some parts of the audit event are temporally
limited in duration or unique to a system. This makes interpretting fields
that are numbers into human readable fields hard or impossible without
running a report at the time of the event or on the machine the event
occurred on.
To address this issue, the audit daemon will get a new mode, enrich, where
the audit trail will be amended as follows at the time an event is logged:
1) Translations will be:
a) appended to the end of the event with the field's name in capital
letters
Please no, let's leave field names case insensitive, perhaps an agreed upon
suffix, e.g. "-trans"?
b) encoded if user controlled data is used for enrichment
2) The auparse library will:
a) preferentially use these fields whenever an interpretation is requested
b) if none exist, look up the fields on the local machine if necessary
I think resolving these fields on the local machine is misleading, and
potentially dangerous; this is especially true with respect to SELinux labels.
If you can't resolve the field, simply display it raw and let the operator
determine how to handle it.
--
paul moore
www.paul-moore.com
7:43 p.m.
On Tue, 2015-12-08 at 19:28 -0500, Paul Moore wrote:
On Tuesday, December 08, 2015 03:25:22 PM Steve Grubb wrote:
> On Tuesday, December 08, 2015 02:58:18 PM Paul Moore wrote:
> > On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
> > > Hello,
> > >
> > > I would like to point out 2 new standards that have been posted to the
> > > linux audit web page. The first establishes the events around system
> > > start up and shutdown. This is important because it sets the session
> > > boundaries for when a system is up or down or crashed.
> > >
> > > http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> > >
> > > The second standard is more of a forward looking standard. It explains
> > > how
> > > the audit daemon and utilities will perform event enrichment before
> > > being
> > > stored long term in an aggregator. The target for implementation is the
> > > 2.5 release of the audit daemon.
> > >
> > > http://people.redhat.com/sgrubb/audit/event-enrichment
> > >
> > > Let me know if anyone has feedback on these standards, especially the
> > > second one.
> >
> > Were these two specification documents created based on published
> > standards from an established standards body, e.g. NIST, IETF, etc?
>
> No. All of the standards published to date is documenting what exists and
> why. The needs are typically driven by common criteria and the need to
> detect certain kinds of events for intrusion detection or anomalous
> conditions.
Okay, let's not call these "standards" and just stick with
"specifications".
The term standards has all sorts of connotations associated with it, both good
and bad, and I think we should be clear when we start talking with other
developers. I think it would also be *very* helpful if you could explain the
motivation behind these specs so we understand what problems you are trying to
solve and what requirements you are trying to meet; you talk about this a bit
in the conclusion, but more background would be nice.
Another nit-picky comment, in the future I would suggest sending the specs
inline in your mail; having to go to my browser to read your document and then
cut-and-paste it into my email to comment on it means your request for
feedback goes to the bottom of my todo list. Lower the bar for collaboration
as much as possible, if you inline the text all we have to do is hit "reply"
to comment on the specs.
Anyway, on to your docs ...
* https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> System Lifecycle Auditing
> =========================
> This document will go over the events that are associated with starting up
> a system and shutting it down. Knowing what events make up these actions
> allows an analytical application to know the boundaries of all sessions and
> actions a user may perform. It also allows identification of crashed systems
> or malfunctioning services. The following table lists the events that make
> up the system lifecycle in the audit trail:
>
> AUDIT_SYSTEM_BOOT - System boot
> AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> AUDIT_DAEMON_START - Audit daemon startup record
> AUDIT_DAEMON_ABORT - Audit daemon error stop record
> AUDIT_SERVICE_START - Service (daemon) start
> AUDIT_SERVICE_STOP - Service (daemon) stop
> AUDIT_SYSTEM_SHUTDOWN - System shutdown
> AUDIT_DAEMON_END - Audit daemon normal stop record
Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just an
AUDIT_DAEMON_STOP with a field to indicate if it was a normal shutdown or a
failure as outlined in the spec? This would be more consistent with the other
daemons and the shutdown result field could potentially be reused by the init
systems for other daemons (assuming the information was conveyed via return
values or some other mechanism).
> Lifecycle of the system
> =======================
> When the system is powered on, control is eventually turned over to an
> init daemon. This daemon is responsible for starting up all other system
> services and performing an order system shutdown when asked. This init
> daemon should send a AUDIT_SYSTEM_BOOT event after it has done its own
> initialization. Most init systems have different targets or modes of
> operation that the system is turned over for interactive sessions. Examples
> are multi-user console, multi-user graphical, etc.
You mention it later, but it might be a good idea to mention in this paragraph
that the audit daemon should be started as early as possible by the init
daemon.
> Init will determine what runlevel the system is ultimately going to try to
> achieve. When gets there or it fails to get there, it shall issue an
> AUDIT_SYSTEM_RUNLEVEL event to denote which mode of operation it was going
> to be in. If an admin requests that the system switch to another runlevel,
> then it should issue another AUDIT_SYSTEM_RUNLEVEL event.
I think it would be good to have a discussion about runlevels that don't
follow the traditional integer numbering, e.g. string based runlevels.
* https://people.redhat.com/sgrubb/audit/event-enrichment
> Audit Event Enrichment
> ======================
>
> There are times when the audit events are stored in another machine and need
> to be searched at a later date. Some parts of the audit event are temporally
> limited in duration or unique to a system. This makes interpretting fields
> that are numbers into human readable fields hard or impossible without
> running a report at the time of the event or on the machine the event
> occurred on.
>
> To address this issue, the audit daemon will get a new mode, enrich, where
> the audit trail will be amended as follows at the time an event is logged:
>
> 1) Translations will be:
> a) appended to the end of the event with the field's name in capital
> letters
Please no, let's leave field names case insensitive, perhaps an agreed upon
suffix, e.g. "-trans"?
> b) encoded if user controlled data is used for enrichment
>
> 2) The auparse library will:
> a) preferentially use these fields whenever an interpretation is requested
> b) if none exist, look up the fields on the local machine if necessary
I think resolving these fields on the local machine is misleading, and
potentially dangerous; this is especially true with respect to SELinux labels.
If you can't resolve the field, simply display it raw and let the operator
determine how to handle it.
Steve,
Can you mock up some examples of an 'enriched' event showing how it is
different from what we have now.
Being one of those people who maintain a central repository of Linux
audit, my main "ingest" concerns are to have data that is simple and,
hence, quick to parse and hence normalize.
I think the risks associated with resolution of raw data can be
mitigated by optionally maintaining the raw value as well when
transmitting the event to a central repository.
Given the above is implemented, then I would recommend the modification
of ausearch to optionally translate a complete raw enriched event into
either a single json or xml record and optionally include raw and
interpreted values. The decision to include both could be driven via
pattern matched directive (eg *id|hostname). In reality, irrespective of
whether the above is implemented or not, I would recommend (and will
probably create the patch).
To me the biggest benefit of Steve's proposal is the near real time
resolution of some values. This is particularly useful for IP addresses
(given one also notes somewhere, the name servers the system uses for
resolution) given their potential reuse in short periods of time.
Regards
Burn
4:49 p.m.
On Wed, 09 Dec 2015 12:43:37 +1100
Burn Alting <burn(a)swtf.dyndns.org> wrote:
On Tue, 2015-12-08 at 19:28 -0500, Paul Moore wrote:
> On Tuesday, December 08, 2015 03:25:22 PM Steve Grubb wrote:
> > On Tuesday, December 08, 2015 02:58:18 PM Paul Moore wrote:
> > > On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb(a)redhat.com>
> > > wrote:
> > > > Hello,
> > > >
> > > > I would like to point out 2 new standards that have been
> > > > posted to the linux audit web page. The first establishes the
> > > > events around system start up and shutdown. This is important
> > > > because it sets the session boundaries for when a system is
> > > > up or down or crashed.
> > > >
> > > > http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> > > >
> > > > The second standard is more of a forward looking standard. It
> > > > explains how
> > > > the audit daemon and utilities will perform event enrichment
> > > > before being
> > > > stored long term in an aggregator. The target for
> > > > implementation is the 2.5 release of the audit daemon.
> > > >
> > > > http://people.redhat.com/sgrubb/audit/event-enrichment
> > > >
> > > > Let me know if anyone has feedback on these standards,
> > > > especially the second one.
> > >
> > > Were these two specification documents created based on
> > > published standards from an established standards body, e.g.
> > > NIST, IETF, etc?
> >
> > No. All of the standards published to date is documenting what
> > exists and why. The needs are typically driven by common criteria
> > and the need to detect certain kinds of events for intrusion
> > detection or anomalous conditions.
>
> Okay, let's not call these "standards" and just stick with
> "specifications". The term standards has all sorts of connotations
> associated with it, both good and bad, and I think we should be
> clear when we start talking with other developers. I think it
> would also be *very* helpful if you could explain the motivation
> behind these specs so we understand what problems you are trying to
> solve and what requirements you are trying to meet; you talk about
> this a bit in the conclusion, but more background would be nice.
>
> Another nit-picky comment, in the future I would suggest sending
> the specs inline in your mail; having to go to my browser to read
> your document and then cut-and-paste it into my email to comment on
> it means your request for feedback goes to the bottom of my todo
> list. Lower the bar for collaboration as much as possible, if you
> inline the text all we have to do is hit "reply" to comment on the
> specs.
>
> Anyway, on to your docs ...
>
> * https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
>
> > System Lifecycle Auditing
> > =========================
> > This document will go over the events that are associated with
> > starting up a system and shutting it down. Knowing what events
> > make up these actions allows an analytical application to know
> > the boundaries of all sessions and actions a user may perform. It
> > also allows identification of crashed systems or malfunctioning
> > services. The following table lists the events that make up the
> > system lifecycle in the audit trail:
> >
> > AUDIT_SYSTEM_BOOT - System boot
> > AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> > AUDIT_DAEMON_START - Audit daemon startup record
> > AUDIT_DAEMON_ABORT - Audit daemon error stop record
> > AUDIT_SERVICE_START - Service (daemon) start
> > AUDIT_SERVICE_STOP - Service (daemon) stop
> > AUDIT_SYSTEM_SHUTDOWN - System shutdown
> > AUDIT_DAEMON_END - Audit daemon normal stop record
>
> Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just
> an AUDIT_DAEMON_STOP with a field to indicate if it was a normal
> shutdown or a failure as outlined in the spec? This would be more
> consistent with the other daemons and the shutdown result field
> could potentially be reused by the init systems for other daemons
> (assuming the information was conveyed via return values or some
> other mechanism).
>
> > Lifecycle of the system
> > =======================
> > When the system is powered on, control is eventually turned over
> > to an init daemon. This daemon is responsible for starting up all
> > other system services and performing an order system shutdown
> > when asked. This init daemon should send a AUDIT_SYSTEM_BOOT
> > event after it has done its own initialization. Most init systems
> > have different targets or modes of operation that the system is
> > turned over for interactive sessions. Examples are multi-user
> > console, multi-user graphical, etc.
>
> You mention it later, but it might be a good idea to mention in
> this paragraph that the audit daemon should be started as early as
> possible by the init daemon.
>
> > Init will determine what runlevel the system is ultimately going
> > to try to achieve. When gets there or it fails to get there, it
> > shall issue an AUDIT_SYSTEM_RUNLEVEL event to denote which mode
> > of operation it was going to be in. If an admin requests that the
> > system switch to another runlevel, then it should issue another
> > AUDIT_SYSTEM_RUNLEVEL event.
>
> I think it would be good to have a discussion about runlevels that
> don't follow the traditional integer numbering, e.g. string based
> runlevels.
>
> * https://people.redhat.com/sgrubb/audit/event-enrichment
>
> > Audit Event Enrichment
> > ======================
> >
> > There are times when the audit events are stored in another
> > machine and need to be searched at a later date. Some parts of
> > the audit event are temporally limited in duration or unique to a
> > system. This makes interpretting fields that are numbers into
> > human readable fields hard or impossible without running a report
> > at the time of the event or on the machine the event occurred on.
> >
> > To address this issue, the audit daemon will get a new mode,
> > enrich, where the audit trail will be amended as follows at the
> > time an event is logged:
> >
> > 1) Translations will be:
> > a) appended to the end of the event with the field's name in
> > capital letters
>
> Please no, let's leave field names case insensitive, perhaps an
> agreed upon suffix, e.g. "-trans"?
>
> > b) encoded if user controlled data is used for enrichment
> >
> > 2) The auparse library will:
> > a) preferentially use these fields whenever an interpretation
> > is requested b) if none exist, look up the fields on the local
> > machine if necessary
>
> I think resolving these fields on the local machine is misleading,
> and potentially dangerous; this is especially true with respect to
> SELinux labels. If you can't resolve the field, simply display it
> raw and let the operator determine how to handle it.
>
Steve,
Can you mock up some examples of an 'enriched' event showing how it is
different from what we have now.
type=LOGIN msg=audit(1449782897.896:2496): pid=1768 uid=0
subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 old-auid=4294967295
auid=4325 old-ses=4294967295 ses=1 res=1 UID="root" OLD-AUID="unset"
AUID="sgrubb"
type=SYSCALL msg=audit(1449778741.412:4952): arch=c000003e syscall=40
success=no exit=-22 a0=3 a1=0 a2=0 a3=4000 items=0 ppid=7362 pid=7994
auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
tty=(none) ses=4294967295 comm="systemd-coredum"
exe="/usr/lib/systemd/systemd-coredump"
subj=system_u:system_r:init_t:s0 key="einval-retcode" ARCH=x86_64
SYSCALL=sendfile AUID="unset" UID="root" GID="root"
EUID="root"
SUID="root" FSUID="root" EGID="root" SGID="root"
FSGID="root"
-Steve
Being one of those people who maintain a central repository of Linux
audit, my main "ingest" concerns are to have data that is simple and,
hence, quick to parse and hence normalize.
I think the risks associated with resolution of raw data can be
mitigated by optionally maintaining the raw value as well when
transmitting the event to a central repository.
Given the above is implemented, then I would recommend the
modification of ausearch to optionally translate a complete raw
enriched event into either a single json or xml record and optionally
include raw and interpreted values. The decision to include both
could be driven via pattern matched directive (eg *id|hostname). In
reality, irrespective of whether the above is implemented or not, I
would recommend (and will probably create the patch).
To me the biggest benefit of Steve's proposal is the near real time
resolution of some values. This is particularly useful for IP
addresses (given one also notes somewhere, the name servers the
system uses for resolution) given their potential reuse in short
periods of time.
4:59 p.m.
On Thu, Dec 10, 2015 at 5:49 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
On Wed, 09 Dec 2015 12:43:37 +1100
Burn Alting <burn(a)swtf.dyndns.org> wrote:
> Steve,
>
> Can you mock up some examples of an 'enriched' event showing how it is
> different from what we have now.
type=LOGIN msg=audit(1449782897.896:2496): pid=1768 uid=0
subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 old-auid=4294967295
auid=4325 old-ses=4294967295 ses=1 res=1 UID="root" OLD-AUID="unset"
AUID="sgrubb"
type=SYSCALL msg=audit(1449778741.412:4952): arch=c000003e syscall=40
success=no exit=-22 a0=3 a1=0 a2=0 a3=4000 items=0 ppid=7362 pid=7994
auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
tty=(none) ses=4294967295 comm="systemd-coredum"
exe="/usr/lib/systemd/systemd-coredump"
subj=system_u:system_r:init_t:s0 key="einval-retcode" ARCH=x86_64
SYSCALL=sendfile AUID="unset" UID="root" GID="root"
EUID="root"
SUID="root" FSUID="root" EGID="root" SGID="root"
FSGID="root"
This could be confusing on a system with "unset" as a user.
--
paul moore
www.paul-moore.com
11:11 p.m.
On 15/12/10, Paul Moore wrote:
On Thu, Dec 10, 2015 at 5:49 PM, Steve Grubb
<sgrubb(a)redhat.com> wrote:
> On Wed, 09 Dec 2015 12:43:37 +1100
> Burn Alting <burn(a)swtf.dyndns.org> wrote:
>
>> Steve,
>>
>> Can you mock up some examples of an 'enriched' event showing how it is
>> different from what we have now.
>
> type=LOGIN msg=audit(1449782897.896:2496): pid=1768 uid=0
> subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 old-auid=4294967295
> auid=4325 old-ses=4294967295 ses=1 res=1 UID="root"
OLD-AUID="unset"
> AUID="sgrubb"
>
> type=SYSCALL msg=audit(1449778741.412:4952): arch=c000003e syscall=40
> success=no exit=-22 a0=3 a1=0 a2=0 a3=4000 items=0 ppid=7362 pid=7994
> auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
> tty=(none) ses=4294967295 comm="systemd-coredum"
> exe="/usr/lib/systemd/systemd-coredump"
> subj=system_u:system_r:init_t:s0 key="einval-retcode" ARCH=x86_64
> SYSCALL=sendfile AUID="unset" UID="root" GID="root"
EUID="root"
> SUID="root" FSUID="root" EGID="root"
SGID="root" FSGID="root"
This could be confusing on a system with "unset" as a user.
As we do with "none", "(unset)" might be better?
paul moore
- RGB
--
Richard Guy Briggs <rbriggs(a)redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
10:35 p.m.
On Tue, 08 Dec 2015 19:28:22 -0500
Paul Moore <paul(a)paul-moore.com> wrote:
Okay, let's not call these "standards" and just stick
with
"specifications". The term standards has all sorts of connotations
associated with it, both good and bad, and I think we should be clear
when we start talking with other developers. I think it would also
be *very* helpful if you could explain the motivation behind these
specs so we understand what problems you are trying to solve and what
requirements you are trying to meet; you talk about this a bit in the
conclusion, but more background would be nice.
Another nit-picky comment, in the future I would suggest sending the
specs inline in your mail;
I think it was updated 6 times between my email and your's. :-) The
link means you review a doc that already is fixed in a number of ways
before your info.
Anyway, on to your docs ...
* https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> System Lifecycle Auditing
> =========================
> This document will go over the events that are associated with
> starting up a system and shutting it down. Knowing what events make
> up these actions allows an analytical application to know the
> boundaries of all sessions and actions a user may perform. It also
> allows identification of crashed systems or malfunctioning
> services. The following table lists the events that make up the
> system lifecycle in the audit trail:
>
> AUDIT_SYSTEM_BOOT - System boot
> AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> AUDIT_DAEMON_START - Audit daemon startup record
> AUDIT_DAEMON_ABORT - Audit daemon error stop record
> AUDIT_SERVICE_START - Service (daemon) start
> AUDIT_SERVICE_STOP - Service (daemon) stop
> AUDIT_SYSTEM_SHUTDOWN - System shutdown
> AUDIT_DAEMON_END - Audit daemon normal stop record
Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just
an AUDIT_DAEMON_STOP with a field to indicate if it was a normal
shutdown or a failure as outlined in the spec?
This is documenting historical behavior that has not changed.
Originally, the event type held more weight about what was happening to
the system. I think there's a few more details for this particular
event, but that is the main reason.
This would be more
consistent with the other daemons and the shutdown result field could
potentially be reused by the init systems for other daemons (assuming
the information was conveyed via return values or some other
mechanism).
> Lifecycle of the system
> =======================
> When the system is powered on, control is eventually turned over to
> an init daemon. This daemon is responsible for starting up all
> other system services and performing an order system shutdown when
> asked. This init daemon should send a AUDIT_SYSTEM_BOOT event after
> it has done its own initialization. Most init systems have
> different targets or modes of operation that the system is turned
> over for interactive sessions. Examples are multi-user console,
> multi-user graphical, etc.
You mention it later, but it might be a good idea to mention in this
paragraph that the audit daemon should be started as early as
possible by the init daemon.
> Init will determine what runlevel the system is ultimately going to
> try to achieve. When gets there or it fails to get there, it shall
> issue an AUDIT_SYSTEM_RUNLEVEL event to denote which mode of
> operation it was going to be in. If an admin requests that the
> system switch to another runlevel, then it should issue another
> AUDIT_SYSTEM_RUNLEVEL event.
I think it would be good to have a discussion about runlevels that
don't follow the traditional integer numbering, e.g. string based
runlevels.
* https://people.redhat.com/sgrubb/audit/event-enrichment
> Audit Event Enrichment
> ======================
>
> There are times when the audit events are stored in another machine
> and need to be searched at a later date. Some parts of the audit
> event are temporally limited in duration or unique to a system.
> This makes interpretting fields that are numbers into human
> readable fields hard or impossible without running a report at the
> time of the event or on the machine the event occurred on.
>
> To address this issue, the audit daemon will get a new mode,
> enrich, where the audit trail will be amended as follows at the
> time an event is logged:
>
> 1) Translations will be:
> a) appended to the end of the event with the field's name in
> capital letters
Please no, let's leave field names case insensitive, perhaps an
agreed upon suffix, e.g. "-trans"?
This is solving multiple issues. Grep auid would also hit auid-trans.
Also, there are a number of strstr in various bits of code. So,
changing to upper case blocks all lower case searches using case
sensitive constructs. Nowhere does the audit system consider the field
names case insensitive. On the contrary, the specifications call out
for field names to be lower case during event creation.
> b) encoded if user controlled data is used for enrichment
>
> 2) The auparse library will:
> a) preferentially use these fields whenever an interpretation is
> requested b) if none exist, look up the fields on the local machine
> if necessary
I think resolving these fields on the local machine is misleading,
and potentially dangerous; this is especially true with respect to
SELinux labels. If you can't resolve the field, simply display it raw
and let the operator determine how to handle it.
Agreed and hence the proposal. I appreciate the feedback. I'll see
about making some changes to the proposal next week.
-Steve
10:50 a.m.
On Wednesday, December 09, 2015 11:35:02 PM Steve Grubb wrote:
On Tue, 08 Dec 2015 19:28:22 -0500
Paul Moore <paul(a)paul-moore.com> wrote:
> Okay, let's not call these "standards" and just stick with
> "specifications". The term standards has all sorts of connotations
> associated with it, both good and bad, and I think we should be clear
> when we start talking with other developers. I think it would also
> be *very* helpful if you could explain the motivation behind these
> specs so we understand what problems you are trying to solve and what
> requirements you are trying to meet; you talk about this a bit in the
> conclusion, but more background would be nice.
>
> Another nit-picky comment, in the future I would suggest sending the
> specs inline in your mail;
I think it was updated 6 times between my email and your's. :-) The
link means you review a doc that already is fixed in a number of ways
before your info.
The link means it is annoying for me to review and comment. The more annoying
something is, the lower the priority I assign it. The lower priority ...
well, you get the idea. At least I hope you get the idea.
When you ask for comments from a group of people you are always going to be in
the position of having to reconcile multiple sets of feedback across multiple
revisions of a document. Hosting a plaintext document over http doesn't solve
that in any meaningful way, in some cases it actually hides changes and makes
the document history less clear.
Just use the mailing list.
> Anyway, on to your docs ...
>
> * https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
>
> > System Lifecycle Auditing
> > =========================
> > This document will go over the events that are associated with
> > starting up a system and shutting it down. Knowing what events make
> > up these actions allows an analytical application to know the
> > boundaries of all sessions and actions a user may perform. It also
> > allows identification of crashed systems or malfunctioning
> > services. The following table lists the events that make up the
> > system lifecycle in the audit trail:
> >
> > AUDIT_SYSTEM_BOOT - System boot
> > AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> > AUDIT_DAEMON_START - Audit daemon startup record
> > AUDIT_DAEMON_ABORT - Audit daemon error stop record
> > AUDIT_SERVICE_START - Service (daemon) start
> > AUDIT_SERVICE_STOP - Service (daemon) stop
> > AUDIT_SYSTEM_SHUTDOWN - System shutdown
> > AUDIT_DAEMON_END - Audit daemon normal stop record
>
> Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just
> an AUDIT_DAEMON_STOP with a field to indicate if it was a normal
> shutdown or a failure as outlined in the spec?
This is documenting historical behavior that has not changed.
Originally, the event type held more weight about what was happening to
the system. I think there's a few more details for this particular
event, but that is the main reason.
It might be nice to add some text about what is currently done in your
document.
> > Audit Event Enrichment
> > ======================
> >
> > There are times when the audit events are stored in another machine
> > and need to be searched at a later date. Some parts of the audit
> > event are temporally limited in duration or unique to a system.
> > This makes interpretting fields that are numbers into human
> > readable fields hard or impossible without running a report at the
> > time of the event or on the machine the event occurred on.
> >
> > To address this issue, the audit daemon will get a new mode,
> > enrich, where the audit trail will be amended as follows at the
> > time an event is logged:
> >
> > 1) Translations will be:
> > a) appended to the end of the event with the field's name in
> >
> > capital letters
>
> Please no, let's leave field names case insensitive, perhaps an
> agreed upon suffix, e.g. "-trans"?
This is solving multiple issues. Grep auid would also hit auid-trans.
If you are simply greping on "auid" without any consideration of the
characters preceding or following that string you are asking for trouble. I
don't buy this as a valid reason to use uppercase for the translated fields.
Also, there are a number of strstr in various bits of code.
The same arguments apply here.
So, changing to upper case blocks all lower case searches using case
sensitive constructs.
You still run into the problem that the searches you propose, e.g. 'grep
"auid"' are bad searches in the first place and moving to uppercase field
names doesn't really solve anything as 'grep "AUID"' is still
problematic.
--
paul moore
www.paul-moore.com
11:40 a.m.
My comments are more from a log user (not developer) perspective. We are
exporting close to 10GB/day of mostly auditd logs. This will potentially go
upto 20GB/day next year.
I'd prefer the ability to translate all auditd logs before they are written
to disk. I believe this is what you have proposed, just wanted to confirm.
This means that uid / gid / auid will resolve on the same machine that the
logs were generated on.
You mentioned IP translation in the enrichment doc. I'm currently tackling
this for automated analysis of these logs (syscall 42). Currently the IPs
are written in hex. It would be nice to have this translated into the IPv4
decimal octets before the logs are written (I haven't checked what is
written for IPv6).
One thing I'm having a very hard time reconciling are the multiple lines
per log event.
Thanks,
-Farhan
On Wed, Dec 9, 2015 at 11:35 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
On Tue, 08 Dec 2015 19:28:22 -0500
Paul Moore <paul(a)paul-moore.com> wrote:
> Okay, let's not call these "standards" and just stick with
> "specifications". The term standards has all sorts of connotations
> associated with it, both good and bad, and I think we should be clear
> when we start talking with other developers. I think it would also
> be *very* helpful if you could explain the motivation behind these
> specs so we understand what problems you are trying to solve and what
> requirements you are trying to meet; you talk about this a bit in the
> conclusion, but more background would be nice.
>
> Another nit-picky comment, in the future I would suggest sending the
> specs inline in your mail;
I think it was updated 6 times between my email and your's. :-) The
link means you review a doc that already is fixed in a number of ways
before your info.
> Anyway, on to your docs ...
>
> * https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
>
> > System Lifecycle Auditing
> > =========================
> > This document will go over the events that are associated with
> > starting up a system and shutting it down. Knowing what events make
> > up these actions allows an analytical application to know the
> > boundaries of all sessions and actions a user may perform. It also
> > allows identification of crashed systems or malfunctioning
> > services. The following table lists the events that make up the
> > system lifecycle in the audit trail:
> >
> > AUDIT_SYSTEM_BOOT - System boot
> > AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> > AUDIT_DAEMON_START - Audit daemon startup record
> > AUDIT_DAEMON_ABORT - Audit daemon error stop record
> > AUDIT_SERVICE_START - Service (daemon) start
> > AUDIT_SERVICE_STOP - Service (daemon) stop
> > AUDIT_SYSTEM_SHUTDOWN - System shutdown
> > AUDIT_DAEMON_END - Audit daemon normal stop record
>
> Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just
> an AUDIT_DAEMON_STOP with a field to indicate if it was a normal
> shutdown or a failure as outlined in the spec?
This is documenting historical behavior that has not changed.
Originally, the event type held more weight about what was happening to
the system. I think there's a few more details for this particular
event, but that is the main reason.
> This would be more
> consistent with the other daemons and the shutdown result field could
> potentially be reused by the init systems for other daemons (assuming
> the information was conveyed via return values or some other
> mechanism).
>
> > Lifecycle of the system
> > =======================
> > When the system is powered on, control is eventually turned over to
> > an init daemon. This daemon is responsible for starting up all
> > other system services and performing an order system shutdown when
> > asked. This init daemon should send a AUDIT_SYSTEM_BOOT event after
> > it has done its own initialization. Most init systems have
> > different targets or modes of operation that the system is turned
> > over for interactive sessions. Examples are multi-user console,
> > multi-user graphical, etc.
>
> You mention it later, but it might be a good idea to mention in this
> paragraph that the audit daemon should be started as early as
> possible by the init daemon.
>
> > Init will determine what runlevel the system is ultimately going to
> > try to achieve. When gets there or it fails to get there, it shall
> > issue an AUDIT_SYSTEM_RUNLEVEL event to denote which mode of
> > operation it was going to be in. If an admin requests that the
> > system switch to another runlevel, then it should issue another
> > AUDIT_SYSTEM_RUNLEVEL event.
>
> I think it would be good to have a discussion about runlevels that
> don't follow the traditional integer numbering, e.g. string based
> runlevels.
>
> * https://people.redhat.com/sgrubb/audit/event-enrichment
>
> > Audit Event Enrichment
> > ======================
> >
> > There are times when the audit events are stored in another machine
> > and need to be searched at a later date. Some parts of the audit
> > event are temporally limited in duration or unique to a system.
> > This makes interpretting fields that are numbers into human
> > readable fields hard or impossible without running a report at the
> > time of the event or on the machine the event occurred on.
> >
> > To address this issue, the audit daemon will get a new mode,
> > enrich, where the audit trail will be amended as follows at the
> > time an event is logged:
> >
> > 1) Translations will be:
> > a) appended to the end of the event with the field's name in
> > capital letters
>
> Please no, let's leave field names case insensitive, perhaps an
> agreed upon suffix, e.g. "-trans"?
This is solving multiple issues. Grep auid would also hit auid-trans.
Also, there are a number of strstr in various bits of code. So,
changing to upper case blocks all lower case searches using case
sensitive constructs. Nowhere does the audit system consider the field
names case insensitive. On the contrary, the specifications call out
for field names to be lower case during event creation.
> > b) encoded if user controlled data is used for enrichment
> >
> > 2) The auparse library will:
> > a) preferentially use these fields whenever an interpretation is
> > requested b) if none exist, look up the fields on the local machine
> > if necessary
>
> I think resolving these fields on the local machine is misleading,
> and potentially dangerous; this is especially true with respect to
> SELinux labels. If you can't resolve the field, simply display it raw
> and let the operator determine how to handle it.
Agreed and hence the proposal. I appreciate the feedback. I'll see
about making some changes to the proposal next week.
-Steve
--
Linux-audit mailing list
Linux-audit(a)redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
9:34 a.m.
On Thursday, December 10, 2015 12:40:55 PM F Rafi wrote:
My comments are more from a log user (not developer) perspective. We
are
exporting close to 10GB/day of mostly auditd logs. This will potentially go
upto 20GB/day next year.
I'd prefer the ability to translate all auditd logs before they are written
to disk. I believe this is what you have proposed, just wanted to confirm.
That is not exactly what I proposed. What I was proposing was to record the
translation of things that could change between systems and thus prevent
correct interpretation later. Doing all translations is technically possible
but would slow down auditd just a bit and increase the amount of data on disk.
But doing this is not really necessary for the native audit tools.
But I guess this gives me an opportunity to ask the community what tools they
are using for audit log collection and viewing? Its been a couple years since
e had this discussion on the mail list and I think some things have changed.
Do people use ELK?
Apache Flume?
Something else?
It might be possible to write a plugin to translate the audit logs into the
native format of these tools.
This means that uid / gid / auid will resolve on the same machine
that the
logs were generated on.
You mentioned IP translation in the enrichment doc. I'm currently tackling
this for automated analysis of these logs (syscall 42). Currently the IPs
are written in hex. It would be nice to have this translated into the IPv4
decimal octets before the logs are written (I haven't checked what is
written for IPv6).
One thing I'm having a very hard time reconciling are the multiple lines
per log event.
In a way, that can't be avoided. I am working on a new representation of the
audit stream that would be just one line in the "cooked" mode.
-Steve
10:38 a.m.
Steve,
The last place I was at heavily used Splunk and then transitioned to dual-routing a
substantial portion of the logs from across the infrastructure to ELK, as well.
-Joe
From: Steve Grubb <sgrubb(a)redhat.com>
To: F Rafi <farhanible(a)gmail.com>; "linux-audit(a)redhat.com"
<linux-audit(a)redhat.com>
Sent: Monday, December 14, 2015 10:34 AM
Subject: Re: New draft standards
But I guess this gives me an opportunity to ask the community what tools they
are using for audit log collection and viewing? Its been a couple years since
e had this discussion on the mail list and I think some things have changed.
Do people use ELK?
Apache Flume?
Something else?
It might be possible to write a plugin to translate the audit logs into the
native format of these tools.
-Steve
11:01 a.m.
ELK
Splunk
We use a proprietary vendor product that migrates data into an HDFS store
via RabbitMQ based collectors and dumps them in raw form. From there I
have access to all the usual "big data" tools albeit I'm not using Flume
just yet, we're still trying to get a handle on operationalizing all the
various big data component so that data science developers can focus on
development instead of operations and support of the hardware/software
ecosystem.
Kevin D Dienst
From: Joe Wulf <joe_wulf(a)yahoo.com>
To: "linux-audit(a)redhat.com" <linux-audit(a)redhat.com>
Date: 12/14/2015 10:51 AM
Subject: Re: New draft standards
Sent by: linux-audit-bounces(a)redhat.com
Steve,
The last place I was at heavily used Splunk and then transitioned to
dual-routing a substantial portion of the logs from across the
infrastructure to ELK, as well.
-Joe
From: Steve Grubb <sgrubb(a)redhat.com>
To: F Rafi <farhanible(a)gmail.com>; "linux-audit(a)redhat.com"
<linux-audit(a)redhat.com>
Sent: Monday, December 14, 2015 10:34 AM
Subject: Re: New draft standards
But I guess this gives me an opportunity to ask the community what tools
they
are using for audit log collection and viewing? Its been a couple years
since
e had this discussion on the mail list and I think some things have
changed.
Do people use ELK?
Apache Flume?
Something else?
It might be possible to write a plugin to translate the audit logs into
the
native format of these tools.
-Steve
--
Linux-audit mailing list
Linux-audit(a)redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
4:12 p.m.
I use a proprietary ELK-like system based on ausearch's -i option. I would
like to see some variant outputs from ausearch that "packages" events into
parse-friendly formats (json, xml) that also incorporates the local
transformations Steve proposes. I believe this would be the most generic
solution to support centralised log management.
I am travelling now, but can write up a specification for review next week.
Burn Alting
On 15 Dec 2015 4:13 am, <Kevin.Dienst(a)usbank.com> wrote:
ELK
Splunk
We use a proprietary vendor product that migrates data into an HDFS store
via
RabbitMQ based collectors and dumps them in raw form. From there I have
access to all the usual "big data" tools albeit I'm not using Flume just
yet, we're still trying to get a handle on operationalizing all the various
big data component so that data science developers can focus on development
instead of operations and support of the hardware/software ecosystem.
Kevin D Dienst
From: Joe Wulf <joe_wulf(a)yahoo.com>
To: "linux-audit(a)redhat.com" <linux-audit(a)redhat.com>
Date: 12/14/2015 10:51 AM
Subject: Re: New draft standards
Sent by: linux-audit-bounces(a)redhat.com
________________________________
Steve,
The last place I was at heavily used Splunk and then transitioned to
dual-routing a
substantial portion of the logs from across the
infrastructure to ELK, as well.
-Joe
________________________________
From: Steve Grubb <sgrubb(a)redhat.com>
To: F Rafi <farhanible(a)gmail.com>; "linux-audit(a)redhat.com" <
linux-audit(a)redhat.com>
Sent: Monday, December 14, 2015 10:34 AM
Subject: Re: New draft standards
But I guess this gives me an opportunity to ask the community what tools
they
are using for audit log collection and viewing? Its been a couple
years
since
e had this discussion on the mail list and I think some things have
changed.
Do people use ELK?
Apache Flume?
Something else?
It might be possible to write a plugin to translate the audit logs into
the
native format of these tools.
-Steve
--
Linux-audit mailing list
Linux-audit(a)redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
--
Linux-audit mailing list
Linux-audit(a)redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
7:46 a.m.
On Tuesday, December 15, 2015 09:12:54 AM Burn Alting wrote:
I use a proprietary ELK-like system based on ausearch's -i
option. I would
like to see some variant outputs from ausearch that "packages" events into
parse-friendly formats (json, xml) that also incorporates the local
transformations Steve proposes. I believe this would be the most generic
solution to support centralised log management.
I am travelling now, but can write up a specification for review next week.
Yes, please do send something to the mail list for people to look at and
comment on.
If anyone wants to help influence future direction and does not want to do it
on the list, please contact me offlist and let me know how you aggregate logs.
We have to address central log aggregation and I would like to see what the
majority are using to know where effort would be best spent.
I did run across this page in my survey:
http://buildoop.github.io/
It mentions audit log processing. No idea if anyone is using this either.
-Steve
On 15 Dec 2015 4:13 am, <Kevin.Dienst(a)usbank.com> wrote:
> ELK
> Splunk
>
> We use a proprietary vendor product that migrates data into an HDFS store
via RabbitMQ based collectors and dumps them in raw form. From there I have
access to all the usual "big data" tools albeit I'm not using Flume just
yet, we're still trying to get a handle on operationalizing all the various
big data component so that data science developers can focus on development
instead of operations and support of the hardware/software ecosystem.
11:12 p.m.
On Tue, 2015-12-15 at 08:46 -0500, Steve Grubb wrote:
On Tuesday, December 15, 2015 09:12:54 AM Burn Alting wrote:
> I use a proprietary ELK-like system based on ausearch's -i option. I would
> like to see some variant outputs from ausearch that "packages" events
into
> parse-friendly formats (json, xml) that also incorporates the local
> transformations Steve proposes. I believe this would be the most generic
> solution to support centralised log management.
>
> I am travelling now, but can write up a specification for review next week.
Yes, please do send something to the mail list for people to look at and
comment on.
All,
To reiterate, my need is to generate easy to parse events over which
local interpretation has been applied, retaining raw input to the some
of the interpretations if required. I want to then transmit the complete
interpreted event to my central event repository.
My proposal is that ausearch gains the following 'interpreted output'
options
--Xo plain|json|xml
generate plain (cf --interpret), xml or json formatted events
--Xr key_a'+'key_b'+'key_c
include raw value for given keys using the the new key
__r_key_a, __r_key_b, etc. The special key __all__ is
interpreted to retain the complete raw record. If the raw value
has no interpreted value, then we will end up with two keys with
the same value.
I have attached the XSD from which the XML and JSON formats could be
defined.
The following provides some examples.
For the raw event (on a system recording execve's and with name_format =
fqd configured)
node=fedora23.a1959.org type=SYSCALL msg=audit(1450409042.880:61882):
arch=c000003e syscall=59 success=yes exit=0 a0=561c5e714d60
a1=561c5e6dbb90 a2=561c5e630920 a3=561c5e6dbb80 items=2 ppid=27269
pid=29282 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0
fsgid=0 tty=pts0 ses=8 comm="ausearch" exe="/usr/sbin/ausearch"
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="cmds"
node=fedora23.a1959.org type=EXECVE msg=audit(1450409042.880:61882):
argc=4 a0="ausearch" a1="-i" a2="-if"
a3="/var/log/audit/audit.log"
node=fedora23.a1959.org type=CWD msg=audit(1450409042.880:61882):
cwd="/home/burn/audit-2.4.4_debug_fix"
node=fedora23.a1959.org type=PATH msg=audit(1450409042.880:61882):
item=0 name="/sbin/ausearch" inode=134573468 dev=fd:00 mode=0100755
ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:bin_t:s0 nametype=NORMAL
node=fedora23.a1959.org type=PATH msg=audit(1450409042.880:61882):
item=1 name="/lib64/ld-linux-x86-64.so.2" inode=134397639 dev=fd:00
mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0
nametype=NORMAL
node=fedora23.a1959.org type=PROCTITLE msg=audit(1450409042.880:61882):
proctitle=6175736561726368002D69002D6966002F7661722F6C6F672F61756469742F61756469742E6C6F67
Running ausearch with the proposed changes becomes, for --Xo plain
node=fedora23.a1959.org type=PROCTITLE msg=audit(12/18/2015
14:24:02.880:61882) : proctitle=ausearch -i
-if /var/log/audit/audit.log
node=fedora23.a1959.org type=PATH msg=audit(12/18/2015
14:24:02.880:61882) : item=1 name=/lib64/ld-linux-x86-64.so.2
inode=134397639 dev=fd:00 mode=file,755 ouid=root ogid=root rdev=00:00
obj=system_u:object_r:ld_so_t:s0 nametype=NORMAL
node=fedora23.a1959.org type=PATH msg=audit(12/18/2015
14:24:02.880:61882) : item=0 name=/sbin/ausearch inode=134573468
dev=fd:00 mode=file,755 ouid=root ogid=root rdev=00:00
obj=system_u:object_r:bin_t:s0 nametype=NORMAL
node=fedora23.a1959.org type=CWD msg=audit(12/18/2015
14:24:02.880:61882) : cwd=/home/burn/audit-2.4.4_debug_fix
node=fedora23.a1959.org type=EXECVE msg=audit(12/18/2015
14:24:02.880:61882) : argc=4 a0=ausearch a1=-i a2=-if
a3=/var/log/audit/audit.log
node=fedora23.a1959.org type=SYSCALL msg=audit(12/18/2015
14:24:02.880:61882) : arch=x86_64 syscall=execve success=yes exit=0
a0=0x561c5e714d60 a1=0x561c5e6dbb90 a2=0x561c5e630920 a3=0x561c5e6dbb80
items=2 ppid=27269 pid=29282 auid=burn uid=root gid=root euid=root
suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts0 ses=8
comm=ausearch exe=/usr/sbin/ausearch
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=cmds
and, for --Xo xml
<event>
<node>fedora23.a1959.org</node>
<time>2015-12-18T14:24:02.880+11:00</time>
<serial>1450409042.880:61882</serial>
<records>
<record>
<type>PROCTITLE</type>
<data name="proctitle" value="ausearch -i
-if /var/log/audit/audit.log" />
</record>
<record>
<type>PATH</type>
<data name="item" value="1" />
<data name="name" value="/lib64/ld-linux-x86-64.so.2" />
<data name="inode" value="134397639" />
<data name="dev" value="fd:00" />
...
<data name="nametype" value="NORMAL" />
</record>
...
<record>
<type>SYSCALL</type>
<data name="arch" value="x86_64" />
<data name="syscall" value="execve" />
..
<data name="subj"
value="unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023" />
<data name="key" value="cmds" />
</record>
</records>
</event>
and, for --Xo json
{
"event": {
"node": "fedora23.a1959.org",
"time": "2015-12-18T14:24:02.880+11:00",
"serial": "1450409042.880:61882",
"records": {
"record": [
{
"type": "PROCTITLE",
"data": {
"_name": "proctitle",
"_value": "ausearch -i -if /var/log/audit/audit.log"
}
},
{
"type": "PATH",
"data": [
{
"_name": "item",
"_value": "1"
},
{
"_name": "name",
"_value": "/lib64/ld-linux-x86-64.so.2"
},
{
"_name": "inode",
"_value": "134397639"
},
{
"_name": "dev",
"_value": "fd:00"
},
...
{
"_name": "nametype",
"_value": "NORMAL"
}
]
},
...
{
"type": "SYSCALL",
"data": [
{
"_name": "arch",
"_value": "x86_64"
},
{
"_name": "syscall",
"_value": "execve"
},
...
{
"_name": "subj",
"_value":
"unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023"
},
{
"_name": "key",
"_value": "cmds"
}
]
}
]
}
}
}
Specifying --Xr __all__ does nothing for plain, but for xml adds a raw
element within each record, as per
...
<records>
<record>
<type>PROCTITLE</type>
<raw>node=fedora23.a1959.org type=PROCTITLE
msg=audit(1450409042.880:61882):
proctitle=6175736561726368002D69002D6966002F7661722F6C6F672F61756469742F61756469742E6C6F67</raw>
...
</record>
<record>
<type>PATH</type>
<raw>node=fedora23.a1959.org type=PATH
msg=audit(1450409042.880:61882): item=1
name="/lib64/ld-linux-x86-64.so.2" inode=134397639 dev=fd:00
mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0
nametype=NORMAL</raw>
...
and for json adds a raw key within each record, as per
"record": [
{
"type": "PROCTITLE",
"raw": "node=fedora23.a1959.org type=PROCTITLE
msg=audit(1450409042.880:61882):
proctitle=6175736561726368002D69002D6966002F7661722F6C6F672F61756469742F61756469742E6C6F67",
"data": {
"_name": "proctitle",
"_value": "ausearch -i -if /var/log/audit/audit.log"
}
},
{
"type": "PATH",
"raw": "node=fedora23.a1959.org type=PATH
msg=audit(1450409042.880:61882): item=1 name=
\"/lib64/ld-linux-x86-64.so.2\" inode=134397639 dev=fd:00 mode=0100755
ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0
nametype=NORMAL",
"data": [
{
...
And if you want arbitrary raw key values as well as their interpreted
value, then provide a '+' separated list of keys. For example,
--Xr auid|syscall|a2
applied to the following raw record
node=fedora23.a1959.org type=SYSCALL msg=audit(1450410618.410:62231):
arch=c000003e syscall=268 success=yes exit=0 a0=ffffffffffffff9c
a1=562de17c80f0 a2=1e8 a3=fffff3ff items=1 ppid=27269 pid=29705
auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
tty=pts0 ses=8 comm="chmod" exe="/usr/bin/chmod"
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
key="perm_mod"
for --Xo plain, we get
node=fedora23.a1959.org type=SYSCALL msg=audit(12/18/2015
14:50:18.410:62231) : arch=x86_64 syscall=fchmodat success=yes exit=0
a0=0xffffffffffffff9c a1=0x562de17c80f0 a2=0750 a3=0xfffff3ff items=1
ppid=27269 pid=29705 auid=burn uid=root gid=root euid=root suid=root
fsuid=root egid=root sgid=root fsgid=root tty=pts0 ses=8 comm=chmod
exe=/usr/bin/chmod
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=perm_mod
__r_syscall=268 __r_auid=1000 __r_a2=1e8
for --Xo xml
...
<data name="syscall" value="fchmodat" />
...
<data name="key" value="perm_mod" />
<data name="__r_syscall" value="268" />
<data name="__r_auid" value="1000" />
<data name="__r_a2" value="1e8" />
for --Xo json
...
{
"_name": "syscall",
"_value": "fchmodat"
},
...
},
{
"_name": "key",
"_value": "perm_mod"
},
{
"_name": "__r_syscall",
"_value": "268"
},
{
"_name": "__r_auid",
"_value": "1000"
},
{
"_name": "__r_a2",
"_value": "1e8"
}
]
4:44 p.m.
On Fri, 2015-12-18 at 16:12 +1100, Burn Alting wrote:
On Tue, 2015-12-15 at 08:46 -0500, Steve Grubb wrote:
> On Tuesday, December 15, 2015 09:12:54 AM Burn Alting wrote:
> > I use a proprietary ELK-like system based on ausearch's -i option. I would
> > like to see some variant outputs from ausearch that "packages" events
into
> > parse-friendly formats (json, xml) that also incorporates the local
> > transformations Steve proposes. I believe this would be the most generic
> > solution to support centralised log management.
> >
> > I am travelling now, but can write up a specification for review next week.
>
> Yes, please do send something to the mail list for people to look at and
> comment on.
>
All,
To reiterate, my need is to generate easy to parse events over which
local interpretation has been applied, retaining raw input to the some
of the interpretations if required. I want to then transmit the complete
interpreted event to my central event repository.
My proposal is that ausearch gains the following 'interpreted output'
options
--Xo plain|json|xml
generate plain (cf --interpret), xml or json formatted events
--Xr key_a'+'key_b'+'key_c
include raw value for given keys using the the new key
__r_key_a, __r_key_b, etc. The special key __all__ is
interpreted to retain the complete raw record. If the raw value
has no interpreted value, then we will end up with two keys with
the same value.
I have attached the XSD from which the XML and JSON formats could be
defined.
Is there any interest in this? If is was available, would people make
use of it?
If so I can modify ausearch and generate a proposed patch over the
Christmas break.
Regards
Burn
10:38 a.m.
On Thursday, December 24, 2015 09:44:00 AM Burn Alting wrote:
On Fri, 2015-12-18 at 16:12 +1100, Burn Alting wrote:
> On Tue, 2015-12-15 at 08:46 -0500, Steve Grubb wrote:
> > On Tuesday, December 15, 2015 09:12:54 AM Burn Alting wrote:
> > > I use a proprietary ELK-like system based on ausearch's -i option. I
> > > would
> > > like to see some variant outputs from ausearch that "packages"
events
> > > into
> > > parse-friendly formats (json, xml) that also incorporates the local
> > > transformations Steve proposes. I believe this would be the most
> > > generic
> > > solution to support centralised log management.
> > >
> > > I am travelling now, but can write up a specification for review next
> > > week.
> >
> > Yes, please do send something to the mail list for people to look at and
> > comment on.
>
> All,
>
> To reiterate, my need is to generate easy to parse events over which
> local interpretation has been applied, retaining raw input to the some
> of the interpretations if required. I want to then transmit the complete
> interpreted event to my central event repository.
>
> My proposal is that ausearch gains the following 'interpreted output'
> options
>
> --Xo plain|json|xml
> generate plain (cf --interpret), xml or json formatted events
>
> --Xr key_a'+'key_b'+'key_c
> include raw value for given keys using the the new key
> __r_key_a, __r_key_b, etc. The special key __all__ is
> interpreted to retain the complete raw record. If the raw value
> has no interpreted value, then we will end up with two keys with
> the same value.
>
> I have attached the XSD from which the XML and JSON formats could be
> defined.
Is there any interest in this? If is was available, would people make
use of it?
I'm somewhat interested in this. I'm just not sure where the best place to do
all this is. Should it be in ausearch? Should it be in auditd? Should it be in
the remote logging plugin? Should audit utilities be modified to accept this
new form of input?
Ultimately, I am wanting to be able to reduce the audit records down to
English sentences something like this:
On 1-node at 2-time 3-subj 4-acting-as 5-results 6-action 7-what 8-using
Which maps to
1) node
2) time
3) auid, failed logins=remote system
4) uid (only when uid != auid) or role (when not unconfined_t)
5) res - successfully / failed to
6) op, syscall, type, key - requires per type classification
7) path,system
8) exe,comm
So, what I was thinking about is looking at the whole event and picking out
the node, time, subject, object, action, and results. The subject and object
would be further broken down to primary identity, secondary identity, and
attributes. I was planning to put this into an extension of auparse so that
events could be dumped out using the classification system.
My thoughts had been to organize the event data to support something along
these lines. I want to get the events easier to understand.
If so I can modify ausearch and generate a proposed patch over the
Christmas break.
At the moment, I'm looking at auditd performance improvements to prepare for
the enrichment of audit records. You're one step ahead of where I am. I hope
to finish this performance work soon so that I can start thinking about the
problem you are. :-)
Of course...we could look at the auditd performance issues together and then
move on to event formatting.
-Steve
6:30 p.m.
On Sat, 2015-12-26 at 11:38 -0500, Steve Grubb wrote:
On Thursday, December 24, 2015 09:44:00 AM Burn Alting wrote:
> On Fri, 2015-12-18 at 16:12 +1100, Burn Alting wrote:
> > On Tue, 2015-12-15 at 08:46 -0500, Steve Grubb wrote:
> > > On Tuesday, December 15, 2015 09:12:54 AM Burn Alting wrote:
> > > > I use a proprietary ELK-like system based on ausearch's -i
option. I
> > > > would
> > > > like to see some variant outputs from ausearch that
"packages" events
> > > > into
> > > > parse-friendly formats (json, xml) that also incorporates the local
> > > > transformations Steve proposes. I believe this would be the most
> > > > generic
> > > > solution to support centralised log management.
> > > >
> > > > I am travelling now, but can write up a specification for review
next
> > > > week.
> > >
> > > Yes, please do send something to the mail list for people to look at and
> > > comment on.
> >
> > All,
> >
> > To reiterate, my need is to generate easy to parse events over which
> > local interpretation has been applied, retaining raw input to the some
> > of the interpretations if required. I want to then transmit the complete
> > interpreted event to my central event repository.
> >
> > My proposal is that ausearch gains the following 'interpreted output'
> > options
> >
> > --Xo plain|json|xml
> > generate plain (cf --interpret), xml or json formatted events
> >
> > --Xr key_a'+'key_b'+'key_c
> > include raw value for given keys using the the new key
> > __r_key_a, __r_key_b, etc. The special key __all__ is
> > interpreted to retain the complete raw record. If the raw value
> > has no interpreted value, then we will end up with two keys with
> > the same value.
> >
> > I have attached the XSD from which the XML and JSON formats could be
> > defined.
>
> Is there any interest in this? If is was available, would people make
> use of it?
Steve,
I'll start with the statement I am happy to enhance the audit capability
of Linux in any way (read that as a direct offer to help).
I'm somewhat interested in this. I'm just not sure where the
best place to do
all this is. Should it be in ausearch? Should it be in auditd? Should it be in
the remote logging plugin? Should audit utilities be modified to accept this
new form of input?
I've concentrated on ausearch as this is the only tool that
comprehensively parses all existing audit records, both well formed and
incorrectly formed. As you know auparse() has difficulties with
mal-formed events. Ausearch also has the benefit of not effecting real
time performance - I'd not like auditd have to wait on an external DNS
service to timeout when attempting to resolve an 'addr' field.
Whatever is done, the code needs to be modular so that any utility, be
it ausearch, auditd or an audispd plugin, or an independent auparse()
based utility can make use of it.
Perhaps to address the higher level audit needs, we can provide an
additional output format to my proposed changes for 'interpretive
formatting' to be that of 'descriptive statements'. This would be
similar to Windows auditing when it allows one to include 'Display
Information' field which provides a 'human readable' description of the
event data.
Perhaps the thrust should be
a. address performance
b. ensure auparse() can better deal with mal-formed events
c. provide interpretative formatting
Regards
Burn
Ultimately, I am wanting to be able to reduce the audit records down
to
English sentences something like this:
On 1-node at 2-time 3-subj 4-acting-as 5-results 6-action 7-what 8-using
Which maps to
1) node
2) time
3) auid, failed logins=remote system
4) uid (only when uid != auid) or role (when not unconfined_t)
5) res - successfully / failed to
6) op, syscall, type, key - requires per type classification
7) path,system
8) exe,comm
So, what I was thinking about is looking at the whole event and picking out
the node, time, subject, object, action, and results. The subject and object
would be further broken down to primary identity, secondary identity, and
attributes. I was planning to put this into an extension of auparse so that
events could be dumped out using the classification system.
My thoughts had been to organize the event data to support something along
these lines. I want to get the events easier to understand.
> If so I can modify ausearch and generate a proposed patch over the
> Christmas break.
At the moment, I'm looking at auditd performance improvements to prepare for
the enrichment of audit records. You're one step ahead of where I am. I hope
to finish this performance work soon so that I can start thinking about the
problem you are. :-)
Of course...we could look at the auditd performance issues together and then
move on to event formatting.
9:06 a.m.
On Sunday, December 27, 2015 11:30:59 AM Burn Alting wrote:
I'll start with the statement I am happy to enhance the audit
capability
of Linux in any way (read that as a direct offer to help).
Thanks!
> I'm somewhat interested in this. I'm just not sure where
the best place to
> do all this is. Should it be in ausearch? Should it be in auditd? Should
> it be in the remote logging plugin? Should audit utilities be modified to
> accept this new form of input?
I've concentrated on ausearch as this is the only tool that
comprehensively parses all existing audit records, both well formed and
incorrectly formed. As you know auparse() has difficulties with
mal-formed events.
Its main problem is dealing with interlaced records. The kernel does not
serialize the audit stream. Whatever thread/process is executing at the moment
can trigger a multi-part or single line event. For example:
type=SYSCALL msg=audit(1396522853.933:313): arch=c000003e syscall=313
success=yes exit=0 a0=0 a1=41a396 a2=0 a3=0 items=1348 ppid=1263 pid=1264
auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
ses=4294967295 tty=(none) comm="modprobe" exe="/usr/bin/kmod"
subj=system_u:system_r:insmod_t:s0 key="module-load"
type=LOGIN msg=audit(1396522854.227:460): pid=1523 uid=0 old auid=4294967295
new auid=4325 old ses=4294967295 new ses=1 res=1
type=PATH msg=audit(1396522853.933:313): item=0 name=(null) inode=165
dev=00:06 mode=040755 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:debugfs_t:s0 nametype=PARENT
type=PATH msg=audit(1396522853.933:313): item=1 name=(null) inode=11206
dev=00:06 mode=040755 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:debugfs_t:s0 nametype=CREATE
type=LOGIN msg=audit(1396522854.315:461): pid=1518 uid=0 old auid=4294967295
new auid=4325 old ses=4294967295 new ses=1 res=1
type=PATH msg=audit(1396522853.933:313): item=2 name=(null) inode=11206
dev=00:06 mode=040755 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:debugfs_t:s0 nametype=PARENT
That should be 3 events.
Ausearch also has the benefit of not effecting real time performance
- I'd not
like auditd have to wait on an external DNS service to timeout when
attempting to resolve an 'addr' field.
Yeah, I'm thinking about just breaking down the SOCKADDR record into src/dst
ip and src/dst port and then leaving it as that for now.
Whatever is done, the code needs to be modular so that any utility,
be
it ausearch, auditd or an audispd plugin, or an independent auparse()
based utility can make use of it.
Perhaps to address the higher level audit needs, we can provide an
additional output format to my proposed changes for 'interpretive
formatting' to be that of 'descriptive statements'. This would be
similar to Windows auditing when it allows one to include 'Display
Information' field which provides a 'human readable' description of the
event data.
I'm not familiar with Windows auditing, but yeah.
Perhaps the thrust should be
a. address performance
I might have this solved. I'll send a separate email.
b. ensure auparse() can better deal with mal-formed events
This would be a big contribution to the project.
c. provide interpretative formatting
Yes. I think this a good order that has the right things in the right
priority. There is one other issue that I need to tackle at some point. The
auparse library does a lot of string manipulation. As such it winds up doing a
lot of strtok/malloc/free. I'd like to see this run faster somehow. Perhaps
moving to ustr could help. Or maybe having a list of pointers/lengths to avoid
malloc/free. But auparse needs a performance boost, too.
-Steve
1:24 a.m.
On Sun, 2015-12-27 at 10:06 -0500, Steve Grubb wrote:
On Sunday, December 27, 2015 11:30:59 AM Burn Alting wrote:
> I'll start with the statement I am happy to enhance the audit capability
> of Linux in any way (read that as a direct offer to help).
Thanks!
> > I'm somewhat interested in this. I'm just not sure where the best place
to
> > do all this is. Should it be in ausearch? Should it be in auditd? Should
> > it be in the remote logging plugin? Should audit utilities be modified to
> > accept this new form of input?
>
> I've concentrated on ausearch as this is the only tool that
> comprehensively parses all existing audit records, both well formed and
> incorrectly formed. As you know auparse() has difficulties with
> mal-formed events.
Its main problem is dealing with interlaced records. The kernel does not
serialize the audit stream. Whatever thread/process is executing at the moment
can trigger a multi-part or single line event. For example:
type=SYSCALL msg=audit(1396522853.933:313): arch=c000003e syscall=313
success=yes exit=0 a0=0 a1=41a396 a2=0 a3=0 items=1348 ppid=1263 pid=1264
auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
ses=4294967295 tty=(none) comm="modprobe" exe="/usr/bin/kmod"
subj=system_u:system_r:insmod_t:s0 key="module-load"
type=LOGIN msg=audit(1396522854.227:460): pid=1523 uid=0 old auid=4294967295
new auid=4325 old ses=4294967295 new ses=1 res=1
type=PATH msg=audit(1396522853.933:313): item=0 name=(null) inode=165
dev=00:06 mode=040755 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:debugfs_t:s0 nametype=PARENT
type=PATH msg=audit(1396522853.933:313): item=1 name=(null) inode=11206
dev=00:06 mode=040755 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:debugfs_t:s0 nametype=CREATE
type=LOGIN msg=audit(1396522854.315:461): pid=1518 uid=0 old auid=4294967295
new auid=4325 old ses=4294967295 new ses=1 res=1
type=PATH msg=audit(1396522853.933:313): item=2 name=(null) inode=11206
dev=00:06 mode=040755 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:debugfs_t:s0 nametype=PARENT
That should be 3 events.
I wasn't too concerned about the interleaved events but the incorrect
use of the audit message as per
type=USER_CHAUTHTOK msg=audit(12/28/2015 18:00:30.858:17862432) :
pid=13418 uid=root auid=burn ses=1
subj=unconfined_u:unconfined_r:passwd_t:s0-s0:c0.c1023 msg='op=change
password id=burn exe=/usr/bin/passwd hostname=? addr=? terminal=pts/0
res=success'
where auparse with discard the string 'password' changing
op=change password id=burn exe=/usr/bin/passwd
to
op=change id=burn exe=/usr/bin/passwd
> Ausearch also has the benefit of not effecting real time performance - I'd not
> like auditd have to wait on an external DNS service to timeout when
> attempting to resolve an 'addr' field.
Yeah, I'm thinking about just breaking down the SOCKADDR record into src/dst
ip and src/dst port and then leaving it as that for now.
> Whatever is done, the code needs to be modular so that any utility, be
> it ausearch, auditd or an audispd plugin, or an independent auparse()
> based utility can make use of it.
>
> Perhaps to address the higher level audit needs, we can provide an
> additional output format to my proposed changes for 'interpretive
> formatting' to be that of 'descriptive statements'. This would be
> similar to Windows auditing when it allows one to include 'Display
> Information' field which provides a 'human readable' description of the
> event data.
I'm not familiar with Windows auditing, but yeah.
> Perhaps the thrust should be
> a. address performance
I might have this solved. I'll send a separate email.
> b. ensure auparse() can better deal with mal-formed events
This would be a big contribution to the project.
> c. provide interpretative formatting
Yes. I think this a good order that has the right things in the right
priority. There is one other issue that I need to tackle at some point. The
auparse library does a lot of string manipulation. As such it winds up doing a
lot of strtok/malloc/free. I'd like to see this run faster somehow. Perhaps
moving to ustr could help. Or maybe having a list of pointers/lengths to avoid
malloc/free. But auparse needs a performance boost, too.
-Steve
1:28 p.m.
On 12/14/2015 08:34 AM, Steve Grubb wrote:
That is not exactly what I proposed. What I was proposing was to
record the
translation of things that could change between systems and thus prevent
correct interpretation later. Doing all translations is technically possible
but would slow down auditd just a bit and increase the amount of data on disk.
But doing this is not really necessary for the native audit tools.
But I guess this gives me an opportunity to ask the community what tools they
are using for audit log collection and viewing? Its been a couple years since
e had this discussion on the mail list and I think some things have changed.
Do people use ELK?
Apache Flume?
Something else?
It might be possible to write a plugin to translate the audit logs into the
native format of these tools.
Sorry for the late reply. Translating the salient details is for me
important.
This is especially true on systems where:
- aggregation is happening from one or more different machines (and
cannot assume federated UIDs), and
- where records are required to be kept over long periods of time
(system updates happen, UIDs are changed, people leave, etc)
I realize it carries a processing burden somewhere; this is inevitable
and I believe we'll need to design for this.
We're auditing for a reason; we need proof of who did what and in
varying degrees I believe this means persistence of accountability.
Because I'm almost a one-stop shop where I work, and the auditing
requirements are specific and particular, I have a homegrown log
collection and viewing solution for now but would prefer to incorporate
a flexible, more useful user tool. So I'm in the "something else"
category but somewhat open to change.
LCB
--
LC (Lenny) Bruzenak
lenny(a)magitekltd.com
2:49 p.m.
On 15/12/08, Steve Grubb wrote:
Hello,
I would like to point out 2 new standards that have been posted to the linux
audit web page. The first establishes the events around system start up and
shutdown. This is important because it sets the session boundaries for when a
system is up or down or crashed.
http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
A couple of very minor corrections to this first one:
--- system-lifecycle.txt.orig 2015-12-08 15:36:34.441782830 -0500
+++ system-lifecycle.txt 2015-12-08 15:38:10.763998066 -0500
@@ -62,7 +62,7 @@
/* boot */
audit_log_user_message (fd, AUDIT_SYSTEM_BOOT, "init", NULL, NULL, NULL, 1);
-/* run leve change */
+/* run level change */
snprintf (buf, sizeof (buf), "old-level=%c new-level=%c", old, level);
audit_log_user_message (fd, AUDIT_SYSTEM_RUNLEVEL, buf, NULL, NULL, NULL, 1);
@@ -77,7 +77,7 @@
audit_log_user_message (fd, AUDIT_SERVICE_START, buf, NULL, NULL, NULL, 1);
free(buf);
-Service stop events should be the same os start with the exception of using
+Service stop events should be the same as start with the exception of using
AUDIT_SERVICE_STOP as the event type. If only the pid is available, record
that as "spid". There must be a way to compare start and stop records to see
that they balance. (There are as many starts as stops.)
The second standard is more of a forward looking standard. It
explains how the
audit daemon and utilities will perform event enrichment before being stored
long term in an aggregator. The target for implementation is the 2.5 release
of the audit daemon.
http://people.redhat.com/sgrubb/audit/event-enrichment
How do you mean for IP address to be "resolved"? Is this simply a
matter of recording it? Or would this be a reverse lookup on the local
machine to get the opinion of what it should be from the DNS perspective
of the local machine, assuming different machines in the logging domain
could potentially have different views of DNS?
Let me know if anyone has feedback on these standards, especially the
second
one.
-Steve
- RGB
--
Richard Guy Briggs <rbriggs(a)redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
3:28 p.m.
On Tuesday, December 08, 2015 03:49:58 PM Richard Guy Briggs wrote:
On 15/12/08, Steve Grubb wrote:
> Hello,
>
> I would like to point out 2 new standards that have been posted to the
> linux audit web page. The first establishes the events around system
> start up and shutdown. This is important because it sets the session
> boundaries for when a system is up or down or crashed.
>
> http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
A couple of very minor corrections to this first one:
Thanks, Applied.
> The second standard is more of a forward looking standard. It
explains how
> the audit daemon and utilities will perform event enrichment before being
> stored long term in an aggregator. The target for implementation is the
> 2.5 release of the audit daemon.
>
> http://people.redhat.com/sgrubb/audit/event-enrichment
How do you mean for IP address to be "resolved"? Is this simply a
matter of recording it? Or would this be a reverse lookup on the local
machine to get the opinion of what it should be from the DNS perspective
of the local machine, assuming different machines in the logging domain
could potentially have different views of DNS?
I think the latter. Bot-nets get shut down. Systems go away. Sometimes
internal names differ from external names.
-Steve
> Let me know if anyone has feedback on these standards,
especially the
> second one.
>
> -Steve
- RGB
--
Richard Guy Briggs <rbriggs(a)redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems,
Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
3280
days inactive
3301
days old
linux-audit@lists.linux-audit.osci.io
24 comments
9 participants
participants (9)
-
Burn Alting -
Burn Alting -
F Rafi -
Joe Wulf -
Kevin.Dienst＠usbank.com -
LC Bruzenak -
Paul Moore -
Richard Guy Briggs -
Steve Grubb