7:12 p.m.
Hi,
I would like to audit all changes to a directory tree using the linux
auditing system[1].
# auditctl -a exit,always -F dir=/etc/ -F perm=wa
It seems like the GNU coreutils are enough to break the audit trail.
The resulting SYSCALL events provide CWD and multiple PATH records,
depending on the syscall. If one of the PATH records is relative, I can
reconstruct the absolute path using the CWD record.
However, that does not work for the whole *at syscall family
(unlinkat(2), renameat(2), linkat(2), ...); accepting paths relative to
a given directory file descriptor. GNU coreutils are prominent users,
for example "rm -r" making use of unlinkat(2) to prevent races.
Things like dup(2) and fd passing via unix domain sockets come to mind.
It's the same old story again: mapping fds to path names is ambiguous at
best, if not impossible.
I wonder why such incomplete file system auditing rules are considered
sufficient in the CAPP/LSPP/NISPOM/STIG rulesets?
Here's a simplified example:
$ cd /tmp
$ mkdir dir
$ touch dir/file
$ ls -ldi /tmp /tmp/dir /tmp/dir/file
2057 drwxrwxrwt 9 root root 380 Sep 17 00:02 /tmp
58781 drwxr-xr-x 2 john john 40 Sep 17 00:02 /tmp/dir
56228 -rw-r--r-- 1 john john 0 Sep 17 00:02 /tmp/dir/file
$ cat > unlinkat.c
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
int dirfd = open("dir", O_RDONLY);
unlinkat(dirfd, "file", 0);
return 0;
}
^D
$ make unlinkat
cc unlinkat.c -o unlinkat
$ sudo autrace ./unlinkat
Waiting to execute: ./unlinkat
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 32121'
$ ls -li dir
total 0
Now, looking at the resulting raw SYSCALL event for unlinkat(2):
type=SYSCALL msg=audit(1316210542.899:779): arch=c000003e syscall=263 success=yes exit=0
a0=3 a1=400690 a2=0 a3=0 items=2 ppid=32106 pid=32121 auid=0 uid=0 gid=0 euid=0 suid=0
fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts12 ses=36 comm="unlinkat"
exe="/tmp/unlinkat" key=(null)
type=CWD msg=audit(1316210542.899:779): cwd="/tmp"
type=PATH msg=audit(1316210542.899:779): item=0 name="/tmp" inode=58781
dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00
type=PATH msg=audit(1316210542.899:779): item=1 name="file" inode=56228
dev=00:0e mode=0100644 ouid=1000 ogid=1000 rdev=00:00
type=EOE msg=audit(1316210542.899:779):
- From this event alone, there's no way to answer "Who unlinked
/tmp/dir/file?". For what it's worth, the provided path names would be
exactly the same if we had unlinked "/tmp/dir/dir/dir/dir/dir/file".
- PATH item 0 reports the inode of "/tmp/dir" (58781, see ls output
above), however, the reported path name is "/tmp" (bug?).
In this example I've used autrace, which traces everything, so I could
possibly search for a previous open(2) of inode 58781. And indeed, there
it is:
type=SYSCALL msg=audit(1316210542.899:778): arch=c000003e syscall=2 success=yes exit=3
a0=40068c a1=0 a2=7fff22724fc8 a3=0 items=1 ppid=32106 pid=32121 auid=0 uid=0 gid=0 euid=0
suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts12 ses=36 comm="unlinkat"
exe="/tmp/unlinkat" key=(null)
type=CWD msg=audit(1316210542.899:778): cwd="/tmp"
type=PATH msg=audit(1316210542.899:778): item=0 name="dir" inode=58781 dev=00:0e
mode=040755 ouid=1000 ogid=1000 rdev=00:00
type=EOE msg=audit(1316210542.899:778):
Great, so inode 58781 was opened using "/tmp/dir", and therefore, the relative
path "file" given to unlinkat(2) above could possibly translate to
"/tmp/dir/path"... not really feeling confident here.
- All file system auditing rules in various rulesets and the examples in
the documentation add the "-F perm=wa" (or similar) filter, so the
open(2) wouldn't even make it into the audit trail.
- If you can handle the volume and log all open(2), what happens if the
open(2) was done hours, days, weeks, ... ago?
- What if the open(2) was done by another process which passed the fd
on a unix domain socket?
It looks like the kernel auditing code should provide
... item=0 name="/tmp/dir" inode=58781 ...
in the unlinkat(2) syscall event above. Looking up the unlinkat(2)
documentation:
int unlinkat(int dirfd, const char *pathname, int flags);
If the pathname given in pathname is relative, then it is
interpreted relative to the directory referred to by the file
descriptor dirfd (rather than relative to the current working
directory of the calling process, as is done by unlink(2) and
rmdir(2) for a relative pathname).
If the pathname given in pathname is relative and dirfd is the
special value AT_FDCWD, then pathname is interpreted relative
to the current working directory of the calling process (like
unlink(2) and rmdir(2)).
As you might see, there's not only the fd->pathname problem, but
also the special case for AT_FDCWD. In this case the kernel side should
probably just duplicate CWD's path name into item 0's path name. But
that's just unlinkat(2), there are a lot more.
What am I missing here? Is there no way to audit a directory tree?
I've looked at alternatives: Inotify watches won't scale to big trees
and events lack so much detail that they can't be used for auditing.
Fanotify, while providing the pid, still lacks a lot of events and
passes fds; the example code relies on readlink("/proc/self/fd/...").
Thanks,
John
[1] http://people.redhat.com/sgrubb/audit/
--
John Feuerstein <john(a)feurix.com>
7:31 a.m.
On Friday, September 16, 2011 08:12:15 PM John Feuerstein wrote:
I would like to audit all changes to a directory tree using the
linux
auditing system[1].
# auditctl -a exit,always -F dir=/etc/ -F perm=wa
It seems like the GNU coreutils are enough to break the audit trail.
I was hoping one of the kernel developers would have got involved with this question.
I pointed out the same problem as you maybe 5 years ago. The people working on it at
the time said that if you really want to know, just add events for opens and then you
can piece it together. In my opinion, that is avoiding the problem and not solving it.
There are way too many opens to put into an audit trail on the odd chance that you
might have needed one. In 5 years, the kernel has changed and so have the people
working on the code. Maybe this problem should be revisited.
-Steve
The resulting SYSCALL events provide CWD and multiple PATH records,
depending on the syscall. If one of the PATH records is relative, I can
reconstruct the absolute path using the CWD record.
However, that does not work for the whole *at syscall family
(unlinkat(2), renameat(2), linkat(2), ...); accepting paths relative to
a given directory file descriptor. GNU coreutils are prominent users,
for example "rm -r" making use of unlinkat(2) to prevent races.
Things like dup(2) and fd passing via unix domain sockets come to mind.
It's the same old story again: mapping fds to path names is ambiguous at
best, if not impossible.
I wonder why such incomplete file system auditing rules are considered
sufficient in the CAPP/LSPP/NISPOM/STIG rulesets?
Here's a simplified example:
$ cd /tmp
$ mkdir dir
$ touch dir/file
$ ls -ldi /tmp /tmp/dir /tmp/dir/file
2057 drwxrwxrwt 9 root root 380 Sep 17 00:02 /tmp
58781 drwxr-xr-x 2 john john 40 Sep 17 00:02 /tmp/dir
56228 -rw-r--r-- 1 john john 0 Sep 17 00:02 /tmp/dir/file
$ cat > unlinkat.c
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
int dirfd = open("dir", O_RDONLY);
unlinkat(dirfd, "file", 0);
return 0;
}
^D
$ make unlinkat
cc unlinkat.c -o unlinkat
$ sudo autrace ./unlinkat
Waiting to execute: ./unlinkat
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 32121'
$ ls -li dir
total 0
Now, looking at the resulting raw SYSCALL event for unlinkat(2):
type=SYSCALL msg=audit(1316210542.899:779): arch=c000003e syscall=263
success=yes exit=0 a0=3 a1=400690 a2=0 a3=0 items=2 ppid=32106 pid=32121
auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts12
ses=36 comm="unlinkat" exe="/tmp/unlinkat" key=(null) type=CWD
msg=audit(1316210542.899:779): cwd="/tmp"
type=PATH msg=audit(1316210542.899:779): item=0 name="/tmp" inode=58781
dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00 type=PATH
msg=audit(1316210542.899:779): item=1 name="file" inode=56228 dev=00:0e
mode=0100644 ouid=1000 ogid=1000 rdev=00:00 type=EOE
msg=audit(1316210542.899:779):
- From this event alone, there's no way to answer "Who unlinked
/tmp/dir/file?". For what it's worth, the provided path names would be
exactly the same if we had unlinked "/tmp/dir/dir/dir/dir/dir/file".
- PATH item 0 reports the inode of "/tmp/dir" (58781, see ls output
above), however, the reported path name is "/tmp" (bug?).
In this example I've used autrace, which traces everything, so I could
possibly search for a previous open(2) of inode 58781. And indeed, there
it is:
type=SYSCALL msg=audit(1316210542.899:778): arch=c000003e syscall=2
success=yes exit=3 a0=40068c a1=0 a2=7fff22724fc8 a3=0 items=1 ppid=32106
pid=32121 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
tty=pts12 ses=36 comm="unlinkat" exe="/tmp/unlinkat" key=(null)
type=CWD
msg=audit(1316210542.899:778): cwd="/tmp"
type=PATH msg=audit(1316210542.899:778): item=0 name="dir" inode=58781
dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00 type=EOE
msg=audit(1316210542.899:778):
Great, so inode 58781 was opened using "/tmp/dir", and therefore, the
relative path "file" given to unlinkat(2) above could possibly translate
to "/tmp/dir/path"... not really feeling confident here.
- All file system auditing rules in various rulesets and the examples in
the documentation add the "-F perm=wa" (or similar) filter, so the
open(2) wouldn't even make it into the audit trail.
- If you can handle the volume and log all open(2), what happens if the
open(2) was done hours, days, weeks, ... ago?
- What if the open(2) was done by another process which passed the fd
on a unix domain socket?
It looks like the kernel auditing code should provide
... item=0 name="/tmp/dir" inode=58781 ...
in the unlinkat(2) syscall event above. Looking up the unlinkat(2)
documentation:
int unlinkat(int dirfd, const char *pathname, int flags);
If the pathname given in pathname is relative, then it is
interpreted relative to the directory referred to by the file
descriptor dirfd (rather than relative to the current working
directory of the calling process, as is done by unlink(2) and
rmdir(2) for a relative pathname).
If the pathname given in pathname is relative and dirfd is the
special value AT_FDCWD, then pathname is interpreted relative
to the current working directory of the calling process (like
unlink(2) and rmdir(2)).
As you might see, there's not only the fd->pathname problem, but
also the special case for AT_FDCWD. In this case the kernel side should
probably just duplicate CWD's path name into item 0's path name. But
that's just unlinkat(2), there are a lot more.
What am I missing here? Is there no way to audit a directory tree?
I've looked at alternatives: Inotify watches won't scale to big trees
and events lack so much detail that they can't be used for auditing.
Fanotify, while providing the pid, still lacks a lot of events and
passes fds; the example code relies on readlink("/proc/self/fd/...").
Thanks,
John
[1] http://people.redhat.com/sgrubb/audit/
2:42 p.m.
On 10/1/2011 5:31 AM, Steve Grubb wrote:
On Friday, September 16, 2011 08:12:15 PM John Feuerstein wrote:
> I would like to audit all changes to a directory tree using the linux
> auditing system[1].
>
> # auditctl -a exit,always -F dir=/etc/ -F perm=wa
>
> It seems like the GNU coreutils are enough to break the audit trail.
I was hoping one of the kernel developers would have got involved with this question.
I pointed out the same problem as you maybe 5 years ago. The people working on it at
the time said that if you really want to know, just add events for opens and then you
can piece it together. In my opinion, that is avoiding the problem and not solving it.
There are way too many opens to put into an audit trail on the odd chance that you
might have needed one. In 5 years, the kernel has changed and so have the people
working on the code. Maybe this problem should be revisited.
Howdy. Kernel developer here.
The problem goes way back. Way, way back. I will do my
best to describe what is going on and why the kernel has
such a problem with pathnames and audit. I am afraid that
you may not be happy with the explanation, but I also
think that you should understand what is going on and why
it has been so difficult to get a satisfactory resolution.
The Linux (and UNIX before it) kernel does not have an
internal concept of a path. Pathname resolution is provided
for the convenience of user space code.
The kernel has a simple view of filesystem objects. They
are inodes and datablocks. So long as there is a name for
the inode somewhere on the system the object is retained,
and once all the names are gone it is expunged. There are
two kinds of names; open file descriptors and directory
entries. A directory entry contains exactly one component
of a pathname. You are not allowed to remove directories
unless they are empty because that would leave objects with
names in an inaccessible state.
The Linux filesystem semantics, inherited in all their
glory from UNIX, permit multiple directory entries to
refer to the same inode. That means that there can be
multiple names for the same object in the filesystem
name space. These names are all peers. None is the "real"
name of the object. The only possible real name for the
object is the inode number (combined with an identification
of the containing filesystem). This identifies the object
even when all entries in the filesystem namespace are
gone but the file is open. Auditible event can occur on
files that are open but have not filesystem entries.
It's a big mess because the auditor obviously wants to
know the name of the file, but it is entirely possible
that there are hundreds of names in the filesystem space
for the object and that there are hundreds of open file
descriptors for the object, none of which were created
by opening pathnames that refer to that object any longer.
The kernel can keep track of the path used to reach an
inode, but with hard links, symlinks, mount points and
namespaces the reality is that you can't identify the
object involved using that information. The best that
can be done is to record the pathname requested, the
pathname resolved, and the inode number. It is impossible
to track objects by pathname because the pathname is
not a kernel concept.
It's been this way forever. UNIX audit systems had/have
the exact same problem. This is why we have AppArmor and
TOMOYO. Unless someone smarter than I am has an outstanding
insight we aren't going to make you happy any time soon.
-Steve
> The resulting SYSCALL events provide CWD and multiple PATH records,
> depending on the syscall. If one of the PATH records is relative, I can
> reconstruct the absolute path using the CWD record.
>
> However, that does not work for the whole *at syscall family
> (unlinkat(2), renameat(2), linkat(2), ...); accepting paths relative to
> a given directory file descriptor. GNU coreutils are prominent users,
> for example "rm -r" making use of unlinkat(2) to prevent races.
>
> Things like dup(2) and fd passing via unix domain sockets come to mind.
> It's the same old story again: mapping fds to path names is ambiguous at
> best, if not impossible.
>
> I wonder why such incomplete file system auditing rules are considered
> sufficient in the CAPP/LSPP/NISPOM/STIG rulesets?
>
> Here's a simplified example:
>
> $ cd /tmp
> $ mkdir dir
> $ touch dir/file
> $ ls -ldi /tmp /tmp/dir /tmp/dir/file
> 2057 drwxrwxrwt 9 root root 380 Sep 17 00:02 /tmp
> 58781 drwxr-xr-x 2 john john 40 Sep 17 00:02 /tmp/dir
> 56228 -rw-r--r-- 1 john john 0 Sep 17 00:02 /tmp/dir/file
> $ cat > unlinkat.c
> #include <unistd.h>
> #include <fcntl.h>
>
> int main(int argc, char **argv)
> {
> int dirfd = open("dir", O_RDONLY);
> unlinkat(dirfd, "file", 0);
> return 0;
> }
> ^D
> $ make unlinkat
> cc unlinkat.c -o unlinkat
> $ sudo autrace ./unlinkat
> Waiting to execute: ./unlinkat
> Cleaning up...
> Trace complete. You can locate the records with 'ausearch -i -p 32121'
> $ ls -li dir
> total 0
>
> Now, looking at the resulting raw SYSCALL event for unlinkat(2):
>
> type=SYSCALL msg=audit(1316210542.899:779): arch=c000003e syscall=263
> success=yes exit=0 a0=3 a1=400690 a2=0 a3=0 items=2 ppid=32106 pid=32121
> auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts12
> ses=36 comm="unlinkat" exe="/tmp/unlinkat" key=(null) type=CWD
> msg=audit(1316210542.899:779): cwd="/tmp"
> type=PATH msg=audit(1316210542.899:779): item=0 name="/tmp" inode=58781
> dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00 type=PATH
> msg=audit(1316210542.899:779): item=1 name="file" inode=56228 dev=00:0e
> mode=0100644 ouid=1000 ogid=1000 rdev=00:00 type=EOE
> msg=audit(1316210542.899:779):
>
> - From this event alone, there's no way to answer "Who unlinked
> /tmp/dir/file?". For what it's worth, the provided path names would be
> exactly the same if we had unlinked "/tmp/dir/dir/dir/dir/dir/file".
>
> - PATH item 0 reports the inode of "/tmp/dir" (58781, see ls output
> above), however, the reported path name is "/tmp" (bug?).
>
> In this example I've used autrace, which traces everything, so I could
> possibly search for a previous open(2) of inode 58781. And indeed, there
> it is:
>
> type=SYSCALL msg=audit(1316210542.899:778): arch=c000003e syscall=2
> success=yes exit=3 a0=40068c a1=0 a2=7fff22724fc8 a3=0 items=1 ppid=32106
> pid=32121 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
> tty=pts12 ses=36 comm="unlinkat" exe="/tmp/unlinkat" key=(null)
type=CWD
> msg=audit(1316210542.899:778): cwd="/tmp"
> type=PATH msg=audit(1316210542.899:778): item=0 name="dir" inode=58781
> dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00 type=EOE
> msg=audit(1316210542.899:778):
>
> Great, so inode 58781 was opened using "/tmp/dir", and therefore, the
> relative path "file" given to unlinkat(2) above could possibly translate
> to "/tmp/dir/path"... not really feeling confident here.
>
> - All file system auditing rules in various rulesets and the examples in
> the documentation add the "-F perm=wa" (or similar) filter, so the
> open(2) wouldn't even make it into the audit trail.
>
> - If you can handle the volume and log all open(2), what happens if the
> open(2) was done hours, days, weeks, ... ago?
>
> - What if the open(2) was done by another process which passed the fd
> on a unix domain socket?
>
> It looks like the kernel auditing code should provide
>
> ... item=0 name="/tmp/dir" inode=58781 ...
>
> in the unlinkat(2) syscall event above. Looking up the unlinkat(2)
> documentation:
>
> int unlinkat(int dirfd, const char *pathname, int flags);
>
> If the pathname given in pathname is relative, then it is
> interpreted relative to the directory referred to by the file
> descriptor dirfd (rather than relative to the current working
> directory of the calling process, as is done by unlink(2) and
> rmdir(2) for a relative pathname).
>
> If the pathname given in pathname is relative and dirfd is the
> special value AT_FDCWD, then pathname is interpreted relative
> to the current working directory of the calling process (like
> unlink(2) and rmdir(2)).
>
> As you might see, there's not only the fd->pathname problem, but
> also the special case for AT_FDCWD. In this case the kernel side should
> probably just duplicate CWD's path name into item 0's path name. But
> that's just unlinkat(2), there are a lot more.
>
> What am I missing here? Is there no way to audit a directory tree?
> I've looked at alternatives: Inotify watches won't scale to big trees
> and events lack so much detail that they can't be used for auditing.
> Fanotify, while providing the pid, still lacks a lot of events and
> passes fds; the example code relies on readlink("/proc/self/fd/...").
>
> Thanks,
> John
>
> [1] http://people.redhat.com/sgrubb/audit/
--
Linux-audit mailing list
Linux-audit(a)redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
12:02 p.m.
On Monday, October 03, 2011 03:42:25 PM Casey Schaufler wrote:
On 10/1/2011 5:31 AM, Steve Grubb wrote:
> On Friday, September 16, 2011 08:12:15 PM John Feuerstein wrote:
>> I would like to audit all changes to a directory tree using the linux
>> auditing system[1].
>>
>> # auditctl -a exit,always -F dir=/etc/ -F perm=wa
>>
>> It seems like the GNU coreutils are enough to break the audit trail.
>
> I was hoping one of the kernel developers would have got involved with
> this question. I pointed out the same problem as you maybe 5 years ago.
> The people working on it at the time said that if you really want to
> know, just add events for opens and then you can piece it together. In
> my opinion, that is avoiding the problem and not solving it. There are
> way too many opens to put into an audit trail on the odd chance that you
> might have needed one. In 5 years, the kernel has changed and so have
> the people working on the code. Maybe this problem should be revisited.
Howdy. Kernel developer here.
Hi Casey,
Glad someone jumped in to this. :)
The problem goes way back. Way, way back. I will do my
best to describe what is going on and why the kernel has
such a problem with pathnames and audit. I am afraid that
you may not be happy with the explanation, but I also
think that you should understand what is going on and why
it has been so difficult to get a satisfactory resolution.
The Linux (and UNIX before it) kernel does not have an
internal concept of a path. Pathname resolution is provided
for the convenience of user space code.
The kernel has a simple view of filesystem objects. They
are inodes and datablocks. So long as there is a name for
the inode somewhere on the system the object is retained,
and once all the names are gone it is expunged. There are
two kinds of names; open file descriptors and directory
entries. A directory entry contains exactly one component
of a pathname. You are not allowed to remove directories
unless they are empty because that would leave objects with
names in an inaccessible state.
The Linux filesystem semantics, inherited in all their
glory from UNIX, permit multiple directory entries to
refer to the same inode. That means that there can be
multiple names for the same object in the filesystem
name space. These names are all peers. None is the "real"
name of the object. The only possible real name for the
object is the inode number (combined with an identification
of the containing filesystem). This identifies the object
even when all entries in the filesystem namespace are
gone but the file is open. Auditible event can occur on
files that are open but have not filesystem entries.
It's a big mess because the auditor obviously wants to
know the name of the file, but it is entirely possible
that there are hundreds of names in the filesystem space
for the object and that there are hundreds of open file
descriptors for the object, none of which were created
by opening pathnames that refer to that object any longer.
The kernel can keep track of the path used to reach an
inode, but with hard links, symlinks, mount points and
namespaces the reality is that you can't identify the
object involved using that information. The best that
can be done is to record the pathname requested, the
pathname resolved, and the inode number. It is impossible
to track objects by pathname because the pathname is
not a kernel concept.
It's been this way forever. UNIX audit systems had/have
the exact same problem. This is why we have AppArmor and
TOMOYO. Unless someone smarter than I am has an outstanding
insight we aren't going to make you happy any time soon.
What I was wondering about is this. Assumption: The openat audit event should not
happen all the time. If it does, then the system is going to perform poorly due to
load. So, if an event fires, we really aren't on the hotpath anymore. Proposal: If we
have an *at syscall event triggered, why can't we look at the fd being passed in and
look it up in the same place that the kernel keeps the path for the /proc/<pid>/fd/#
?
-Steve
5:09 p.m.
Casey,
thanks for your explanation.
On Mon, 03 Oct 2011 12:42:25 -0700, Casey Schaufler wrote:
The Linux filesystem semantics, inherited in all their
glory from UNIX, permit multiple directory entries to
refer to the same inode. That means that there can be
multiple names for the same object in the filesystem
name space. These names are all peers. None is the "real"
name of the object. The only possible real name for the
object is the inode number (combined with an identification
of the containing filesystem). This identifies the object
even when all entries in the filesystem namespace are
gone but the file is open. Auditible event can occur on
files that are open but have not filesystem entries.
Linux does not support multiple hardlinks to a directory[1] though?
Since the first argument to *at(2) syscalls is a dirfd, would it not be
possible to do something similar to getcwd(2)?
1. identify the root directory -> root
2. identify the given directory using the dirfd -> dir
3. until we reach root:
- open ".." -> parent
- scan for a dentry that matches dir
- dir = parent
4. reconstruct path from dentry components
d_path() in fs/dcache.c[2] seems to implement that.
I understand that this is ambiguous because of directory symlinks, but
it's better than the current situation. It would work out fine on
filesystems without symlinks (AFAIK this is only possible using FUSE on
Linux as of now, FreeBSD has had mount -o nosymlink for ages).
However, I'm not sure if it's worth the performance penalty. What about
making this configurable with sysctl? If enabled, PATH records for
syscall arguments consisting of a directory file descriptor will get
their name field reconstructed (best-effort/ambiguous). If disabled, the
name field will simply remain empty, instead of falling back to the cwd.
[1] http://lwn.net/Articles/249607/
[2]
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=fs/...
8:50 a.m.
Casey only talked about the easy part of the reason the pathnames are
useless. He forgot to mention that the linux kernel has mount
namespaces. There is absolutely no reason why one could not mount a FS
in the init namespace, launch a whole 'virtual machine' in that new FS,
and then unmount the FS from the initial namespace. Now we have 2
COMPLETELY disjoint 'filesystems'.
The audit logs, and things like /proc/pid/fd or dpath functions are all
going to be relative to the local FS namespace. Sometimes it just quite
simply can't be resolved. So now inside virtual machine namespace they
might read/modify /etc/shadow and that file IS /etc/shadow. There is no
other 'path' for that file. True its not the same /etc/shadow as the
one in the init fs namespace. And at some point there may have existed
a path in the init namespace /mnt/virt1/etc/shadow which also
represented that inode, but at this point in time the ONLY path which
represents this file is /etc/shadow.
Audit logs based on name are wrong and misleading. There's a reason the
auditable object is the inode and fs details Casey mentioned. We might
be able to usually give me information, but that information cannot EVER
be used for anything useful. Its unreliable. Exposing it only leads
one to believe they have knowledge they don't.
-Eric
9:04 a.m.
On Friday, October 07, 2011 09:50:58 AM Eric Paris wrote:
Audit logs based on name are wrong and misleading. There's a
reason the
auditable object is the inode and fs details Casey mentioned. We might
be able to usually give me information, but that information cannot EVER
be used for anything useful. Its unreliable. Exposing it only leads
one to believe they have knowledge they don't.
But also depending on an inode leads you to have knowledge when you don't. Inodes get
reassigned. Depending on when you look at your logs, that inode could have been used
in 5 different file names between the event and now. So, it is important to snapshot
what the inode was associated with at a given time.
My thinking is that we can make the system more useful even with name spaces. By
recording the current association we are more certain about what was being accessed.
We should be able to record the mount command that caused the new namespace so that we
can reconstruct a meaningful path. Maybe we need to change the session id on that
occurance so that child processes can be properly identified.
But you raise a new point that I think we eventally have to address and that is
containers. I think we will have to record container IDs and the like so that an audit
event can have a proper context for what it means. We at some point could have
duplicate uids and pids that are disjointed - not just files.
We have to address this some time.
-Steve
12:20 p.m.
On 10/7/2011 6:50 AM, Eric Paris wrote:
Casey only talked about the easy part of the reason the pathnames
are
useless. He forgot to mention
I didn't forgot to mention the whole mount point thingy.
People always get hung up in coming up with ways to explain
around the problem, and having already identified the root
cause of the problem I figured we might avoid yet another
round of clever and convoluted arguments around identifying
mount points.
that the linux kernel has mount
namespaces. There is absolutely no reason why one could not mount a FS
in the init namespace, launch a whole 'virtual machine' in that new FS,
and then unmount the FS from the initial namespace. Now we have 2
COMPLETELY disjoint 'filesystems'.
You don't even need all this to demonstrate the problem. A simple
chroot (or worse yet, fchroot (do we have that? I forget)) gives
most audit tools the willies.
The audit logs, and things like /proc/pid/fd or dpath functions are
all
going to be relative to the local FS namespace. Sometimes it just quite
simply can't be resolved. So now inside virtual machine namespace they
might read/modify /etc/shadow and that file IS /etc/shadow. There is no
other 'path' for that file. True its not the same /etc/shadow as the
one in the init fs namespace. And at some point there may have existed
a path in the init namespace /mnt/virt1/etc/shadow which also
represented that inode, but at this point in time the ONLY path which
represents this file is /etc/shadow.
Audit logs based on name are wrong and misleading. There's a reason the
auditable object is the inode and fs details Casey mentioned. We might
be able to usually give me information, but that information cannot EVER
be used for anything useful. Its unreliable. Exposing it only leads
one to believe they have knowledge they don't.
The unfortunate reality is that audit requirements dating back to
the 1985 Orange Book clearly require audit be done by name. The
intent has always been clear that the name should be the name by
which people reference the object. We have been arguing that
providing the dev/inode pair meets the requirements for decades
now and only getting by with it by promising to provide mappings
to real pathnames.
The best you can do, and Irix(tm) did (does in some remote backwaters)
is report the dev/inode, the pathname requested and the pathname
resolved with indications of components that are symlinks and mount
points.
This is why there is value in pathname based access controls.
In many cases you don't care so much which object was accessed
as you do that the object is accessible using the name /etc/shadow.
Of course, once you say that, you really care about the object
because it may be accessed by other names as well and if you
care about the object named /etc/shadow you care about it regardless
of the name used to access it.
I would be delighted if someone came up with the fiendishly
clever solution to the issue. I am not going to bet on one
in my lifetime.
-Eric
1:02 p.m.
On Friday, October 07, 2011 01:20:23 PM Casey Schaufler wrote:
I would be delighted if someone came up with the fiendishly
clever solution to the issue. I am not going to bet on one
in my lifetime.
It doesn't even need to be fiendishly clever to be useful. Using the /etc/shadow
analogy, What we get now is just shadow. Shadow where? /etc? /var/chroot/bind/etc?
/backup/etc? Any clue would be helpful. Bind mounts, chroot, and namespaces all make
it interesting, but just adding the dir as an aux record would make things so much
better. We can solve the other problem another day.
-Steve
1:27 p.m.
On Fri, 2011-10-07 at 10:20 -0700, Casey Schaufler wrote:
On 10/7/2011 6:50 AM, Eric Paris wrote:
> Casey only talked about the easy part of the reason the pathnames are
> useless. He forgot to mention
I didn't forgot to mention the whole mount point thingy.
People always get hung up in coming up with ways to explain
around the problem, and having already identified the root
cause of the problem
Ok fair enough. I guess I just saw two root problems not just one. You
mentioned there existing multiple names for the same object. I was
thinking of the of there not existing any name for an object which makes
sense at a 'system wide' level. In any case. We might be able to get
some more pathname like info, but it's never (like Casey so sagely said)
going to be truely useful....
-Eric
4:38 p.m.
On 10/7/2011 11:27 AM, Eric Paris wrote:
On Fri, 2011-10-07 at 10:20 -0700, Casey Schaufler wrote:
> On 10/7/2011 6:50 AM, Eric Paris wrote:
>> Casey only talked about the easy part of the reason the pathnames are
>> useless. He forgot to mention
> I didn't forgot to mention the whole mount point thingy.
> People always get hung up in coming up with ways to explain
> around the problem, and having already identified the root
> cause of the problem
Ok fair enough. I guess I just saw two root problems not just one. You
mentioned there existing multiple names for the same object. I was
thinking of the of there not existing any name for an object which makes
sense at a 'system wide' level. In any case. We might be able to get
some more pathname like info, but it's never (like Casey so sagely said)
going to be truely useful....
The worst case is 4000 processes that opened the file under 4000
different pathnames, all of which have since been unlinked, doing
fchmod. At the time of fchmod there is no pathname that refers to
the file, although 4000 pathnames are associated with the object
whose mode is getting changed. The dev/inode pair is the only
externally visible identifier that could possibly be used to
name the file in the log, and as you point out, the dev is not
reliable.
Now even with that, a path name could be useful. It just can't
be considered definitive or unique. As for audit tracking, you
really ought to be able to say things like "show me everything
that happens to the file that is currently called /etc/shadow"
and "show me everything that happens to any file called /etc/shadow",
even though the two statements are radically different underneath.
The problem is that 99 44/100% of the people looking at or setting
up audit trails are going to be disinterested in or possibly
incapable of making the distinction. Let's face it, most people
shouldn't be using computers capable of running anything except
AngryBirds.
-Eric
7:54 a.m.
On Friday, October 07, 2011 05:38:41 PM Casey Schaufler wrote:
On 10/7/2011 11:27 AM, Eric Paris wrote:
> On Fri, 2011-10-07 at 10:20 -0700, Casey Schaufler wrote:
>> On 10/7/2011 6:50 AM, Eric Paris wrote:
>>> Casey only talked about the easy part of the reason the pathnames are
>>> useless. He forgot to mention
>>
>> I didn't forgot to mention the whole mount point thingy.
>> People always get hung up in coming up with ways to explain
>> around the problem, and having already identified the root
>> cause of the problem
>
> Ok fair enough. I guess I just saw two root problems not just one. You
> mentioned there existing multiple names for the same object. I was
> thinking of the of there not existing any name for an object which makes
> sense at a 'system wide' level. In any case. We might be able to get
> some more pathname like info, but it's never (like Casey so sagely said)
> going to be truely useful....
The worst case is 4000 processes that opened the file under 4000
different pathnames, all of which have since been unlinked, doing
fchmod. At the time of fchmod there is no pathname that refers to
the file, although 4000 pathnames are associated with the object
whose mode is getting changed. The dev/inode pair is the only
externally visible identifier that could possibly be used to
name the file in the log, and as you point out, the dev is not
reliable.
Now even with that, a path name could be useful. It just can't
be considered definitive or unique. As for audit tracking, you
really ought to be able to say things like "show me everything
that happens to the file that is currently called /etc/shadow"
and "show me everything that happens to any file called /etc/shadow",
even though the two statements are radically different underneath.
The problem is that 99 44/100% of the people looking at or setting
up audit trails are going to be disinterested in or possibly
incapable of making the distinction. Let's face it, most people
shouldn't be using computers capable of running anything except
AngryBirds.
I think that you missed the point of this problem report. There is a problem in path
name resolution as has been discussed. But the problem is that we only get partial
path information for some syscalls. The *at syscalls take 2 parameters: a previously
opened fd and a path relative to the fd. The audit system is recording the relative
path fine. What we don't have is what the fd points to.
It would be unreasonable to expect a watch on /etc be added on the off chance that you
might need 1 of those million opens to reconstruct an event. Right now we don't get
any information about the parent directory. Neither the directory's device or inode
gets recorded. Everything you point out is another problem.
-Steve
6:09 p.m.
On Fri, Sep 16, 2011 at 5:12 PM, John Feuerstein <john(a)feurix.com> wrote:
Hi,
I would like to audit all changes to a directory tree using the linux
auditing system[1].
# auditctl -a exit,always -F dir=/etc/ -F perm=wa
It seems like the GNU coreutils are enough to break the audit trail.
The resulting SYSCALL events provide CWD and multiple PATH records,
depending on the syscall. If one of the PATH records is relative, I can
reconstruct the absolute path using the CWD record.
However, that does not work for the whole *at syscall family
(unlinkat(2), renameat(2), linkat(2), ...); accepting paths relative to
a given directory file descriptor. GNU coreutils are prominent users,
for example "rm -r" making use of unlinkat(2) to prevent races.
Things like dup(2) and fd passing via unix domain sockets come to mind.
It's the same old story again: mapping fds to path names is ambiguous at
best, if not impossible.
I wonder why such incomplete file system auditing rules are considered
sufficient in the CAPP/LSPP/NISPOM/STIG rulesets?
Here's a simplified example:
$ cd /tmp
$ mkdir dir
$ touch dir/file
$ ls -ldi /tmp /tmp/dir /tmp/dir/file
2057 drwxrwxrwt 9 root root 380 Sep 17 00:02 /tmp
58781 drwxr-xr-x 2 john john 40 Sep 17 00:02 /tmp/dir
56228 -rw-r--r-- 1 john john 0 Sep 17 00:02 /tmp/dir/file
$ cat > unlinkat.c
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
int dirfd = open("dir", O_RDONLY);
unlinkat(dirfd, "file", 0);
return 0;
}
^D
$ make unlinkat
cc unlinkat.c -o unlinkat
$ sudo autrace ./unlinkat
Waiting to execute: ./unlinkat
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 32121'
$ ls -li dir
total 0
Now, looking at the resulting raw SYSCALL event for unlinkat(2):
type=SYSCALL msg=audit(1316210542.899:779): arch=c000003e syscall=263 success=yes exit=0
a0=3 a1=400690 a2=0 a3=0 items=2 ppid=32106 pid=32121 auid=0 uid=0 gid=0 euid=0 suid=0
fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts12 ses=36 comm="unlinkat"
exe="/tmp/unlinkat" key=(null)
type=CWD msg=audit(1316210542.899:779): cwd="/tmp"
type=PATH msg=audit(1316210542.899:779): item=0 name="/tmp" inode=58781
dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00
type=PATH msg=audit(1316210542.899:779): item=1 name="file" inode=56228
dev=00:0e mode=0100644 ouid=1000 ogid=1000 rdev=00:00
type=EOE msg=audit(1316210542.899:779):
- From this event alone, there's no way to answer "Who unlinked
/tmp/dir/file?". For what it's worth, the provided path names would be
exactly the same if we had unlinked "/tmp/dir/dir/dir/dir/dir/file".
- PATH item 0 reports the inode of "/tmp/dir" (58781, see ls output
above), however, the reported path name is "/tmp" (bug?).
In this example I've used autrace, which traces everything, so I could
possibly search for a previous open(2) of inode 58781. And indeed, there
it is:
type=SYSCALL msg=audit(1316210542.899:778): arch=c000003e syscall=2 success=yes exit=3
a0=40068c a1=0 a2=7fff22724fc8 a3=0 items=1 ppid=32106 pid=32121 auid=0 uid=0 gid=0 euid=0
suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts12 ses=36 comm="unlinkat"
exe="/tmp/unlinkat" key=(null)
type=CWD msg=audit(1316210542.899:778): cwd="/tmp"
type=PATH msg=audit(1316210542.899:778): item=0 name="dir" inode=58781
dev=00:0e mode=040755 ouid=1000 ogid=1000 rdev=00:00
type=EOE msg=audit(1316210542.899:778):
Great, so inode 58781 was opened using "/tmp/dir", and therefore, the relative
path "file" given to unlinkat(2) above could possibly translate to
"/tmp/dir/path"... not really feeling confident here.
- All file system auditing rules in various rulesets and the examples in
the documentation add the "-F perm=wa" (or similar) filter, so the
open(2) wouldn't even make it into the audit trail.
- If you can handle the volume and log all open(2), what happens if the
open(2) was done hours, days, weeks, ... ago?
- What if the open(2) was done by another process which passed the fd
on a unix domain socket?
It looks like the kernel auditing code should provide
... item=0 name="/tmp/dir" inode=58781 ...
in the unlinkat(2) syscall event above. Looking up the unlinkat(2)
documentation:
int unlinkat(int dirfd, const char *pathname, int flags);
If the pathname given in pathname is relative, then it is
interpreted relative to the directory referred to by the file
descriptor dirfd (rather than relative to the current working
directory of the calling process, as is done by unlink(2) and
rmdir(2) for a relative pathname).
If the pathname given in pathname is relative and dirfd is the
special value AT_FDCWD, then pathname is interpreted relative
to the current working directory of the calling process (like
unlink(2) and rmdir(2)).
As you might see, there's not only the fd->pathname problem, but
also the special case for AT_FDCWD. In this case the kernel side should
probably just duplicate CWD's path name into item 0's path name. But
that's just unlinkat(2), there are a lot more.
What am I missing here? Is there no way to audit a directory tree?
I've looked at alternatives: Inotify watches won't scale to big trees
and events lack so much detail that they can't be used for auditing.
Fanotify, while providing the pid, still lacks a lot of events and
passes fds; the example code relies on readlink("/proc/self/fd/...").
After posting something similar to linux-audit the other day, I
rechecked the archives and found this. I'm hitting the same issue.
I've seen the same thing happen also with chmodat and chownat (and
those are even uglier since you don't have the chain of inodes in the
audit logs like you do with unlinkat). I'd mistakenly thought it was
due to recursion, but that's clearly not the case (the binaries in
question must just be using the *at() versions of the syscalls for
recursive calls). Did you find any sort of workaround for this? I
didn't see a relevant-looking patch in the more recent LKML archives,
though it could've been titled something I didn't interpret as
relevant.
If you see my recent linux-audit posting, another related thing (at
least as far as missing relevant information in the logs) is that the
audit logs are logging pathnames relative to the chroot, instead of
the pathnames relative to the root of the OS itself. You'd expect a
process chroot'd to /chroot, accessing (from the perspective of the
OS) /chroot/etc/password would get logged as /chroot/etc/password but
is rather logged as /etc/password.
I don't have a working LXC install handy, but I'd imagine the audit
subsystem would log relative to the container's / instead of the
host's / too.
6:29 p.m.
On Tue, Oct 09, 2012 at 04:09:18PM -0700, Mark Moseley wrote:
On Fri, Sep 16, 2011 at 5:12 PM, John Feuerstein
<john(a)feurix.com> wrote:
> Hi,
>
> I would like to audit all changes to a directory tree using the linux
> auditing system[1].
>
> # auditctl -a exit,always -F dir=/etc/ -F perm=wa
>
> It seems like the GNU coreutils are enough to break the audit trail.
>
> The resulting SYSCALL events provide CWD and multiple PATH records,
> depending on the syscall. If one of the PATH records is relative, I can
> reconstruct the absolute path using the CWD record.
>
> However, that does not work for the whole *at syscall family
> (unlinkat(2), renameat(2), linkat(2), ...); accepting paths relative to
> a given directory file descriptor. GNU coreutils are prominent users,
> for example "rm -r" making use of unlinkat(2) to prevent races.
>
> Things like dup(2) and fd passing via unix domain sockets come to mind.
> It's the same old story again: mapping fds to path names is ambiguous at
> best, if not impossible.
Your point being? Even if you do get all pathnames, you *can't* reconstruct
the changes of filesystem tree, period. Pathname resolution is not atomic.
Can't be made such, either - not without serializing all system calls, which
will hurt too damn much.
You can tell when something happens to filesystem *object*. Which audit,
lousy as it is, allows to do. Anything that hopes to reconstruct the
history of changes based on fully timestamped history of syscalls is
inherently unreliable.
Again, pathname resolution is not atomic at all and neither is reconstructing
pathname by object (i.e. by vfsmount/dentry pair).
6:39 p.m.
On Tue, Oct 09, 2012 at 04:09:18PM -0700, Mark Moseley wrote:
If you see my recent linux-audit posting, another related thing (at
least as far as missing relevant information in the logs) is that the
audit logs are logging pathnames relative to the chroot, instead of
the pathnames relative to the root of the OS itself. You'd expect a
process chroot'd to /chroot, accessing (from the perspective of the
OS) /chroot/etc/password would get logged as /chroot/etc/password but
is rather logged as /etc/password.
I don't have a working LXC install handy, but I'd imagine the audit
subsystem would log relative to the container's / instead of the
host's / too.
BTW, what makes you think that container's root is even reachable from
"the host's /"? There is no such thing as "root of the OS
itself"; different
processes can (and in case of containers definitely do) run in different
namespaces. With entirely different filesystems mounted in those, and
no promise whatsoever that any specific namespace happens to have all
filesystems mounted somewhere in it...
6:47 p.m.
On Tue, Oct 9, 2012 at 4:39 PM, Al Viro <viro(a)zeniv.linux.org.uk> wrote:
On Tue, Oct 09, 2012 at 04:09:18PM -0700, Mark Moseley wrote:
> If you see my recent linux-audit posting, another related thing (at
> least as far as missing relevant information in the logs) is that the
> audit logs are logging pathnames relative to the chroot, instead of
> the pathnames relative to the root of the OS itself. You'd expect a
> process chroot'd to /chroot, accessing (from the perspective of the
> OS) /chroot/etc/password would get logged as /chroot/etc/password but
> is rather logged as /etc/password.
>
> I don't have a working LXC install handy, but I'd imagine the audit
> subsystem would log relative to the container's / instead of the
> host's / too.
BTW, what makes you think that container's root is even reachable from
"the host's /"? There is no such thing as "root of the OS
itself"; different
processes can (and in case of containers definitely do) run in different
namespaces. With entirely different filesystems mounted in those, and
no promise whatsoever that any specific namespace happens to have all
filesystems mounted somewhere in it...
Nothing beyond guesswork, since it's been a while since I've played
with LXC. In any case, I was struggling a bit for the correct
terminology.
Am I similarly off-base with regards to the chroot'd scenario?
6:54 p.m.
On Tue, Oct 09, 2012 at 04:47:17PM -0700, Mark Moseley wrote:
> BTW, what makes you think that container's root is even
reachable from
> "the host's /"? There is no such thing as "root of the OS
itself"; different
> processes can (and in case of containers definitely do) run in different
> namespaces. With entirely different filesystems mounted in those, and
> no promise whatsoever that any specific namespace happens to have all
> filesystems mounted somewhere in it...
Nothing beyond guesswork, since it's been a while since I've played
with LXC. In any case, I was struggling a bit for the correct
terminology.
Am I similarly off-base with regards to the chroot'd scenario?
chroot case is going to be reachable from namespace root, but I seriously
doubt that pathname relative to that will be more useful...
Again, relying on pathnames for forensics (or security in general) is
a serious mistake (cue unprintable comments about apparmor and similar
varieties of snake oil). And using audit as poor man's ktrace analog
is... misguided, to put it very mildly.
5:45 p.m.
On Tue, Oct 9, 2012 at 4:54 PM, Al Viro <viro(a)zeniv.linux.org.uk> wrote:
On Tue, Oct 09, 2012 at 04:47:17PM -0700, Mark Moseley wrote:
> > BTW, what makes you think that container's root is even reachable from
> > "the host's /"? There is no such thing as "root of the OS
itself"; different
> > processes can (and in case of containers definitely do) run in different
> > namespaces. With entirely different filesystems mounted in those, and
> > no promise whatsoever that any specific namespace happens to have all
> > filesystems mounted somewhere in it...
>
> Nothing beyond guesswork, since it's been a while since I've played
> with LXC. In any case, I was struggling a bit for the correct
> terminology.
>
> Am I similarly off-base with regards to the chroot'd scenario?
chroot case is going to be reachable from namespace root, but I seriously
doubt that pathname relative to that will be more useful...
Possibly not, but it'd still be good to have some sort of indicator
that this entry is being logged relative to the chroot, like an
additional item in the audit entry or even some kind of flag. But in
this case, and far more so in the unlinkat/chmodat/chownat case, I'd
think the least surprising thing (to me, at least) would be for the
directory item in the audit entry to have a pathname relative to
namespace root.
Again, relying on pathnames for forensics (or security in general)
is
a serious mistake (cue unprintable comments about apparmor and similar
varieties of snake oil). And using audit as poor man's ktrace analog
is... misguided, to put it very mildly.
Caveat: I'm just a sysadmin, so this stuff is as darn near "magic" as
I get to see on a regular basis, so it's safe to expect some naivety
and/or misguidedness on my part :)
I'm just using it as a log of files that have been written/changed on
moderately- to heavily-used systems. If there's another in-kernel
mechanism that'd be better suited for that sort of thing (at least
without adding a lot of overhead), I'd be definitely eager to know
about it. It's a web hosting environment, with customer files all
solely on NFS, so writes to the same directory can come from an
arbitrary number of servers. When they get swamped with write
requests, the amount of per-client stats exposed by our Netapp and
Oracle NFS servers is often only enough to point us at a client server
with an abusive user on it (but not much more, without turning on
debugging). Having logs of who's doing writes would be quite useful,
esp when writes aren't happening at that exact moment and wouldn't
show up in tools like iotop. The audit subsystem seemed like the best
fit for this kind of thing, but I'm more than open to whatever works.
6 p.m.
On Wednesday, October 10, 2012 03:45:08 PM Mark Moseley wrote:
On Tue, Oct 9, 2012 at 4:54 PM, Al Viro
<viro(a)zeniv.linux.org.uk> wrote:
> Again, relying on pathnames for forensics (or security in general) is
> a serious mistake (cue unprintable comments about apparmor and similar
> varieties of snake oil). And using audit as poor man's ktrace analog
> is... misguided, to put it very mildly.
Caveat: I'm just a sysadmin, so this stuff is as darn near "magic" as
I get to see on a regular basis, so it's safe to expect some naivety
and/or misguidedness on my part :)
I'm just using it as a log of files that have been written/changed on
moderately- to heavily-used systems. If there's another in-kernel
mechanism that'd be better suited for that sort of thing (at least
without adding a lot of overhead), I'd be definitely eager to know
about it. It's a web hosting environment, with customer files all
solely on NFS, so writes to the same directory can come from an
arbitrary number of servers. When they get swamped with write
requests, the amount of per-client stats exposed by our Netapp and
Oracle NFS servers is often only enough to point us at a client server
with an abusive user on it (but not much more, without turning on
debugging). Having logs of who's doing writes would be quite useful,
esp when writes aren't happening at that exact moment and wouldn't
show up in tools like iotop. The audit subsystem seemed like the best
fit for this kind of thing, but I'm more than open to whatever works.
The audit system is the best fit. But I think Al is saying there are some
limitations. i know that Eric pushed some patches a while back that makes a
stronger effort at collecting some of this information. What kernel are you
using?
-Steve
6:07 p.m.
On Wed, Oct 10, 2012 at 4:00 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
On Wednesday, October 10, 2012 03:45:08 PM Mark Moseley wrote:
> On Tue, Oct 9, 2012 at 4:54 PM, Al Viro <viro(a)zeniv.linux.org.uk> wrote:
> > Again, relying on pathnames for forensics (or security in general) is
> > a serious mistake (cue unprintable comments about apparmor and similar
> > varieties of snake oil). And using audit as poor man's ktrace analog
> > is... misguided, to put it very mildly.
>
> Caveat: I'm just a sysadmin, so this stuff is as darn near "magic" as
> I get to see on a regular basis, so it's safe to expect some naivety
> and/or misguidedness on my part :)
>
> I'm just using it as a log of files that have been written/changed on
> moderately- to heavily-used systems. If there's another in-kernel
> mechanism that'd be better suited for that sort of thing (at least
> without adding a lot of overhead), I'd be definitely eager to know
> about it. It's a web hosting environment, with customer files all
> solely on NFS, so writes to the same directory can come from an
> arbitrary number of servers. When they get swamped with write
> requests, the amount of per-client stats exposed by our Netapp and
> Oracle NFS servers is often only enough to point us at a client server
> with an abusive user on it (but not much more, without turning on
> debugging). Having logs of who's doing writes would be quite useful,
> esp when writes aren't happening at that exact moment and wouldn't
> show up in tools like iotop. The audit subsystem seemed like the best
> fit for this kind of thing, but I'm more than open to whatever works.
The audit system is the best fit. But I think Al is saying there are some
limitations. i know that Eric pushed some patches a while back that makes a
stronger effort at collecting some of this information. What kernel are you
using?
Yup, understood. I've been playing with a variety of boxes, but mostly
within the 3.0.x and 3.2.x series. I'll drop 3.5.6 on some of these
boxes and see if my issues are already fixed (and proceed directly to
foot-in-mouth chagrined stage -- usually takes slightly longer to get
to that stage).
12:27 p.m.
On Wed, Oct 10, 2012 at 4:07 PM, Mark Moseley <moseleymark(a)gmail.com> wrote:
On Wed, Oct 10, 2012 at 4:00 PM, Steve Grubb
<sgrubb(a)redhat.com> wrote:
> On Wednesday, October 10, 2012 03:45:08 PM Mark Moseley wrote:
>> On Tue, Oct 9, 2012 at 4:54 PM, Al Viro <viro(a)zeniv.linux.org.uk> wrote:
>> > Again, relying on pathnames for forensics (or security in general) is
>> > a serious mistake (cue unprintable comments about apparmor and similar
>> > varieties of snake oil). And using audit as poor man's ktrace analog
>> > is... misguided, to put it very mildly.
>>
>> Caveat: I'm just a sysadmin, so this stuff is as darn near "magic"
as
>> I get to see on a regular basis, so it's safe to expect some naivety
>> and/or misguidedness on my part :)
>>
>> I'm just using it as a log of files that have been written/changed on
>> moderately- to heavily-used systems. If there's another in-kernel
>> mechanism that'd be better suited for that sort of thing (at least
>> without adding a lot of overhead), I'd be definitely eager to know
>> about it. It's a web hosting environment, with customer files all
>> solely on NFS, so writes to the same directory can come from an
>> arbitrary number of servers. When they get swamped with write
>> requests, the amount of per-client stats exposed by our Netapp and
>> Oracle NFS servers is often only enough to point us at a client server
>> with an abusive user on it (but not much more, without turning on
>> debugging). Having logs of who's doing writes would be quite useful,
>> esp when writes aren't happening at that exact moment and wouldn't
>> show up in tools like iotop. The audit subsystem seemed like the best
>> fit for this kind of thing, but I'm more than open to whatever works.
>
> The audit system is the best fit. But I think Al is saying there are some
> limitations. i know that Eric pushed some patches a while back that makes a
> stronger effort at collecting some of this information. What kernel are you
> using?
Yup, understood. I've been playing with a variety of boxes, but mostly
within the 3.0.x and 3.2.x series. I'll drop 3.5.6 on some of these
boxes and see if my issues are already fixed (and proceed directly to
foot-in-mouth chagrined stage -- usually takes slightly longer to get
to that stage).
Just gave 3.5.6 a shot and in these two particular cases, the result
is the same: chroot'd actions are logged in the audit entry relative
to the chroot, and the unlinkat/chmodat/chownat audit log entries only
have one item with the bare filename and no indication of directory.
8:12 p.m.
On Thu, Oct 11, 2012 at 10:27 AM, Mark Moseley <moseleymark(a)gmail.com> wrote:
On Wed, Oct 10, 2012 at 4:07 PM, Mark Moseley
<moseleymark(a)gmail.com> wrote:
> On Wed, Oct 10, 2012 at 4:00 PM, Steve Grubb <sgrubb(a)redhat.com> wrote:
>> On Wednesday, October 10, 2012 03:45:08 PM Mark Moseley wrote:
>>> On Tue, Oct 9, 2012 at 4:54 PM, Al Viro <viro(a)zeniv.linux.org.uk>
wrote:
>>> > Again, relying on pathnames for forensics (or security in general) is
>>> > a serious mistake (cue unprintable comments about apparmor and similar
>>> > varieties of snake oil). And using audit as poor man's ktrace
analog
>>> > is... misguided, to put it very mildly.
>>>
>>> Caveat: I'm just a sysadmin, so this stuff is as darn near
"magic" as
>>> I get to see on a regular basis, so it's safe to expect some naivety
>>> and/or misguidedness on my part :)
>>>
>>> I'm just using it as a log of files that have been written/changed on
>>> moderately- to heavily-used systems. If there's another in-kernel
>>> mechanism that'd be better suited for that sort of thing (at least
>>> without adding a lot of overhead), I'd be definitely eager to know
>>> about it. It's a web hosting environment, with customer files all
>>> solely on NFS, so writes to the same directory can come from an
>>> arbitrary number of servers. When they get swamped with write
>>> requests, the amount of per-client stats exposed by our Netapp and
>>> Oracle NFS servers is often only enough to point us at a client server
>>> with an abusive user on it (but not much more, without turning on
>>> debugging). Having logs of who's doing writes would be quite useful,
>>> esp when writes aren't happening at that exact moment and wouldn't
>>> show up in tools like iotop. The audit subsystem seemed like the best
>>> fit for this kind of thing, but I'm more than open to whatever works.
>>
>> The audit system is the best fit. But I think Al is saying there are some
>> limitations. i know that Eric pushed some patches a while back that makes a
>> stronger effort at collecting some of this information. What kernel are you
>> using?
Would you happen to have a pointer to those patches? I've been surfing
the archives and not gotten lucky yet with finding the applicable
patchset.
> Yup, understood. I've been playing with a variety of boxes,
but mostly
> within the 3.0.x and 3.2.x series. I'll drop 3.5.6 on some of these
> boxes and see if my issues are already fixed (and proceed directly to
> foot-in-mouth chagrined stage -- usually takes slightly longer to get
> to that stage).
Just gave 3.5.6 a shot and in these two particular cases, the result
is the same: chroot'd actions are logged in the audit entry relative
to the chroot, and the unlinkat/chmodat/chownat audit log entries only
have one item with the bare filename and no indication of directory.
renameat seems to be the toughest of all of them (where
unlinkat/chmodat/chownat give you a hint in another audit entry). This
is doing a renameat(), from /home/moseley/tmp/tmp/renameat/1/a1 to
/home/moseley/tmp/tmp/renameat/2/a2
type=SYSCALL msg=audit(1351557710.520:74211): arch=c000003e
syscall=264 success=yes exit=0 a0=3 a1=40075c a2=4 a3=400759 items=4
ppid=22742 pid=15181 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0
sgid=0 fsgid=0 tty=pts17 ses=1727 comm="rename" exe="/tmp/rename"
key=(null)
type=CWD msg=audit(1351557710.520:74211): cwd="/tmp"
type=PATH msg=audit(1351557710.520:74211): item=0 name="/tmp"
inode=2367550 dev=08:02 mode=040775 ouid=1000 ogid=1000 rdev=00:00
type=PATH msg=audit(1351557710.520:74211): item=1 name="/tmp"
inode=2367551 dev=08:02 mode=040775 ouid=1000 ogid=1000 rdev=00:00
type=PATH msg=audit(1351557710.520:74211): item=2 name="a1"
inode=2367552 dev=08:02 mode=0100664 ouid=1000 ogid=1000 rdev=00:00
type=PATH msg=audit(1351557710.520:74211): item=3 name="a2"
inode=2367552 dev=08:02 mode=0100664 ouid=1000 ogid=1000 rdev=00:00
Anything else I could/should be trying? I'm more than willing to
experiment. I just always assume I'm missing some key flag or
something.
Here's the simple example code ... and, yes, I *do* know how to use
variables, just didn't bother here ;)
#include <fcntl.h>
#include <stdio.h>
#include <sys/types.h>
#include <dirent.h>
int main() {
DIR *a;
DIR *b;
char* dir1 = "/home/moseley/tmp/tmp/renameat/1";
char* dir2 = "/home/moseley/tmp/tmp/renameat/2";
a = opendir( dir1 );
b = opendir( dir2 );
int afd = dirfd( a );
int bfd = dirfd( b );
renameat( afd, "a1", bfd, "a2" );
}
4435
days inactive
4844
days old
linux-audit@lists.linux-audit.osci.io
21 comments
6 participants
participants (6)
-
Al Viro -
Casey Schaufler -
Eric Paris -
John Feuerstein -
Mark Moseley -
Steve Grubb