On Tue, 2007-07-03 at 09:46 +1200, Darryl Dixon - Winterhouse Consulting wrote: > Scenario: > A very large filesystem with potentially millions of files in an ad-hoc, > unordered directory structure. The requirement is to be able to audit > any > action on any file in this filesystem (moves, adds, changes, deletes, > etc). In auditfs terms, there is a requirement to have a 'watch' on > every > single file (millions of files), and on any new file that is added. > > Hypothetical solution: > Clearly, scanning the filesystem with `find` and adding calling auditctl > with the appropriate arguments to generate a watch on every singly file > is > totally infeasible (find takes almost an hour to run, and in the > meantime > stuff is potentially changing...). Instead, I envision it would make > better sense to simply audit every call to write(), open(), rename(), > etc, > and then filter backwards from there with ausearch and a filter based on > the path. On Solaris with BSM, this is possible. My problem is that this > doesn't seem possible with the Linux Audit subsystem, as the arguments > to > the system calls are not decoded (eg, the audit records for write() > include only an opaque filehandle and pointer to the written data, etc). This is entirely feasible, with the exception that you probably don't want to audit write() calls generally. That would be a truly insane amount of traffic. A highly effective heuristic is to look at the arguments to the open() system call. If the file was opened for write (a1 | 1 == 1 || a1 | 2 == 2), then assume that the file was altered. Take the following example: type=SYSCALL msg=audit(1183414428.656:2779): arch=40000003 syscall=5 success=yes exit=3 a0=91d4088 a1=8000 a2=0 a3=8000 items =1 ppid=16166 pid=16270 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 comm="vi" exe="/bin/vi" sub j=user_u:system_r:unconfined_t:s0 key=(null) type=CWD msg=audit(1183414428.656:2779): cwd="/root" type=PATH msg=audit(1183414428.656:2779): item=0 name="install.log" inode=720898 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00 :00 obj=root:object_r:user_home_t:s0 You can tell the filesystem from the dev field of the PATH record: [mbooth@mbooth ~]$ ls -l /dev/mapper/vg_local-root brw-rw---- 1 root disk 253, 0 Jul 2 19:53 /dev/mapper/vg_local-root 253, 0 == fd:00 You can tell it was opened for read, because the last 2 bits of a1 in the SYSCALL record are 00. I guess doing this automatically is the crux of your question. For ease of path handling, you're also going to want a solution to the absolute path question. As an aside, adding the rules you mention above with a standard audit configuration is likely to kill your system as soon as you put load on it. The principal reason for this is the frequent flushing of the log file. If integrity of the audit trail isn't that important, tone down the flushing on the audit log and you'll find that it can sustain pretty intense throughput. If you need both integrity and performance, the best solution I've come up with is to send audit records to a remote host via syslog in real time over a network which we trust not to drop packets, and not write anything to disk. Matt

On Tue, 2007-07-03 at 10:48 +1200, Darryl Dixon - Winterhouse Consulting wrote: > Hi Matt, > > Thank you for your very thorough response. What you say about not being > able to audit 'write()' is worrying to me. The problem with auditing write > by inference from open(), is that one doesn't know *when* the file was > written, or even if it really ever was at all (eg, was data written > continuously from open() to close(), or only sporadically over the course > of hours or days?). Auditing for actual alterations is definitely > something that we need to be able to track. Assuming for a moment that we > have beefy enough hardware ( heh ), can the path be extracted from write() > as with your example for open() above? My assumption would have been that > CWD reflected only where the exe was launched from, and not necessarily > where the write()-en file was located... Well, you can audit write(). The question is whether you can handle the resulting data volume. read() and write() don't generate a PATH record. The only time at which a path is relevant is at the time the file is opened. Once the file is opened, it can have zero or more paths, which can all change without affecting the open file. An open file is genuinely divorced from its path. That said, it's *not* divorced from its filesystem, which I understand is what you're really looking for. Unfortunately that information doesn't appear to live on its own in the audit trail, and it's not available to filter on for these calls. This would leave you having to audit all read() and write() calls and filtering them in post-processing. I doubt this would be a practical solution.

One caveat though - auditing of write() won't catch all possible ways of modifying the file data, e.g. one could mmap() the file with MAP_SHARED and then write to the memory, followed by msync or munmap.

What you say about not being able to audit 'write()' is worrying to me. The problem with auditing write by inference from open(), is that one doesn't know *when* the file was written,

My assumption would have been that CWD reflected only where the exe was launched from, and not necessarily where the write()-en file was located...