I set heartbeat to 60 on the client and idle to 120 on the server.  Reconnects seem fine now, although I never did nail down the exact conditions under which reconnects failed.

But I still have the problem of weird buffering on the client side.  If I run `sudo ls` on the client side, locally I get something like:

    node=grax.sea.marchex.com type=SYSCALL msg=audit(1468448156.161:3288765): arch=c000003e syscall=59 success=yes exit=0 a0=8f81e8 a1=8f7578 a2=8febf0 a3=7ffd3d956370 items=2 ppid=19387 pid=19388 auid=2288 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts4 ses=399 comm="ls" exe="/bin/ls" key="rootcmd"
    node=grax.sea.marchex.com type=EXECVE msg=audit(1468448156.161:3288765): argc=1 a0="ls"
    node=grax.sea.marchex.com type=CWD msg=audit(1468448156.161:3288765):  cwd="/tmp"
    node=grax.sea.marchex.com type=PATH msg=audit(1468448156.161:3288765): item=0 name="/bin/ls" inode=132438 dev=09:01 mode=0100755 ouid=0 ogid=0 rdev=00:00
    node=grax.sea.marchex.com type=PATH msg=audit(1468448156.161:3288765): item=1 name=(null) inode=71179 dev=09:01 mode=0100755 ouid=0 ogid=0 rdev=00:00

But remotely, I just get:

    node=grax.sea.marchex.com type=SYSCALL msg=audit(1468448156.161:3288765): arch=c000003e syscall=59 success=yes exit=0 a0=8f81e8 a1=8f7578 a2=8febf0 a3=7ffd3d956370 items=2 ppid=19387 pid=19388 auid=2288 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts4 ses=399 comm="ls" exe="/bin/ls" key="rootcmd"

Just the first line of the audit record.  No matter how long I wait.  If I then run `sudo ls` again, *then* the rest of the lines show up in the server's log.

The buffering appears to be on the client side, because if I restart the server's auditd, those lines are not lost: they still appear in the remote log ... but not until the next time I run `sudo ls` on the client side.

This is on 1.7.x.  This does not happen on 2.4.x or 2.6.x.







On Wed, Jul 13, 2016 at 11:38 AM, Steve Grubb <sgrubb@redhat.com> wrote:
On Wednesday, July 13, 2016 10:51:07 AM EDT Chris Nandor wrote:
> The only reason I am even upgrading is because of the issues with
> audisp-remote, the not-reconnecting, and the apparent client-side
> buffering, that went away with 2.4.x and 2.6.x.  So if we decide to ship
> logs a different way than with audisp-remote, then it might be best to
> stick with 1.7.x.

This sounds a lot like the idle detection is not set right. In audisp-
remote.conf there is a setting heartbeat_timeout. This should be set to
something like 60 or 120. Then on the server in auditd.conf there is a setting
tcp_client_max_idle which should be over twice as high as heartbeat_timeout.
So, you'd set it to 180 or 300.

> That said, so far I see no issues, so we're going to forge ahead and see
> what happens.  I just need to keep in mind what our mitigation plan would
> be if we do run into issues.

Old utilities won't know what to do with enriched events. AFAICS, that would
be the long term issue. You'll need to do aperl, awk, or cut command to trim
off the unknown part of the event in your logs.

-Steve