I was looking at the case where a user boots up with audit daemon installed. It turns on auditing. This means that all processes that fork will start getting a context built. Then the user decides to do a benchmark and turns the audit system off by auditctl -e 0. The system doesn't really get performance back as if auditing was never turned on. If you look at audit_syscall_exit, there is this check: if (likely(!context)) goto out; Don't all the running processes still have a context?

Shouldn't this also have a check that if audit_enabled == 0, that the context is reclaimed and context set to NULL? What reaps the context for these processes. They all still seem to be penalized.

On Thursday 08 September 2005 19:31, Chris Wright wrote: > It seems odd a benchmark that runs after -e0 (that's creating only new > processes) would be that penalized. Agreed, that's why I think we need to dig into this to see where it leads.

> Profiles would be helpful. Actually, it'd be interesting to see overhead of > audit turned on, but not generating any records (no rules loaded, no avc > messages). True. I'll see if we can get those.

> Not sure what you mean by reap, normal task destruction will still reap > those. I mean claim the memory used by the context and set context = NULL so that syscall_exit no longer gets penalized.

> But it's a valid question if disabling audit should disable > audit_exit. Any pending audit records would be lost. The act of issuing auditctl -e 0 is not precise, there could always be something lost. > Trying to proactively reclaim contexts when disabled would mean long running > processes would no longer get audited if an admin did disable/enable.

It would seem that whatever made the program auditable would still be true and a context would eventually get generated for it if it indeed was of interest. Maybe changing the if (likely(!context)) to if (likely(context == NULL || audit_enabled == 0)) would solve the problem.

On Thursday 08 September 2005 21:44, Steve Grubb wrote: >>Profiles would be helpful. Actually, it'd be interesting to see overhead >>of audit turned on, but not generating any records (no rules loaded, no >>avc messages). > >True. I'll see if we can get those. OK, I do have some profile data from booting with profile=1. When audit was never enabled, the test ran in 22.6 seconds. When audit had run and been disabled, the test ran in 23.8 seconds. With audit enabled, the test took 24.7 seconds. This is repeatable. Now some data...from audit not enabled: 22579 total 0.0088 2987 __d_lookup 4.1602 2939 system_call 58.7800 2477 __link_path_walk 0.5132 2139 strncpy_from_user 23.7667 1129 inode_has_perm 8.8898 1120 avc_has_perm_noaudit 5.3589 959 kmem_cache_alloc 13.1370 913 kmem_cache_free 19.4255 807 dput 0.4667 763 selinux_inode_permission 4.6524 731 _atomic_dec_and_lock 2.2562 614 memcpy 14.9756 571 permission 3.1547 485 __might_sleep 2.8869 boot with audit, then disabled: 23841 total 0.0093 2729 strncpy_from_user 30.3222 2584 __d_lookup 3.5989 2525 system_call 50.5000 2358 __link_path_walk 0.4885 1222 inode_has_perm 9.6220 1131 avc_has_perm_noaudit 5.4115 1103 kmem_cache_alloc 15.1096 995 kmem_cache_free 21.1702 752 memcpy 18.3415 693 _atomic_dec_and_lock 2.1389 663 __might_sleep 3.9464 640 syscall_exit_work 26.6667 606 permission 3.3481 591 dput 0.3418 546 selinux_inode_permission 3.3293 511 avc_lookup 3.3399 And audit on: 24672 total 0.0096 3198 system_call 63.9600 2764 __d_lookup 3.8496 2556 __link_path_walk 0.5295 1427 strncpy_from_user 15.8556 1303 kmem_cache_free 27.7234 1112 audit_syscall_exit 0.8355 1013 avc_has_perm_noaudit 4.8469 974 inode_has_perm 7.6693 974 _atomic_dec_and_lock 3.0062 812 kmem_cache_alloc 11.1233 665 dput 0.3846 624 syscall_exit_work 26.0000 545 memcpy 13.2927 464 selinux_inode_permission 2.8293 463 __might_sleep 2.7560 The big difference between 1 & 2 is more kmem_cache_alloc & free, and syscall_exit_work. The syscall_exit_work comes from this test: 227 movl TI_flags(%ebp), %ecx 228 testw $_TIF_ALLWORK_MASK, %cx 229 jne syscall_exit_work So, Amy is right, the TIF_AUDIT flag has some bearing on the benchmarks. The big difference between runs 2 & 3 are _atomic_dec_and_lock being higher and audit_syscall_exit showing up. system_syscall was higher, too. Not sure where it picked up cycles. I think all of these increases are to be expected. These are run with no rules loaded. -Steve -- Linux-audit mailing list Linux-audit(a)redhat.com http://www.redhat.com/mailman/listinfo/linux-audit

OK, just don't clutter core code with audit enabled tests.

* Steve Grubb (sgrubb(a)redhat.com) wrote: >So, what about re-enabling these for existing processes when audit_enabled >changes to 1 again? That's the part I was kinda stuck at. I don't think we >constantly want to set the thread info. fresh out of good ideas ;-) that's partly why i'm curious if that patch makes a difference. if it doesn't then we can go with current method. same issue for lsm, and the rule of thumb is to make sure you're enabled from bootup, otherwise you have to check every process either at load time or lazily at syscall entrance. doing it at load time is ugly and discouraged (requires walking tasklist), and lazy method undoes the benefits of the patch.

* Linda Knippers (linda.knippers(a)hp.com) wrote: >Would it be better to not allow auditing to be enabled after boot >then? I'm concerned about the case where auditing isn't started >at boot time but enabled later. There could be alot of processes >that won't be audited. If things can't be both dynamic and correct >then I vote for correct. That would also mean you can't disable dynamically (as it would be a reboot to turn it back on). This sounds overally restrictive.

I'd vote for documented and left up to admin (with sane default).

It's pretty useful (at least from development perspective ;-) to disable, then re-enable.

Understood, it's arguably violating the principle of least surprise to allow disable but not enable.

... For CAPP, you wouldn't want to use nfs,

Why not always enable the audit subsystem at boot (if it's configured) always and then in rc.local or whatever, disable it via auditctl. That way if you re-enable later, those processes can be audited.

On Thu, Sep 08, 2005 at 04:31:02PM -0700, Chris Wright wrote: > * Steve Grubb (sgrubb(a)redhat.com) wrote: > > Don't all the running processes still have a context? > > Already running processes would (except those with 'never' rules), but > news ones shouldn't (new meaning after -e0). It seems odd a benchmark > that runs after -e0 (that's creating only new processes) would be that > penalized. New processes would have no context, and existing processes > would bail out of audit_syscall_entry. Profiles would be helpful. The TIF_SYSCALL_AUDIT flag isn't explicitly cleared in copy_process, so it continues to be propagated to new tasks once audit has been enabled, regardless of the current value of the audit_enabled flag. After -e0, new tasks still incur some extra overhead from trace syscall processing instead of jumping out early. It would be interesting to see some numbers on that.

The only problem I see is when audit is re-enabled, we need a way to start getting the TIF_SYSCALL_AUDIT flag set again for already running processes. For example, suppose apache was of interest and audit was disabled. The above code would remove the flag. Then when audit is re-enabled, we need to set the flag again. I'm looking for a low impact way of doing this. Still thinking.