Hello Paul, On 08/12/2015 17:42, Paul Moore wrote: > To the best of our knowledge, everyone who enables audit at compile > time also enables syscall auditing; this patch simplifies the Kconfig > menus by removing the option to disable syscall auditing when audit > is selected and the target arch supports it. > > Signed-off-by: Paul Moore <pmoore(a)redhat.com&gt; this patch is responsible for massive performance degradation for those who used only CONFIG_SECURITY_APPARMOR. And the numbers are, take the following test for instance: dd if=/dev/zero of=/dev/null count=2M ARM64: 500MB/s -> 350MB/s ARM: 400MB/s -> 300MB/s

Hello Paul, On 28/01/2019 15:19, Paul Moore wrote: >>> time also enables syscall auditing; this patch simplifies the Kconfig >>> menus by removing the option to disable syscall auditing when audit >>> is selected and the target arch supports it. >>> >>> Signed-off-by: Paul Moore <pmoore(a)redhat.com&gt; >> this patch is responsible for massive performance degradation for those >> who used only CONFIG_SECURITY_APPARMOR. >> >> And the numbers are, take the following test for instance: >> >> dd if=/dev/zero of=/dev/null count=2M >> >> ARM64: 500MB/s -> 350MB/s >> ARM: 400MB/s -> 300MB/s > Hi there. > > Out of curiosity, what kernel/distribution are you running, or is this > a custom kernel compile? Can you also share the output of 'auditctl This test was carried out with Linux 4.9. Custom built.

> -l' from your system? The general approach taken by everyone to > turn-off the per-syscall audit overhead is to add the "-a never,task" > rule to their audit configuration: > > # auditctl -a never,task > > If you are using Fedora/CentOS/RHEL, or a similarly configured system, This is an embedded distribution. We are not using auditctl or any other audit-related user-space packages. > you can find this configuration in the /etc/audit/audit.rules file (be > warned, that file is automatically generated based on > /etc/audit/rules.d). I suppose in this case rule list must be empty. Is there a way to check this without extra user-space packages?

Hello Paul, On 28/01/2019 15:52, Paul Moore wrote: >>>>> time also enables syscall auditing; this patch simplifies the Kconfig >>>>> menus by removing the option to disable syscall auditing when audit >>>>> is selected and the target arch supports it. >>>>> >>>>> Signed-off-by: Paul Moore <pmoore(a)redhat.com&gt; >>>> this patch is responsible for massive performance degradation for those >>>> who used only CONFIG_SECURITY_APPARMOR. >>>> >>>> And the numbers are, take the following test for instance: >>>> >>>> dd if=/dev/zero of=/dev/null count=2M >>>> >>>> ARM64: 500MB/s -> 350MB/s >>>> ARM: 400MB/s -> 300MB/s >>> Hi there. >>> >>> Out of curiosity, what kernel/distribution are you running, or is this >>> a custom kernel compile? Can you also share the output of 'auditctl >> This test was carried out with Linux 4.9. Custom built. > I suspected that was the case, thanks. > >>> -l' from your system? The general approach taken by everyone to >>> turn-off the per-syscall audit overhead is to add the "-a never,task" >>> rule to their audit configuration: >>> >>> # auditctl -a never,task >>> >>> If you are using Fedora/CentOS/RHEL, or a similarly configured system, >> This is an embedded distribution. We are not using auditctl or any other >> audit-related user-space packages. >> >>> you can find this configuration in the /etc/audit/audit.rules file (be >>> warned, that file is automatically generated based on >>> /etc/audit/rules.d). >> I suppose in this case rule list must be empty. Is there a way to check >> this without extra user-space packages? > Yes, unless you are loading rules through some other method I would > expect that your audit rule list is empty. > > I'm not aware of any other tools besides auditctl to load audit rules > into the kernel, although I haven't ever had a need for another tool > so I haven't looked very hard. If you didn't want to bring auditctl > into your distribution, I expect it would be a rather trivial task to > create a small tool to load a single "-a never,task" into the kernel. I've done a quick test on my x86_64 PC and got the following results:

Which brings me to an idea, that the subject patch should have been accompanied by a default "never,task" rule inside the kernel, otherwise you require an extra user-space package (audit) just to bring Linux 4.5+ to 4.4 performance levels.

On Mon, 28 Jan 2019 11:26:51 -0500 Paul Moore <paul(a)paul-moore.com&gt; wrote: > On Mon, Jan 28, 2019 at 10:38 AM Sverdlin, Alexander (Nokia - DE/Ulm) > <alexander.sverdlin(a)nokia.com&gt; wrote: > > Hello Paul, > > > > On 28/01/2019 15:52, Paul Moore wrote: > > >>>>> time also enables syscall auditing; this patch simplifies the > > >>>>> Kconfig menus by removing the option to disable syscall > > >>>>> auditing when audit is selected and the target arch supports > > >>>>> it. > > >>>>> > > >>>>> Signed-off-by: Paul Moore <pmoore(a)redhat.com&gt; > > >>>> this patch is responsible for massive performance degradation > > >>>> for those who used only CONFIG_SECURITY_APPARMOR. > > >>>> > > >>>> And the numbers are, take the following test for instance: > > >>>> > > >>>> dd if=/dev/zero of=/dev/null count=2M > > >>>> > > >>>> ARM64: 500MB/s -> 350MB/s > > >>>> ARM: 400MB/s -> 300MB/s > > >>> Hi there. > > >>> > > >>> Out of curiosity, what kernel/distribution are you running, or > > >>> is this a custom kernel compile? Can you also share the output > > >>> of 'auditctl > > >> This test was carried out with Linux 4.9. Custom built. > > > I suspected that was the case, thanks. > > > > > >>> -l' from your system? The general approach taken by everyone to > > >>> turn-off the per-syscall audit overhead is to add the "-a > > >>> never,task" rule to their audit configuration: > > >>> > > >>> # auditctl -a never,task > > >>> > > >>> If you are using Fedora/CentOS/RHEL, or a similarly configured > > >>> system, > > >> This is an embedded distribution. We are not using auditctl or > > >> any other audit-related user-space packages. > > >> > > >>> you can find this configuration in the /etc/audit/audit.rules > > >>> file (be warned, that file is automatically generated based on > > >>> /etc/audit/rules.d). > > >> I suppose in this case rule list must be empty. Is there a way > > >> to check this without extra user-space packages? > > > Yes, unless you are loading rules through some other method I > > > would expect that your audit rule list is empty. > > > > > > I'm not aware of any other tools besides auditctl to load audit > > > rules into the kernel, although I haven't ever had a need for > > > another tool so I haven't looked very hard. If you didn't want > > > to bring auditctl into your distribution, I expect it would be a > > > rather trivial task to create a small tool to load a single "-a > > > never,task" into the kernel. > > > > I've done a quick test on my x86_64 PC and got the following > > results: > > ... > > > Which brings me to an idea, that the subject patch should have been > > accompanied by a default "never,task" rule inside the kernel, > > otherwise you require an extra user-space package (audit) just to > > bring Linux 4.5+ to 4.4 performance levels. > > [NOTE: I dropped pmoore(a)redhat.com from the To/CC line, I left Red Hat > for greener pastures several months ago.] > > Well, it generally hasn't been an issue as 1) most people that enable > audit also enable syscall auditing and 2) most people that enable > audit have some sort of audit userspace tools to work with the audit > logs (and configure audit if necessary). I don't want to diminish > your report, but this change was made several years ago and we really > haven't heard of many issues surrounding the change. If we can ever > get all of the different arches to support syscall auditing, I'd love > to get rid of the syscall auditing Kconfig knob entirely. > > If you wanted to put together a patch that added a single "-a > never,task" rule on boot I could get behind that, just make it default > to off. That will make processes unauditable for everyone. That's how it gets a speedup is not entering into the audit machinery.

I suppose its possible that people may want MAC hardwired events but no syscall events. I don't know if there are other kconfig audit options. but maybe getting it down to audit enabled and syscall auditing as the only choices is probably the most performant. -Steve

On Mon, 28 Jan 2019 15:08:56 -0500 Paul Moore <paul(a)paul-moore.com&gt; wrote: > On Mon, Jan 28, 2019 at 3:03 PM Steve Grubb <sgrubb(a)redhat.com&gt; wrote: > > On Mon, 28 Jan 2019 11:26:51 -0500 > > Paul Moore <paul(a)paul-moore.com&gt; wrote: > > > > > On Mon, Jan 28, 2019 at 10:38 AM Sverdlin, Alexander (Nokia - > > > DE/Ulm) <alexander.sverdlin(a)nokia.com&gt; wrote: > > > > Hello Paul, > > > > > > > > On 28/01/2019 15:52, Paul Moore wrote: > > > > >>>>> time also enables syscall auditing; this patch simplifies > > > > >>>>> the Kconfig menus by removing the option to disable > > > > >>>>> syscall auditing when audit is selected and the target > > > > >>>>> arch supports it. > > > > >>>>> > > > > >>>>> Signed-off-by: Paul Moore <pmoore(a)redhat.com&gt; > > > > >>>> this patch is responsible for massive performance > > > > >>>> degradation for those who used only > > > > >>>> CONFIG_SECURITY_APPARMOR. > > > > >>>> > > > > >>>> And the numbers are, take the following test for instance: > > > > >>>> > > > > >>>> dd if=/dev/zero of=/dev/null count=2M > > > > >>>> > > > > >>>> ARM64: 500MB/s -> 350MB/s > > > > >>>> ARM: 400MB/s -> 300MB/s > > > > >>> Hi there. > > > > >>> > > > > >>> Out of curiosity, what kernel/distribution are you running, > > > > >>> or is this a custom kernel compile? Can you also share the > > > > >>> output of 'auditctl > > > > >> This test was carried out with Linux 4.9. Custom built. > > > > > I suspected that was the case, thanks. > > > > > > > > > >>> -l' from your system? The general approach taken by > > > > >>> everyone to turn-off the per-syscall audit overhead is to > > > > >>> add the "-a never,task" rule to their audit configuration: > > > > >>> > > > > >>> # auditctl -a never,task > > > > >>> > > > > >>> If you are using Fedora/CentOS/RHEL, or a similarly > > > > >>> configured system, > > > > >> This is an embedded distribution. We are not using auditctl > > > > >> or any other audit-related user-space packages. > > > > >> > > > > >>> you can find this configuration in > > > > >>> the /etc/audit/audit.rules file (be warned, that file is > > > > >>> automatically generated based on /etc/audit/rules.d). > > > > >> I suppose in this case rule list must be empty. Is there a > > > > >> way to check this without extra user-space packages? > > > > > Yes, unless you are loading rules through some other method I > > > > > would expect that your audit rule list is empty. > > > > > > > > > > I'm not aware of any other tools besides auditctl to load > > > > > audit rules into the kernel, although I haven't ever had a > > > > > need for another tool so I haven't looked very hard. If you > > > > > didn't want to bring auditctl into your distribution, I > > > > > expect it would be a rather trivial task to create a small > > > > > tool to load a single "-a never,task" into the kernel. > > > > > > > > I've done a quick test on my x86_64 PC and got the following > > > > results: > > > > > > ... > > > > > > > Which brings me to an idea, that the subject patch should have > > > > been accompanied by a default "never,task" rule inside the > > > > kernel, otherwise you require an extra user-space package > > > > (audit) just to bring Linux 4.5+ to 4.4 performance levels. > > > > > > [NOTE: I dropped pmoore(a)redhat.com from the To/CC line, I left > > > Red Hat for greener pastures several months ago.] > > > > > > Well, it generally hasn't been an issue as 1) most people that > > > enable audit also enable syscall auditing and 2) most people that > > > enable audit have some sort of audit userspace tools to work with > > > the audit logs (and configure audit if necessary). I don't want > > > to diminish your report, but this change was made several years > > > ago and we really haven't heard of many issues surrounding the > > > change. If we can ever get all of the different arches to > > > support syscall auditing, I'd love to get rid of the syscall > > > auditing Kconfig knob entirely. > > > > > > If you wanted to put together a patch that added a single "-a > > > never,task" rule on boot I could get behind that, just make it > > > default to off. > > > > That will make processes unauditable for everyone. That's how it > > gets a speedup is not entering into the audit machinery. > > That is pretty much what is being asked for in this thread. It's > really no different from what Fedora/CentOS/RHEL (and who knows how > many others) ship with their default audit config. It's important to > note that you could always delete this rule at runtime; all that is > being proposed is to have the kernel populate the the audit ruleset > with the "-a never,task" rule *IF* the proposed kernel Kconfig option > is enabled (and it would default to being off, you would have to turn > it on during build). Yes, but you can't add the auditable flag back to the task struct because syscalls never enter audit machinery where it can be added back. Meaning that even if you wanted to audit, there will be processes you never can audit. Whereas deciding via auditd, they become unauditable only after auditd loads the rule. Prior to that they are and all processes are auditable if you decide to audit. So, this is fundamentally different than what fedora does.

-Steve > > I suppose its possible that people may want MAC hardwired events > > but no syscall events. I don't know if there are other kconfig > > audit options. but maybe getting it down to audit enabled and > > syscall auditing as the only choices is probably the most > > performant. > > > > -Steve

>> To the best of our knowledge, everyone who enables audit at compile >> time also enables syscall auditing; this patch simplifies the Kconfig >> menus by removing the option to disable syscall auditing when audit >> is selected and the target arch supports it. >> >> Signed-off-by: Paul Moore <pmoore(a)redhat.com&gt; > this patch is responsible for massive performance degradation for those > who used only CONFIG_SECURITY_APPARMOR. > > And the numbers are, take the following test for instance: > > dd if=/dev/zero of=/dev/null count=2M > > ARM64: 500MB/s -> 350MB/s > ARM: 400MB/s -> 300MB/s Hi there. Out of curiosity, what kernel/distribution are you running, or is this a custom kernel compile? Can you also share the output of 'auditctl -l' from your system? The general approach taken by everyone to turn-off the per-syscall audit overhead is to add the "-a never,task" rule to their audit configuration: # auditctl -a never,task If you are using Fedora/CentOS/RHEL, or a similarly configured system, you can find this configuration in the /etc/audit/audit.rules file (be warned, that file is automatically generated based on /etc/audit/rules.d).