Hi all,

 

I've seen a several systems crash with what appears to a page fault on audit_prune_tree. The systems are 2.6.18-164.11.1.el5 and audit-1.7.17-3.el5. I've seen crashes with traces like this:

 

crash> bt

PID: 16570  TASK: ffff810108f447a0  CPU: 6   COMMAND: "audit_prune_tre"

#0 [ffff81012e009bd0] crash_kexec at ffffffff800ad060

#1 [ffff81012e009c90] __die at ffffffff80065157

#2 [ffff81012e009cd0] do_page_fault at ffffffff80066dd7

#3 [ffff81012e009dc0] error_exit at ffffffff8005dde9

    [exception RIP: pin_inotify_watch+20]

    RIP: ffffffff800f6034  RSP: ffff81012e009e70  RFLAGS: 00010282

    RAX: 0000000000000190  RBX: ffff8100af8f4ec0  RCX: ffff8100af8f4ef0

    RDX: ffffffff8030fbe0  RSI: ffff810108f447a0  RDI: ffffffff80311d70

    RBP: ffff8100383bb730   R8: ffff81012e008000   R9: 000000000000003a

    R10: ffff81012fd54038  R11: 0000000000000280  R12: ffff8100383bb720

    R13: 0000000000000000  R14: ffff8100383bb730  R15: ffffffff8009fea7

    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0000

#4 [ffff81012e009e88] untag_chunk at ffffffff800b835d

#5 [ffff81012e009ec8] prune_one at ffffffff800b8711

#6 [ffff81012e009ed8] prune_tree_thread at ffffffff800b8f93

#7 [ffff81012e009ee8] kthread at ffffffff8003298c

#8 [ffff81012e009f48] kernel_thread at ffffffff8005dfb1

crash>

 

I looked on bugzilla and did not see anything related to this.

I looked at the kernel stack dump below and saw that there appeared to be duplicates of the audit processes running (auditd, audispd, audisp-prelide) I'm wondering if for some reason a second set of audit processes got started up and that caused a race condition and the kernel crashed.

 

  15599      1   3  ffff8100171e3860  IN   0.0   92684   1676  auditd

  15600      1   1  ffff8100171e3100  IN   0.0   92684   1676  auditd

  15601  15599   5  ffff8100b5952820  IN   0.0  147336    964  audispd

  15602  15601   4  ffff8100b59520c0  IN   0.1   38252   2504  audisp-prelude

  15603  15599   7  ffff81004a964860  IN   0.0  147336    964  audispd

  15606  15601   4  ffff81004a964100  IN   0.1   38252   2504  audisp-prelude

  16554  14228   6  ffff81002a875820  IN   0.0    8700    952  module:auditd

  16555  16554   6  ffff8100a66c9860  IN   0.0   66212   1548  auditd

  16559  16555   1  ffff81010be17100  UN   0.0    6176    816  auditctl

> 16570     73   6  ffff810108f447a0  RU   0.0       0      0  [audit_prune_tre]

 

Any thoughts are appreciated.

 

Best regards,

 

Gary Smith