On Fri, 2005-07-08 at 14:18 -0600, Timothy R. Chavez wrote: >These don't look familiar to me, but I think it'd be good to send them >out to everyone to take a look... > > HTML-ised, word-wrapped, 'deepSkyBlue' oopses? Can I have some of what you lot are smoking? :) The first panic is similar to something else I've seen but not managed to make any progress with. In that case I was told the precise kernel and was able to determine that the oops in proc_get_inode was due to an invalid ->owner field in try_module_get(). That one was sent to me as an OpenOffice document, but I'll do the world a favour and reproduce it here as text... Oops: 0000 [1] SMP inode=2 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00 CPU 7 Modules linked in: michael_mic parport_pc lp parport netconsole netdump autofs4 i2c_dev i2c_core sunrpc ds yenta_socket pcmcia_core button battery ac md5 ipv6 ohci_hcd ehci_hcd tg3 dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod aacraid sd_mod scsi_mod Pid: 10704, comm: mount_test Not tainted 2.6.9-11.EL.audit.74smp RIP: 0010:[<ffffffff801a6764>] <ffffffff801a6764>{proc_get_inode+199} RSP: 0018:0000010068eb9ce8 EFLAGS: 00010282 RAX: 0000000000000007 RBX: 000001007cce6d70 RCX: 0000000000000000 RDX: ffffffffa01a5180 RSI: 000001000314b478 RDI: ffffffff80476380 RBP: 0000010068a1e060 R08: 0000010068eb9ca8 R09: 0000000000000000 RBP: 0000010068a1e060 R08: 0000010068eb9ca8 R09: 0000000000000000 R10: 000001006d81c950 R11: 0000000000000058 R12: 000001007ff05538 R13: 0000000000000000 R14: 00000000ffffffea R15: 0000010068eb9d98 FS: 0000002a95583b00(0000) GS:ffffffff804c6f80(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffa01a5180 CR3: 000000007fe9a000 CR4: 00000000000006e0 Process mount_test (pid: 10704, threadinfo 0000010068eb8000, task 000001006c5be030) Stack: 000001007cce6d70 00000100679ed260 000001006963e600 ffffffff801a91da fffffffffffffff4 00000100679ed260 000001006963e600 00000100679ed368 0000010068eb9e58 ffffffff80182060 Call Trace: <ffffffff801a91da>{proc_lookup+246} <ffffffff80182060>{do_lookup+230} <ffffffff80182c76>{link_path_walk+2508} <ffffffff801832a7>{path_lookup+451} <ffffffff80183553>{__user_walk+47} <ffffffff8017e037>{vfs_lstat+21} <ffffffff80154234>{audit_syscall_entry+306} <ffffffff8017e369>{sys_newlstat+17} <ffffffff801141e4>{syscall_trace_enter+161} <ffffffff80110142>{tracesys+113} <ffffffff801101a2>{tracesys+209} Code: 83 3a 02 74 32 89 c0 48 c1 e0 07 48 8d 04 02 ff 80 00 01 00 RIP <ffffffff801a6764>{proc_get_inode+199} RSP <0000010068eb9ce8> CR2: ffffffffa01a5180 In this case, the owner field of the proc directory in question is set to 0xffffffffa01a5180, when it _should_ have been a pointer to a valid 'struct module'. We don't seem to have been given the faulting address in the panic you show, but I'm fairly sure it'll be the same thing. Can you tell me which /proc file was being accessed when this happened? This is happening _before_ the audit hooks in path_lookup() are reached; there shouldn't be anything happening in this particular code path which is audit-related. There's probably been some problem _beforehand_. Jeff, wasn't there a netdump in the x86_64 case above?

The second and third oopses are basically the same as each other. I'm inclined to suspect that it's the call to fops_get() in dentry_open(), which is actually another call to try_module_get(). But in that case it's a _different_ pointer to a struct module, not one in a proc_dir_entry but one in a struct file_operations. Again, what file is being access? One in /proc, I'd imagine?

How easy is it to reproduce these oopsen? Can it be done with audit disabled? Can it be done on the base U1 kernel?

> I can write a test that will try and read all files in /proc. May or > may not provide > any data. Attached is the test program they were running. Kris Wilson sent it to me. I've tried it a few times on an ia64 system with the .74 kernel but haven't seen any problems. Maybe you'll have better luck. -- ljk ------------------------------------------------------------------------ ########################################################################## # Copyright (C) International Business Machines Corp., 2004 # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See # the GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # # # # FILE : Makefile # # PURPOSE: The default "all" target uses the gcc compiler to compile all # test files in the current directory. # # The "clean" target removes all files the end in ".o" (object # files). # # # HISTORY: # 10/04 originated by Dan Jones (danjones(a)us.ibm.com) # ########################################################################## TOPDIR= ../.. LOG=log CC=gcc EXE=testfileperms DIRS= proc \ sys all:: $(EXE) run:: all -grep "^fpermu" /etc/passwd | awk -F: '{print $$1}' | xargs -i userdel -r {} -grep "^fpermg" /etc/group | awk -F: '{print $$1}' | xargs -i groupdel {} useradd fpermu &>/dev/null groupadd fpermg &>/dev/null -for i in $(DIRS); do ./$(EXE) /$$i fpermu fpermg &>$(EXE).$$i.$(LOG); done -for i in $(DIRS); do grep "FAIL" $(EXE).$$i.$(LOG) > $(EXE).$$i.FAIL; done -for i in $(DIRS); do grep "SKIP" $(EXE).$$i.$(LOG) > $(EXE).$$i.SKIP; done groupdel fpermg &>/dev/null userdel -fr fpermu &>/dev/null rmlogs:: rm -f *log &>/dev/null rm -f *FAIL &>/dev/null rm -f *SKIP &>/dev/null clean:: rm -f $(EXE) # include $(TOPDIR)/rules.mk ------------------------------------------------------------------------ /********************************************************************** ** Copyright (C) International Business Machines Corp., 2004 ** ** This program is free software; you can redistribute it and/or modify ** it under the terms of the GNU General Public License as published by ** the Free Software Foundation; either version 2 of the License, or ** (at your option) any later version. ** ** This program is distributed in the hope that it will be useful, ** but WITHOUT ANY WARRANTY; without even the implied warranty of ** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See ** the GNU General Public License for more details. ** ** You should have received a copy of the GNU General Public License ** along with this program; if not, write to the Free Software ** Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ** ** ** ** FILE : testfileperms.c ** ** PURPOSE: The purpose of this test is to verify the file permissions ** of all files under a given directory. The test makes 2 passes ** through all the files. ** Pass 1: ** Using the existing file attributes, verify file ** access as the file owner, group owner and other. ** The results of "open" are used to determine access. ** If the file owner is root, it is expected that access ** will be granted even if permissions are explicitly ** denied. ** Pass 2: ** An attempt is made to chown the file to the provided ** testuser and testgroup. ** If the chown fails, a message is logged and the file ** is skipped. ** If the chown succeeds, a stat is performed to verify ** the chown was effective. If the file owner and group ** has not been modified, a message is logged and the file ** is skipped. ** Once the file is chowned, file permissions are verified ** as the testuser/testgroup. ** ** In all cases, links are skipped. ** ** ** HISTORY: ** 10/04 originated by Dan Jones (danjones(a)us.ibm.com) ** A1 11/18/04 Kris Wilson (krisw(a)us.ibm.com) If access is expected ** but denied, this is not a security problem, it is a ** problem if denial is expected but access allowed. So if ** pass is expected but fail received, this has been deemed ** OK for certification purposes. **********************************************************************/ #include <sys/types.h> #include <sys/dir.h> #include <sys/param.h> #include <sys/stat.h> #include <stdio.h> #include <unistd.h> #include <string.h> #include <pwd.h> #include <grp.h> #include <fcntl.h> #include <errno.h> extern int alphasort (); static int perms[] = {S_IRUSR, S_IWUSR, S_IXUSR, S_IRGRP, S_IWGRP, S_IXGRP, S_IROTH, S_IWOTH, S_IXOTH}; static char *ptext[] = {"r", "w", "x", "r", "w", "x", "r", "w", "x"}; int totalpass = 0; int totalfail = 0; uid_t uid_nobody = 65534; gid_t gid_nobody = 65533; int test = 0; static char *test_description[] = {"Check default permissions", "chown files to testuser/testgroup"}; /* * Do not include . or .. in directory list. */ int file_select (struct direct *entry) { if ((strcmp (entry->d_name, ".") == 0) || (strcmp (entry->d_name, "..") == 0)) return (0); else return (1); } /* * Set euid, egid */ void setids(uid_t uid, gid_t gid) { int rc = 0; if ((rc = setegid(gid)) == -1) printf("\nERROR: unable to set gid. errno = %d\n", errno); if ((rc = seteuid(uid)) == -1) printf("\nERROR: unable to set uid. errno = %d\n", errno); return; } /* * Check actual vs. expected access using open system call */ void testaccess(char *pathname, int mode, int expected, char *outbuf) { int testrc = 0; int myerr = 0; if (expected == -1) { strcat(outbuf, "expected: fail "); } else { strcat(outbuf, "expected: pass "); } if ((testrc = open(pathname, mode)) == -1) { myerr = errno; strcat(outbuf, "actual: fail"); } else { strcat(outbuf, "actual: pass"); close(testrc); } if (myerr == ENODEV) { sprintf(&(outbuf[strlen(outbuf)]), "\tresult: SKIP : no device : %s\n", pathname); } else if (myerr == EBUSY) { sprintf(&(outbuf[strlen(outbuf)]), "\tresult: SKIP : device busy : %s\n", pathname); } else if ((testrc == expected) || ((expected == 0) && (testrc != -1))) { strcat(outbuf, "\tresult: PASS\n"); totalpass++; } else { /* A1 OK if expected pass but got fail, but need to log so can manually check errno. */ if ((expected == 0) && (testrc == -1)) { sprintf(&(outbuf[strlen(outbuf)]), "\tresult: FAIL/PASS : errno = %d : %s\n", errno, pathname); totalpass++; } else { sprintf(&(outbuf[strlen(outbuf)]), "\tresult: FAIL : errno = %d : %s\n", errno, pathname); totalfail++; } } printf("%s", outbuf); return; } /* * Test access for owner, group, other */ void testall(struct stat *ostatbufp, char *pathname, uid_t uid, gid_t gid) { int i; int rc = 0; char outbuf[256]; struct passwd passwd; struct passwd *passwdp; struct group group; struct group *groupp; struct stat statbuf; struct stat *statbufp = &statbuf; char *pbuf; char *gbuf; passwdp = (struct passwd *)malloc(sizeof(passwd)); groupp = (struct group *)malloc(sizeof(group)); pbuf = (char *)malloc(4096); gbuf = (char *)malloc(4096); memset(pbuf, '\0', 4096); memset(gbuf, '\0', 4096); setids(0,0); printf("\n%s\n", pathname); // For test 1 we chown the file owner/group if (test == 1) { if ((rc = chown(pathname, uid, gid)) == -1) { printf("ERROR: unable to chown %s to %d:%d\n", pathname, uid, gid); goto EXIT; } } // Start with clean buffers memset(&statbuf, '\0', sizeof(statbuf)); // Get file stat info to determine actual owner and group stat(pathname, &statbuf); // If we successfully chown'd the file, but the owner hasn't changed // log it and skip. if ((test == 1) && ((statbufp->st_uid != uid) || (statbufp->st_gid != gid))) { printf("INFO: chown success, but file owner did not change: %s\n", pathname); goto EXIT; } memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "MODE: "); for (i = 0; i < sizeof(perms)/sizeof(int); i++) { if (statbufp->st_mode & perms[i]) { strcat(outbuf, ptext[i]); } else { strcat(outbuf, "-"); } } getpwuid_r(statbufp->st_uid, &passwd, pbuf, 4096, &passwdp); getgrgid_r(statbufp->st_gid, &group, gbuf, 4096, &groupp); sprintf(&(outbuf[strlen(outbuf)]), " %s:%s\n", passwd.pw_name, group.gr_name); printf("%s", outbuf); // Check owner access for read/write setids(statbufp->st_uid, gid_nobody); memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "Owner read\t"); // If we are root, we expect to succeed event // without explicit permission. if ((statbufp->st_mode & S_IRUSR) || (statbufp->st_uid == 0)) { testaccess(pathname, O_RDONLY, 0, outbuf); } else { testaccess(pathname, O_RDONLY, -1, outbuf); } memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "Owner write\t"); // If we are root, we expect to succeed event // without explicit permission. if ((statbufp->st_mode & S_IWUSR) || (statbufp->st_uid == 0)) { testaccess(pathname, O_WRONLY, 0, outbuf); } else { testaccess(pathname, O_WRONLY, -1, outbuf); } // Check group access for read/write setids(0, 0); setids(uid_nobody, statbufp->st_gid); memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "Group read\t"); if (statbufp->st_mode & S_IRGRP) { testaccess(pathname, O_RDONLY, 0, outbuf); } else { testaccess(pathname, O_RDONLY, -1, outbuf); } memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "Group write\t"); if (statbufp->st_mode & S_IWGRP) { testaccess(pathname, O_WRONLY, 0, outbuf); } else { testaccess(pathname, O_WRONLY, -1, outbuf); } // Check other access for read/write setids(0, 0); setids(uid_nobody, gid_nobody); memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "Other read\t"); if (statbufp->st_mode & S_IROTH) { testaccess(pathname, O_RDONLY, 0, outbuf); } else { testaccess(pathname, O_RDONLY, -1, outbuf); } memset(outbuf, '\0', sizeof(outbuf)); strcat(outbuf, "Other write\t"); if (statbufp->st_mode & S_IWOTH) { testaccess(pathname, O_WRONLY, 0, outbuf); } else { testaccess(pathname, O_WRONLY, -1, outbuf); } setids(0, 0); if (test == 1) { chown(pathname, ostatbufp->st_uid, ostatbufp->st_gid); } EXIT: return; } /* * Check access. * * This method check a file or recursively scan directories and verify * the file access modes are enforced. * */ void check_access (char *pathname, uid_t uid, gid_t gid) { int count = 0; int i = 0; int rc = 0; int file_select (); char entry[MAXPATHLEN]; struct dirent **entries; struct stat statbuf; // Start with clean buffers memset(&statbuf, '\0', sizeof(statbuf)); // Get file stat info. if ((rc = lstat(pathname, &statbuf)) == -1) { printf("\nERROR: %s. Could not obtain file status. errno = %d\n", pathname, errno); goto EXIT; } // If link, skip it. if (S_ISLNK(statbuf.st_mode)) { printf("Link: skipping %s\n", entry); goto EXIT; } // If not a directory, check it and leave. if (!(S_ISDIR(statbuf.st_mode))) { testall(&statbuf, pathname, uid, gid); goto EXIT; } // If directory, recurse through all subdirectories, checking all files. if ((count = scandir (pathname, &entries, file_select, alphasort)) == -1) { printf("\nERROR: %s. Could not scandir. errno = %d\n", pathname, errno); goto EXIT; } for (i = 0; i < count; i++) { sprintf(entry, "%s/%s", pathname, entries[i]->d_name); // If link, skip it // Else if directory, call check_access() recursively if (entries[i]->d_type == DT_LNK) { printf("Link: skipping %s\n", entry); continue; } else if (entries[i]->d_type == DT_DIR) { check_access(entry, uid, gid); continue; } // Clean the buffer memset(&statbuf, '\0', sizeof(statbuf)); // Get file stat info. if ((rc = lstat(entry, &statbuf)) == -1) { printf("\nERROR: %s. Could not obtain file status. errno = %d\n", pathname, errno); continue; } // The directory entry doesn't always seem to have the // right info. So we check again after the stat(). // // If link, skip it // Else if directory, call check_access() recursively // Else check access if (S_ISLNK(statbuf.st_mode)) { printf("Link: (2) skipping %s\n", entry); continue; } else if (S_ISDIR(statbuf.st_mode)) { check_access(entry, uid, gid); continue; } else { testall(&statbuf, entry, uid, gid); continue; } } EXIT: return; } int main (int argc, char *argv[]) { int i = 0; struct passwd *pw; struct group *gr; if (argc != 4) { printf("usage: checkchown <directory> <testuser> <testgroup>\n"); goto EXIT; } if ((pw = getpwnam(argv[2])) == NULL) { printf("ERROR: invalid username %s\n", argv[2]); goto EXIT; } if ((gr = getgrnam(argv[3])) == NULL) { printf("ERROR: invalid group %s\n", argv[3]); goto EXIT; } for (i = 0; i < 2; i++) { totalpass = 0; totalfail = 0; test = i; printf("Test: %s\n\n", test_description[i]); check_access(argv[1], pw->pw_uid, gr->gr_gid); printf("\nPASS = %d, FAIL = %d\n\n", totalpass, totalfail); } EXIT: return (0); }

> Is this test out of the additions to the LTP test suite for audit? I assume so, although the test itself isn't actually looking for audit records. Someone from IBM would need to say for sure. I just asked for the test that was running so I could try to reproduce it and they sent it to me. -- ljk -- Linux-audit mailing list Linux-audit(a)redhat.com http://www.redhat.com/mailman/listinfo/linux-audit

> However, audit was enabled while running the test. What were the audit rules at the time? -- ljk

> Is this test out of the additions to the LTP test suite for audit? I assume so, although the test itself isn't actually looking for audit records. Someone from IBM would need to say for sure. I just asked for the test that was running so I could try to reproduce it and they sent it to me.