Patches to get Arla running on FreeBSD 8-CURRENT
Locked
123
123
 
|
Dear Arla folk, I've spent the evening getting Arla (checked out of anoncvs) up and running on FreeBSD 8-CURRENT, and have it at least minimally running (/afs mounted, some directory access, read a few files, etc), although arlad core dumped fairly quickly during use. I've not yet attempted to debug that problem though. I've attached a patch that does a few things: First, it does a few minor Arla cleanups that appear to be necessary to build on FreeBSD 8: A few general Arla ifdef fixes, etc, such as testing defined(SunOS) before using the value, likewise on __NetBSD_Version__, OpenBSD, etc. Fix two build dependency issues I ran into regarding building arlalib before dependent tools and an include that broke (at least on FreeBSD). Second, it adds new autoconf and ifdef parts to get Arla building on FreeBSD 8, such as handling VFS changes that appeared in FreeBSD 7.x and 8.x, the priv(9) kernel privilege framework, some include problems I ran into with using /usr/src/sys before /usr/include/sys (which doesn't work for generated files such as vnode_if.h), and things along those lines. Unfortunately, I'm not set up to easily build test on other platforms, and I've also not had a chance to try this on FreeBSD 7 -- my guess is some minor tweaks may be required with respect to both of those, but hopefully these are steps in the right direction and someone with a bit more Arla experience can sort out what I've done into things with keeping and things with fixing :-). I'll investigate the arlad crash tomorrow. Patch also up at: http://www.watson.org/~robert/freebsd/20080216-arla.diff Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-afs To unsubscribe, send any mail to "[hidden email]" |
20080216-arla.diff (33K)
|
|
|
|
Robert, I've been playing with your patches, etc. on 8-CURRENT.
I've been having to tweak up include/config.h a little so I wonder if you missed a few autoconf things... but more important, running "ls /afs" results in: nnpfs: cdev: 0, syscall: 339 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x26c fault code = supervisor read, page not present instruction pointer = 0x20:0xc2a7f1fc stack pointer = 0x28:0xcd412a40 frame pointer = 0x28:0xcd412a5c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 4822 (ls) [thread pid 4822 tid 100176 ] Stopped at nnpfs_getattr_common+0xc: movl 0x26c(%eax),%eax db> bt Tracing pid 4822 tid 100176 td 0xc2783440 nnpfs_getattr_common(c297b110,cd412ab4,c24f1700,c2783440,cd412aa0,...) at nnpfs_getattr_common+0xc nnpfs_getattr(cd412aa0,0,c0b10208,cd412b48,cd412b28,...) at nnpfs_getattr+0x33 VOP_GETATTR_APV(c2a854a0,cd412aa0,c0bd91a0,c297b110,cd412ab4,...) at VOP_GETATTR_APV+0xa5 vn_stat(c297b110,cd412b48,c24f1700,0,c2783440,...) at vn_stat+0x49 kern_stat(c2783440,8113138,0,cd412c18,c0c5d140,...) at kern_stat+0x81 stat(c2783440,cd412cfc,8,cd412d38,c0ba1e60,...) at stat+0x2f syscall(cd412d38) at syscall+0x2b3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (188, FreeBSD ELF32, stat), eip = 0x281a3acb, esp = 0xbfbfe54c, ebp = 0xbfbfe5d8 --- db> If I'me reading this (and nnpfs_vnodeops_common.c) right, this is probably a NULL value for (xn or perhaps vap) in *vap = xn->attr; (line 765) Any thoughts? Or tips on what I can do to help? I don't have gdb ready to go yet---but I probably can this weekend. -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
On 2008-02-22 06:52, wrote:
> Robert, I've been playing with your patches, etc. on 8-CURRENT. [chop] I take it back... boldly adding printf all over says: vp->v_mount is null in the NNPFS_FROM_VNODE macro. Adding if (vp->v_mount) return EDOOFUS does: ls: /afs: Programming error Attempting to umount /afs then panics most likely for the same reason (v_mount is null): Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x26c fault code = supervisor read, page not present instruction pointer = 0x20:0xc2a26dfc stack pointer = 0x28:0xcd49bb00 frame pointer = 0x28:0xcd49bb2c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 5754 (umount) [thread pid 5754 tid 100206 ] Stopped at nnpfs_inactive_common+0xc: movl 0x26c(%eax),%eax db> bt Tracing pid 5754 tid 100206 td 0xc298e220 nnpfs_inactive_common(c2a58440,c298e220,cd49bb54,c0a68485,cd49bb6c,...) at nnpfs_inactive_common+0xc nnpfs_inactive(cd49bb6c,c2a584c8,c2a58440,c2a584c8,cd49bb84,...) at nnpfs_inactive+0x1e VOP_INACTIVE_APV(c2a2b4a0,cd49bb6c,c0afa47c,8fd,c0bd93e0,...) at VOP_INACTIVE_APV+0xa5 vinactive(c2a584c8,0,c0afa47c,86d,c253b880,...) at vinactive+0x91 vrele(c2a58440,c2a2ba70,0,c2a2800b,0,...) at vrele+0x18b nnpfs_free_all_nodes(c2a2b6e0,0,1,c2a58330,c2a2b6e0,...) at nnpfs_free_all_nodes+0xa5 nnpfs_unmount_common(c231329c,8000000,0,c298e220,cd49bc54,...) at nnpfs_unmount_common+0x4e nnpfs_unmount(c231329c,8000000,c298e220,4f0,4da,...) at nnpfs_unmount+0x49 dounmount(c231329c,8000000,c298e220,482,8,...) at dounmount+0x426 unmount(c298e220,cd49bcfc,8,3d90d,c0ba0ed0,...) at unmount+0x2e0 syscall(cd49bd38) at syscall+0x2b3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280c62db, esp = 0xbfbfe4dc, ebp = 0xbfbfe598 --- db> -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
|
On 2008-02-23 09:28, Robert Watson wrote:
> This is a symptom of a failure to insmntque a vnode after creating it, a > new requirement for vnodes in FreeBSD 7.x/8.x; previously, vnodes were > automatically inserted on the mount vnode queue and had their mount pointer > set up during getnewvnode(), but closing certain races motivated this > change. The reason I know this is that I remember adding two calls to > insmntque to nnpfs in my patches, so there are three possibilities: (1) I > missed a call to getnewvnode, (2) the patches I posted didn't include that > change, or (3) the patch didn't apply properly. I'll investigate this > later today once I've given my FOSDEM talk; chances are it's an issue with > the patch. #define HAVE_KERNEL_INSMNTQUE 1 in it. I'll add it to my list of configure-related issues. Recompiling now... -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
On 2008-02-23 04:29, Alec wrote:
> > #define HAVE_KERNEL_INSMNTQUE 1 > > in it. I'll add it to my list of configure-related issues. > Recompiling now... > oldothello% uname -a FreeBSD oldothello.setfilepointer.com 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Wed Feb 20 13:45:22 CST 2008 [hidden email]:/usr/local/obj/usr/src/sys/GENERIC i386 oldothello% ls /afs andrew.cmu.edu grand.central.org openafs.org athena.mit.edu hallf.kth.se research.company.com cern.ch isk.kth.se rose-hulman.edu deek.org it.kth.se setfilepointer.com dementia.org md.kth.se stacken.kth.se dev.mit.edu mech.kth.se e.kth.se mekinok.com oldothello% Neat! Heimdal gets tokens correctly, access to private directories works. I'm running the arla test suite now; I'm up to hardlink3 with no problems. I did notice that, when umounting /afs, I get a lock order reversal: lock order reversal: 1st 0xc231329c vfslock (vfslock) @ /usr/src/sys/kern/vfs_mount.c:1242 2nd 0xc2dabaf8 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:2156 KDB: stack backtrace: db_trace_self_wrapper(c0af1b78,cd4b3b2c,c07a1a6e,c0af3fc2,c2dabaf8,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0af3fc2,c2dabaf8,c0afad01,c0afad01,c0afa47c,...) at kdb_backtrace+0x29 witness_checkorder(c2dabaf8,9,c0afa47c,86c,c0c16cb4,...) at witness_checkorder+0x6de _lockmgr(c2dabaf8,2002,c2dabb28,c0afa47c,86c,...) at _lockmgr+0x43c vop_stdlock(cd4b3bc4,c0afa47c,c07a1338,2002,c2dabaa0,...) at vop_stdlock+0x39 VOP_LOCK1_APV(c0bb0980,cd4b3bc4,84f,cd4b3be4,c2dabb28,...) at VOP_LOCK1_APV+0xa5 _vn_lock(c2dabaa0,2002,c0afa47c,86c,0,...) at _vn_lock+0xf2 vrele(c2dabaa0,0,c0af9de1,4f0,4da,...) at vrele+0x142 dounmount(c231329c,8000000,c356b000,482,8,...) at dounmount+0x372 unmount(c356b000,cd4b3cfc,8,cd4b3d38,c0ba0ed0,...) at unmount+0x2e0 syscall(cd4b3d38) at syscall+0x2b3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280c62db, esp = 0xbfbfe4ec, ebp = 0xbfbfe5a8 --- I see a few other lock order reversals on -CURRENT right now anyway, so I'm not sure if it's cause for concern or even caused by arla. I'm going to move on to trying everything on 7.x. I know there are autoconf/etc. things that need to be fixed. Do you want me to work on that or are you likely to have it done already. I do have a tweaked version of your patch---as I recall it contains just one typo fix---at http://setfilepointer.com/pub/arla/20080218-arla.diff There's also the snapshot of the sources I used: http://setfilepointer.com/pub/arla/arla-20080118.tar.bz2 Assuming I get things working on 7.x, I'll update the port I built later today. :) -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
Sorry for the chatter... arla makes me excited. :)
I've gotten Robert's patches compiled and running under -CURRENT, RELENG_7, and RELENG_6. A FreeBSD port and FreeBSD packages are available at http://setfilepointer.com/pub/afs/FreeBSD/ All of the heavy lifting is done in the big patch at http://setfilepointer.com/pub/arla/20080223-arla.diff This was generated by taking a current snapshot from arla's CVS, applying Robert's original patch, and then adding a handful of fixes of my own. The jist of my changes are autoconf corrections, and one test for NNPFS_DEBUG_PRIV which is undefined on -CURRENT. There's a general autoconf problem running around that a few tests require -Werror to work correctly some can't have -Werror to work correctly. I've run the "-all -fast" tests which don't cause any major issues on RELENG_7 and -CURRENT, but I do seem to be able to panic 6.x with those tests. More testers and feedback would be appreciated. Can anyone from Arla comment on the chances for incorporating these patches into Arla itself? It'd be nice to have these changes in Arla itself prior to submitting the port to FreeBSD. And, can anyone comment on need for the arla port to include 0.43, which works on FreeBSD 5.x? Robert, thanks so much for your time. I owe you (at least) a beer. -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
|
|
|
|
|
|
On 2008-02-25 00:22, Palle Girgensohn wrote:
> I'm delighted to see all this chatter about arla. Any chance we can get a > server working as well? I reckon that should, in principle, be easier, > since it is not as much kernel involvement, right? But the best bet is > OpenAFS, and it is different animal, so the KTH guys are not as deep into > this code? > > /Palle <[hidden email]> > > And hi, Rasmus! :-) > > cells with multiple terabytes of data on FreeBSD only. The OpenAFS param files tend to be out of date but I'm planning on getting on top of that too. There's a port of OpenAFS 1.5.30 I put together available right next to the arla stuff here: http://setfilepointer.com/pub/afs/FreeBSD/ I tend to install the server everywhere I install arla because the vos and bos commands in arla in incomplete compared to their implementation in openafs. I suppose I should nag someone at OpenAFS to include the new param files I made for newer FreeBSDs (not that the new param files are particularly inspired). It'd be great if a few people would comment on this port and then I guess I'll create a PR to get it included in the ports tree at the same time as arla. Sound good? Also, who's writing the FreeBSD wiki? I'm embarrased to tell people to go look at the gentoo documentation for AFS. :) -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
|
|
|
|
|
|
|
First off, I've got a few hours to work on this tomorrow night, so
if we could decide on an approach (and Robert can help me with the heavy lifting) before then, that'd be super. On 2008-02-25 21:19, Robert Watson wrote: > On Sun, 24 Feb 2008, Tomas Olsson wrote: > > >Looks mostly ok, but I do have some questions. > > > >1) appl/fs/fs_local.h: would that break `fs nnpfsdeb all`? > > > >2) cf/try-compile-kernel.m4: why do we need /usr/include? It sounds scary > >given that we may want to compile using random kernel trees. Same goes for > >nnpfs/freebsd/FreeBSD-Makefile. I don't know much about kernel build magic. > > > >3) cf/bsd-vop-unlock.m4: do we need it? I don't care about older versions > >of FreeBSD than 6.x; traditionally we try to support latest stable > >OS-release plus -CURRENT but maybe that's a bit limiting. Perphaps > >nnpfs_vfs_unlock solves part of the problem? > > Just back from FOSDEM, and am 3-4 days behind on e-mail, so will need to > investigate (1) and (2) in a day or two. > > If I've done (1) correctly then, in practice, it shouldn't change things at > all, except that on FreeBSD 7.x and higher, it will use the priv(9) > interface to check for privilege rather than suser(9). While there are > plans for further privilege semantic changes, the interface change so far > is actually a syntactic change -- the policy remains the same, but > information about the check is managed differently, hence the change to the > interface. This is a precursor to more fine-grained privileges in the > kernel. nnpfsdeb almost-all has no effect. I added the check for PRIV_NNPFS_DEBUG in nnpfs_common-bsd.c: +#elif defined(HAVE_KERNEL_PRIV_CHECK) && defined(PRIV_NNPFS_DEBUG) because on my -current box PRIV_NNPFS_DEBUG isn't defined. I thought it might be an OpenBSD-ism. Regardless, I would think it *should* fall back to checking with suser() but apparently it doesn't. I can investigate a bit more, but removing nnpfs_deb.h must have broader impact than we though. Robert, any thoughts about what PRIV_NNPFS_DEBUG should be? > With respect to (2), I need to look at the details, but I believe this has > to do with the fact that nnpfs is relying on generated files that may not > be present in a kernel source tree. The more right fix may be to force > generation of the files (if we can) in the nnpfs build, as we already do > for vnode_if.h, but I'll have to look in more detail. I think this is correct too. Things like machine/endian.h aren't in the kernel tree. I should be able to autoconf this for just FreeBSD if that's how we want to approach this. If you want to have configure generate these headers like vnode_if.h, I'll probably need a few hints, but I'll do what I can. > With respect to (3) -- that has to do with support for 8-CURRENT, not > pre-6.x. It looks like the VFS folks are in the middle of dropping > unnecessary thread arguments from various locking interfaces in VFS, > including lock/unlock/assert/etc (these will now all be curthread > implicitly). I just saw a couple more such changes trickle in today, so > I'll probably have some more patches, sadly. And I added the cf check... I found HAVE_THREE_ARGUMENT_VOP_UNLOCK was undefined in ./nnpfs/bsd/nnpfs_vnodeops-common.c and figured it's smarter to just add the check than assume something about VOP_UNLOCK from the VOP_LOCK macros. -- Alec Kloss [hidden email] IM: [hidden email] PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E "No Bunny!" -- Simon, from Frisky Dingo |
|
|
Loading...
| Powered by Nabble | Edit this page |