UFS2 with SAN
Locked
123
123
UFS2 with SAN
 
|
Hello,
I am looking into setting up a SAN with several web servers that will be clustered. It would be a FC network using Qlogic cards in each of those FreeBSD web servers. It would be about 5+ of those web servers. I want to have the capability to share the same web data across those web servers. I have scorched the entire mailing list and found that there were some work on GFS porting over to FreeBSD. It seems like that it is just all talk and if I am wrong, could you have my head turned over to where I can find out how to enable GFS on those FreeBSD systems. If GFS is out of question, which file system am I recommendeded to attempt to use for this SAN setup? My first thought to use UFS2 and attempt is to allow only one web server to have a write/read access while the reminder would be read only access. That should prevent from lockings that is similar on NFS/NAS. Chris _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
...you format a SAN target with UFS2, nobody else can touch it..think of it
as a SATA drive. How many computers can plug into the same SATA drive? If you wanna share, youre looking at NFS somewhere in there. On 2/10/07, Chris Haulmark <[hidden email]> wrote: > > Hello, > > I am looking into setting up a SAN with several web servers that > will be clustered. It would be a FC network using Qlogic cards > in each of those FreeBSD web servers. It would be about 5+ > of those web servers. > > I want to have the capability to share the same web data across > those web servers. I have scorched the entire mailing list and > found that there were some work on GFS porting over to FreeBSD. > It seems like that it is just all talk and if I am wrong, could > you have my head turned over to where I can find out how to enable > GFS on those FreeBSD systems. > > If GFS is out of question, which file system am I recommendeded > to attempt to use for this SAN setup? > > My first thought to use UFS2 and attempt is to allow only one web > server to have a write/read access while the reminder would be > read only access. That should prevent from lockings that is similar > on NFS/NAS. > > Chris > _______________________________________________ > [hidden email] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "[hidden email]" > [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
In reply to this post by Chris Haulmark
On 02/09/07 19:30, Chris Haulmark wrote:
> Hello, > > I am looking into setting up a SAN with several web servers that > will be clustered. It would be a FC network using Qlogic cards > in each of those FreeBSD web servers. It would be about 5+ > of those web servers. > > I want to have the capability to share the same web data across > those web servers. I have scorched the entire mailing list and > found that there were some work on GFS porting over to FreeBSD. > It seems like that it is just all talk and if I am wrong, could > you have my head turned over to where I can find out how to enable > GFS on those FreeBSD systems. GFS on FreeBSD is indeed dead. Not enough people stepped up to help port it. > If GFS is out of question, which file system am I recommendeded > to attempt to use for this SAN setup? NFS. > My first thought to use UFS2 and attempt is to allow only one web > server to have a write/read access while the reminder would be > read only access. That should prevent from lockings that is similar > on NFS/NAS. This will result it the read/write system seeing the data ok, and the rest getting corrupt data without knowing it, and probably crashing. UFS2 is not cluster aware. You could mount all the hosts read only, and then update the mount point on one to rw, makes changes, then back to ro, then unmount/remount on the other boxes. That's all still a kludge to simulate what NFS will do for you. Why won't NFS work for you? I agree that it would be fantastic to have a clustered file system for FreeBSD, and I've done lot's of hunting and nagging vendors to support it - but it's just not there. Eric _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
RE: UFS2 with SAN
|
|
> -----Original Message----- > From: Eric Anderson [mailto:[hidden email]] > Sent: Saturday, February 10, 2007 12:48 AM > To: Chris Haulmark > Cc: [hidden email] > Subject: Re: UFS2 with SAN > > On 02/09/07 19:30, Chris Haulmark wrote: > > Hello, > > > > I am looking into setting up a SAN with several web servers that > > will be clustered. It would be a FC network using Qlogic cards > > in each of those FreeBSD web servers. It would be about 5+ > > of those web servers. > > > > I want to have the capability to share the same web data across > > those web servers. I have scorched the entire mailing list and > > found that there were some work on GFS porting over to FreeBSD. > > It seems like that it is just all talk and if I am wrong, could > > you have my head turned over to where I can find out how to enable > > GFS on those FreeBSD systems. > > GFS on FreeBSD is indeed dead. Not enough people stepped up to help > port it. I really feared to hear that! > > > If GFS is out of question, which file system am I recommendeded > > to attempt to use for this SAN setup? > > NFS. > > > My first thought to use UFS2 and attempt is to allow only one web > > server to have a write/read access while the reminder would be > > read only access. That should prevent from lockings that is similar > > on NFS/NAS. > > This will result it the read/write system seeing the data ok, and the > rest getting corrupt data without knowing it, and probably crashing. > UFS2 is not cluster aware. You could mount all the hosts read only, > and > then update the mount point on one to rw, makes changes, then back to > ro, then unmount/remount on the other boxes. That's my original idea if I do not have anything else better to go with. > > That's all still a kludge to simulate what NFS will do for you. Why > won't NFS work for you? I have a client who wants to go from NAS to a true SAN solution with full fibre channel network. I would hate to lose the opportunity for this client to continue using FreeBSD as the choice of OS for his web servers. Currently, his set up is using NAS with NFS. He complains of locking files that occurs too often. I had hoped to find more better solution and make this client much more happier with all the FreeBSD support that can be provided. > > I agree that it would be fantastic to have a clustered file system for > FreeBSD, and I've done lot's of hunting and nagging vendors to support > it - but it's just not there. We should get few bandwagons and get in circle. It could be likely that I could provide access for the developers to test and get whatever file system and other necessaries needed to be working. :) Thanks for your reply. > > Eric _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
On 02/10/07 00:54, Chris Haulmark wrote:
> >> -----Original Message----- >> From: Eric Anderson [mailto:[hidden email]] >> Sent: Saturday, February 10, 2007 12:48 AM >> To: Chris Haulmark >> Cc: [hidden email] >> Subject: Re: UFS2 with SAN >> >> On 02/09/07 19:30, Chris Haulmark wrote: >>> Hello, >>> >>> I am looking into setting up a SAN with several web servers that >>> will be clustered. It would be a FC network using Qlogic cards >>> in each of those FreeBSD web servers. It would be about 5+ >>> of those web servers. >>> >>> I want to have the capability to share the same web data across >>> those web servers. I have scorched the entire mailing list and >>> found that there were some work on GFS porting over to FreeBSD. >>> It seems like that it is just all talk and if I am wrong, could >>> you have my head turned over to where I can find out how to enable >>> GFS on those FreeBSD systems. >> GFS on FreeBSD is indeed dead. Not enough people stepped up to help >> port it. > > I really feared to hear that! > >>> If GFS is out of question, which file system am I recommendeded >>> to attempt to use for this SAN setup? >> NFS. >> >>> My first thought to use UFS2 and attempt is to allow only one web >>> server to have a write/read access while the reminder would be >>> read only access. That should prevent from lockings that is similar >>> on NFS/NAS. >> This will result it the read/write system seeing the data ok, and the >> rest getting corrupt data without knowing it, and probably crashing. >> UFS2 is not cluster aware. You could mount all the hosts read only, >> and >> then update the mount point on one to rw, makes changes, then back to >> ro, then unmount/remount on the other boxes. > > That's my original idea if I do not have anything else better to go > with. > >> That's all still a kludge to simulate what NFS will do for you. Why >> won't NFS work for you? > > I have a client who wants to go from NAS to a true SAN solution with > full > fibre channel network. I would hate to lose the opportunity for this > client > to continue using FreeBSD as the choice of OS for his web servers. > Currently, > his set up is using NAS with NFS. He complains of locking files that > occurs > too often. > > I had hoped to find more better solution and make this client much more > happier > with all the FreeBSD support that can be provided. Well, I'm not sure what issues they had, but have had fantastic success with NFS and FreeBSD. FreeBSD with the right hardware and tweaks can make some NetApp boxes look weak. *cough* WAFL *cough* >> I agree that it would be fantastic to have a clustered file system for >> FreeBSD, and I've done lot's of hunting and nagging vendors to support >> it - but it's just not there. > > We should get few bandwagons and get in circle. It could be likely that > I could > provide access for the developers to test and get whatever file system > and other > necessaries needed to be working. :) The problem isn't the environment or hardware, it's developers skilled to do the work. They're all either in NDA's, off writing something else, or just too busy to provide any amount of input. Eric _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
|
In reply to this post by Chris Haulmark
On Sat, 10 Feb 2007 07:54:57 +0100, Chris Haulmark <[hidden email]> wrote:
> > >> -----Original Message----- >> From: Eric Anderson [mailto:[hidden email]] >> Sent: Saturday, February 10, 2007 12:48 AM >> To: Chris Haulmark >> Cc: [hidden email] >> Subject: Re: UFS2 with SAN >> >> On 02/09/07 19:30, Chris Haulmark wrote: >> > Hello, >> > >> > I am looking into setting up a SAN with several web servers that >> > will be clustered. It would be a FC network using Qlogic cards >> > in each of those FreeBSD web servers. It would be about 5+ >> > of those web servers. >> > >> > I want to have the capability to share the same web data across >> > those web servers. I have scorched the entire mailing list and >> > found that there were some work on GFS porting over to FreeBSD. >> > It seems like that it is just all talk and if I am wrong, could >> > you have my head turned over to where I can find out how to enable >> > GFS on those FreeBSD systems. >> >> GFS on FreeBSD is indeed dead. Not enough people stepped up to help >> port it. > > I really feared to hear that! > If it was possible to use OCFS2 then thats a cluster-fs that can handle reasonable traffic. Does it work in FreeBSD? >> >> > If GFS is out of question, which file system am I recommendeded >> > to attempt to use for this SAN setup? >> >> NFS. >> >> > My first thought to use UFS2 and attempt is to allow only one web >> > server to have a write/read access while the reminder would be >> > read only access. That should prevent from lockings that is similar >> > on NFS/NAS. >> >> This will result it the read/write system seeing the data ok, and the >> rest getting corrupt data without knowing it, and probably crashing. >> UFS2 is not cluster aware. You could mount all the hosts read only, >> and >> then update the mount point on one to rw, makes changes, then back to >> ro, then unmount/remount on the other boxes. > > That's my original idea if I do not have anything else better to go > with. > >> >> That's all still a kludge to simulate what NFS will do for you. Why >> won't NFS work for you? > > I have a client who wants to go from NAS to a true SAN solution with > full > fibre channel network. I would hate to lose the opportunity for this > client > to continue using FreeBSD as the choice of OS for his web servers. > Currently, > his set up is using NAS with NFS. He complains of locking files that > occurs > too often. > > I had hoped to find more better solution and make this client much more > happier > with all the FreeBSD support that can be provided. > >> >> I agree that it would be fantastic to have a clustered file system for >> FreeBSD, and I've done lot's of hunting and nagging vendors to support >> it - but it's just not there. > > We should get few bandwagons and get in circle. It could be likely that > I could > provide access for the developers to test and get whatever file system > and other > necessaries needed to be working. :) > > Thanks for your reply. > >> >> Eric > Vasek _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
In reply to this post by Eric Anderson-13
Well, I'm not sure what issues they had, but have had fantastic success
with NFS and FreeBSD. FreeBSD with the right hardware and tweaks can make some NetApp boxes look weak. *cough* WAFL *cough* === Eric, I look forward to your white paper on that. When will we all see it? Did you fix Fbsd's nasty ../..caching issue on the client yet? _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
In reply to this post by Eric Anderson-13
Eric Anderson <[hidden email]> writes:
> Well, I'm not sure what issues they had, but have had fantastic > success with NFS and FreeBSD. FreeBSD with the right hardware and > tweaks can make some NetApp boxes look weak. *cough* WAFL *cough* I'd be very surprised, considering that NetApp filers run FreeBSD. DES -- Dag-Erling Smørgrav - [hidden email] _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
Dag-Erling Smørgrav wrote:
> Eric Anderson <[hidden email]> writes: >> Well, I'm not sure what issues they had, but have had fantastic >> success with NFS and FreeBSD. FreeBSD with the right hardware and >> tweaks can make some NetApp boxes look weak. *cough* WAFL *cough* > > I'd be very surprised, considering that NetApp filers run FreeBSD. not 100 % true.. Until 6 months ago they were based on a proprietary kernel with NetBSD networking code grafted into it. Their newest stuff is based on FreeBSD, but any older model stuff they sell is still the old OS. > > DES _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
In reply to this post by Dag-Erling Smørgrav
Incorrect.
WAFL and the associated raid layer(s), etc, are -not- Freebsd. GX..is different, but is still NOT a pure Fbsd6 playground. http://en.wikipedia.org/wiki/FreeBSD#Derivatives On 2/12/07, Dag-Erling Smørgrav <[hidden email]> wrote: > > Eric Anderson <[hidden email]> writes: > > Well, I'm not sure what issues they had, but have had fantastic > > success with NFS and FreeBSD. FreeBSD with the right hardware and > > tweaks can make some NetApp boxes look weak. *cough* WAFL *cough* > > I'd be very surprised, considering that NetApp filers run FreeBSD. > > DES > -- > Dag-Erling Smørgrav - [hidden email] > _______________________________________________ > [hidden email] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "[hidden email]" > _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
In reply to this post by Eric Anderson-13
--- Eric Anderson <[hidden email]> wrote:
> On 02/10/07 00:54, Chris Haulmark wrote: > > > >> -----Original Message----- > >> From: Eric Anderson [mailto:[hidden email]] > >> Sent: Saturday, February 10, 2007 12:48 AM > >> To: Chris Haulmark > >> Cc: [hidden email] > >> Subject: Re: UFS2 with SAN > >> > >> On 02/09/07 19:30, Chris Haulmark wrote: > >>> Hello, > >>> > >>> I am looking into setting up a SAN with several > web servers that > >>> will be clustered. It would be a FC network > using Qlogic cards > >>> in each of those FreeBSD web servers. It would > be about 5+ > >>> of those web servers. > >>> > >>> I want to have the capability to share the same > web data across > >>> those web servers. I have scorched the entire > mailing list and > >>> found that there were some work on GFS porting > over to FreeBSD. > >>> It seems like that it is just all talk and if I > am wrong, could > >>> you have my head turned over to where I can find > out how to enable > >>> GFS on those FreeBSD systems. > >> GFS on FreeBSD is indeed dead. Not enough people > stepped up to help > >> port it. > > > > I really feared to hear that! > > > >>> If GFS is out of question, which file system am > I recommendeded > >>> to attempt to use for this SAN setup? > >> NFS. > >> > >>> My first thought to use UFS2 and attempt is to > allow only one web > >>> server to have a write/read access while the > reminder would be > >>> read only access. That should prevent from > lockings that is similar > >>> on NFS/NAS. > >> This will result it the read/write system seeing > the data ok, and the > >> rest getting corrupt data without knowing it, and > probably crashing. > >> UFS2 is not cluster aware. You could mount all > the hosts read only, > >> and > >> then update the mount point on one to rw, makes > changes, then back to > >> ro, then unmount/remount on the other boxes. > > > > That's my original idea if I do not have anything > else better to go > > with. > > > >> That's all still a kludge to simulate what NFS > will do for you. Why > >> won't NFS work for you? > > > > I have a client who wants to go from NAS to a true > SAN solution with > > full > > fibre channel network. I would hate to lose the > opportunity for this > > client > > to continue using FreeBSD as the choice of OS for > his web servers. > > Currently, > > his set up is using NAS with NFS. He complains of > locking files that > > occurs > > too often. > > > > I had hoped to find more better solution and make > this client much more > > happier > > with all the FreeBSD support that can be provided. > > > Well, I'm not sure what issues they had, but have > had fantastic success > with NFS and FreeBSD. FreeBSD with the right > hardware and tweaks can > make some NetApp boxes look weak. *cough* WAFL > *cough* > > > >> I agree that it would be fantastic to have a > clustered file system for > >> FreeBSD, and I've done lot's of hunting and > nagging vendors to support > >> it - but it's just not there. > > > > We should get few bandwagons and get in circle. > It could be likely that > > I could > > provide access for the developers to test and get > whatever file system > > and other > > necessaries needed to be working. :) > > > The problem isn't the environment or hardware, it's > developers skilled > to do the work. They're all either in NDA's, off > writing something > else, or just too busy to provide any amount of > input. > > Eric I have a set of servers NFS mounted to a Netapp and after hurs of tuning with netapp's help. (after getting through the idiots adking what FreeBSd was) I got very low performance. I was of course then told by Netapp to switch to Linux for better NFS support. I would love for any help with tuning this further, but I cannot say that FreeBSD with Netapp NFS will be great. Of course, I have not been able to test if indeed Linux would be any better. I will say however that I have a large number of small files which tends to not do well with NFS. Nicole _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
In reply to this post by Dag-Erling Smørgrav
Dag-Erling Smørgrav wrote:
> Eric Anderson <[hidden email]> writes: > > Well, I'm not sure what issues they had, but have had fantastic > > success with NFS and FreeBSD. FreeBSD with the right hardware and > > tweaks can make some NetApp boxes look weak. *cough* WAFL *cough* > > I'd be very surprised, considering that NetApp filers run FreeBSD. AFAIK they run a heavily modified NetBSD, but at least it's some sort of BSD. ;-) In fact I'm very satisfied with NetApp filers and their WAFL system. We have NetApp Filer clusters in production at customers, they exhibit exceptional reliability, even if a disk, a disk shelf, or an entire filer head fails, the cluster keeps running. Not to mention that it submits a support mail automatically and you get the replacement without having to do anything yourself. I'm afraid it is not possible to implement such a redundant NFS cluster with FreeBSD. Not to mention snapshots ... Years before FreeBSD intro- duced snapshots, WAFL supported them, and they work a whole lot better. Creating a snapshot takes just a few seconds, and file system access never freezes during that period. It works so well that you can create hourly snapshots for simple "undelete" usage without affecting the usability and performance of the file system at all. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart Any opinions expressed in this message are personal to the author and may not necessarily reflect the opinions of secnetix GmbH & Co KG in any way. FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd Python is executable pseudocode. Perl is executable line noise. _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
In reply to this post by N. Harrington
For certain workloads, linux is far better tuned for NFS than Fbsd..netapp
as a server or not. On 2/12/07, Nicole Harrington <[hidden email]> wrote: > > --- Eric Anderson <[hidden email]> wrote: > > > On 02/10/07 00:54, Chris Haulmark wrote: > > > > > >> -----Original Message----- > > >> From: Eric Anderson [mailto:[hidden email]] > > >> Sent: Saturday, February 10, 2007 12:48 AM > > >> To: Chris Haulmark > > >> Cc: [hidden email] > > >> Subject: Re: UFS2 with SAN > > >> > > >> On 02/09/07 19:30, Chris Haulmark wrote: > > >>> Hello, > > >>> > > >>> I am looking into setting up a SAN with several > > web servers that > > >>> will be clustered. It would be a FC network > > using Qlogic cards > > >>> in each of those FreeBSD web servers. It would > > be about 5+ > > >>> of those web servers. > > >>> > > >>> I want to have the capability to share the same > > web data across > > >>> those web servers. I have scorched the entire > > mailing list and > > >>> found that there were some work on GFS porting > > over to FreeBSD. > > >>> It seems like that it is just all talk and if I > > am wrong, could > > >>> you have my head turned over to where I can find > > out how to enable > > >>> GFS on those FreeBSD systems. > > >> GFS on FreeBSD is indeed dead. Not enough people > > stepped up to help > > >> port it. > > > > > > I really feared to hear that! > > > > > >>> If GFS is out of question, which file system am > > I recommendeded > > >>> to attempt to use for this SAN setup? > > >> NFS. > > >> > > >>> My first thought to use UFS2 and attempt is to > > allow only one web > > >>> server to have a write/read access while the > > reminder would be > > >>> read only access. That should prevent from > > lockings that is similar > > >>> on NFS/NAS. > > >> This will result it the read/write system seeing > > the data ok, and the > > >> rest getting corrupt data without knowing it, and > > probably crashing. > > >> UFS2 is not cluster aware. You could mount all > > the hosts read only, > > >> and > > >> then update the mount point on one to rw, makes > > changes, then back to > > >> ro, then unmount/remount on the other boxes. > > > > > > That's my original idea if I do not have anything > > else better to go > > > with. > > > > > >> That's all still a kludge to simulate what NFS > > will do for you. Why > > >> won't NFS work for you? > > > > > > I have a client who wants to go from NAS to a true > > SAN solution with > > > full > > > fibre channel network. I would hate to lose the > > opportunity for this > > > client > > > to continue using FreeBSD as the choice of OS for > > his web servers. > > > Currently, > > > his set up is using NAS with NFS. He complains of > > locking files that > > > occurs > > > too often. > > > > > > I had hoped to find more better solution and make > > this client much more > > > happier > > > with all the FreeBSD support that can be provided. > > > > > > Well, I'm not sure what issues they had, but have > > had fantastic success > > with NFS and FreeBSD. FreeBSD with the right > > hardware and tweaks can > > make some NetApp boxes look weak. *cough* WAFL > > *cough* > > > > > > >> I agree that it would be fantastic to have a > > clustered file system for > > >> FreeBSD, and I've done lot's of hunting and > > nagging vendors to support > > >> it - but it's just not there. > > > > > > We should get few bandwagons and get in circle. > > It could be likely that > > > I could > > > provide access for the developers to test and get > > whatever file system > > > and other > > > necessaries needed to be working. :) > > > > > > The problem isn't the environment or hardware, it's > > developers skilled > > to do the work. They're all either in NDA's, off > > writing something > > else, or just too busy to provide any amount of > > input. > > > > Eric > > > I have a set of servers NFS mounted to a Netapp and > after hurs of tuning with netapp's help. (after > getting through the idiots adking what FreeBSd was) > I got very low performance. I was of course then told > by Netapp to switch to Linux for better NFS support. > > I would love for any help with tuning this further, > but I cannot say that FreeBSD with Netapp NFS will be > great. Of course, I have not been able to test if > indeed Linux would be any better. > I will say however that I have a large number of > small files which tends to not do well with NFS. > > > Nicole > > > _______________________________________________ > [hidden email] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "[hidden email]" > [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
RE: UFS2 with SAN
|
|
In reply to this post by N. Harrington
> -----Original Message----- > From: Nicole Harrington [mailto:[hidden email]] > Sent: Tuesday, February 13, 2007 1:56 AM > To: Eric Anderson; Chris Haulmark > Cc: [hidden email] > Subject: Re: UFS2 with SAN > > --- Eric Anderson <[hidden email]> wrote: > > > On 02/10/07 00:54, Chris Haulmark wrote: > > > > > >> -----Original Message----- > > >> From: Eric Anderson [mailto:[hidden email]] > > >> Sent: Saturday, February 10, 2007 12:48 AM > > >> To: Chris Haulmark > > >> Cc: [hidden email] > > >> Subject: Re: UFS2 with SAN > > >> > > >> On 02/09/07 19:30, Chris Haulmark wrote: > > >>> Hello, > > >>> > > >>> I am looking into setting up a SAN with several > > web servers that > > >>> will be clustered. It would be a FC network > > using Qlogic cards > > >>> in each of those FreeBSD web servers. It would > > be about 5+ > > >>> of those web servers. > > >>> > > >>> I want to have the capability to share the same > > web data across > > >>> those web servers. I have scorched the entire > > mailing list and > > >>> found that there were some work on GFS porting > > over to FreeBSD. > > >>> It seems like that it is just all talk and if I > > am wrong, could > > >>> you have my head turned over to where I can find > > out how to enable > > >>> GFS on those FreeBSD systems. > > >> GFS on FreeBSD is indeed dead. Not enough people > > stepped up to help > > >> port it. > > > > > > I really feared to hear that! > > > > > >>> If GFS is out of question, which file system am > > I recommendeded > > >>> to attempt to use for this SAN setup? > > >> NFS. > > >> > > >>> My first thought to use UFS2 and attempt is to > > allow only one web > > >>> server to have a write/read access while the > > reminder would be > > >>> read only access. That should prevent from > > lockings that is similar > > >>> on NFS/NAS. > > >> This will result it the read/write system seeing > > the data ok, and the > > >> rest getting corrupt data without knowing it, and > > probably crashing. > > >> UFS2 is not cluster aware. You could mount all > > the hosts read only, > > >> and > > >> then update the mount point on one to rw, makes > > changes, then back to > > >> ro, then unmount/remount on the other boxes. > > > > > > That's my original idea if I do not have anything > > else better to go > > > with. > > > > > >> That's all still a kludge to simulate what NFS > > will do for you. Why > > >> won't NFS work for you? > > > > > > I have a client who wants to go from NAS to a true > > SAN solution with > > > full > > > fibre channel network. I would hate to lose the > > opportunity for this > > > client > > > to continue using FreeBSD as the choice of OS for > > his web servers. > > > Currently, > > > his set up is using NAS with NFS. He complains of > > locking files that > > > occurs > > > too often. > > > > > > I had hoped to find more better solution and make > > this client much more > > > happier > > > with all the FreeBSD support that can be provided. > > > > > > Well, I'm not sure what issues they had, but have > > had fantastic success > > with NFS and FreeBSD. FreeBSD with the right > > hardware and tweaks can > > make some NetApp boxes look weak. *cough* WAFL > > *cough* > > > > > > >> I agree that it would be fantastic to have a > > clustered file system for > > >> FreeBSD, and I've done lot's of hunting and > > nagging vendors to support > > >> it - but it's just not there. > > > > > > We should get few bandwagons and get in circle. > > It could be likely that > > > I could > > > provide access for the developers to test and get > > whatever file system > > > and other > > > necessaries needed to be working. :) > > > > > > The problem isn't the environment or hardware, it's > > developers skilled > > to do the work. They're all either in NDA's, off > > writing something > > else, or just too busy to provide any amount of > > input. > > > > Eric > > > I have a set of servers NFS mounted to a Netapp and > after hurs of tuning with netapp's help. (after > getting through the idiots adking what FreeBSd was) > I got very low performance. I was of course then told > by Netapp to switch to Linux for better NFS support. That is what I would like to avoid telling my client to do The same thing. "Stay with NFS and tolerate it." I had hoped a SAN solution would be possible for FreeBSD. So far, it appears that it is not possible to share the same file system across several web servers. Chris > > I would love for any help with tuning this further, > but I cannot say that FreeBSD with Netapp NFS will be > great. Of course, I have not been able to test if > indeed Linux would be any better. > I will say however that I have a large number of > small files which tends to not do well with NFS. > > > Nicole > _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
Its pretty much that simple.
You cannot share SAN data..well..you -can- make a LUN appear as a shared NFS or CIFS share on a Netapp, but I havent tried it in a while..but you -can-. But in the normal world..you cannot...you would have to make it available via NFS to other client. Thats the key difference between SAN and NAS. On 2/13/07, Chris Haulmark <[hidden email]> wrote: > > > > > -----Original Message----- > > From: Nicole Harrington [mailto:[hidden email]] > > Sent: Tuesday, February 13, 2007 1:56 AM > > To: Eric Anderson; Chris Haulmark > > Cc: [hidden email] > > Subject: Re: UFS2 with SAN > > > > --- Eric Anderson <[hidden email]> wrote: > > > > > On 02/10/07 00:54, Chris Haulmark wrote: > > > > > > > >> -----Original Message----- > > > >> From: Eric Anderson [mailto:[hidden email]] > > > >> Sent: Saturday, February 10, 2007 12:48 AM > > > >> To: Chris Haulmark > > > >> Cc: [hidden email] > > > >> Subject: Re: UFS2 with SAN > > > >> > > > >> On 02/09/07 19:30, Chris Haulmark wrote: > > > >>> Hello, > > > >>> > > > >>> I am looking into setting up a SAN with several > > > web servers that > > > >>> will be clustered. It would be a FC network > > > using Qlogic cards > > > >>> in each of those FreeBSD web servers. It would > > > be about 5+ > > > >>> of those web servers. > > > >>> > > > >>> I want to have the capability to share the same > > > web data across > > > >>> those web servers. I have scorched the entire > > > mailing list and > > > >>> found that there were some work on GFS porting > > > over to FreeBSD. > > > >>> It seems like that it is just all talk and if I > > > am wrong, could > > > >>> you have my head turned over to where I can find > > > out how to enable > > > >>> GFS on those FreeBSD systems. > > > >> GFS on FreeBSD is indeed dead. Not enough people > > > stepped up to help > > > >> port it. > > > > > > > > I really feared to hear that! > > > > > > > >>> If GFS is out of question, which file system am > > > I recommendeded > > > >>> to attempt to use for this SAN setup? > > > >> NFS. > > > >> > > > >>> My first thought to use UFS2 and attempt is to > > > allow only one web > > > >>> server to have a write/read access while the > > > reminder would be > > > >>> read only access. That should prevent from > > > lockings that is similar > > > >>> on NFS/NAS. > > > >> This will result it the read/write system seeing > > > the data ok, and the > > > >> rest getting corrupt data without knowing it, and > > > probably crashing. > > > >> UFS2 is not cluster aware. You could mount all > > > the hosts read only, > > > >> and > > > >> then update the mount point on one to rw, makes > > > changes, then back to > > > >> ro, then unmount/remount on the other boxes. > > > > > > > > That's my original idea if I do not have anything > > > else better to go > > > > with. > > > > > > > >> That's all still a kludge to simulate what NFS > > > will do for you. Why > > > >> won't NFS work for you? > > > > > > > > I have a client who wants to go from NAS to a true > > > SAN solution with > > > > full > > > > fibre channel network. I would hate to lose the > > > opportunity for this > > > > client > > > > to continue using FreeBSD as the choice of OS for > > > his web servers. > > > > Currently, > > > > his set up is using NAS with NFS. He complains of > > > locking files that > > > > occurs > > > > too often. > > > > > > > > I had hoped to find more better solution and make > > > this client much more > > > > happier > > > > with all the FreeBSD support that can be provided. > > > > > > > > > Well, I'm not sure what issues they had, but have > > > had fantastic success > > > with NFS and FreeBSD. FreeBSD with the right > > > hardware and tweaks can > > > make some NetApp boxes look weak. *cough* WAFL > > > *cough* > > > > > > > > > >> I agree that it would be fantastic to have a > > > clustered file system for > > > >> FreeBSD, and I've done lot's of hunting and > > > nagging vendors to support > > > >> it - but it's just not there. > > > > > > > > We should get few bandwagons and get in circle. > > > It could be likely that > > > > I could > > > > provide access for the developers to test and get > > > whatever file system > > > > and other > > > > necessaries needed to be working. :) > > > > > > > > > The problem isn't the environment or hardware, it's > > > developers skilled > > > to do the work. They're all either in NDA's, off > > > writing something > > > else, or just too busy to provide any amount of > > > input. > > > > > > Eric > > > > > > I have a set of servers NFS mounted to a Netapp and > > after hurs of tuning with netapp's help. (after > > getting through the idiots adking what FreeBSd was) > > I got very low performance. I was of course then told > > by Netapp to switch to Linux for better NFS support. > > That is what I would like to avoid telling my client to do > The same thing. "Stay with NFS and tolerate it." > > I had hoped a SAN solution would be possible for > FreeBSD. So far, it appears that it is not possible to > share the same file system across several web servers. > > Chris > > > > I would love for any help with tuning this further, > > but I cannot say that FreeBSD with Netapp NFS will be > > great. Of course, I have not been able to test if > > indeed Linux would be any better. > > I will say however that I have a large number of > > small files which tends to not do well with NFS. > > > > > > Nicole > > > > _______________________________________________ > [hidden email] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "[hidden email]" > [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
RE: UFS2 with SAN
|
|
> From: Jeff Mohler [mailto:[hidden email]] > Sent: Tuesday, February 13, 2007 10:40 AM > To: Chris Haulmark > Cc: Nicole Harrington; Eric Anderson; [hidden email] > Subject: Re: UFS2 with SAN > > Its pretty much that simple. > > You cannot share SAN data..well..you -can- make a LUN appear as a > shared NFS or CIFS share on a Netapp, but I havent tried it in a > while..but you -can-. included 8 nodes. > > But in the normal world..you cannot...you would have to make it > available via NFS to other client. Thats the key difference between > SAN and NAS. No. SAN and NAS is different based on their access. SAN was dependent on either SCSI or Fibre until iSCSI (IPSAN) was introduced. NAS will always require an IP address. If you were using Fibre and wanted to use NAS, you would have to use iFCP or FCIP protocols. I do not need to explain the limitation of using SCSI with NAS. If your responses on this SAN thread would not be productive, please stay out. I am not interested to hear lectures about what is impossible. I asked about if anyone has tried to use UFS2 with only one node to have write/read only while the rest would be read only. Thank you, Chris > On 2/13/07, Chris Haulmark <[hidden email]> wrote: > > > > -----Original Message----- > > From: Nicole Harrington [mailto:[hidden email]] > > Sent: Tuesday, February 13, 2007 1:56 AM > > To: Eric Anderson; Chris Haulmark > > Cc: [hidden email] > > Subject: Re: UFS2 with SAN > > > > --- Eric Anderson <[hidden email]> wrote: > > > > > On 02/10/07 00:54, Chris Haulmark wrote: > > > > > > > >> -----Original Message----- > > > >> From: Eric Anderson [mailto:[hidden email] ] > > > >> Sent: Saturday, February 10, 2007 12:48 AM > > > >> To: Chris Haulmark > > > >> Cc: [hidden email] > > > >> Subject: Re: UFS2 with SAN > > > >> > > > >> On 02/09/07 19:30, Chris Haulmark wrote: > > > >>> Hello, > > > >>> > > > >>> I am looking into setting up a SAN with several > > > web servers that > > > >>> will be clustered. It would be a FC network > > > using Qlogic cards > > > >>> in each of those FreeBSD web servers. It would > > > be about 5+ > > > >>> of those web servers. > > > >>> > > > >>> I want to have the capability to share the same > > > web data across > > > >>> those web servers. I have scorched the entire > > > mailing list and > > > >>> found that there were some work on GFS porting > > > over to FreeBSD. > > > >>> It seems like that it is just all talk and if I > > > am wrong, could > > > >>> you have my head turned over to where I can find > > > out how to enable > > > >>> GFS on those FreeBSD systems. > > > >> GFS on FreeBSD is indeed dead. Not enough people > > > stepped up to help > > > >> port it. > > > > > > > > I really feared to hear that! > > > > > > > >>> If GFS is out of question, which file system am > > > I recommendeded > > > >>> to attempt to use for this SAN setup? > > > >> NFS. > > > >> > > > >>> My first thought to use UFS2 and attempt is to > > > allow only one web > > > >>> server to have a write/read access while the > > > reminder would be > > > >>> read only access. That should prevent from > > > lockings that is similar > > > >>> on NFS/NAS. > > > >> This will result it the read/write system seeing > > > the data ok, and the > > > >> rest getting corrupt data without knowing it, and > > > probably crashing. > > > >> UFS2 is not cluster aware. You could mount all > > > the hosts read only, > > > >> and > > > >> then update the mount point on one to rw, makes > > > changes, then back to > > > >> ro, then unmount/remount on the other boxes. > > > > > > > > That's my original idea if I do not have anything > > > else better to go > > > > with. > > > > > > > >> That's all still a kludge to simulate what NFS > > > will do for you. Why > > > >> won't NFS work for you? > > > > > > > > I have a client who wants to go from NAS to a true > > > SAN solution with > > > > full > > > > fibre channel network. I would hate to lose the > > > opportunity for this > > > > client > > > > to continue using FreeBSD as the choice of OS for > > > his web servers. > > > > Currently, > > > > his set up is using NAS with NFS. He complains of > > > locking files that > > > > occurs > > > > too often. > > > > > > > > I had hoped to find more better solution and make > > > this client much more > > > > happier > > > > with all the FreeBSD support that can be provided. > > > > > > > > > Well, I'm not sure what issues they had, but have > > > had fantastic success > > > with NFS and FreeBSD. FreeBSD with the right > > > hardware and tweaks can > > > make some NetApp boxes look weak. *cough* WAFL > > > *cough* > > > > > > > > > >> I agree that it would be fantastic to have a > > > clustered file system for > > > >> FreeBSD, and I've done lot's of hunting and > > > nagging vendors to support > > > >> it - but it's just not there. > > > > > > > > We should get few bandwagons and get in circle. > > > It could be likely that > > > > I could > > > > provide access for the developers to test and get > > > whatever file system > > > > and other > > > > necessaries needed to be working. :) > > > > > > > > > The problem isn't the environment or hardware, it's > > > developers skilled > > > to do the work. They're all either in NDA's, off > > > writing something > > > else, or just too busy to provide any amount of > > > input. > > > > > > Eric > > > > > > I have a set of servers NFS mounted to a Netapp and > > after hurs of tuning with netapp's help. (after > > getting through the idiots adking what FreeBSd was) > > I got very low performance. I was of course then told > > by Netapp to switch to Linux for better NFS support. > > That is what I would like to avoid telling my client to do > The same thing. "Stay with NFS and tolerate it." > > I had hoped a SAN solution would be possible for > FreeBSD. So far, it appears that it is not possible to > share the same file system across several web servers. > > Chris > > > > I would love for any help with tuning this further, > > but I cannot say that FreeBSD with Netapp NFS will be > > great. Of course, I have not been able to test if > > indeed Linux would be any better. > > I will say however that I have a large number of > > small files which tends to not do well with NFS. > > > > > > Nicole > > > > _______________________________________________ > [hidden email] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "[hidden email]" _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
Chris Haulmark wrote:
> If your responses on this SAN thread would not be productive, please > stay out. I am not interested to hear lectures about what is impossible. But maybe others are interested to hear that information. > I asked about if anyone has tried to use UFS2 with only one node to > have write/read only while the rest would be read only. And you already got the answer, see Eric's mail. He wrote: | This will result it the read/write system seeing the data ok, | and the rest getting corrupt data without knowing it, and | probably crashing. UFS2 is not cluster aware. Eric is right. For UFS to be cluster-aware, it would need to implement a cache-coherency protocol, so every node knows what data is up-to-date and what data is stale. UFS doesn't do that at all because it was designed as a local-only file system. The only way it would work is to mount _all_ of the nodes read-only. When you need to update data, you must remount the file system read-write on one node _and_ unmount it on all other nodes. I don't think that you want to do that, though. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart Any opinions expressed in this message are personal to the author and may not necessarily reflect the opinions of secnetix GmbH & Co KG in any way. FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." -- Doug Gwyn _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
|
In reply to this post by Chris Haulmark
On 02/13/07 11:00, Chris Haulmark wrote:
>> From: Jeff Mohler [mailto:[hidden email]] >> Sent: Tuesday, February 13, 2007 10:40 AM >> To: Chris Haulmark >> Cc: Nicole Harrington; Eric Anderson; [hidden email] >> Subject: Re: UFS2 with SAN >> >> Its pretty much that simple. >> >> You cannot share SAN data..well..you -can- make a LUN appear as a >> shared NFS or CIFS share on a Netapp, but I havent tried it in a >> while..but you -can-. > > I set up a Linux cluster with GFS on a single SAN file system. It > included 8 nodes. Yep, and there's a handful of other linux supported cluster file systems out there, some good, some not so good. >> But in the normal world..you cannot...you would have to make it >> available via NFS to other client.  Thats the key difference between >> SAN and NAS. > > No. SAN and NAS is different based on their access. SAN was dependent > on either SCSI or Fibre until iSCSI (IPSAN) was introduced. NAS will > always require an IP address. If you were using Fibre and wanted to > use NAS, you would have to use iFCP or FCIP protocols. I do not need to > explain the limitation of using SCSI with NAS. > > If your responses on this SAN thread would not be productive, please > stay out. I am not interested to hear lectures about what is impossible. > > I asked about if anyone has tried to use UFS2 with only one node to > have write/read only while the rest would be read only. Right, and the answer comes down to - UFS is not cluster aware, and so it doesn't know to talk to other nodes to tell them to reload data from disk, nor does it know if two nodes are writing to the same file at the same time (think: appending to a log file from many nodes), locking, etc. Sharing the storage works (I have lots of FreeBSD boxen using the same fiber backend, with disks visible to all the nodes). You can mount the file system RO on many systems just fine too. *BUT* as soon as you mount an fs rw on another box, it's going to start making changes to the file system, and those changes will not be see on the other nodes. I have indeed tried it. You might look at tdfs (fuse module from Ivan Voras I believe). It's probably horribly beta (not knocking it Ivan - I'm sure it's an amazing start), so lots of testing would need to be done. It's also most likely not high performance. Do you need one node to be doing *lots* of writes to the file system, or just nightly updates, etc? Eric >> On 2/13/07, Chris Haulmark <[hidden email]> wrote: >> >> >>> -----Original Message----- >>> From: Nicole Harrington [mailto:[hidden email]] >>> Sent: Tuesday, February 13, 2007 1:56 AM >>> To: Eric Anderson; Chris Haulmark >>> Cc: [hidden email] >>> Subject: Re: UFS2 with SAN >>> >>> --- Eric Anderson <[hidden email]> wrote: >>> >>>> On 02/10/07 00:54, Chris Haulmark wrote: >>>>>> -----Original Message----- >>>>>> From: Eric Anderson [mailto:[hidden email] ] >>>>>> Sent: Saturday, February 10, 2007 12:48 AM >>>>>> To: Chris Haulmark >>>>>> Cc: [hidden email] >>>>>> Subject: Re: UFS2 with SAN >>>>>> >>>>>> On 02/09/07 19:30, Chris Haulmark wrote: >>>>>>> Hello, >>>>>>> >>>>>>> I am looking into setting up a SAN with several >>>> web servers that >>>>>>> will be clustered.  It would be a FC network >>>> using Qlogic cards >>>>>>> in each of those FreeBSD web servers.  It would >>>> be about 5+ >>>>>>> of those web servers. >>>>>>> >>>>>>> I want to have the capability to share the same >>>> web data across >>>>>>> those web servers.  I have scorched the entire >>>> mailing list and >>>>>>> found that there were some work on GFS porting >>>> over to FreeBSD. >>>>>>> It seems like that it is just all talk and if I >>>> am wrong, could >>>>>>> you have my head turned over to where I can find >>>> out how to enable >>>>>>> GFS on those FreeBSD systems. >>>>>> GFS on FreeBSD is indeed dead.  Not enough people >>>> stepped up to help >>>>>> port it. >>>>> I really feared to hear that! >>>>> >>>>>>> If GFS is out of question, which file system am >>>> I recommendeded >>>>>>> to attempt to use for this SAN setup? >>>>>> NFS. >>>>>> >>>>>>> My first thought to use UFS2 and attempt is to >>>> allow only one web >>>>>>> server to have a write/read access while the >>>> reminder would be >>>>>>> read only access. That should prevent from >>>> lockings that is similar >>>>>>> on NFS/NAS. >>>>>> This will result it the read/write system seeing >>>> the data ok, and the >>>>>> rest getting corrupt data without knowing it, and >>>> probably crashing. >>>>>> UFS2 is not cluster aware.  You could mount all >>>> the hosts read only, >>>>>> and >>>>>> then update the mount point on one to rw, makes >>>> changes, then back to >>>>>> ro, then unmount/remount on the other boxes. >>>>> That's my original idea if I do not have anything >>>> else better to go >>>>> with. >>>>> >>>>>> That's all still a kludge to simulate what NFS >>>> will do for you.  Why >>>>>> won't NFS work for you? >>>>> I have a client who wants to go from NAS to a true >>>> SAN solution with >>>>> full >>>>> fibre channel network.  I would hate to lose the >>>> opportunity for this >>>>> client >>>>> to continue using FreeBSD as the choice of OS for >>>> his web servers. >>>>> Currently, >>>>> his set up is using NAS with NFS.  He complains of >>>> locking files that >>>>> occurs >>>>> too often. >>>>> >>>>> I had hoped to find more better solution and make >>>> this client much more >>>>> happier >>>>> with all the FreeBSD support that can be provided. >>>> >>>> Well, I'm not sure what issues they had, but have >>>> had fantastic success >>>> with NFS and FreeBSD.  FreeBSD with the right >>>> hardware and tweaks can >>>> make some NetApp boxes look weak. *cough* WAFL >>>> *cough* >>>> >>>> >>>>>> I agree that it would be fantastic to have a >>>> clustered file system for >>>>>> FreeBSD, and I've done lot's of hunting and >>>> nagging vendors to support >>>>>> it - but it's just not there. >>>>> We should get few bandwagons and get in circle. >>>> It could be likely that >>>>> I could >>>>> provide access for the developers to test and get >>>> whatever file system >>>>> and other >>>>> necessaries needed to be working. :) >>>> >>>> The problem isn't the environment or hardware, it's >>>> developers skilled >>>> to do the work.  They're all either in NDA's, off >>>> writing something >>>> else, or just too busy to provide any amount of >>>> input. >>>> >>>> Eric >>> >>> I have a set of servers NFS mounted to a Netapp and >>> after hurs of tuning with netapp's help. (after >>> getting through the idiots adking what FreeBSd was) >>>   I got very low performance. I was of course then told >>>   by Netapp to switch to Linux for better NFS support. >> That is what I would like to avoid telling my client to do >> The same thing. "Stay with NFS and tolerate it." >> >> I had hoped a SAN solution would be possible for >> FreeBSD.  So far, it appears that it is not possible to >> share the same file system across several web servers. >> >> Chris >>>   I would love for any help with tuning this further, >>> but I cannot say that FreeBSD with Netapp NFS will be >>> great. Of course, I have not been able to test if >>> indeed Linux would be any better. >>>   I will say however that I have a large number of >>> small files which tends to not do well with NFS. >>> >>> >>>   Nicole >>> >> _______________________________________________ >> [hidden email] mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "[hidden email]" > _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
RE: UFS2 with SAN
|
|
> -----Original Message----- > From: Eric Anderson [mailto:[hidden email]] > Sent: Tuesday, February 13, 2007 11:19 AM > To: Chris Haulmark > Cc: [hidden email] > Subject: Re: UFS2 with SAN > > On 02/13/07 11:00, Chris Haulmark wrote: > >> From: Jeff Mohler [mailto:[hidden email]] > >> Sent: Tuesday, February 13, 2007 10:40 AM > >> To: Chris Haulmark > >> Cc: Nicole Harrington; Eric Anderson; [hidden email] > >> Subject: Re: UFS2 with SAN > >> > >> Its pretty much that simple. > >> > >> You cannot share SAN data..well..you -can- make a LUN appear as a > >> shared NFS or CIFS share on a Netapp, but I havent tried it in a > >> while..but you -can-. > > > > I set up a Linux cluster with GFS on a single SAN file system. It > > included 8 nodes. > > > Yep, and there's a handful of other linux supported cluster file > systems > out there, some good, some not so good. > > > >> But in the normal world..you cannot...you would have to make it > >> available via NFS to other client.Ã' Ã' Thats the key difference > between > >> SAN and NAS. > > > > No. SAN and NAS is different based on their access. SAN was > dependent > > on either SCSI or Fibre until iSCSI (IPSAN) was introduced. NAS will > > always require an IP address. If you were using Fibre and wanted to > > use NAS, you would have to use iFCP or FCIP protocols. I do not need > to > > explain the limitation of using SCSI with NAS. > > > > If your responses on this SAN thread would not be productive, please > > stay out. I am not interested to hear lectures about what is > impossible. > > > > I asked about if anyone has tried to use UFS2 with only one node to > > have write/read only while the rest would be read only. > > > Right, and the answer comes down to - UFS is not cluster aware, and so > it doesn't know to talk to other nodes to tell them to reload data from > disk, nor does it know if two nodes are writing to the same file at the > same time (think: appending to a log file from many nodes), locking, > etc. > I figured that if UFS was my only solution, I would write up a script that would handle the talking to the other nodes to umount/mount the file system on demand. > Sharing the storage works (I have lots of FreeBSD boxen using the same > fiber backend, with disks visible to all the nodes). You can mount the > file system RO on many systems just fine too. *BUT* as soon as you > mount an fs rw on another box, it's going to start making changes to > the > file system, and those changes will not be see on the other nodes. I > have indeed tried it. > > You might look at tdfs (fuse module from Ivan Voras I believe). It's > probably horribly beta (not knocking it Ivan - I'm sure it's an amazing > start), so lots of testing would need to be done. It's also most > likely > not high performance. Thanks for the tip! > > Do you need one node to be doing *lots* of writes to the file system, > or > just nightly updates, etc? Just updates from the staging production. I doubt it would happen as often as the nightly updates. The client mentioned having a staging environment so I am sure that the site would be tested before doing a final update. It feels more likely to be weekly updates. Oliver, thanks too. Chris > > > Eric > > > >> On 2/13/07, Chris Haulmark <[hidden email]> wrote: > >> > >> > >>> -----Original Message----- > >>> From: Nicole Harrington [mailto:[hidden email]] > >>> Sent: Tuesday, February 13, 2007 1:56 AM > >>> To: Eric Anderson; Chris Haulmark > >>> Cc: [hidden email] > >>> Subject: Re: UFS2 with SAN > >>> > >>> --- Eric Anderson <[hidden email]> wrote: > >>> > >>>> On 02/10/07 00:54, Chris Haulmark wrote: > >>>>>> -----Original Message----- > >>>>>> From: Eric Anderson [mailto:[hidden email] ] > >>>>>> Sent: Saturday, February 10, 2007 12:48 AM > >>>>>> To: Chris Haulmark > >>>>>> Cc: [hidden email] > >>>>>> Subject: Re: UFS2 with SAN > >>>>>> > >>>>>> On 02/09/07 19:30, Chris Haulmark wrote: > >>>>>>> Hello, > >>>>>>> > >>>>>>> I am looking into setting up a SAN with several > >>>> web servers that > >>>>>>> will be clustered.Ã' Ã' It would be a FC network > >>>> using Qlogic cards > >>>>>>> in each of those FreeBSD web servers.Ã' Ã' It would > >>>> be about 5+ > >>>>>>> of those web servers. > >>>>>>> > >>>>>>> I want to have the capability to share the same > >>>> web data across > >>>>>>> those web servers.Ã' Ã' I have scorched the entire > >>>> mailing list and > >>>>>>> found that there were some work on GFS porting > >>>> over to FreeBSD. > >>>>>>> It seems like that it is just all talk and if I > >>>> am wrong, could > >>>>>>> you have my head turned over to where I can find > >>>> out how to enable > >>>>>>> GFS on those FreeBSD systems. > >>>>>> GFS on FreeBSD is indeed dead.Ã' Ã' Not enough people > >>>> stepped up to help > >>>>>> port it. > >>>>> I really feared to hear that! > >>>>> > >>>>>>> If GFS is out of question, which file system am > >>>> I recommendeded > >>>>>>> to attempt to use for this SAN setup? > >>>>>> NFS. > >>>>>> > >>>>>>> My first thought to use UFS2 and attempt is to > >>>> allow only one web > >>>>>>> server to have a write/read access while the > >>>> reminder would be > >>>>>>> read only access. That should prevent from > >>>> lockings that is similar > >>>>>>> on NFS/NAS. > >>>>>> This will result it the read/write system seeing > >>>> the data ok, and the > >>>>>> rest getting corrupt data without knowing it, and > >>>> probably crashing. > >>>>>> UFS2 is not cluster aware.Ã' Ã' You could mount all > >>>> the hosts read only, > >>>>>> and > >>>>>> then update the mount point on one to rw, makes > >>>> changes, then back to > >>>>>> ro, then unmount/remount on the other boxes. > >>>>> That's my original idea if I do not have anything > >>>> else better to go > >>>>> with. > >>>>> > >>>>>> That's all still a kludge to simulate what NFS > >>>> will do for you.Ã' Ã' Why > >>>>>> won't NFS work for you? > >>>>> I have a client who wants to go from NAS to a true > >>>> SAN solution with > >>>>> full > >>>>> fibre channel network.Ã' Ã' I would hate to lose the > >>>> opportunity for this > >>>>> client > >>>>> to continue using FreeBSD as the choice of OS for > >>>> his web servers. > >>>>> Currently, > >>>>> his set up is using NAS with NFS.Ã' Ã' He complains of > >>>> locking files that > >>>>> occurs > >>>>> too often. > >>>>> > >>>>> I had hoped to find more better solution and make > >>>> this client much more > >>>>> happier > >>>>> with all the FreeBSD support that can be provided. > >>>> > >>>> Well, I'm not sure what issues they had, but have > >>>> had fantastic success > >>>> with NFS and FreeBSD.Ã' Ã' FreeBSD with the right > >>>> hardware and tweaks can > >>>> make some NetApp boxes look weak. *cough* WAFL > >>>> *cough* > >>>> > >>>> > >>>>>> I agree that it would be fantastic to have a > >>>> clustered file system for > >>>>>> FreeBSD, and I've done lot's of hunting and > >>>> nagging vendors to support > >>>>>> it - but it's just not there. > >>>>> We should get few bandwagons and get in circle. > >>>> It could be likely that > >>>>> I could > >>>>> provide access for the developers to test and get > >>>> whatever file system > >>>>> and other > >>>>> necessaries needed to be working. :) > >>>> > >>>> The problem isn't the environment or hardware, it's > >>>> developers skilled > >>>> to do the work.Ã' Ã' They're all either in NDA's, off > >>>> writing something > >>>> else, or just too busy to provide any amount of > >>>> input. > >>>> > >>>> Eric > >>> > >>> I have a set of servers NFS mounted to a Netapp and > >>> after hurs of tuning with netapp's help. (after > >>> getting through the idiots adking what FreeBSd was) > >>> Ã' Ã' I got very low performance. I was of course then told > >>> Ã' Ã' by Netapp to switch to Linux for better NFS support. > >> That is what I would like to avoid telling my client to do > >> The same thing. "Stay with NFS and tolerate it." > >> > >> I had hoped a SAN solution would be possible for > >> FreeBSD.Ã' Ã' So far, it appears that it is not possible to > >> share the same file system across several web servers. > >> > >> Chris > >>> Ã' Ã' I would love for any help with tuning this further, > >>> but I cannot say that FreeBSD with Netapp NFS will be > >>> great. Of course, I have not been able to test if > >>> indeed Linux would be any better. > >>> Ã' Ã' I will say however that I have a large number of > >>> small files which tends to not do well with NFS. > >>> > >>> > >>> Ã' Ã' Nicole > >>> > >> _______________________________________________ > >> [hidden email] mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs- > [hidden email]" > > _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
Re: UFS2 with SAN
|
In reply to this post by Jeff Mohler
> On Tue, 13 Feb 2007, Jeff Mohler wrote: > For certain workloads, linux is far better tuned for NFS than Fbsd..netapp > as a server or not. > On 2/12/07, Nicole Harrington <[hidden email]> wrote: >> >> >> I have a set of servers NFS mounted to a Netapp and >> after hurs of tuning with netapp's help. (after >> getting through the idiots adking what FreeBSd was) >> I got very low performance. I was of course then told >> by Netapp to switch to Linux for better NFS support. >> >> I would love for any help with tuning this further, >> but I cannot say that FreeBSD with Netapp NFS will be >> great. Of course, I have not been able to test if >> indeed Linux would be any better. >> I will say however that I have a large number of >> small files which tends to not do well with NFS. >> >> >> Nicole I don't know if any of this will help, but... My experience is that abismal NFS perf is usually caused by one of: - Transport/Network problems: If the network interconnect introduces delays or drops packets, you've got to fix it. This includes network device drivers that aren't up to it. If you use NFS over UDP, one thing to check for is the # of "fragments dropped due to timeout" in the IP section of "netstat -s". If there are any of those happenning, you need to fix your network, or at least switch to TCP or reduce your read/write data size. (Make rsize, wsize as large as you can, given what your network can handle.) - Byte range locking (the NLM protocol, not really NFS). It's a dog in mho. All you can do is avoid apps that do byte range locking. (NFSv4 has good byte range locking support, but the client in FreeBSD isn't useful for production work yet, from what I've seem.) Beyond that, I'm not sure there is much you can do, rick _______________________________________________ [hidden email] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "[hidden email]" |
«
Return to freebsd-fs
|
1 view|%1 views
Loading...
| Powered by Nabble | Edit this page |