Replacing dead drives in ZRAID2

[Simon-14]

Hello,

I decided to give ZFS ZRAID2 a shot after getting fed up with some legacy
hardware RAID cards that don't properly perform, or at all, patrol-reads +
consistency checking. So...

I can't seem to figure out the proper way to replace a dead drive in a running
system with SCSI+SES enclosure. I tried:

zpool detach zroot baddrive
camcontrol stop baddrive

At this point when I pull the drive out, I get bus reset errors, etc... I will go
into details after someone confirms whether the above 2 steps suffice
to pull a dead/malfunctioning drive out from SES enclosure or am I missing
something?

I tried to simulate drive failure by pulling 2 out of 3 ZRAID2 drives. The first
drive pull went smoothly. System noticed drive disconnected and stopped
using it. The 2nd drive pull resulted in bunch of errors and system froze
completely. I can't pull 2 drives out in ZRAID2 system and expect the
machine to continue to function? what am I missing :\

PS: I couldn't find dedicated FreeBSD ZFS email list, is there one?

Thank you very much!
Simon


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Freddie Cash-8]

On Tue, May 1, 2012 at 1:57 PM, Simon <[hidden email]> wrote:
> I decided to give ZFS ZRAID2 a shot after getting fed up with some legacy
> hardware RAID cards that don't properly perform, or at all, patrol-reads +
> consistency checking. So...
>
> I can't seem to figure out the proper way to replace a dead drive in a running
> system with SCSI+SES enclosure. I tried:
>
> zpool detach zroot baddrive
> camcontrol stop baddrive

You can't detach drives from raidz vdevs.  The correct process is:

zpool offline zroot <baddrive>
<pull drive from system>
<insert new drive>
<do any partitioning, labelling, etc to suit your env>
zpool replace zroot <baddrive> <newdrive>

"zpool detach" is only used for mirror vdevs.

--
Freddie Cash
[hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

Sorry, I meant to say zpool offline.

After I take the drive out marked as offline, and put it back in, the system spits
the following:

ahd0: someone reset channel A
ahd0: WARNING no command for scb 242 (cmdcmplt)
QOUTPOS = 283

>>>>>>>>>>>>>>>>>>>Dumpt Card State Begins>>>>>>>>>>>>
ahd0: dumping card state.... followed by a lard amount of data.

It then freezes and won't executed any new commands.

beta_srv# uname -a
FreeBSD beta_srv 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:15:25
UTC 2012     [hidden email]:/usr/obj/usr/src/sys/GENERIC  i386

beta_srv# dmesg | grep ses
ses0 at ahd0 bus 0 scbus0 target 6 lun 0
ses0: <SUPER GEM318 0> Fixed Processor SCSI-2 device
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device

Thanks,
Simon

On Tue, 1 May 2012 14:26:35 -0700, Freddie Cash wrote:

>On Tue, May 1, 2012 at 1:57 PM, Simon <[hidden email]> wrote:
>> I decided to give ZFS ZRAID2 a shot after getting fed up with some legacy
>> hardware RAID cards that don't properly perform, or at all, patrol-reads +
>> consistency checking. So...
>>
>> I can't seem to figure out the proper way to replace a dead drive in a running
>> system with SCSI+SES enclosure. I tried:
>>
>> zpool detach zroot baddrive
>> camcontrol stop baddrive

>You can't detach drives from raidz vdevs.  The correct process is:

>zpool offline zroot <baddrive>
><pull drive from system>
><insert new drive>
><do any partitioning, labelling, etc to suit your env>
>zpool replace zroot <baddrive> <newdrive>

>"zpool detach" is only used for mirror vdevs.

>--
>Freddie Cash
>[hidden email]



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Rich-3]

What card is this?

- Rich

On Tue, May 1, 2012 at 6:15 PM, Simon <[hidden email]> wrote:

>
> Sorry, I meant to say zpool offline.
>
> After I take the drive out marked as offline, and put it back in, the system spits
> the following:
>
> ahd0: someone reset channel A
> ahd0: WARNING no command for scb 242 (cmdcmplt)
> QOUTPOS = 283
>
>>>>>>>>>>>>>>>>>>>>Dumpt Card State Begins>>>>>>>>>>>>
> ahd0: dumping card state.... followed by a lard amount of data.
>
> It then freezes and won't executed any new commands.
>
> beta_srv# uname -a
> FreeBSD beta_srv 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:15:25
> UTC 2012     [hidden email]:/usr/obj/usr/src/sys/GENERIC  i386
>
> beta_srv# dmesg | grep ses
> ses0 at ahd0 bus 0 scbus0 target 6 lun 0
> ses0: <SUPER GEM318 0> Fixed Processor SCSI-2 device
> ses0: 3.300MB/s transfers
> ses0: SAF-TE Compliant Device
>
> Thanks,
> Simon
>
> On Tue, 1 May 2012 14:26:35 -0700, Freddie Cash wrote:
>
>>On Tue, May 1, 2012 at 1:57 PM, Simon <[hidden email]> wrote:
>>> I decided to give ZFS ZRAID2 a shot after getting fed up with some legacy
>>> hardware RAID cards that don't properly perform, or at all, patrol-reads +
>>> consistency checking. So...
>>>
>>> I can't seem to figure out the proper way to replace a dead drive in a running
>>> system with SCSI+SES enclosure. I tried:
>>>
>>> zpool detach zroot baddrive
>>> camcontrol stop baddrive
>
>>You can't detach drives from raidz vdevs.  The correct process is:
>
>>zpool offline zroot <baddrive>
>><pull drive from system>
>><insert new drive>
>><do any partitioning, labelling, etc to suit your env>
>>zpool replace zroot <baddrive> <newdrive>
>
>>"zpool detach" is only used for mirror vdevs.
>
>>--
>>Freddie Cash
>>[hidden email]
>
>
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter>
on board of Super X5DPR-8G2+ motherboard.

Also would like to add that when I plug a drive into this running machine, it
prints the following, but does not come up under /dev unless I issue reset
using camcontrol.

ahd0: Someone reset channel A
(da0:ahd0:0:0:0): WRITE(10). CDB: 2a 0 1 10 23 f0 0 0 80 0
(da0:ahd0:0:0:0): CAM status: SCSI Status Error
(da0:ahd0:0:0:0): SCSI status: Check Condition
(da0:ahd0:0:0:0): SCSI sense: UNIT ATTENTION asc:29,1 (Power on occurred)
(da0:ahd0:0:0:0): Field Replaceable Unit: 1

-Simon

On Tue, 1 May 2012 20:57:52 -0400, Rich wrote:

>What card is this?

>- Rich

>On Tue, May 1, 2012 at 6:15 PM, Simon <[hidden email]> wrote:
>>
>> Sorry, I meant to say zpool offline.
>>
>> After I take the drive out marked as offline, and put it back in, the system spits
>> the following:
>>
>> ahd0: someone reset channel A
>> ahd0: WARNING no command for scb 242 (cmdcmplt)
>> QOUTPOS = 283
>>
>>>>>>>>>>>>>>>>>>>>>Dumpt Card State Begins>>>>>>>>>>>>
>> ahd0: dumping card state.... followed by a lard amount of data.
>>
>> It then freezes and won't executed any new commands.
>>
>> beta_srv# uname -a
>> FreeBSD beta_srv 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:15:25
>> UTC 2012     [hidden email]:/usr/obj/usr/src/sys/GENERIC  i386
>>
>> beta_srv# dmesg | grep ses
>> ses0 at ahd0 bus 0 scbus0 target 6 lun 0
>> ses0: <SUPER GEM318 0> Fixed Processor SCSI-2 device
>> ses0: 3.300MB/s transfers
>> ses0: SAF-TE Compliant Device
>>
>> Thanks,
>> Simon
>>
>> On Tue, 1 May 2012 14:26:35 -0700, Freddie Cash wrote:
>>
>>>On Tue, May 1, 2012 at 1:57 PM, Simon <[hidden email]> wrote:
>>>> I decided to give ZFS ZRAID2 a shot after getting fed up with some legacy
>>>> hardware RAID cards that don't properly perform, or at all, patrol-reads +
>>>> consistency checking. So...
>>>>
>>>> I can't seem to figure out the proper way to replace a dead drive in a running
>>>> system with SCSI+SES enclosure. I tried:
>>>>
>>>> zpool detach zroot baddrive
>>>> camcontrol stop baddrive
>>
>>>You can't detach drives from raidz vdevs.  The correct process is:
>>
>>>zpool offline zroot <baddrive>
>>><pull drive from system>
>>><insert new drive>
>>><do any partitioning, labelling, etc to suit your env>
>>>zpool replace zroot <baddrive> <newdrive>
>>
>>>"zpool detach" is only used for mirror vdevs.
>>
>>>--
>>>Freddie Cash
>>>[hidden email]
>>
>>
>>
>> _______________________________________________
>> [hidden email] mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

Am I the only one using this SCSI card and ZFS? does anyone use any type of
Adaptec U320 SCSI card? I hardly doubt the behaviour I'm experiencing is strictly
related to ahd driver.

When I first read about ZFS I've built up high hopes, but now they are slowly
fading away.

-Simon

On Tue, 01 May 2012 21:31:20 -0400, Simon wrote:


>ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter>
>on board of Super X5DPR-8G2+ motherboard.

>Also would like to add that when I plug a drive into this running machine, it
>prints the following, but does not come up under /dev unless I issue reset
>using camcontrol.

>ahd0: Someone reset channel A
>(da0:ahd0:0:0:0): WRITE(10). CDB: 2a 0 1 10 23 f0 0 0 80 0
>(da0:ahd0:0:0:0): CAM status: SCSI Status Error
>(da0:ahd0:0:0:0): SCSI status: Check Condition
>(da0:ahd0:0:0:0): SCSI sense: UNIT ATTENTION asc:29,1 (Power on occurred)
>(da0:ahd0:0:0:0): Field Replaceable Unit: 1

>-Simon

>On Tue, 1 May 2012 20:57:52 -0400, Rich wrote:

>>What card is this?

>>- Rich

>>On Tue, May 1, 2012 at 6:15 PM, Simon <[hidden email]> wrote:
>>>
>>> Sorry, I meant to say zpool offline.
>>>
>>> After I take the drive out marked as offline, and put it back in, the system spits
>>> the following:
>>>
>>> ahd0: someone reset channel A
>>> ahd0: WARNING no command for scb 242 (cmdcmplt)
>>> QOUTPOS = 283
>>>
>>>>>>>>>>>>>>>>>>>>>>Dumpt Card State Begins>>>>>>>>>>>>
>>> ahd0: dumping card state.... followed by a lard amount of data.
>>>
>>> It then freezes and won't executed any new commands.
>>>
>>> beta_srv# uname -a
>>> FreeBSD beta_srv 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:15:25
>>> UTC 2012     [hidden email]:/usr/obj/usr/src/sys/GENERIC  i386
>>>
>>> beta_srv# dmesg | grep ses
>>> ses0 at ahd0 bus 0 scbus0 target 6 lun 0
>>> ses0: <SUPER GEM318 0> Fixed Processor SCSI-2 device
>>> ses0: 3.300MB/s transfers
>>> ses0: SAF-TE Compliant Device
>>>
>>> Thanks,
>>> Simon
>>>
>>> On Tue, 1 May 2012 14:26:35 -0700, Freddie Cash wrote:
>>>
>>>>On Tue, May 1, 2012 at 1:57 PM, Simon <[hidden email]> wrote:
>>>>> I decided to give ZFS ZRAID2 a shot after getting fed up with some legacy
>>>>> hardware RAID cards that don't properly perform, or at all, patrol-reads +
>>>>> consistency checking. So...
>>>>>
>>>>> I can't seem to figure out the proper way to replace a dead drive in a running
>>>>> system with SCSI+SES enclosure. I tried:
>>>>>
>>>>> zpool detach zroot baddrive
>>>>> camcontrol stop baddrive
>>>
>>>>You can't detach drives from raidz vdevs.  The correct process is:
>>>
>>>>zpool offline zroot <baddrive>
>>>><pull drive from system>
>>>><insert new drive>
>>>><do any partitioning, labelling, etc to suit your env>
>>>>zpool replace zroot <baddrive> <newdrive>
>>>
>>>>"zpool detach" is only used for mirror vdevs.
>>>
>>>>--
>>>>Freddie Cash
>>>>[hidden email]
>>>
>>>
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>> To unsubscribe, send any mail to "[hidden email]"

>_______________________________________________
>[hidden email] mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>To unsubscribe, send any mail to "[hidden email]"



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Freddie Cash-8]

On Wed, May 2, 2012 at 9:46 AM, Simon <[hidden email]> wrote:
> Am I the only one using this SCSI card and ZFS? does anyone use any type of
> Adaptec U320 SCSI card? I hardly doubt the behaviour I'm experiencing is strictly
> related to ahd driver.

Try the following:
  - format a drive or two using UFS
  - unplug the drive from the controller
  - plug it back in

Do you get the same errors?  If so, it's a driver/controller issue.

> When I first read about ZFS I've built up high hopes, but now they are slowly
> fading away.

You can't expect miracles from ZFS if the underlying hardware has issues.  :)

--
Freddie Cash
[hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

I don't expect miracles from ZFS if hardware is not compatible. However,
I did expect for ahd driver and Adaptec U320 to still be fairly common and
well aged, thus being compatible, but perhaps that isn't the case.

Is there a list of compatible controllers?

I do not understand how formatting a drive using UFS would solve my
issues. I pulled 2 drives out from functional zraid2 to simulate drive failure
and the system froze. Clearly there is an issue using this controller with ZFS
in ZRAID setup. It shouldn't matter what I have installed on a drive.
When a drive is pulled out of hot-plug enclosure, it should get marked as
offline.

-Simon

On Wed, 2 May 2012 09:50:41 -0700, Freddie Cash wrote:

>On Wed, May 2, 2012 at 9:46 AM, Simon <[hidden email]> wrote:
>> Am I the only one using this SCSI card and ZFS? does anyone use any type of
>> Adaptec U320 SCSI card? I hardly doubt the behaviour I'm experiencing is strictly
>> related to ahd driver.

>Try the following:
>  - format a drive or two using UFS
>  - unplug the drive from the controller
>  - plug it back in

>Do you get the same errors?  If so, it's a driver/controller issue.

>> When I first read about ZFS I've built up high hopes, but now they are slowly
>> fading away.

>You can't expect miracles from ZFS if the underlying hardware has issues.  :)

>--
>Freddie Cash
>[hidden email]



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Freddie Cash-8]

On Wed, May 2, 2012 at 10:27 AM, Simon <[hidden email]> wrote:

> I don't expect miracles from ZFS if hardware is not compatible. However,
> I did expect for ahd driver and Adaptec U320 to still be fairly common and
> well aged, thus being compatible, but perhaps that isn't the case.
>
> Is there a list of compatible controllers?
>
> I do not understand how formatting a drive using UFS would solve my
> issues. I pulled 2 drives out from functional zraid2 to simulate drive
> failure and the system froze. Clearly there is an issue using this controller with
> ZFS in ZRAID setup. It shouldn't matter what I have installed on a drive.
> When a drive is pulled out of hot-plug enclosure, it should get marked as
> offline.

Until you format drives with something other than ZFS and test
removing/plugging in a drive, you cannot say for certain that it's a
ZFS issue.  The error messages shown all come from ahd which is way
below ZFS.

--
Freddie Cash
[hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

I'm not saying it is ZFS. What I am saying is that I have setup ZRAID2 using
3x U320 SCSI drives connected to SAF-TE backplane connected to Adaptec
7902 controller. When I pull 2 out of 3 drives out, the system freezes where
no new requests are being processed and the ahb driver starts throwing a fit.

Is this normal/expected behaviour? I cannot pull drives out from hotswappable
enclosure to simulate drive failure? if so, what would happen if a drive did fail
while inside the enclosure? what would be different? I cannot trust RAID setup
where I cannot pull/disconnect a drive and expect the system to continue to
run smoothly.

BTW, I'm not sure if this matters or not but I'm setting up ZRAID2 using GPT
partitions as shown here:

http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE

Can someone confirm whether or not I can use Adaptec 7902 with ZRAID?

-Simon

On Wed, 2 May 2012 10:50:48 -0700, Freddie Cash wrote:

>On Wed, May 2, 2012 at 10:27 AM, Simon <[hidden email]> wrote:
>> I don't expect miracles from ZFS if hardware is not compatible. However,
>> I did expect for ahd driver and Adaptec U320 to still be fairly common and
>> well aged, thus being compatible, but perhaps that isn't the case.
>>
>> Is there a list of compatible controllers?
>>
>> I do not understand how formatting a drive using UFS would solve my
>> issues. I pulled 2 drives out from functional zraid2 to simulate drive
>> failure and the system froze. Clearly there is an issue using this controller with
>> ZFS in ZRAID setup. It shouldn't matter what I have installed on a drive.
>> When a drive is pulled out of hot-plug enclosure, it should get marked as
>> offline.

>Until you format drives with something other than ZFS and test
>removing/plugging in a drive, you cannot say for certain that it's a
>ZFS issue.  The error messages shown all come from ahd which is way
>below ZFS.

>--
>Freddie Cash
>[hidden email]



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Mark Felder-4]

First of all, you're testing on decade old SCSI hardware that probably  
hasn't seen any serious use on a newer FreeBSD install in a very, very  
long time.

Secondly, I'm confused about the concept of a "3 drive RAIDZ2". How is  
that even possible? Two drives have to be parity, so the last drive is...  
the entire dataset? Why aren't you just doing a 3-way mirror?

And finally yes, you can just yank drives in a ZFS array to simulate a  
failure. After reinsertion you have to manually add them back to the pool,  
but it certainly works.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Rich]

It's perfectly possible to do a 3-way RAID-Z2 - just not useful, AFAIK.

Also, Simon, I think the disconnect between you and the mailing list
is that you are observing bad behavior of the system when you remove
the drives, and you think this is a ZFS problem. This is not a ZFS
problem - if the underlying storage driver (ahd) freezes up and stops
handling requests (which is what it sounds like you're describing
here), there's not much ZFS can do about it.

- Rich

On Wed, May 2, 2012 at 5:05 PM, Mark Felder <[hidden email]> wrote:

> First of all, you're testing on decade old SCSI hardware that probably
> hasn't seen any serious use on a newer FreeBSD install in a very, very long
> time.
>
> Secondly, I'm confused about the concept of a "3 drive RAIDZ2". How is that
> even possible? Two drives have to be parity, so the last drive is... the
> entire dataset? Why aren't you just doing a 3-way mirror?
>
> And finally yes, you can just yank drives in a ZFS array to simulate a
> failure. After reinsertion you have to manually add them back to the pool,
> but it certainly works.
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

I'm trying to figure out why it's not working with ahd driver. It's beginning to look
more and more as if I shouldn't use ZFS with ahd driver. That's why I asked for
someone to confirm whether or not the two can be used together. It also seems
that everyone moved on to SAS/SATA technology, albeit it's hard to believe.
There are few machine I have from early 2007, so 5 years old, that have the
same controller onboard. So while u320 is a decade old technology, many
servers been built using it in 2006+

Unless someone can confirm otherwise, I will assume ahd and ZFS do not
work well together when it comes to dying/swapping drives.

Having said the above, is there any FreeBSD RAID software I can use with
ahb driver that won't give me same issues I'm experiencing with ahd and ZFS?
vinum? gmirror? Can I expect them to work better with ahd given ahd cannot
handle pulling of working drives out of SAF-TE aware enclosure?

Thank you,
Simon

On Wed, 2 May 2012 17:18:22 -0400, Rich wrote:

>It's perfectly possible to do a 3-way RAID-Z2 - just not useful, AFAIK.

>Also, Simon, I think the disconnect between you and the mailing list
>is that you are observing bad behavior of the system when you remove
>the drives, and you think this is a ZFS problem. This is not a ZFS
>problem - if the underlying storage driver (ahd) freezes up and stops
>handling requests (which is what it sounds like you're describing
>here), there's not much ZFS can do about it.

>- Rich

>On Wed, May 2, 2012 at 5:05 PM, Mark Felder <[hidden email]> wrote:
>> First of all, you're testing on decade old SCSI hardware that probably
>> hasn't seen any serious use on a newer FreeBSD install in a very, very long
>> time.
>>
>> Secondly, I'm confused about the concept of a "3 drive RAIDZ2". How is that
>> even possible? Two drives have to be parity, so the last drive is... the
>> entire dataset? Why aren't you just doing a 3-way mirror?
>>
>> And finally yes, you can just yank drives in a ZFS array to simulate a
>> failure. After reinsertion you have to manually add them back to the pool,
>> but it certainly works.
>>
>> _______________________________________________
>> [hidden email] mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "[hidden email]"
>_______________________________________________
>[hidden email] mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>To unsubscribe, send any mail to "[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Mark Felder-4]

On Wed, 02 May 2012 17:06:04 -0500, <[hidden email]> wrote:

>
> Having said the above, is there any FreeBSD RAID software I can use with
> ahb driver that won't give me same issues I'm experiencing with ahd and  
> ZFS?
> vinum? gmirror? Can I expect them to work better with ahd given ahd  
> cannot
> handle pulling of working drives out of SAF-TE aware enclosure?

We're pretty sure it's not the /RAID Software/ and that it's a driver  
issue. I don't think with your version of FreeBSD that you can do *any*  
hot swapping of drives with that controller.

Ask the maintainer of the ahd driver to take a look -- he might be able to  
provide a patch.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

Thank you Mark, that's what I thought, too. My major concern is the failure
of drives, not so much the hot-swapping. I can live with taking the machine
offline, replacing the drive(s), and booting up. Yanking the drives out from live
system is the only way I can simulate a hard failure. Without this working, I
cannot be sure the system will remain running should a drive go bad. I'm
not sure how the system would handle this given ahd cannot handle pulling
of hot drives out.  I will try to get in touch with the driver maintainer.

Thanks again!
-Simon

On Wed, 2 May 2012 17:21:14 -0500, Mark Felder wrote:

>On Wed, 02 May 2012 17:06:04 -0500, <[hidden email]> wrote:

>>
>> Having said the above, is there any FreeBSD RAID software I can use with
>> ahb driver that won't give me same issues I'm experiencing with ahd and  
>> ZFS?
>> vinum? gmirror? Can I expect them to work better with ahd given ahd  
>> cannot
>> handle pulling of working drives out of SAF-TE aware enclosure?

>We're pretty sure it's not the /RAID Software/ and that it's a driver  
>issue. I don't think with your version of FreeBSD that you can do *any*  
>hot swapping of drives with that controller.

>Ask the maintainer of the ahd driver to take a look -- he might be able to  
>provide a patch.
>_______________________________________________
>[hidden email] mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>To unsubscribe, send any mail to "[hidden email]"



_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Daryl Richards]

On 12-05-02 6:34 PM, Simon wrote:

> Thank you Mark, that's what I thought, too. My major concern is the failure
> of drives, not so much the hot-swapping. I can live with taking the machine
> offline, replacing the drive(s), and booting up. Yanking the drives out from live
> system is the only way I can simulate a hard failure. Without this working, I
> cannot be sure the system will remain running should a drive go bad. I'm
> not sure how the system would handle this given ahd cannot handle pulling
> of hot drives out.  I will try to get in touch with the driver maintainer.
>
> Thanks again!
> -Simon

One way to simulate a "failure" with ZFS is to use dd to write trash data to the underlying drive, and then scrub it to detect the errors and have it correct itself.


--
Daryl Richards
Isle Technical Services Inc.

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Simon-14]

On Wed, 02 May 2012 19:06:56 -0400, Daryl Richards wrote:

>One way to simulate a "failure" with ZFS is to use dd to write trash data to the
> underlying drive, and then scrub it to detect the errors and have it correct itself.



Sure, but to me this is a soft failure: the drive is still powered-on, spinning,
responding, etc... this works fine. I was trying to simulate hard failure,
drive power failure, motor broke, ECB burnt, etc... with hardware RAID
I would just yank the drive out.

-Simon


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"

[Daniel Kalchev]

Simon,

ZFS is known to put much more stress on hardware than UFS (in typical
usage) and any flaky hardware setup very quickly breaks up. When I
started playing with ZFS, I was dismayed that few drives that were
otherwise perfectly OK and never, ever showed any indication of
malfunction did show errors with ZFS. Same with controllers.

Just don't panic :)

ZFS is very reliable otherwise, sometimes in the 'magical' realm (but of
course, don't ever imagine things, when it comes to your data).

My guess is, that you simulated the failing drives by pulling them out
too quickly. That is, if the Adaptec controller or rather the drives
doesn't fully support hot-swap you may need to do things more 'safely',
by waiting a bit between pulls, doing scsi bus reset (and waiting for it
to enumerate again), rescans etc.

One can sometimes hot-swap even non-hot swappable SATA this way :)
Sometimes..

Have you tried repeating this experiment?

Daniel

On 03.05.12 01:34, Simon wrote:

> Thank you Mark, that's what I thought, too. My major concern is the failure
> of drives, not so much the hot-swapping. I can live with taking the machine
> offline, replacing the drive(s), and booting up. Yanking the drives out from live
> system is the only way I can simulate a hard failure. Without this working, I
> cannot be sure the system will remain running should a drive go bad. I'm
> not sure how the system would handle this given ahd cannot handle pulling
> of hot drives out.  I will try to get in touch with the driver maintainer.
>
> Thanks again!
> -Simon
>
> On Wed, 2 May 2012 17:21:14 -0500, Mark Felder wrote:
>
>> On Wed, 02 May 2012 17:06:04 -0500,<[hidden email]>  wrote:
>>> Having said the above, is there any FreeBSD RAID software I can use with
>>> ahb driver that won't give me same issues I'm experiencing with ahd and
>>> ZFS?
>>> vinum? gmirror? Can I expect them to work better with ahd given ahd
>>> cannot
>>> handle pulling of working drives out of SAF-TE aware enclosure?
>> We're pretty sure it's not the /RAID Software/ and that it's a driver
>> issue. I don't think with your version of FreeBSD that you can do *any*
>> hot swapping of drives with that controller.
>> Ask the maintainer of the ahd driver to take a look -- he might be able to
>> provide a patch.
>> _______________________________________________
>> [hidden email] mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "[hidden email]"
>
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"