Quantcast

`zpool create' fails on geli ...

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

`zpool create' fails on geli ...

Zeus Panchenko
hi all,

SYNOPSIS: `zpool create poolname device.eli' available on .eli device only after dd some
          random data to .eli first

I am trying to get ZFS on GELI disk ...

Here is the issue:

#> uname -a
FreeBSD 9.0-RELEASE #0 amd64

for /dev/ada2 I do:

#> geli init -K /path/key -s 4096 -a hmac/sha256 -e aes-xts /dev/ada2
Enter new passphrase:
Reenter new passphrase:

Metadata backup can be found in /var/backups/ada2.eli and
can be restored with the following command:

        # geli restore /var/backups/ada2.eli /dev/ada2


#> geli attach -k /path/key /dev/ada2

now I have .eli device

#> ls -al /dev/*eli
lrwxr-xr-x  1 root  wheel            8 Aug 16 15:43 /dev/ad14.eli -> ada2.eli
crw-r-----  1 root  operator    0,  99 Aug 16 15:43 /dev/ada2.eli

now I am trying to create zfs on it:

>  zpool create geliz /dev/ada2.eli
cannot create 'geliz': one or more devices is currently unavailable

`zpool create -f ...' gave the same result and in messages I have plenty
rows like these:

cat /var/log/messages
...
GEOM_ELI: ada2.eli: 131072 bytes corrupted at offset 444539600896.
GEOM_ELI: ada2.eli: 131072 bytes corrupted at offset 444539863040.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 270336.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 444539609088.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 444539871232.
GEOM_ELI: ada2.eli: 4096 bytes corrupted at offset 444540313600.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 65536.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 8192.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 0.
GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 262144.
...



but after
#> dd if=/dev/random of=/dev/ada2.eli bs=10m count=10
10+0 records in
10+0 records out
104857600 bytes transferred in 7.124000 secs (14718922 bytes/sec)

I was able to do it!

#> zpool create geliz /dev/ada2.eli

pool was successfully created

but pool status looks weird for me:

#> zpool status geliz
  pool: geliz
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        geliz       ONLINE       0     0     0
          ada2.eli  ONLINE      10     0     0

errors: No known data errors

after `zscub' and `zpool clear' I have clean pool:

#> zpool status geliz
  pool: geliz
 state: ONLINE
 scan: scrub repaired 0 in 0h0m with 0 errors on Thu Aug 16 16:36:44 2012
config:

        NAME        STATE     READ WRITE CKSUM
        geliz       ONLINE       0     0     0
          ada2.eli  ONLINE       0     0     0

errors: No known data errors


QUESTION:

1. Am I correct to think I really have correct ZFS over GELI set?

2. Why it was needed to dd? What am I missing here, please?


may somebody explain that for me please ...?

--
Zeus V. Panchenko jid:[hidden email]
IT Dpt., I.B.S. LLC  GMT+2 (EET)
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: `zpool create' fails on geli ...

Fabian Keil-2
Zeus Panchenko <[hidden email]> wrote:

> I am trying to get ZFS on GELI disk ...

Good idea, I never use ZFS without it.
 

> Here is the issue:
>
> #> uname -a
> FreeBSD 9.0-RELEASE #0 amd64
>
> for /dev/ada2 I do:
>
> #> geli init -K /path/key -s 4096 -a hmac/sha256 -e aes-xts /dev/ada2
> Enter new passphrase:
> Reenter new passphrase:
ZFS already provides checksums, so why do you want to use
checksums for geli as well?

In my opinion "-a hmac/sha256" doesn't add any protection in
your case, while reducing the space that is available for
ZFS and wasting cpu cycles.

I'm not aware of any problem that can be detected by geli's
integrity checks but wouldn't be detected by ZFS anyway.
ZFS checksums actually offer better protection, as geli only
checksums single sectors.

> Metadata backup can be found in /var/backups/ada2.eli and
> can be restored with the following command:
>
>         # geli restore /var/backups/ada2.eli /dev/ada2
>
>
> #> geli attach -k /path/key /dev/ada2
>
> now I have .eli device
>
> #> ls -al /dev/*eli
> lrwxr-xr-x  1 root  wheel            8 Aug 16 15:43 /dev/ad14.eli -> ada2.eli
> crw-r-----  1 root  operator    0,  99 Aug 16 15:43 /dev/ada2.eli
>
> now I am trying to create zfs on it:
>
> >  zpool create geliz /dev/ada2.eli
> cannot create 'geliz': one or more devices is currently unavailable
>
> `zpool create -f ...' gave the same result and in messages I have plenty
> rows like these:
>
> cat /var/log/messages
> ...
> GEOM_ELI: ada2.eli: 131072 bytes corrupted at offset 444539600896.
> GEOM_ELI: ada2.eli: 131072 bytes corrupted at offset 444539863040.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 270336.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 444539609088.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 444539871232.
> GEOM_ELI: ada2.eli: 4096 bytes corrupted at offset 444540313600.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 65536.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 8192.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 0.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 262144.
> ...
Quoting geli(8):

| DATA AUTHENTICATION
| [..]
| It is recommended to write to the whole provider before first use, in
| order to make sure that all sectors and their corresponding checksums are
| properly initialized into a consistent state.  One can safely ignore data
| authentication errors that occur immediately after the first time a
| provider is attached and before it is initialized in this way.

> but after
> #> dd if=/dev/random of=/dev/ada2.eli bs=10m count=10
> 10+0 records in
> 10+0 records out
> 104857600 bytes transferred in 7.124000 secs (14718922 bytes/sec)
>
> I was able to do it!

Because this forced geli to create the checksums for the first 100m.
Using /dev/zero as source should have worked the same.

> #> zpool create geliz /dev/ada2.eli
>
> pool was successfully created
>
> but pool status looks weird for me:
>
> #> zpool status geliz
>   pool: geliz
>  state: ONLINE
> status: One or more devices has experienced an unrecoverable error.  An
>         attempt was made to correct the error.  Applications are unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
>         using 'zpool clear' or replace the device with 'zpool replace'.
>    see: http://www.sun.com/msg/ZFS-8000-9P
>  scan: none requested
> config:
>
>         NAME        STATE     READ WRITE CKSUM
>         geliz       ONLINE       0     0     0
>           ada2.eli  ONLINE      10     0     0
>
> errors: No known data errors
>
> after `zscub' and `zpool clear' I have clean pool:
>
> #> zpool status geliz
>   pool: geliz
>  state: ONLINE
>  scan: scrub repaired 0 in 0h0m with 0 errors on Thu Aug 16 16:36:44 2012
> config:
>
> NAME        STATE     READ WRITE CKSUM
> geliz       ONLINE       0     0     0
>  ada2.eli  ONLINE       0     0     0
>
> errors: No known data errors
I assume this is the result of not forcing geli to generate the
checksums for the whole provider as described in the man page.

Fabian

signature.asc (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: `zpool create' fails on geli ...

Alaksiej C
In reply to this post by Zeus Panchenko
Geli doesn't initialize checksums, when geli device is created. They
will be calculated only on write. That's why these "XXX bytes
corrupted" messages appeared.

I believe it's better to fill your whole geli device with any data
before use with ZFS, if integrity verification (-a) was enabled for
it.

On Tue, Aug 21, 2012 at 7:07 PM, Zeus Panchenko <[hidden email]> wrote:

> hi all,
>
> SYNOPSIS: `zpool create poolname device.eli' available on .eli device only after dd some
>           random data to .eli first
>
> I am trying to get ZFS on GELI disk ...
>
> Here is the issue:
>
> #> uname -a
> FreeBSD 9.0-RELEASE #0 amd64
>
> for /dev/ada2 I do:
>
> #> geli init -K /path/key -s 4096 -a hmac/sha256 -e aes-xts /dev/ada2
> Enter new passphrase:
> Reenter new passphrase:
>
> Metadata backup can be found in /var/backups/ada2.eli and
> can be restored with the following command:
>
>         # geli restore /var/backups/ada2.eli /dev/ada2
>
>
> #> geli attach -k /path/key /dev/ada2
>
> now I have .eli device
>
> #> ls -al /dev/*eli
> lrwxr-xr-x  1 root  wheel            8 Aug 16 15:43 /dev/ad14.eli -> ada2.eli
> crw-r-----  1 root  operator    0,  99 Aug 16 15:43 /dev/ada2.eli
>
> now I am trying to create zfs on it:
>
>>  zpool create geliz /dev/ada2.eli
> cannot create 'geliz': one or more devices is currently unavailable
>
> `zpool create -f ...' gave the same result and in messages I have plenty
> rows like these:
>
> cat /var/log/messages
> ...
> GEOM_ELI: ada2.eli: 131072 bytes corrupted at offset 444539600896.
> GEOM_ELI: ada2.eli: 131072 bytes corrupted at offset 444539863040.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 270336.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 444539609088.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 444539871232.
> GEOM_ELI: ada2.eli: 4096 bytes corrupted at offset 444540313600.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 65536.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 8192.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 0.
> GEOM_ELI: ada2.eli: 8192 bytes corrupted at offset 262144.
> ...
>
>
>
> but after
> #> dd if=/dev/random of=/dev/ada2.eli bs=10m count=10
> 10+0 records in
> 10+0 records out
> 104857600 bytes transferred in 7.124000 secs (14718922 bytes/sec)
>
> I was able to do it!
>
> #> zpool create geliz /dev/ada2.eli
>
> pool was successfully created
>
> but pool status looks weird for me:
>
> #> zpool status geliz
>   pool: geliz
>  state: ONLINE
> status: One or more devices has experienced an unrecoverable error.  An
>         attempt was made to correct the error.  Applications are unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
>         using 'zpool clear' or replace the device with 'zpool replace'.
>    see: http://www.sun.com/msg/ZFS-8000-9P
>  scan: none requested
> config:
>
>         NAME        STATE     READ WRITE CKSUM
>         geliz       ONLINE       0     0     0
>           ada2.eli  ONLINE      10     0     0
>
> errors: No known data errors
>
> after `zscub' and `zpool clear' I have clean pool:
>
> #> zpool status geliz
>   pool: geliz
>  state: ONLINE
>  scan: scrub repaired 0 in 0h0m with 0 errors on Thu Aug 16 16:36:44 2012
> config:
>
>         NAME        STATE     READ WRITE CKSUM
>         geliz       ONLINE       0     0     0
>           ada2.eli  ONLINE       0     0     0
>
> errors: No known data errors
>
>
> QUESTION:
>
> 1. Am I correct to think I really have correct ZFS over GELI set?
>
> 2. Why it was needed to dd? What am I missing here, please?
>
>
> may somebody explain that for me please ...?
>
> --
> Zeus V. Panchenko                               jid:[hidden email]
> IT Dpt., I.B.S. LLC                                       GMT+2 (EET)
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-geom
> To unsubscribe, send any mail to "[hidden email]"
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: `zpool create' fails on geli ...

Xin LI-2
In reply to this post by Zeus Panchenko
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

On 08/21/12 09:07, Zeus Panchenko wrote:

> hi all,
>
> SYNOPSIS: `zpool create poolname device.eli' available on .eli
> device only after dd some random data to .eli first
>
> I am trying to get ZFS on GELI disk ...
>
> Here is the issue:
>
> #> geli init -K /path/key -s 4096 -a hmac/sha256 -e aes-xts
> /dev/ada2 Enter new passphrase: Reenter new passphrase:
[...]
> #> geli attach -k /path/key /dev/ada2

Normally you will want to fill the device with random data before
using.  Note that you have specified -a, which makes geli to do
checksum authentication, that's not needed because ZFS have built-in
end-to-end checksums already.

> now I have .eli device
>
> #> ls -al /dev/*eli lrwxr-xr-x  1 root  wheel            8 Aug 16
> 15:43 /dev/ad14.eli -> ada2.eli crw-r-----  1 root  operator    0,
> 99 Aug 16 15:43 /dev/ada2.eli
>
> now I am trying to create zfs on it:
>
>> zpool create geliz /dev/ada2.eli
> cannot create 'geliz': one or more devices is currently
> unavailable
>
> `zpool create -f ...' gave the same result and in messages I have
> plenty rows like these:

These are expected behavior.

> cat /var/log/messages ... GEOM_ELI: ada2.eli: 131072 bytes
> corrupted at offset 444539600896. GEOM_ELI: ada2.eli: 131072 bytes
> corrupted at offset 444539863040.
[...]

> ...
>
> but after #> dd if=/dev/random of=/dev/ada2.eli bs=10m count=10
> 10+0 records in 10+0 records out 104857600 bytes transferred in
> 7.124000 secs (14718922 bytes/sec)
>
> I was able to do it!
>
> #> zpool create geliz /dev/ada2.eli
>
> pool was successfully created
>
> but pool status looks weird for me:
>
> #> zpool status geliz pool: geliz state: ONLINE status: One or more
> devices has experienced an unrecoverable error.  An attempt was
> made to correct the error.  Applications are unaffected. action:
> Determine if the device needs to be replaced, and clear the errors
> using 'zpool clear' or replace the device with 'zpool replace'.
> see: http://www.sun.com/msg/ZFS-8000-9P scan: none requested
> config:
>
> NAME        STATE     READ WRITE CKSUM geliz       ONLINE       0
> 0     0 ada2.eli  ONLINE      10     0     0
>
> errors: No known data errors
>
> after `zscub' and `zpool clear' I have clean pool:

Did you see any GELI checksum errors when having this?

> #> zpool status geliz pool: geliz state: ONLINE scan: scrub
> repaired 0 in 0h0m with 0 errors on Thu Aug 16 16:36:44 2012
> config:
>
> NAME        STATE     READ WRITE CKSUM geliz       ONLINE       0
> 0     0 ada2.eli  ONLINE       0     0     0
>
> errors: No known data errors
>
>
> QUESTION:
>
> 1. Am I correct to think I really have correct ZFS over GELI set?
>
> 2. Why it was needed to dd? What am I missing here, please?

My suggestions:

1. Don't use -a, it's a waste of CPU cycle (and disk space) to do
checksums twice -- this won't give more redundancy or more chances to
recover data in case of a hardware failure.

2. Do use dd to initialize the GELI device before use.  There are
several benefits of doing this -- the most important two are -- it
wipes existing, possibly sensitive data, and make it harder for
attackers to tell where is the important data.

Cheers,
- --
Xin LI <[hidden email]>    https://www.delphij.net/
FreeBSD - The Power to Serve!           Live free or die
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBCAAGBQJQM8kpAAoJEG80Jeu8UPuzeOAH/i2kG/jN3j58wTe/gG2teKoM
08xy+Lv9lhljihJkUhRx1hAPtYdK1oMKVg7mnQbohSRzjGGqBRnT25ZUD8kbusmW
ULDOmSBbnraStNQbBSpnyik/y2trzfne9YzjhH4aB1CKVJ2X4cHTaJIaGv9iQqI3
S8QjEpKCDcpKlEyGlhJ9TPaCqyzpJbw6p5TDGoVEsq9YIiE7BAbrjfw5Pe87HKK0
BAsLqmJYmQSjjLp/g4FK5vjr/zVpGgPcwP7oD0iSXCX7UI7M/Rhj8Rqyai1cv2/g
ES7uhpy5ifAUalcuJjIFqox7QC5h2uT0e5/DPNttmXfL1d0yb3FdLPgWkV0GDF0=
=v/ZJ
-----END PGP SIGNATURE-----
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: `zpool create' fails on geli ...

Zeus Panchenko
In reply to this post by Alaksiej C
thanks much to all for help

now everything is clear to me and works fine!

resume:

if geli was initialized with `-a' than we need to fill whole of it to
initialize checksums what will make it possible to `zpool create ...'

something like this:

geli init -K /path/key -s 4096 -a hmac/sha256 -e aes-xts /dev/adaX
geli attach -k /path/key /dev/adaX
dd if=/dev/zero of=/dev/adaX.eli bs=10m
zpool create geliz /dev/adaX.eli

but it's better to

geli init -K /path/key -s 4096 -e aes-xts /dev/adaX
geli attach -k /path/key /dev/adaX
zpool create geliz /dev/adaX.eli

since `geli -a ...' in this case, is a waste of CPU cycles and disk
space.

--
Zeus V. Panchenko jid:[hidden email]
IT Dpt., I.B.S. LLC  GMT+2 (EET)
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: `zpool create' fails on geli ...

Fabian Keil-2
Zeus Panchenko <[hidden email]> wrote:

> resume:
>
> if geli was initialized with `-a' than we need to fill whole of it to
> initialize checksums what will make it possible to `zpool create ...'
>
> something like this:
>
> geli init -K /path/key -s 4096 -a hmac/sha256 -e aes-xts /dev/adaX
> geli attach -k /path/key /dev/adaX
> dd if=/dev/zero of=/dev/adaX.eli bs=10m
> zpool create geliz /dev/adaX.eli
>
> but it's better to
>
> geli init -K /path/key -s 4096 -e aes-xts /dev/adaX
Does your disk actually use 4k sectors? Otherwise it's not clear
to me that "-s 4096" makes sense when using ZFS.

I'm not claiming that it's obviously wrong, but I'm not aware of
any benchmarks that show that it's better than the default in
any way.

Fabian

signature.asc (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: `zpool create' fails on geli ...

Flemming Jacobsen
Fabian Keil wrote:
> Zeus Panchenko <[hidden email]> wrote:
> > geli init -K /path/key -s 4096 -e aes-xts /dev/adaX
>
> Does your disk actually use 4k sectors? Otherwise it's not clear
> to me that "-s 4096" makes sense when using ZFS.
>
> I'm not claiming that it's obviously wrong, but I'm not aware of
> any benchmarks that show that it's better than the default in
> any way.

It is my understanding that creating a 4K setup will prepare you
for the day when your replacement drive is a 4K one.
No benefit today, but also no real performance hit. And we avoid
a real performance hit later.

If I am mistaken, then I wold love to hear about it.


Regards,
Flemming

--
Flemming Jacobsen                                  Email: [hidden email]

"I don't need The Media to tell me that I should be outraged about a
brutal murder. All I need is to be informed that it has happened, and
I'll form my own opinion about it."
-- The_Morlock (http://slashdot.org/comments.pl?sid=00%2F02%2F21%2F1125208)
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "[hidden email]"
Loading...