Quantcast

Time to bump default VM_SWZONE_SIZE_MAX?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Time to bump default VM_SWZONE_SIZE_MAX?

Colin Percival-4
Hi all,

If I'm understanding things correctly, the "maxswzone" value -- set by the
kern.maxswzone loader tunable or to VM_SWZONE_SIZE_MAX by default -- should
be approximately 9 MiB per GiB of swap space.

The current default for VM_SWZONE_SIZE_MAX was set in August 2002 to 32 MiB;
meaning that anyone who wants to use more than ~ 3.5 GB of swap space ought
to set kern.maxswzone in /boot/loader.conf.

Is it time to increase this default on amd64?  (I understand that keeping the
value low on i386 is important due to KVA limitations, but amd64 has far more
address space available...)

--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Alan Cox-9
On Sun, Aug 12, 2012 at 5:44 PM, Colin Percival <[hidden email]>wrote:

> Hi all,
>
> If I'm understanding things correctly, the "maxswzone" value -- set by the
> kern.maxswzone loader tunable or to VM_SWZONE_SIZE_MAX by default -- should
> be approximately 9 MiB per GiB of swap space.
>
> The current default for VM_SWZONE_SIZE_MAX was set in August 2002 to 32
> MiB;
> meaning that anyone who wants to use more than ~ 3.5 GB of swap space ought
> to set kern.maxswzone in /boot/loader.conf.
>
> Is it time to increase this default on amd64?  (I understand that keeping
> the
> value low on i386 is important due to KVA limitations, but amd64 has far
> more
> address space available...)
>
>
The short answer is "Yes."  The long answer is that amd64, ia64, etc.
shouldn't have this limit at all.  The swap zone objects are small objects
that could be allocated by uma_small_alloc() and accessed through the
direct map.

Alan
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Peter Jeremy-8
In reply to this post by Colin Percival-4
On 2012-Aug-12 15:44:07 -0700, Colin Percival <[hidden email]> wrote:
>If I'm understanding things correctly, the "maxswzone" value -- set by the
>kern.maxswzone loader tunable or to VM_SWZONE_SIZE_MAX by default -- should
>be approximately 9 MiB per GiB of swap space.

I'm not sure how you got that value.  By default, struct swblock is
288 bytes (280 bytes on 32-bit archs) and can store up to 32 pages of
swap (the comment in vm/swap_pager.c:swap_pager_swap_init() is wrong).
For x86, this is 2.25 MiB per GiB (best case).

>The current default for VM_SWZONE_SIZE_MAX was set in August 2002 to 32 MiB;
>meaning that anyone who wants to use more than ~ 3.5 GB of swap space ought
>to set kern.maxswzone in /boot/loader.conf.

In practice, you can't fully populate each swblock.  I did a test on
my amd64 box by running multiple copies of a program that allocates
and dirties a big chunk of RAM and then pause()s.  That gave me a 90%
swblock utilisation - which I suspect is higher than a typical
scenario where memory pressure pushes more randomly unused pages out.

Realistically, I'd say that the default VM_SWZONE_SIZE_MAX can handle
about 9GB swap (at least, that was my experience).

BTW, if you plan on allocating lots of swap, be aware that each swap
device is limited to 32GiB - see vm/swap_pager.c:swaponsomething().

--
Peter Jeremy

attachment0 (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Dag-Erling Smørgrav
In reply to this post by Colin Percival-4
Colin Percival <[hidden email]> writes:
> If I'm understanding things correctly, the "maxswzone" value -- set by the
> kern.maxswzone loader tunable or to VM_SWZONE_SIZE_MAX by default -- should
> be approximately 9 MiB per GiB of swap space.

Far less, in fact.  As mentioned in loader(8), the default 32 MB limit
is enough for slightly more than 7 GB of swap space - assuming 100%
efficient use of swblocks.  The man page recommends not using more than
half the theoretical limit, or, with 16 pages per swblock,

                          1     maxswzone x s
                         --- x ---------------
                          2          16

where s = 276 on 32-bit systems and 288 on 64-bit systems.

> The current default for VM_SWZONE_SIZE_MAX was set in August 2002 to 32 MiB;
> meaning that anyone who wants to use more than ~ 3.5 GB of swap space ought
> to set kern.maxswzone in /boot/loader.conf.
>
> Is it time to increase this default on amd64?  (I understand that keeping the
> value low on i386 is important due to KVA limitations, but amd64 has far more
> address space available...)

There is no reason to keep this limit at all.  The algorithm used to
size the zone is "physpages / 2 * sizeof(struct swblock) or maxswzone,
whichever is smaller" where maxswzone == VM_SWZONE_SIZE_MAX unless
kern.maxswzone was set in loader.conf.  On amd64, the cutoff point is
slightly below 1 GB of physical memory (233,016 4,096-byte pages), so
the limit doesn't help machines that actually need to conserve memory,
and it hurts machines that have plenty of it and therefore also plenty
of swap (assuming the user followed the old "twice the amount of RAM"
rule of thumb).

DES
--
Dag-Erling Smørgrav - [hidden email]


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"

maxswzone-no-default.diff (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Dag-Erling Smørgrav
Slightly better patch (improved documentation)

DES
--
Dag-Erling Smørgrav - [hidden email]


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"

maxswzone-no-default.diff (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Colin Percival-4
In reply to this post by Peter Jeremy-8
On 08/13/12 14:23, Peter Jeremy wrote:

> On 2012-Aug-12 15:44:07 -0700, Colin Percival <[hidden email]>
> wrote:
>> If I'm understanding things correctly, the "maxswzone" value -- set by
>> the kern.maxswzone loader tunable or to VM_SWZONE_SIZE_MAX by default --
>> should be approximately 9 MiB per GiB of swap space.
>
> I'm not sure how you got that value.  By default, struct swblock is 288
> bytes (280 bytes on 32-bit archs) and can store up to 32 pages of swap (the
> comment in vm/swap_pager.c:swap_pager_swap_init() is wrong). For x86, this
> is 2.25 MiB per GiB (best case).

I got that value from a previous mailing list discussion -- I think it was
based on the (incorrect) value of 16 pages per swblock and not exceeding
50% swblock utilization.

> Realistically, I'd say that the default VM_SWZONE_SIZE_MAX can handle about
> 9GB swap (at least, that was my experience).

The 50% utilization rule of thumb would make it 7 GB, but yes, same order
of magnitude.

> BTW, if you plan on allocating lots of swap, be aware that each swap device
> is limited to 32GiB - see vm/swap_pager.c:swaponsomething().

Yes, noted.  I'm not actually using lots of swap myself, but I was writing
code for EC2 instances to set up "reasonable" amounts of swap at boot time,
and I don't want to accidentally autoconfigure more swap than FreeBSD can
safely use.

--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

John Baldwin
In reply to this post by Dag-Erling Smørgrav
On Tuesday, August 14, 2012 02:47:47 AM Dag-Erling Smørgrav wrote:

> Colin Percival <[hidden email]> writes:
> > If I'm understanding things correctly, the "maxswzone" value -- set by
> > the kern.maxswzone loader tunable or to VM_SWZONE_SIZE_MAX by default --
> > should be approximately 9 MiB per GiB of swap space.
>
> Far less, in fact.  As mentioned in loader(8), the default 32 MB limit
> is enough for slightly more than 7 GB of swap space - assuming 100%
> efficient use of swblocks.  The man page recommends not using more than
> half the theoretical limit, or, with 16 pages per swblock,
>
>                           1     maxswzone x s
>                          --- x ---------------
>                           2          16
>
> where s = 276 on 32-bit systems and 288 on 64-bit systems.
>
> > The current default for VM_SWZONE_SIZE_MAX was set in August 2002 to 32
> > MiB; meaning that anyone who wants to use more than ~ 3.5 GB of swap
> > space ought to set kern.maxswzone in /boot/loader.conf.
> >
> > Is it time to increase this default on amd64?  (I understand that keeping
> > the value low on i386 is important due to KVA limitations, but amd64 has
> > far more address space available...)
>
> There is no reason to keep this limit at all.  The algorithm used to
> size the zone is "physpages / 2 * sizeof(struct swblock) or maxswzone,
> whichever is smaller" where maxswzone == VM_SWZONE_SIZE_MAX unless
> kern.maxswzone was set in loader.conf.  On amd64, the cutoff point is
> slightly below 1 GB of physical memory (233,016 4,096-byte pages), so
> the limit doesn't help machines that actually need to conserve memory,
> and it hurts machines that have plenty of it and therefore also plenty
> of swap (assuming the user followed the old "twice the amount of RAM"
> rule of thumb).

Hmm, this is not true on i386 where the problem is not just the physical
RAM required, but also address space.  (The swap zone is all mapped into KVA
even if it isn't used.)  This is why Alan's e-mail specifically
mentioned amd64, ia64, etc. but not i386 in his list.  I think i386 still
needs this limit, and I think your commit jumped the gun a bit.

--
John Baldwin
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Dag-Erling Smørgrav
John Baldwin <[hidden email]> writes:
> Hmm, this is not true on i386 where the problem is not just the physical
> RAM required, but also address space.  (The swap zone is all mapped into KVA
> even if it isn't used.)  This is why Alan's e-mail specifically
> mentioned amd64, ia64, etc. but not i386 in his list.  I think i386 still
> needs this limit, and I think your commit jumped the gun a bit.

How about we reinstate the limit on i386, but increase it to 64 MB?
That would increase the theoretical maximum to ~15 GB.  People with 8 GB
swap would get a warning, but would be unlikely to run into trouble.

(or we could increase the limit to 72351744 bytes, which is the precise
amount required to support 16 GB)

DES
--
Dag-Erling Smørgrav - [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

John Baldwin
On Friday, August 24, 2012 5:44:48 am Dag-Erling Smørgrav wrote:

> John Baldwin <[hidden email]> writes:
> > Hmm, this is not true on i386 where the problem is not just the physical
> > RAM required, but also address space.  (The swap zone is all mapped into KVA
> > even if it isn't used.)  This is why Alan's e-mail specifically
> > mentioned amd64, ia64, etc. but not i386 in his list.  I think i386 still
> > needs this limit, and I think your commit jumped the gun a bit.
>
> How about we reinstate the limit on i386, but increase it to 64 MB?
> That would increase the theoretical maximum to ~15 GB.  People with 8 GB
> swap would get a warning, but would be unlikely to run into trouble.
>
> (or we could increase the limit to 72351744 bytes, which is the precise
> amount required to support 16 GB)

Note that on i386 you can't get more than 4GB of RAM without PAE, and if you
have any modern x86 box with > 4GB of RAM, you are most likely running amd64
on it, not i386.  I think i386 would be fine to just keep the limit it had.

--
John Baldwin
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Dag-Erling Smørgrav
John Baldwin <[hidden email]> writes:
> Note that on i386 you can't get more than 4GB of RAM without PAE, and if you
> have any modern x86 box with > 4GB of RAM, you are most likely running amd64
> on it, not i386.  I think i386 would be fine to just keep the limit it had.

The limit we had was insufficient for 8 GB of swap.

DES
--
Dag-Erling Smørgrav - [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

John Baldwin
On Friday, August 24, 2012 8:45:43 am Dag-Erling Smørgrav wrote:
> John Baldwin <[hidden email]> writes:
> > Note that on i386 you can't get more than 4GB of RAM without PAE, and if you
> > have any modern x86 box with > 4GB of RAM, you are most likely running amd64
> > on it, not i386.  I think i386 would be fine to just keep the limit it had.
>
> The limit we had was insufficient for 8 GB of swap.

In absolute or practical terms?  Not all swap blocks are fully utilized.  At
Y! the install script we used would compute the maximum theoretical swap zone
needed and then cut it in half, and this worked quite well.  Also, keep in mind,
this is for i386, not amd64.  At this point i386 is going to be used on smaller
systems (e.g. netbooks, etc.), not servers that have lots of swap.

--
John Baldwin
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Colin Percival-4
On 08/24/12 07:13, John Baldwin wrote:

> On Friday, August 24, 2012 8:45:43 am Dag-Erling Smørgrav wrote:
>> John Baldwin <[hidden email]> writes:
>>> Note that on i386 you can't get more than 4GB of RAM without PAE, and if you
>>> have any modern x86 box with > 4GB of RAM, you are most likely running amd64
>>> on it, not i386.  I think i386 would be fine to just keep the limit it had.
>>
>> The limit we had was insufficient for 8 GB of swap.
>
> In absolute or practical terms?  Not all swap blocks are fully utilized.  At
> Y! the install script we used would compute the maximum theoretical swap zone
> needed and then cut it in half, and this worked quite well.  Also, keep in mind,
> this is for i386, not amd64.  At this point i386 is going to be used on smaller
> systems (e.g. netbooks, etc.), not servers that have lots of swap.

I'd like to see i386 bumped slightly, just so that the rule of "allocate swap
space equal to max(RAM, min(2*RAM, 8 GB))" (which I've seen in lots of places)
is more likely to be safe.  If I'm understanding things correctly, bumping from
32 MB up to 34.5 MB should give us a theoretical 16 GiB or a "safe" 8 GiB limit
on swap usage (2^17 structures which are 276 bytes each on i386).

But I agree that the real issue was with amd64, not i386.

--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Dag-Erling Smørgrav
In reply to this post by John Baldwin
John Baldwin <[hidden email]> writes:
> Dag-Erling Smørgrav <[hidden email]> writes:
> > The limit we had was insufficient for 8 GB of swap.
> In absolute or practical terms?

This whole thing started because I have a machine with 8 GB swap that
ran out of swzone.

> At this point i386 is going to be used on smaller systems
> (e.g. netbooks, etc.), not servers that have lots of swap.

I don't think it's unreasonable for an i386 box to be maxed out on RAM
(4 GB) and have twice that amount of swap.

DES
--
Dag-Erling Smørgrav - [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Time to bump default VM_SWZONE_SIZE_MAX?

Dag-Erling Smørgrav
In reply to this post by Dag-Erling Smørgrav
Dag-Erling Smørgrav <[hidden email]> writes:
> (or we could increase the limit to 72351744 bytes, which is the precise
> amount required to support 16 GB)

Correction, 36175872 - there are actually 32 pages per entry, not 16.

DES
--
Dag-Erling Smørgrav - [hidden email]
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "[hidden email]"
Loading...