RLIMIT_DATA and malloc(3) use of mmap(2)
Locked
 
|
I was reminded about the patch I wrote for Igor Sysoev some time ago.
The issue the patch tries to handle is that jemalloc uses mmap() instead of sbrk() for pages allocation, and thus RLIMIT_DATA limit is no longer effective to put the bounds on the process heap. Since reverting to sbrk for such purpose is worse then the issue itself, I proposed a solution of 'self-restricting malloc'. The patch adds a flag MAP_DATALIMIT to the flags argument of mmap(2), which instructs the system to account the mapping in the RLIMIT_DATA resource count. The malloc(3) also gets new option 'L' to enable passing MAP_DATALIMIT to mmap() when allocating pages. By default, the 'L' option is not activated. Now, if user wants to ensure that process heap is restricted by the ulimit -d and still use mmap() for jemalloc, he supplies the option using any mechanism. The behaviour is voluntaristic, to prevent the trashing use RLIMIT_SWAP. Do people consider the facility useful ? Any comments for the patch itself ? http://people.freebsd.org/~kib/misc/map_datalimit.1.patch |
attachment0 (203 bytes)
|
|
|
On Tue, Nov 22, 2011 at 07:43:57PM +0400, Maxim Dounin wrote:
> Hello! > > On Tue, Nov 22, 2011 at 02:44:10PM +0200, Kostik Belousov wrote: > > > I was reminded about the patch I wrote for Igor Sysoev some time ago. > > The issue the patch tries to handle is that jemalloc uses mmap() instead > > of sbrk() for pages allocation, and thus RLIMIT_DATA limit is no longer > > effective to put the bounds on the process heap. Since reverting to sbrk > > for such purpose is worse then the issue itself, I proposed a solution > > of 'self-restricting malloc'. > > Just a little clarification for others: currently, there is no way > to *safely* limit memory usage of a process while using jemalloc > with mmap(). > > The only thing available is RLIMIT_VMEM, but it's not safe as it > may be reached on stack grow (leaving no possibility for an > application to handle this). > > > The patch adds a flag MAP_DATALIMIT to the flags argument of mmap(2), > > which instructs the system to account the mapping in the RLIMIT_DATA > > resource count. The malloc(3) also gets new option 'L' to enable > > passing MAP_DATALIMIT to mmap() when allocating pages. By default, > > the 'L' option is not activated. > > > > Now, if user wants to ensure that process heap is restricted by the > > ulimit -d and still use mmap() for jemalloc, he supplies the option > > using any mechanism. The behaviour is voluntaristic, to prevent the > > trashing use RLIMIT_SWAP. > > > > Do people consider the facility useful ? > > Yes, at least some way to safely limit process memory usage is > certainly needed. > > It's a bit sad this isn't enabled by default, but it's probably > too late for this. RLIMIT_DATA was (almost) a noop for too long > and making it work again to limit all memory allocations will > break POLA. > > > Any comments for the patch itself ? > > > > http://people.freebsd.org/~kib/misc/map_datalimit.1.patch > > Patch looks ok for me (though I'm not a VM expert). > > Another possible aproach would be to introduce separate anonymous > (private?) mmap limit, this will allow to do essentially the same > thing in a bit more consistent (IMHO) manner. swap limit to the per-process limit raises the architectural questions, due to shadow chains of the backing vm objects. In particular, we have to account the anonymous memory used for backing the changed pages from the writeable private mapping of the files. Similar issues appear due to fork(). Anyway, the patch needs testers before I will push it forward. |
|
|
|
|
On Thu, Nov 24, 2011 at 09:38:19PM +0400, Maxim Konovalov wrote:
> Hi Kostik, > > On Wed, 23 Nov 2011, 11:22+0400, Maxim Konovalov wrote: > > > [...] > > > Anyway, the patch needs testers before I will push it forward. > > > > > [igor's email was corrected] > > > > We will test it in out environment and let you know. > > > It seems I don't understand how it works. > > Here is a test program: http://maxim.int.ru/stuff/malloc_test.c.txt > > It allocates successfully 64x1MB chunks by malloc(), reallocf() and > realloc() with the following command line: > > MALLOC_OPTIONS=JM limits -d 10m ./malloc_test > > When we add L flag to the MALLOC_OPTIONS it starts to work strange: > > It allocates 64MB via malloc() but fails to allocate more than 3MB by > realloc() and reallocf(). instead of new entry created due to mmap(2). The other part, in particular, the failure after 3MB, is in fact the correct behaviour. Jemalloc() caches allocation of the pages, and it allocates more then asked in the request. ktrace(1) shows the whole history. Malloc() first allocated 4MB for the needs of libc etc. Then, it allocated 4MB chunk which was used for satisfying the requests of 1-3MB. When the 4MB request came in, the allocation for 8MB was attempted, and failed, since 4MB (libc etc) + 8MB = 12MB > data limit. > > More funny, the result varies from time to time: I have no explanation for this, and cannot reproduce the issue. Look at the ktrace. Overall, the test is quite curious but absolutely unrealistic. Fixed patch is available at http://people.freebsd.org/~kib/misc/map_datalimit.2.patch |
|
|
|
|
|
|
Loading...
| Powered by Nabble | Edit this page |