Imported Upstream version 1.7.0+dfsg1

[rustc.git] / src / doc / book / custom-allocators.md

% Custom Allocators

Allocating memory isn't always the easiest thing to do, and while Rust generally
takes care of this by default it often becomes necessary to customize how
allocation occurs. The compiler and standard library currently allow switching
out the default global allocator in use at compile time. The design is currently
spelled out in [RFC 1183][rfc] but this will walk you through how to get your
own allocator up and running.

[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1183-swap-out-jemalloc.md

# Default Allocator

The compiler currently ships two default allocators: `alloc_system` and
`alloc_jemalloc` (some targets don't have jemalloc, however). These allocators
are normal Rust crates and contain an implementation of the routines to
allocate and deallocate memory. The standard library is not compiled assuming
either one, and the compiler will decide which allocator is in use at
compile-time depending on the type of output artifact being produced.

Binaries generated by the compiler will use `alloc_jemalloc` by default (where
available). In this situation the compiler "controls the world" in the sense of
it has power over the final link. Primarily this means that the allocator
decision can be left up the compiler.

Dynamic and static libraries, however, will use `alloc_system` by default. Here
Rust is typically a 'guest' in another application or another world where it
cannot authoritatively decide what allocator is in use. As a result it resorts
back to the standard APIs (e.g. `malloc` and `free`) for acquiring and releasing
memory.

# Switching Allocators

Although the compiler's default choices may work most of the time, it's often
necessary to tweak certain aspects. Overriding the compiler's decision about
which allocator is in use is done simply by linking to the desired allocator:

```rust,no_run
#![feature(alloc_system)]

extern crate alloc_system;

fn main() {
    let a = Box::new(4); // allocates from the system allocator
    println!("{}", a);
}
```

In this example the binary generated will not link to jemalloc by default but
instead use the system allocator. Conversely to generate a dynamic library which
uses jemalloc by default one would write:

```rust,ignore
#![feature(alloc_jemalloc)]
#![crate_type = "dylib"]

extern crate alloc_jemalloc;

pub fn foo() {
    let a = Box::new(4); // allocates from jemalloc
    println!("{}", a);
}
# fn main() {}
```

# Writing a custom allocator

Sometimes even the choices of jemalloc vs the system allocator aren't enough and
an entirely new custom allocator is required. In this you'll write your own
crate which implements the allocator API (e.g. the same as `alloc_system` or
`alloc_jemalloc`). As an example, let's take a look at a simplified and
annotated version of `alloc_system`

```rust,no_run
# // only needed for rustdoc --test down below
# #![feature(lang_items)]
// The compiler needs to be instructed that this crate is an allocator in order
// to realize that when this is linked in another allocator like jemalloc should
// not be linked in
#![feature(allocator)]
#![allocator]

// Allocators are not allowed to depend on the standard library which in turn
// requires an allocator in order to avoid circular dependencies. This crate,
// however, can use all of libcore.
#![no_std]

// Let's give a unique name to our custom allocator
#![crate_name = "my_allocator"]
#![crate_type = "rlib"]

// Our system allocator will use the in-tree libc crate for FFI bindings. Note
// that currently the external (crates.io) libc cannot be used because it links
// to the standard library (e.g. `#![no_std]` isn't stable yet), so that's why
// this specifically requires the in-tree version.
#![feature(libc)]
extern crate libc;

// Listed below are the five allocation functions currently required by custom
// allocators. Their signatures and symbol names are not currently typechecked
// by the compiler, but this is a future extension and are required to match
// what is found below.
//
// Note that the standard `malloc` and `realloc` functions do not provide a way
// to communicate alignment so this implementation would need to be improved
// with respect to alignment in that aspect.

#[no_mangle]
pub extern fn __rust_allocate(size: usize, _align: usize) -> *mut u8 {
    unsafe { libc::malloc(size as libc::size_t) as *mut u8 }
}

#[no_mangle]
pub extern fn __rust_deallocate(ptr: *mut u8, _old_size: usize, _align: usize) {
    unsafe { libc::free(ptr as *mut libc::c_void) }
}

#[no_mangle]
pub extern fn __rust_reallocate(ptr: *mut u8, _old_size: usize, size: usize,
                                _align: usize) -> *mut u8 {
    unsafe {
        libc::realloc(ptr as *mut libc::c_void, size as libc::size_t) as *mut u8
    }
}

#[no_mangle]
pub extern fn __rust_reallocate_inplace(_ptr: *mut u8, old_size: usize,
                                        _size: usize, _align: usize) -> usize {
    old_size // this api is not supported by libc
}

#[no_mangle]
pub extern fn __rust_usable_size(size: usize, _align: usize) -> usize {
    size
}

# // only needed to get rustdoc to test this
# fn main() {}
# #[lang = "panic_fmt"] fn panic_fmt() {}
# #[lang = "eh_personality"] fn eh_personality() {}
# #[lang = "eh_unwind_resume"] extern fn eh_unwind_resume() {}
# #[no_mangle] pub extern fn rust_eh_register_frames () {}
# #[no_mangle] pub extern fn rust_eh_unregister_frames () {}
```

After we compile this crate, it can be used as follows:

```rust,ignore
extern crate my_allocator;

fn main() {
    let a = Box::new(8); // allocates memory via our custom allocator crate
    println!("{}", a);
}
```

# Custom allocator limitations

There are a few restrictions when working with custom allocators which may cause
compiler errors:

* Any one artifact may only be linked to at most one allocator. Binaries,
  dylibs, and staticlibs must link to exactly one allocator, and if none have
  been explicitly chosen the compiler will choose one. On the other hand rlibs
  do not need to link to an allocator (but still can).

* A consumer of an allocator is tagged with `#![needs_allocator]` (e.g. the
  `liballoc` crate currently) and an `#[allocator]` crate cannot transitively
  depend on a crate which needs an allocator (e.g. circular dependencies are not
  allowed). This basically means that allocators must restrict themselves to
  libcore currently.