pic
Personal
Website

7f. Preliminaries on Memory Allocations

Martin Alfaro
PhD in Economics

Introduction

The previous sections explored Julia's type system and function calls, which laid the groundwork for understanding type stability. We now shift our focus to memory allocations, a subject that it's important for performance, although not directly related to type stability. Since and memory allocations could be a symptom of type stability

When working with data, the approach chosen to reserve memory space can significantly impact performance. Specifically, we'll show that memory allocations on the heap, simply referred to as memory allocations, incur a substantial cost. The reason is that managing memory in this way requires the CPU to execute additional instructions.

From a practical standpoint, disproportionate memory allocation is a red flag: when two approaches for an identical operation exhibit big differences in memory allocation, they'll likely differ in execution speed. Despite this, the relationship between memory allocation and performance is complex. In fact, reducing memory allocation is neither necessary nor sufficient for speeding up computations—we'll present instances where allocating more memory turns out to be faster. Basically, the trade-off arises because new objects store data in contiguous blocks of memory, allowing the CPU to access information more efficiently.

Heap vs Stack

User-defined objects are stored in the Random Access Memory (RAM). For its part, the RAM is divided into two logical areas: the stack and the heap. These areas aren't physical divisions of memory, but rather data structures that govern how memory is allocated and deallocated.

Remark
Throughout this website, we'll refer to memory allocations as heap allocations exclusively. This is a common approach adopted in programming, including Julia, as allocations on the heap are those that slow down computations.

Stack Allocations

The stack is known for its speed, thanks to its simple memory management operations. [note] The stack follows a data structure known as Last-In-First-Out (LIFO), meaning that objects are allocated and deallocated in a sequential order. This makes the management of the memory relatively easy. They typically store basic data types such as integers, floats, characters, as well as small fixed-size collections like tuples. Objects on the stack must have a fixed length, thus precluding the possibility of dynamically growing or shrinking in size.

The main limitation of the stack is its limited capacity, making it suitable only for objects containing a few elements. Indeed, attempting to allocate more memory than the stack can accommodate will result in a "stack overflow" error, causing the program to crash. Moreover, even if the stack can fit the object, more than a handful of elements will degrade performance significantly. [note] To reap the benefits of stack allocation, there's no hard and fast rule about how many elements are "too many". Benchmarking code is the only way to determine the performance consequences for each particular case. However, to provide a rough sense, objects with more than 100 elements will certainly suffer from poor performance, while those with fewer than 15 elements will likely benefit from stack allocation.

Heap Allocations

The heap is specifically designed to accommodate larger data structures, allowing objects as large as the available RAM. Additionally, it's capable of handling objects with variable sizes that can grow or shrink. Typical variables stored on the heap are arrays (vectors and matrices) and strings.

Although the heap provides greater flexibility than the stack, it comes at the cost of slower performance due to its more complex memory management. [note] To handle the memory of heap-allocated objects, Julia uses what's known as a garbage collector. This allows Julia to automatically identify and free up memory no longer in use or needed. The operations performed by the garbage collector are costly. As a result, it's recommended to minimize heap allocations. One strategy for achieving this is to rely on the stack whenever possible. An alternative is to reuse heap-allocated objects, which entails favoring the mutation of existing vectors over the creation of new ones.