7e. Functions: Type Inference and Multiple Dispatch

Function's Variable Scope

To fully appreciate why functions in Julia are central to performance, we must first revisit the notion of variable scope. A function's local variables encompass all variables defined within a function's scope, including its arguments and any variable created inside the function body. These variables exist only during the function's execution and are inaccessible from outside. Global variables, on the other hand, refer to any variable defined outside a function's scope and remain accessible throughout the program's execution.

From a practical perspective, a key takeaway from this section is that wrapping code in functions is crucial for high performance in Julia. Instead, relying on global variables imposes a significant performance penalty.

Note that the use of global variables isn't confined to operations in the global scope. It also takes place when a function references variables that aren't passed as arguments. This is why the detrimental effect of global variables will appear in all the following cases.

x     = 2

y     = 3 * x

x     = 2

foo() = 3 * x

Recall that an expression like x = 2 is shorthand for x::Any = 2, reflecting that global variables default to Any if they aren't explicitly type-annotated. Also remember that only concrete types can be instantiated, meaning that values can only adopt a concrete type. Consequently, x::Any shouldn't be understood as x having type Any, but rather that x can take on any concrete type that's a subtype of Any. Since Any sits at the top of Julia's type hierarchy, this simply means that x's types are unrestricted.

Working with a global variable like x prevents specialization of *. The reason is that Julia treats global variables as potentially embodying any value and therefore any type. The underlying logic for this behavior is that, even if a variable holds some value at a specific moment, the user may reassign it at any point in the program. Because of this possibility, Julia must consider multiple possible methods for its computation, one for each possible concrete type of x. In practice, this results in Julia generating code with multiple branches, potentially involving type checks, conversions, and object creations. The consequence is degraded performance.

Even if we had type-annotated x with a concrete type like x::Int64 = 2, the performance limitations wouldn't completely go away. Certain optimizations can only be implemented when both the scope of variables is clearly delimited and values are known. When this is the case, Julia can gain a comprehensive view of all the operations to be performed, creating opportunities for further optimizations.

Functions in Julia were designed to address all these considerations. They achieve this following a series of steps, which we cover next.

Function Calls

Building upon our understanding of function definition and methods, let's now analyze the process triggered when a function is called. In the following, all our explanations will be based on the following function foo:

foo(a, b) = 2 + a * b

In Julia, defining a function like foo(a, b) is shorthand for creating a method with the signature foo(a::Any, b::Any). Thus, the function body foo(a,b) holds for all possible type combinations of a and b.

When foo(1, 2) is called, Julia evaluates the expression 2 + a * b by following a series of steps.

The process begins with what's known as multiple dispatch, where Julia selects which method of a function to execute. Importantly, this decision is based solely on the types of the arguments, not their values. Specifically, Julia begins by identifying the concrete types of the function arguments. In our example where a = 1 and b = 2, both are identified as Int64. The information on types is then used to select a method, which defines the function body and hence the operations to be performed. This process involves searching through the available methods of foo to find the most specific one for the concrete types of a and b. In our example, foo has only one method foo(a,b) = 2 + a * b, which is defined for all argument types, including a::Int64 and b::Int64. Therefore, the corresponding function body is 2 + a * b.

The specific version of this method for the signature foo(a::Int64, b::Int64) is known as a method instance. If the code for the method instance foo(a::Int64, b::Int64) already exists, Julia will directly employ it to compute foo(1,2). Otherwise, the compiler generates optimized code for that method instance, stores it in memory for future use, and Julia executes it.

The following diagram depicts the process unfolded when foo(1,2) is executed.

Multiple Dispatch

The process outlined has implications for how the language works. When a function is called with a particular combination of argument types for the first time, Julia incurs an additional cost because it must generate specialized code for those types. This initial delay is often referred to as Time To First Plot, a phrase that highlights how the compilation overhead becomes noticeable in interactive workflows such as plotting. Once the code has been compiled, however, Julia stores the resulting method instance, so that subsequent calls with the same argument types can reuse it. This eliminates the compilation step and leads to much faster execution.

The example with foo illustrates this process clearly. After evaluating foo(1, 2), Julia has already compiled a method instance for the signature foo(a::Int64, b::Int64). This is why the subsequent call foo(3, 2) can be executed immediately by invoking the cached method instance, without any need for recompilation. Instead, the execution of foo(3.0, 2) introduces a new combination of argument types, where a::Float64 and b::Int64. Because no compiled method instance yet exists for this signature, Julia must generate one before executing the function.

Remarks on Type Inference

Below, we provide various remarks about type inference that are worth keeping in mind for next sections.

Functions Do Not Guarantee Identification of Concrete Types

Merely wrapping code in a function doesn't guarantee that the compiler will identify concrete types for all operations. The following example illustrates this.

x      = [1, 2, "hello"]    # Vector{Any}

foo(x) = x[1] + x[2]        # type unstable

The issue in the example arises because the compiler assigns the type Any to both x[1] and x[2], as they come from an object with type Vector{Any}. As a consequece, the compiler can't specialize the computation of the operation +. The example also highlights that compilation is exclusively based on types, not values. Thus, the generated code ignores the actual values x[1] = 1 and x[2] = 2, which would otherwise indicate a type Int64.

Global Variables Inherit Their Global Type

Type inference is restricted to local variables, with any global variable having its type inherited from the global scope. For instance, consider the following example.

a         = 2
b         = 1

foo(a)    = a * b

a         = 2
b::Number = 1

foo(a)    = a * b

In both examples b is a global variable. Consequently, the compiler infers b's type as Any in the first tab and as Number in the second tab.

Type-Annotating Function Arguments Does Not Improve Performance

Identifying concrete types is crucial for achieving high performance. At first glance, this might suggest that explicitly annotating function arguments is necessary for performance, or at least beneficial. However, thanks to type inference, such annotations are redundant. In fact, adding them can be counterproductive, as they unnecessarily restrict the types accepted by a function, thereby limiting its flexibility and potential applications.

To better appreciate this loss of flexibility, compare the following scripts.

foo2(a, b) = a * b

foo1(a::Float64, b::Float64) = a * b

The function on the first tab only accepts arguments of type Float64, implying that even integer variables are disallowed. By contrast, the function on the second tab mirrors the behavior with Float64 inputs, but additionally allows for other types. This flexibility stems from the implicit type annotation Any on the function arguments.

Packages Commonly Type-Annotate Function Arguments

When inspecting the code of packages, you may notice that function arguments are often type-annotated. The reason for this isn't related to performance, but rather to ensure the function's intended usage, safeguarding against inadvertent type mismatches.

For instance, suppose a function that computes the revenue of a theater via nr_tickets * price. Importantly, the operator * in Julia not only implements product of numbers, but also concatenates words when applied to expressions with type String. Without type-annotations, the function could potentially be misused. This is exemplified in the first tab below, with the second tab precluding this possibility by asserting types.

Unannotated Function Type-Annotated Function

revenue1(nr_tickets, price) = nr_tickets * price

Output in REPL

julia>

revenue1(3, 2)

6

julia>

revenue1("this is ", "allowed")

"this is allowed"

revenue2(nr_tickets::Int64, price::Number) = nr_tickets * price

Output in REPL

julia>

revenue2(3, 2)

6

julia>

revenue2("this is ", "not allowed")

ERROR: MethodError: no method matching revenue2(::String, ::String)