pic
Personal
Website

Conflicting Packages Preventing Updating

Martin Alfaro - PhD in Economics
June 2, 2023

Description of the Issue Covered

Remark
If you stumble upon issues when updating packages, this post guides you on how to handle the upgrade process. Otherwise, this post may not be relevant to you.

Packages in Julia rely on other packages, known as their dependencies. When a package is downloaded, Julia also installs any dependencies required by the package.

To better understand this process, consider two packages, SomePackage and Statistics. Suppose additionaly that SomePackage uses version 1.2 of Statistics as a dependency. If the developers of Statistics released a new version (let's say version 1.3), the authors of SomePackage must update their package to version 1.3 of Statistics. Not doing so will result in Julia impeding the update of Statistics when you execute Pkg.update(), leaving you stuck on version 1.2.

This is a consequence of how Julia updates packages. When you execute Pkg.update(), Julia identifies the oldest versions of each package, including their dependencies. Then, it proceeds to install the oldest version identified. While preventing the update of some package may seem tedious, the aim is to avoid introducing breaking changes in your code. By sticking to the minimum version used as a dependency, Julia ensures that all your projects will still run.

In this context, you have two options. If you need version 1.3 of Statistics, you'll have to remove SomePackage by executing Pkg.rm("SomePackage"). Alternatively, you can use version 1.2. of Statistics, and wait until the developers of SomePackage update their dependencies (well-maintained packages are typically updated within a few days).

The problem with taking the first path is that Julia can be somewhat cryptic about what package is preventing the update. In terms of the example, identifying SomePackage is not trivial, as it may be the dependency of a dependency that is impeding the update. Next, we show how to address this issue.

Miscellaneous (OPTIONAL)

To facilitate the explanation, let me describe a recent experience I had, after not using one of my computers for a long time. I ran Pkg.update() and checked the installed versions on my computer by using Pkg.status(). The results were as follows.

Code

Pkg.status()

Output in REPL

As can be seen, the installed version of DataFrames was not up-to-date, and Julia wouldn't let me update it in isolation. Moreover, I needed DataFrames for the project I was working on, as the version installed was too old. Thus, I needed to i) identify what packages were preventing the update of DataFrames, ii) and remove those packages. To gain more insight into the packages in conflict, we can use the option outdated of the command Pkg.status.

Code

Pkg.status(outdated=true)

Output in REPL

Let's analyze this information with the specific goal of updating DataFrames in mind. The output indicates that the installed version of DataFrames in the system is v0.21.8, whereas the last available version is v1.4.4. Although not important for our purposes, it also reveals that DataFrames is blocking the update of CategoricalArrays. Consequently, once we solve the update of DataFrames, we'll simultaneously solve the issue with CategoricalArrays.

However, this information doesn't indicate what packages are conflicting with the update of DataFrames. To identify this, we'll use Pkg.add, but with an option that specifies the version of DataFrames to be installed (v1.4.4 in this case). This won't allow the update to occur, but it'll reveal the cause.

Code

Pkg.add(Pkg.PackageSpec(; name="DataFrames", version="1.4.4"))

Output in REPL

The output shows that the package TexTables uses DataFrames as a dependency and hasn't been maintained for a while. By removing it, we'll be able to update DataFrames.

Code

Pkg.rm("TexTables")
Pkg.update()

Output in REPL

Notice we executed Pkg.update() rather than Pkg.update("DataFrames"). In this way, other packages that couldn't be updated will be indirectly updated as well. In our case, this made it possible to update CategoricalArrays, which in turn allowed for the update of Distributions and FixedEffectModels.

Conclusion

To avoid dealing with issues of this kind (or at least minimizing them), we should follow the following two steps.

  1. Install in Base only those packages that are essential
This means that we should keep a minimum set of officially supported packages in Base. For the rest, we should create environments.

  1. Create Environments
Julia lets you create what's called a package environment. This allows the user to install a set of packages and versions for each script you're working on. In this way, each project will have its own set of packages and versions, preventing you from accummulating "legacy" packages that are no longer maintained. You can find more information on package environments in Pkg's official documentation.