Path-agnostic binaries, co-installable libraries, and How To Have Nice Things

Eric Myhre

Playlists: 'asg2018' videos starting here / audio / related events

Portability is a shining goal for all software -- an objective since the beginning of computing through the present. And yet, it also remains illusive. We once sought to develop software "portable" between whole OSes and kernels; today, we've lost control of our build and distribution pipelines so completely that portability of a binary between two barely-different distributions of linux is considered radical and nearly impossible. Even "portability" of a binary when moving between two directories on the same host is often a trial by fire. In this talk, we will define and discuss two specific kinds of portability -- what it means to build path-agnostic libraries and what it means to support co-installable libraries -- then explore what this kind of freedom could do for us, and most importantly, practical ways to achieve these goals within existing systems.

Computers are general purpose computers -- they can run any program.

But theory aside, is that really true? Forget tivoization and the rise of walled-garden app stores -- when considering new software, often one of the first questions we ask ourselves as linux users is "Is it available on ${distro|packman}?"... We've already been doing this to ourselves for decades. Source builds are possible, but remain full of "well actually" moments; containerization hints at the sheer popularity of The Dream of escaping this madness; and yet, most of our day to day business is guided and limited by what's available in a package manager, and our machines as a whole are limited by which package manager we've chosen.

Distributions become inherently balkanized: no one can install anyone else's packages with anything short of a full chroot. Why? Nobody *wants* this. The fracturing creeps in from somewhere. Is it something we can fix?

What would it look like if we tried to define packages without distros? Packages without the need for management, beyond drag and drop? Packages with *bring your own* management? Easily relocatable, path-agnostic binaries, and co-installable libraries -- all of them able to work together, or be shipped independently at the user's choice, free of version conflicts in either form?

We can have all these things. And if you think this is going to be a talk about static linking, think again: there are more options, and there are things we can do now, and we can do them within the ecosystems and infrastructure of existing distros non-disruptively.

There are already a variety of approaches to producing relocatable binaries: from static linking, to snapshots and images which require chroots to be executable, to content-addressable (but still absolute-pathed) dynamic linking -- distros and tools using all of these approaches exist. In this talk, we'll survey all of them, plus go beyond into approaches to dynamic linking which are free from absolute paths completely. Finally, we'll compare and contrast all these approaches, map any limitations and assumptions, and try to plot a road ahead.