(This was somewhat hastily written up in response to a question I once got. I may revisit it in the future.)
The operating system
Let’s begin by defining what an operating system is. Off to a good start, right? I realize that it’s not possible to say anything here that can please everyone, but bear with me here.
For our purposes, let’s define an OS as a full suite of software for a computer system with the ultimate goal of turning the machine into something perfectly useful. After all, what good is a computer that only shows a blinking cursor? This software covers all the basics: means for user interaction, utilities for file management and system configuration, and a kernel to provide a base for these and other programs to to run on and share resources.
With this definition, systems such as Solaris, HP-UX, Darwin (macOS) and even Windows are considered to be full-fledged operating systems. The notable exception is Linux, but more on that later. For each of these examples, the entire system - the bootloader, kernel, utilities (file manager, text editor, etc.), and the shell - are developed in tandem by a single entity and then released as one complete product.
There’s only so much one can do with just a minimal operating system installation. Any decent system in this day and age provides a way of quickly and easily installing additional software. If a satisfactory one doesn’t exist, you can bet that a dedicated group of people will rise up to the challenge of making an unofficial one.
The package manager
At its essence, a package management system is responsible for only one thing: managing additional software. The details on how this is accomplished is beyond the scope of this write-up. Given the many package formats and the wide array of package managers out there, we would very quickly get off topic.
A package typically comes as a single file, often a compressed archive. As one would expect, these archives contain the files that make up that particular piece of software. Additionally, they include metadata: the package name, its version, cryptographic proof of who assembled them. They also contain other instructions for the package manager: how to install, how to properly upgrade, which tasks to perform afterwards, and even which other packages it expects to have installed first.
Packages are obtained from repositories. How these repositories are implemented depends on the package manager being used. These days, there exist official repositories hosted by the OS distributor, or by a volunteer who keeps a copy of these files on the distributor’s behalf. Before most users had the luxury of a constant Internet connection, repositories were also held on the very CD-ROMs used to install the base system. Third-party repositories exist as well which can provide software and utilities that the OS distributor may not support for various reasons.
More often than not, external and third-party software is installed in a designated directory, /opt. Should a software package rely on functionality that doesn’t exist in the base system, placing needed features there ensures that potential bugs aren’t accidentally integrated into the base system proper. It avoids “poisoning the well”, so to speak.
Linux and distributions
A.K.A. “Why are package managers such a big deal with Linux?”
Linux is different. It does not perfectly fit into the definition of an operating system that we defined above. To reiterate: OSes are shipped out as one cohesive whole, with everything in the base system being developed entirely by a single entity with the aim to have one consistent release that’s perfectly usable as-is. Linux, however, is just a kernel.
To quote Matt Fuller: “Linux has never had any sort of separation between what is the ‘base system’ and what is ‘addon utilities’. The entire system is ‘addon utilities’.”
Whereas other systems may include package management software for convenience, such a thing is one of the defining components of a Linux distribution.
Authors of these distributions (or distros) take a different approach to building an OS. In place of developing a base system from scratch, all the basic components are sourced from other projects (referred to as “upstream”). These projects are then modified to varying degrees so all components can better suit the goal of that particular distro. It’s for this reason that a package meant for one Linux system may not properly work in another one. A program built for one may have problems running on another due to subtle differences in how the base system has been modified between the two.
An example: Ubuntu has had a reputation for the better part of a decade to depart from traditional user interface standards in an effort to modernize it. Beginning in Ubuntu 9.10 (2009), the notification area was deprecated. In its place, a unified menu system was created as a means to enforce interface consistency (ref). At the time, only this version of Ubuntu used this system. To ensure its immediate use, many third-party applications within Ubuntu’s repositories were modified either slightly or drastically to make full use of it in time for its release.
Conclusion
Package management systems free the user from a the messy and error-prone installation process for external software. They are a powerful yet accessible means of automating the installation of software in such a way that allows the system to remain stable while remaining highly customizable. Given their purpose, it’s little wonder that package management systems are practically a requirement for any computer power user, regardless of their preferred OS.