Friday, October 4, 2013

Micro Kernel, a Historical Perspective of Distributed Systems

In the 1980's, microkernels were developed as a response to the complexities and flakiness of a monolithic kernel. By introducing modularity and abstraction at the run-time level, the fundamental functionalities of a kernel is broken down into manageable services that are easier to maintain. These distributed services communicate by remote procedure call. Remote procedure call differs from local procedure call in that data dependency of the function call argument has to be made explicit because the two services are in different address spaces, not necessarily sharing the same memory. One way to satisfy data dependency is by copying, but the overhead of copying might be unacceptable if a lot of data is transferred this way. Another approach is the out of band buffer transfer. I have written about these issues of interprocess communication before.

These days, programming languages have the ability to enforce modularity and abstraction at compile time, so each module could be run safely in the same address space even without hardware protection. As long as someone is willing to port a monolithic kernel to a new language (i.e. rewriting it), the initial motivations of the microkernel—modularity and abstraction—are largely addressed. But this is not to say that microkernel has become irrelevant.

Around 1970's, there have been effort to network computers that can talk to one another. In the 80's, it was recognized that a cluster of interconnected computers should work together in parallel in order to speed up computation and scale up the amount of data that could be processed. They coordinate computation, again, by interprocess communication. Scaling by parallelism continued as computers are also made faster, denser, and smaller. Fast forward until today, the speed of a single computer has approached its quantum limit, so processors have gone multi-core, which is to make a processor a package of multiple computing units, each capable of acting as an autonomous computer.

The difference of modern multi-core with the 80's design is the level of abstraction at which interprocess communication takes place. In a modern computer, even when the CPU and the main memory are all distributed nodes, the hardware hides much of this and still presents the illusion of a single computer. But scaling general purpose hardware is much more difficult than scaling software which tends to be domain specific. Domain specific hardware does exist when the economy sustains it, for example the torus network topology is designed for parallelizing convolution problems such as weather simulation, funded by national weather service, and the GPU is designed for graphics rendering, which is an embarrassingly parallel problem, in video games, movies, as well as industrial visualization.

On the one hand, we have single computers that have become a distributed system of multi-core nodes using a vendor-specific interconnect (QPI for Intel and HyperTransport for AMD). Barrelfish is an operating system that has recognized and embraced this in its design. On the other hand, we have cheap system-on-a-chip (SoC) computers like Raspberry Pi that can be built into a cluster of computers over the Ethernet, using some TCP/IP based RPC for distributed computing as back in the 80's. As one of the operating system's purpose is to abstract the underlying hardware, if we were to reconcile the differences of these disparagingly different yet similar architectures, we arrive at a design of distributed processes coordinated through interprocess communication, similar to that of a microkernel.

Even in the 80's through 90's, visionary researchers have foreseen this need, and as a result we have distributed operating systems like Plan 9 (which begets the programming language Go) and Amoeba (which begets Python). Although the distributed operating systems have never been very popular because the hardware running them weren't, now that the hardware is becoming more popular, so will the distributed operating systems.