Can you read a project's reliability from its history?
The Linux kernel changes constantly. Across 2019 and 2020 alone its history holds roughly 18,000 commits over 10,000 files, from nearly 7,000 different contributors who range from domain experts to drive-by patchers to the occasional bad actor. So how do you form an honest opinion of whether a codebase like that is reliable? You can’t read all of it, and you can’t vouch for every hand that touched it.
The bet I chased during an internship at SRI, part of a DARPA effort on open-source integrity, was that the history itself is a signal. Not just the code as it stands, but how its APIs get used, and misused, as it evolves.
Code as graphs, over time
The idea is to stop treating source as text and start treating it as graphs of API usage.
- An API usage graph captures how a piece of code uses an API: the calls, and the control and data flow between them.
- A change graph connects two of those, before and after a commit, so you can see exactly how a usage shifted.
- Roll that forward across a project’s history and you get a trajectory of how each API is really used over time.
Misuse detection then becomes a pattern problem. Mine the normal way an API gets used from files you have reason to trust, like the ones nobody has needed to touch in years, then flag the usages that diverge from it. A divergence is a candidate for a bug, a wrong design choice, or something worse.
Making it work on the kernel
The kernel is a hostile target for this. It leans on macros and inline assembly, and its toolchain shifts underneath you as the code and standards move. I built SENSOR to preprocess kernel source down to LLVM IR, which also meant it wasn’t C-only (Rust and Swift come along for free), and wrapped the whole IR-generation step in a configurable container so it could keep pace with the kernel’s moving toolchain. On a real slice of history it flagged an API misuse in a xen grant-table function, which was the encouraging sign that the approach wasn’t only theory. Code.
It was an internship project and never grew into a full paper, so the results are promising rather than proven.