Google Spanner (warning: PDF) has made some noise recently, and I got around to giving their paper a quick read. I was happy to discover that it describes a solution to a problem I've been thinking about for awhile. I was happy to read that my musing were heading in the right direction. The key (as the Googlers are at pains to point out, and I was also considering) is how to handle distributed co-ordination of commits (in a scalable manner).
Spanner introduces the concept of tracking time by a combination of instants, intervals, and assertions about the two - on the understanding that there's uncertainty of the "current" time. As I read through the paper, I tend to think that there's no particular need for atomic and GPS clocks (although they're definitely desirable); so long as the confidence in the current time can be established. The effect of rising uncertainty in the current time would be a reduction in performance, I imagine...
It turns out that the venerable NTP protocol is able to provide measures of uncertainty; so one could re-implement Spanner without atomic clocks (at least, initially), and could then assault the various questions of electing "leaders" and manging reads and writes in a scalable manner.
Apache Zookeeper is the first thought for the leader election, but they talk about clusters of 4 machines being a large deployment, and about "within this data centre", etc. Let's reserve Zookeeper for a use case that fits better; something without Spanner's global ambitions. Spanner's architecture means that this is a key real concern. A closer reading of the paper is necessary to figure out what's involved here; and how it intimately it's related to uncertain timing.
The distributed reads and writes problem does look a bit like Ceph...
Spanner may not be so intractable, after all.
