• Faults and Partial Failures

  • Software and hardware are faulty, and failures can happen in a nondeterministic way.
  • Cloud computing → Many things can go wrong.

• Unreliable Networks → If you send a packet, many things can go wrong (lost, waiting in queue, failed, node stopped responding, etc.), and we don’t know if it was delivered. Public cloud providers experience numerous transient network issues. We must be able to handle them.

  • Detecting faults. If the node process failed, but the OS is still running, a script can notify about the crash, etc.
  • Timeouts. There is no simple answer to how long the timeout should be. On the public cloud, choose experimentally.
  • Sync vs. async networks. ISDN is an example of a synchronous network; it runs at a fixed rate of 4000 frames per second when a call is established.

• Unreliable Clocks

  • Some clocks measure durations (the time interval between a request being sent and a response received)
  • This is a problem in distributed systems because communication is not instantaneous, as messages travel across the network.
  • Each machine has its own clock, which is usually not very accurate.
  • Modern computers have monotonic and time-of-day clocks.
    • Time of day returns the current date and time according to the calendar. They are usually synced with NTP.
    • Monotonic is suitable for measuring duration, such as timeouts, etc. (nanoseconds). You check the duration between the two times. They don’t need to be synced.

• Knowledge, Truth, and Lies

  • A node cannot trust its assessment of a situation. A node may think it's the leader, while the other nodes have elected a new one; it may think it's alive, while other nodes have declared it dead. As a result, many distributed algorithms rely on a quorum for making decisions, i.e., decisions require a minimum number of votes from several nodes to reduce dependence on a single node.
  • A system is Byzantine fault-tolerant if it continues to operate correctly even if some of the nodes are malfunctioning and not obeying the protocol, or if malicious attackers are interfering with the network.
  • System Model and Reality. When developing distributed systems algorithms, we need to write them in a way that doesn't depend too heavily on hardware and software details. Essentially, we require an abstraction for what the algorithm can assume and the types of faults that we can expect in a system. This abstraction is known as a system model.

My note: It looks like Fallacies of Distributed Computing, but it doesn’t mention it.