Skip to content
Nobody Is Awake All fifty notes

When it breaks

When It Breaks an Hour Before the Handover

An incident opening near the end of a working day is the most expensive event on a team with no shared hour, and the cost is set in the last twenty minutes.

What was being doneElapsed
An incident handed over liveWedThu1 night
A decision only one half can makeThuSat2 nights
Waking somebody at 03:00Neverno relay
Somebody awake and working on itOne night: nobody awake who can move itCannot be done by relay at all

three things · three nights · one that no number of nights settles

Four shapes, and the last is a decision about people rather than about systems.

Home / When it breaks

Reference

Four incidents that started near the end of a working day, and what each cost

What was being doneElapsed
Started at 16:40, still open at 17:30WedThu1 night

Handed over live, with the diagnosis half finished and the next step named

Started at 17:05, still open at 17:30ThuSat2 nights

Handed over with no diagnosis at all; the other end restarted the investigation

Somebody awake and working on itOne night: nobody awake who can move itCannot be done by relay at all

two things · three nights

Four incidents, three nights, and 25 minutes of investigation separate the third row from the fourth. An incident handed over with a hypothesis moves; an incident handed over as raw symptoms is a new incident at the other end. One team's own month, logged by whoever was on at the time.

Things break at inconvenient times, and on a team with a gap there is one particularly inconvenient time: the last hour before everybody at one end stops.

What happens in that hour determines whether the other half inherits a problem that is moving or a problem that has to be started again.

Why this hour is different

An incident at eleven in the morning has the whole of a working day in front of it. Most are resolved by the people who found them.

An incident at twenty past four has fifty minutes, and then it crosses to people who did not see it start, do not know what has been tried, and are arriving at the beginning of their own day with their own plans.

The transfer is not like a handover of work. Work has a state that can be inspected. An incident has a state that exists mostly in the head of whoever has been looking at it, and that state is the expensive thing to lose.

The first decision: stay or hand over

This should be made explicitly and early, not discovered at 17:25.

Staying means the person who found it holds it past the end of their day until it is resolved or until there is something solid to pass. This is appropriate when the thing is close to resolution, when the diagnosis is nearly there, or when the person at the other end would have no way in.

Handing over means stopping at a defined point and writing. This is appropriate when the problem is understood, when the next steps are mechanical, or when the person holding it is too tired to be useful — which is a real condition and is better admitted than pushed through.

What produces bad outcomes is neither: working on until the last minute intending to resolve it, failing, and then writing four lines in a hurry.

The twenty minutes that matter

Whichever choice is made, the last twenty minutes before the handover go on the transfer rather than on the problem.

That feels wrong. There is a strong pull to spend every remaining minute investigating, and the investigation is more urgent than the writing. It is also, at that point, less valuable: twenty more minutes of investigation by somebody who has had the problem for an hour is worth considerably less than twenty minutes spent ensuring the next person does not start from zero.

The useful framing is that the incident now has two people working on it, in sequence, and the handover is the only part either of them controls.

What has to be written

Five things, in this order, because the order is the order of usefulness if the reader only gets partway.

What is wrong, in observable terms. What a user or a monitor sees, not what you think is causing it.

What is affected and what is not. The second half of that sentence matters as much as the first and is usually omitted.

What you have ruled out. Of the five this one carries the most and is omitted the most often. Each hypothesis already eliminated is an hour the next person will not spend eliminating it again.

Your current best guess, labelled as a guess.

What you would do next.

  1. What is wrong, in observable terms.
  2. What is affected, and what is not.
  3. What you have ruled out.
  4. Your current best guess, labelled as a guess.
  5. What you would do next.

The hypothesis is the thing

The difference between a one-night incident and a two-night incident is almost always whether a hypothesis crossed with it.

Raw symptoms arriving at the other end produce a fresh investigation, and the fresh investigation repeats most of the first one — the same logs, the same first three checks, the same dead ends — because there is no record of what was already done.

A stated hypothesis, even a wrong one, gives the next person somewhere to start and something to disprove. Disproving a wrong hypothesis is fast and informative. Starting from nothing is neither.

This is why the last twenty minutes are better spent getting to a guess than to a conclusion.

Telling people it is coming

The warning is not optional here.

Somebody at the other end needs to clear their morning, and if they discover the incident when they wake, the clearing happens an hour later than it should and after they have already started something.

Sending "this may still be open when you start" as soon as it looks plausible — not when it becomes certain — costs nothing when the problem resolves and buys an hour when it does not.

The part that cannot be fixed

Some of this cost is structural and no practice removes it.

An incident that spans a handover is worked on by two people who never speak, each holding half the picture, with fifteen hours between them. That is worse than one person holding it throughout, and it will stay worse.

The achievable goal is not to make the split free. It is to make the second person's first hour productive rather than repeated, and that is decided entirely by twenty minutes of writing at the end of somebody's long day.