There is a cemetery in Vienna where the composers draw visitors all year. Beethoven, Brahms, Schubert, a Strauss or two. A short walk from them, in a quieter row, stands a headstone with a bust on top and something carved beneath it that nobody puts on a grave.
An equation.
S = k log W
The man under it is Ludwig Boltzmann, who died in 1906. The popular version of his story has him taking his own life because the scientific world refused to believe in atoms. Historians of science mostly do not accept that account, and the chronology does not help it: Einstein had published on Brownian motion the year before, and the question was closing rather than hardening. What actually happened is simpler and sadder and belongs to him rather than to physics.
The equation is his in substance: he established the relation in a paper of 1877. But the compact form carved on the stone, constant and all, is Planck’s, and it was Planck who named that constant after him.
What the equation says
Take a room full of air. There is a description of it you can fit in a sentence: temperature, pressure, volume. And there is a description of the same room listing where every molecule is and how fast it is going, a number so large it is of no use to anyone.
W is how many of the second kind of description are compatible with one of the first. How many arrangements of the invisible parts would give you exactly this visible room.
S is the logarithm of that count, times a constant.
Which means entropy is not a fluid, a force, or a tendency. It is a number of ways. When a physicist says entropy increases, the claim is that a system drifts toward the descriptions there are more of, for the same reason a shuffled deck comes out disordered. There are vastly more disordered arrangements than ordered ones and nothing is steering.
Now notice what the equation needs in order to mean anything at all. To count arrangements you must have things to arrange. Boltzmann wrote this while the existence of atoms was still being argued about, which makes the carving on the headstone less a tribute than a verdict.
Eighty years later, from a telephone wire
In 1948 Claude Shannon, at Bell Labs, was working on a problem with nothing to do with gases: how much channel you need to carry a message. He arrived at a quantity measuring how much a message narrows down what it could have been, and called it H.
Put the two side by side.
The same function. Not similar, not analogous in spirit. The same mathematics, reached twice, by two people who were not looking at the same thing and one of whom was not born when the other died.
Shannon did not choose the name himself. He told the story to Myron Tribus in April 1961, and Tribus printed it in Scientific American ten years after that. He had thought of calling the quantity information and rejected the word as overused. He settled on uncertainty. Then he mentioned it to John von Neumann, who told him to call it entropy for two reasons: the function already had that name in statistical mechanics, and, more usefully, nobody knows what entropy really is, so in a debate he would always have the advantage.
Nobody knows what entropy really is, so in a debate you will always have the advantage.
The story gets repeated as though someone had minuted it. It is a recollection at two removes, printed a decade after the conversation that recalled it. It is probably true. It is not documented the way its circulation implies, and that matters more here than it usually would, because this anecdote is load-bearing: it is the origin of the word that made everyone assume the two quantities were one thing.
When is a resemblance an identity?
That the two expressions are formally identical is disputed by nobody. What follows from it has been argued for seventy years.
E. T. Jaynes argued from 1957 that the resemblance is not a coincidence at all. On his account, statistical mechanics is properly understood as a form of inference, and thermodynamic entropy is an information measure applied to a physical system. The two entropies are one thing wearing two names, and physicists were doing information theory before it existed.
The opposing view is that a shared form is mathematics doing what mathematics does. The same function turns up in ecology, in population genetics, in linguistics, and nobody concludes from this that a forest is a message.
I have no settlement to offer and would distrust anyone who did. What I want to name is what this essay is actually about, because it is something I keep doing here and have never examined.
Every few weeks in this newsletter I put two things next to each other and notice that they rhyme. An etymology that lines up too well with a piece of engineering. A Sanskrit term for sustained attention that behaves suspiciously like a thermodynamic operation. Each time there is a sentence somewhere saying that the resemblance is not the argument.
Here is the strongest resemblance available anywhere in science. Not a likeness of theme, not a similarity of shape. The same function, arrived at independently, eighty years and an ocean apart, by people with no common problem.
And even that does not settle itself.
What would settle it
So here is a criterion, and it applies to my resemblances at least as much as to this one.
A shared form is not evidence that two things are one thing. It becomes evidence when treating them as one lets you predict something that treating them as two does not.
That is a demand rather than a mood. It says: show me the case where the identity does work. Where assuming that the entropy of a gas and the entropy of a message are the same quantity produces a number you could not otherwise have got, and then go and measure the number.
Jaynes believed he had that case. Whether he did is worth its own essay.
In the meantime, the test I am setting myself, and handing you to hold me to. When I put two things side by side here and they rhyme, ask me what the identity would predict that the separation would not. If I cannot answer, then I have shown you a coincidence and called it a finding, and you should say so.
Earlier in this thread:
The Being With Neat Fingers
On the eleventh of December 1867, James Clerk Maxwell sat down to write to his friend Peter Guthrie Tait. He was thirty six. What he wanted, he told Tait, was to pick a hole in the second law of thermodynamics.
What a Bit Actually Is
On a Saturday evening in February 1946, a new program went out on the Mutual Broadcasting System from a theater on West 48th Street in New York. Four panelists. One hidden object, mailed in by a listener. Twenty questions, each answerable only yes or no. The show was called Twenty Questions, and it ran, in one format or another, for the better part of a…
I believe we share something in our approach to nature, science, and humanity. I also wrote on this really striking chain of thoughts. I believe re-examing some of the things we think we know can help us understand ourselves better and maybe even help us navigate some of the difficulties we find ourselves in.
This is exactly the sort of thing my brain does constantly, although admittedly with considerably less mathematics involved, I am admittedly more linguistically inclined then mathematically inclined.
I make connections between things all the time. A smell reminds me of a place, which reminds me of a person, which reminds me of something I read 20 years ago, and suddenly I've constructed this whole chain in my head. Sometimes there's genuinely something connecting all of it. And sometimes I've apparently just found two things that happen to rhyme.
I really like your test at the end for that reason. "These two things look remarkably alike" is interesting. "Treating them as the same thing actually allows us to discover something we couldn't otherwise" is an entirely different proposition.
Also, putting an equation on your gravestone is one hell of a way to have the last word.