Kaluza-Klein Theory — String Theory's Predecessor

One of the earliest attempts to unify gravity and electromagnetic forces came in the form of Kaluza-Klein theory, a short-lived theory that again unified the forces by introducing an extra space dimension. In this theory, the extra space dimension was curled up to a microscopic size. Though it failed, many of the same concepts were eventually applied in the study of string theory.
Einstein’s theory had proved so elegant in explaining gravity that physicists wanted to apply it to the other force known at the time — the electromagnetic force. Was it possible that this other force was also a manifestation of the geometry of space-time?
In 1915, even before Einstein completed his general relativity field equations, the British mathematician David Hilbert said that research by Nordstrom and others indicated “that gravitation and electrodynamics are not really different.” Einstein responded, “I have often tortured my mind in order to bridge the gap between gravitation and electromagnetism.”
One theory in this regard was developed and presented to Einstein in 1919 by German mathematician Theodor Kaluza. In 1914, Nordstrom had written Maxwell’s equations in five dimensions and had obtained the gravity equations (see the section “Pulled in another direction: Einstein’s competition for a theory of gravity”). Kaluza took the gravitational field equations of general relativity and wrote them in five dimensions, obtaining results that included Maxwell’s equations of electromagnetism!
When Kaluza wrote to Einstein to present the idea, the founder of relativity replied by saying that increasing the dimensions “never dawned on me” (which means he must have been unaware of Nordstrom’s attempt to unify electromagnetism and gravity, even though he was clearly aware of Nordstrom’s theory of gravity).
In Kaluza’s view, the universe was a 5-dimensional cylinder and our 4-dimensional world was a projection on its surface. Einstein wasn’t quite ready to take that leap without any evidence for the extra dimension. Still, he incorporated some of Kaluza’s concepts into his own unified field theory that he published and almost immediately recanted in 1925.
A year later, in 1926, Swedish physicist Oskar Klein dusted off Kaluza’s theory and reworked it into the form that has come to be known as the Kaluza-Klein theory. Klein introduced the idea that the fourth space dimension was rolled up into a tiny circle, so small that there was essentially no way for us to detect it directly.
In Kaluza-Klein theory, the geometry of this extra, hidden space dimension dictated the properties of the electromagnetic force — the size of the circle, and a particle’s motion in that extra dimension, related to the electrical charge of a particle. The physics fell apart on this level because the predictions of an electron’s charge and mass never worked out to match the true value. Also, many physicists initially intrigued with the Kaluza-Klein theory became far more intrigued with the growing field of quantum mechanics, which had actual experimental evidence (as you see in topic 7).
Another problem with the theory is that it predicted a particle with zero mass, zero spin, and zero charge. Not only was this particle never observed (despite the fact that it should have been, because it’s a low-energy particle), but the particle corresponded to the radius of the extra dimensions. It didn’t make sense to add a theory with extra dimensions and then have a result be that the extra dimensions effectively didn’t exist.
There is another (though less conventional) way to describe the failure of Kaluza-Klein theory, viewing it as a fundamental theoretical limitation: For electromagnetism to work, the extra dimension’s geometry had to be completely fixed.
In this view, tacking an extra dimension onto a theory of dynamic space should result in a theory that is still dynamic. Having a fifth dimension that’s fixed (while the other four dimensions are flexible) just doesn’t make sense from this point of view. This concept, called background dependence, returns as a serious criticism of string theory in topic 17.
Whatever the ultimate reason for its failure, Kaluza-Klein theory lasted for only a short time, although there are indications that Einstein continued to tinker with it off and on until the early 1940s, incorporating elements into his various failed unified field theory attempts.
In the 1970s, as physicists began to realize that string theory contained extra dimensions, the original Kaluza-Klein theory served as an example from the past. Physicists once again curled up the extra dimensions, as Klein had done, so they were essentially undetectable (I explain this in more detail in topic 10). Such theories are called Kaluza-Klein theories.