James Sethna > Sloppy Models > Why is science possible?

Why is science possible?

Why is science possible? Why is the world comprehensible? How is it that we can capture the behavior of a cell with thousands of different interacting proteins, RNA, and DNA using models that ignore most of this complexity? Why is it possible to make theories about ecosystems, or the US economy?

Our group has discovered a striking similarity between theories of physics and more general models used to study complex systems such as in systems biology, economics, ecology, climate science, etc.

Over the past decade, we have been studying a wonderful commonality in these more general systems. Their collective behavior depends mostly on a few "stiff' combinations of rules; most other combinations are sloppy, with little or no influence on system behavior.

Parameter Space to Behavior Space
Sloppy models are multiparameter models, whose behavior depends only on a few stiff combinations of parameters, with many sloppy parameter directions largely unimportant for model predictions. We've found them ubiquitous in systems biology and in other fields of science where models are fit to data.

We describe the possible system behaviors as points in a 'behavior' space, and we find that they form a hyper-ribbon, long along stiff combinations and very thin (reflecting unchanging behavior) when sloppy combinations are changed (fig below right). Theories of cell behavior (and presumably ecosystems and economics) don't need to reflect all the details; a useful theory needs only to capture the collective, 'stiff' behaviors.
Hyperribbon structure in Behavior Space
Hyperribbon structure in behavior space, for the diffusion equation after three evolution steps. Note that it is longer than it is wide, and it is very thin -- like a ribbon.
Theoretical physics, in fact, works precisely for this same reason. We don't need to know the shapes and sizes of individual molecules to make a theory of sound waves; only the overall density and compressibility (weight and squeezability) matter. High energy physicists don't need to solve string theory to predict the Higgs boson, or to describe quarks.

Indeed, theoretical physics is like a tree (figures below). Our high-energy colleagues study the limbs of the tree, searching for more unified theories closer to the trunk. We in condensed matter physics build outward, searching for 'emergent' branches and leaves -- effective theories describing sound, semiconductors, and superfluids.

Does physics share the 'sloppy model' behavior we found in other sciences? We generalized our information geometry methods and applied them to two standard physics models -- diffusion and the Ising model (figure at left). The diffusion equation describes how perfume goes from her skin to your nose (if the air is still). It is usually derived as a continuum limit, using methods similar to those we use for describing many other phases and phenomena in condensed matter physics -- sound, magnets, and superconductors. The Ising model of magnetism, which becomes fractal, is usually analyzed using renormalization group similar to methods used in high-energy physics. Our method, different from continuum limits and renormalization group methods, examines different combinations of the microscopic rules, finding the stiff and sloppy directions. The figure at left shows that the diffusion equation and the Ising model share the same spread of stiff and sloppy combinations as do systems drawn from other branches of science. Moreover, the stiff directions directly correspond to the important combinations that the traditional continuum limit and renormalization-group methods identified as the collective, emergent control parameters.

High Energy Physics
High Energy Physics. Theories in high-energy physics form a nested hierarchy. Each theory is derived from a more fundamental, unified theory, describing behavior at higher energy scales (demanding bigger particle accellerators). The unified theory explains key parameters in the derived theory: quantum chromodynamics and the electroweak theory tell you the masses of the nuclei and electron.
Condensed Matter Physics
Condensed-matter physics. Theories in condensed-matter physics form a nested hierarchy. Each theory emerges from a more microscopic and complicated theory 'below' it, providing a simpler and more beautiful description. The emergent theory compresses the microscopic details into a few governing parameters that efficiently describe the behavior at longer distances, longer times, and lower temperatures.

So, physics, systems biology, and presumably much of the rest of science all rely on a kind of information compression about the system rules. The collective system behavior in each case relies on only a few 'stiff' combinations of parameters from the many variables in the full microscopic description. In physics, we have systematic methods for extracting these emergent, collective theories from the microscopic complexity. In other fields, we don't have such tools -- but the theories show the same kind of independence from the microscopic details.

If one needed to extract every detail of the true underlying theory to make a useful theory, science would be impossible. Sloppiness makes science possible.
Sloppy Eigenvalues
Sloppy Eigenvalues. We use our 'sloppy model' analysis tools to study the hierarchy of models in theoretical physics. We find exactly the same parameter compression behavior in physics as we found in systems biology and other fields of science. Only a few 'stiff' combinations of parameters determine the system behavior.

References and Videos

More on sloppiness:

Last Modified: October 23, 2013

James P. Sethna, sethna@lassp.cornell.edu; This work supported by the Division of Materials Research of the U.S. National Science Foundation, through grant DMR-1005479.

Statistical Mechanics: Entropy, Order Parameters, and Complexity, now available at Oxford University Press (USA, Europe).