Why is this? The simple answer is that it's an illusion. What is commonly called beauty in nature (or "elegance") - and why physics is called more beautiful than biology - is really just selection bias.
In physics we are looking at very large and very small phenomenon, far from human experience.[1] We can take our time, and we will naturally notice and focus on those phenomena that fit neatly into our cognitive machinery. Collison here notes that physics is beautiful because there are complex phenomena that can be explained with very few parameters - these are the parameters that are easy for us to notice and, our cognition being computationally bounded, we will especially notice those which require fewer parameters. Beauty in nature is really just selection bias: selecting for things which we can understand more easily.[2]
It's obvious why biology seems uglier: it's a complicated mess. But there are lots of messes in understanding reality that we can ignore, whereas we notice biology, not because it fits neatly into our brains, but because we have to - it has more to do with us. If we want to treat diseases, we have no choice but to descend into the massively complex "towers of hacks" that evolution has produced. Yes, it's uncomfortable because of its complexity. But we're forced to pay attention to it, because it's important. Notice that the techniques we use to learn about the towers of hacks are often beautiful. They have to be, for us to understand them.
If natural phenomena exist on a spectrum of beautiful and elegant, to ugly and complex, this is exactly what we should expect to see in our scientific enterprise. The more disconnected a field is from human experience, the more beautiful it will seem on average (because we can pursue those things that fit our cognition.) The more a field has to do with us, the more we will be forced to grapple with complex ugliness, because we can't just pick and choose those things that we notice and that fit. Another way of putting it: in the hard sciences (far from human experience) there is strong selection for things we understand; for the soft sciences, there is not, because these things are relevant to us regardless of their messiness.[3] We could disprove this theory by creating a scatterplot of scientific fields on the axes of elegance-messiness and having to do with human experience vs not; if there are about as many fields that have little to do with human experience, but are about as messy as those that do, that would count against this theory.
It would be nice if beauty and truth had a relationship. But we think they do, only because we're just marveling at the outline of our own reflection.
[1] What we've been able to do with these beautiful principles obviously has impacted the human experience (eg, solid state theory and semiconductors.) But we're talking about nature, not engineering. When first studied in nature, energy levels of electrons had very little to do with conscious human experience.
[2] Addressing the statement that Watson and Crick knew DNA was the heredity molecule because it was beautiful: how many times has a beautiful theory or model or molecule turned out to be tragically incorrect or irrelevant? Many many times, vastly outnumbering the experience of something beautiful being true. You yourself have probably experienced this. We don't hear about those vastly more numerous instances, effectively due publication bias.
[3] Therefore, even in biology and softer sciences, we should still expect occasional accidental gifts of elegant principles: e.g., evolution and DNA.
