Linear modeling has had us searching for coefficient parameter estimates with occasional interaction terms for decades. Here I show the poverty of the polynomial with a lesson for over 90% of science.
I have a feeling that almost all (if not almost all, then all) public health officials - who are merely political figureheads, not medical or mathematical geniuses - do not even have the capacity to understand the import of this issue. Or do they understand its importance well enough to realize that their job is convincing the public that it doesn't exist?
The problem in academia is that there's not enough of a feedback mechanism. During my six years of engineering school, there was one and only one professor who made the biggest contribution to the skills I would eventually need in the industrial world. The differentiator is that he was the one and only professor who indeed worked twenty years in that industrial world. He taught science as it was applied in the real world, connecting theory with reality in a way the other teachers couldn't. His students learned many tricks of trade that he had learned along the way.
This is not to speak badly of my other engineering professors. To a person, they were either good or very good at teaching the university curriculum. Problem was, neither the professors nor the curriculum had the ability to differentiate what was important from what was not.
The value of the academic environment is that once freed from everyday constraints, some useful and ultimately productive out-of-the-box ideas emerge from time to time. There is indeed a value in that,
But the typical student is more interested in finding a job following graduation. And the "hit rate" on crazy concept variants escaping from the university environment these days seems to far outweigh the number of concepts practical and useful to graduates and the outside environs.
If I could make one change to the university system, I would require ten years of real world experience to qualify for an academic professorship. If you wish to improve upon the real world in any way, shape, or form, you might at least learn the hardheaded realities of what you're dealing with.
Courage is even less common then sense. This is the result, in my neanderthal thinking, of decades of trying to engineer the danger out every aspect of life. We have diluted and disfigured primal urges and instincts and as a result have accreted profound tinder ready to be sparked.
This reminds me of the toolbox analogy. We might have a wrench in our toolbox and so go to try to tinker with something we don’t fully comprehend; we start to turn nuts and bolts, tighten and loosen, in an exploration. Our first trials are successful—we’re able to make some observable improvements based on our tinkering. Unfortunately, this minor success leads us to believe we can fix all sorts of things with our wrench, and so we start using it wherever we can, even when it becomes obvious that it isn’t the right tool for the problem at hand. Sill not fully comprehending what we’re working on, we might force things, misusing the tool, until confronted by our hubris, we break something and make the situation worse.
I am not a scientist and your article was way over my head but the intuitive brain sat in the back of my head and nodded and I felt myself thinking -"I don't know how to verbalise what I just imbibed but I know it was VERY important".
I did come away with a critical insight though - the funding that sits above science has it captured and idealised outcomes although realised do not always come to pass.
This is related to something I’ve been thinking of writing about — signals. I haven’t yet written it because I’m still contemplating. So some of this is still being formed in my mind.
Science is currently focused on extracting signals from noise, then making inferences, all the while ignoring all the things contributing to the formation of that signal. Consider, for example, some seed that might sometimes be useful in treating certain sicknesses. Scientists will first scoff at the idea. Then someone will think to (metaphorically) “open” the seed and determine what it’s made of. Then they find many components/compounds, but one in particular stands out. So they decide to isolate and test that signal by synthesizing it, then administering the synthetic form, and measure safety and efficacy. The pill will have side effects that the seed doesn’t have. Perhaps, we should instead use those resources to determine when the seed is useful, for whom, under what conditions, etc. Then give the seed as the medicine if/when conditions make sense. Synthetic forms should only be used where potency is a relevant aspect.
To a mathematician, nothing you’ve written here is surprising. I don’t mean to be condescending, but this phenomenon is well understood in mathematics. The data that the scientist sees look like A signal, but may be composed of several signals. The examples you showed as startling are simply showing how one constituent signal can dominate, but still be influenced by interacting with other smaller signals (one example showed that many similar signals can overwhelm a contradictory signal). Scientists have a tendency to aim for the dominant signal and call it a day. I applaud you for calling that into question in this way!
Some technical notes that will probably just make me look like a jerk. But I can’t not say these things. Sorry.
- it appears you forgot a “+b” in your hyperplane Y-hat. But it’s safe to ignore for your discussion.
- if the powers aren’t nonnegative integers, they’re not polynomial functions but power functions.
- polynomial approximations are notoriously bad, while biologists are beginning to make use of power functions (a pop culture source would be Scale by Geoffrey West (spelling?)).
Additionally, I don’t know if you’ve thought about this, but you’re sort of circling around the related topic of Fourier Series/Analysis, which decomposes signals into constituent waves. None of the waves by themselves looks particularly meaningful, but when selectively combined, they can be made into any shape. 3Blue1Brown has some excellent videos about this on YouTube. I’m sure you could find others that have done it, or devise ways for yourself, to decompose a signal into constituent power functions in a similar manner. But there’d be a lot to work through.
I have a feeling that almost all (if not almost all, then all) public health officials - who are merely political figureheads, not medical or mathematical geniuses - do not even have the capacity to understand the import of this issue. Or do they understand its importance well enough to realize that their job is convincing the public that it doesn't exist?
The problem in academia is that there's not enough of a feedback mechanism. During my six years of engineering school, there was one and only one professor who made the biggest contribution to the skills I would eventually need in the industrial world. The differentiator is that he was the one and only professor who indeed worked twenty years in that industrial world. He taught science as it was applied in the real world, connecting theory with reality in a way the other teachers couldn't. His students learned many tricks of trade that he had learned along the way.
This is not to speak badly of my other engineering professors. To a person, they were either good or very good at teaching the university curriculum. Problem was, neither the professors nor the curriculum had the ability to differentiate what was important from what was not.
The value of the academic environment is that once freed from everyday constraints, some useful and ultimately productive out-of-the-box ideas emerge from time to time. There is indeed a value in that,
But the typical student is more interested in finding a job following graduation. And the "hit rate" on crazy concept variants escaping from the university environment these days seems to far outweigh the number of concepts practical and useful to graduates and the outside environs.
If I could make one change to the university system, I would require ten years of real world experience to qualify for an academic professorship. If you wish to improve upon the real world in any way, shape, or form, you might at least learn the hardheaded realities of what you're dealing with.
Courage is even less common then sense. This is the result, in my neanderthal thinking, of decades of trying to engineer the danger out every aspect of life. We have diluted and disfigured primal urges and instincts and as a result have accreted profound tinder ready to be sparked.
For us dummies, if you could add what GLM's means as well, thanks for your rationalisms, JLW.
Linear modeling and polynomial are racist and bigoted!
(We can model to produce any outcome we want! Until we can pay bills with model outputs, I diss all models - except those sexy kind!)
This reminds me of the toolbox analogy. We might have a wrench in our toolbox and so go to try to tinker with something we don’t fully comprehend; we start to turn nuts and bolts, tighten and loosen, in an exploration. Our first trials are successful—we’re able to make some observable improvements based on our tinkering. Unfortunately, this minor success leads us to believe we can fix all sorts of things with our wrench, and so we start using it wherever we can, even when it becomes obvious that it isn’t the right tool for the problem at hand. Sill not fully comprehending what we’re working on, we might force things, misusing the tool, until confronted by our hubris, we break something and make the situation worse.
I am not a scientist and your article was way over my head but the intuitive brain sat in the back of my head and nodded and I felt myself thinking -"I don't know how to verbalise what I just imbibed but I know it was VERY important".
I did come away with a critical insight though - the funding that sits above science has it captured and idealised outcomes although realised do not always come to pass.
James I hope your ideals come to pass.
This is related to something I’ve been thinking of writing about — signals. I haven’t yet written it because I’m still contemplating. So some of this is still being formed in my mind.
Science is currently focused on extracting signals from noise, then making inferences, all the while ignoring all the things contributing to the formation of that signal. Consider, for example, some seed that might sometimes be useful in treating certain sicknesses. Scientists will first scoff at the idea. Then someone will think to (metaphorically) “open” the seed and determine what it’s made of. Then they find many components/compounds, but one in particular stands out. So they decide to isolate and test that signal by synthesizing it, then administering the synthetic form, and measure safety and efficacy. The pill will have side effects that the seed doesn’t have. Perhaps, we should instead use those resources to determine when the seed is useful, for whom, under what conditions, etc. Then give the seed as the medicine if/when conditions make sense. Synthetic forms should only be used where potency is a relevant aspect.
To a mathematician, nothing you’ve written here is surprising. I don’t mean to be condescending, but this phenomenon is well understood in mathematics. The data that the scientist sees look like A signal, but may be composed of several signals. The examples you showed as startling are simply showing how one constituent signal can dominate, but still be influenced by interacting with other smaller signals (one example showed that many similar signals can overwhelm a contradictory signal). Scientists have a tendency to aim for the dominant signal and call it a day. I applaud you for calling that into question in this way!
Some technical notes that will probably just make me look like a jerk. But I can’t not say these things. Sorry.
- it appears you forgot a “+b” in your hyperplane Y-hat. But it’s safe to ignore for your discussion.
- if the powers aren’t nonnegative integers, they’re not polynomial functions but power functions.
- polynomial approximations are notoriously bad, while biologists are beginning to make use of power functions (a pop culture source would be Scale by Geoffrey West (spelling?)).
Additionally, I don’t know if you’ve thought about this, but you’re sort of circling around the related topic of Fourier Series/Analysis, which decomposes signals into constituent waves. None of the waves by themselves looks particularly meaningful, but when selectively combined, they can be made into any shape. 3Blue1Brown has some excellent videos about this on YouTube. I’m sure you could find others that have done it, or devise ways for yourself, to decompose a signal into constituent power functions in a similar manner. But there’d be a lot to work through.
Thanks for this excursion!