Enjoy but Beware

Regression is an impressive analytical technique that works well where natural laws reliability and consistently influences the topic under study. In my field of engineering, regression analysis and their resulting models are phenomenally successful in analyzing and predicting outcomes. We use regression and models to automatically target processes and to uncover problems. We have recently worked with some supplier firms that will take all the data and measurements collected during the production processes to create self-correcting models that would keep product in spec.

I feel, though, that the technique is over used in financial markets. Armies of statisticians and economist have worked to create predictive models to explain markets, all to no avail. Certainly they have been lured by stories of models that can predict Supreme Court decisions. Talib, the author of Black Swan and Fooled by Randomness, has been very critical of regression analysis on social science topics. Any enthusiast of regression must read his book to understand the limitations of the technique.

But because you work in the physicals science and engineering fields, thoroughly learn the techniques and enjoy the fruits of prediction. To be the best in one’s field, though, take the time to try to explain the fundamental science principles that influence the outcome and give the models validity.