This post from John Whitehead at the Environmental Economics Blog brought to mind concerns that I have had for years about statistical packages as black boxes. From his post:
---------
The more troubling thing is the difference in the Limdep and Stata coefficient estimates. The consumer surplus for each seafood meal is about $27 (1/.0372) with Limdep and $1.72 (1/.580) with Stata. This would seem to have policy implications. Any ideas on why the coefficient estimates differ so much?
---------
In our first econometrics classes at the University of Michigan Ph.D Economics program in 2000/2001, in addition to unending theory and proof-making, we wrote our own estimation code in, I think, Matlab, rather than using canned routines or canned packages like Stata. The educational purpose was to reinforce exactly where the estimates come from. Since I had already learned Stata as an undergraduate, I thought, why write all this code when we don't have to? Another lesson, though, that has stuck with me (even as I continue to use Stata almost exclusively in my research) is that when we use statistical packages we usually have no idea what the underlying code is for the estimation routines and just assume it to be accurate. One wonders, especially when seeing situations like that above, if that is a good assumption! How much error have various packages built up in our literature?
This is why R (http://www.r-project.org/ ) is becoming the standard for serious applied statistical analysis. It is open source, uses a reasonably easy to learn scripting language, and provides a script record of analysis and output for archiving. Anyone can create and contribute a new R application, but for it to be included by R-project it has to go through review.
ReplyDeleteIn teaching and research, we should be moving to a an open-source scripted statistical platform: Matlab or R based.