The research is using SAT score as a proxy for general intelligence.
Despite a number of statements to the contrary in the various comments here, taking SAT scores as an informative correlate (proxy) of what psychologists call "general intelligence" is a procedure often found in the professional literature of psychology, with the warrant of studies specifically on that issue. Note that it is standard usage among psychologists to treat "general intelligence" as a term that basically equates with "scoring well on IQ tests and good proxies of IQ tests," which is why the submitted article has a point.
"Indeed, research suggests that SAT scores load highly on the first principal factor of a factor analysis of cognitive measures; a finding that strongly suggests that the SAT is g loaded (Frey & Detterman, 2004)."
"Furthermore, the SAT is largely a measure of general intelligence. Scores on the SAT correlate very highly with scores on standardized tests of intelligence, and like IQ scores, are stable across time and not easily increased through training, coaching or practice."
"Numeracy’s effects can be examined when controlling for other proxies of general intelligence (e.g., SAT scores; Stanovich & West, 2008)."
As I have heard the issue discussed in the local "journal club" I participate in with professors and graduate students of psychology who focus on human behavioral genetics (including the genetics of IQ), one thing that makes the SAT a very good proxy of general intelligence is that its item content is disclosed (in released previous tests that can be used as practice tests), so that almost the only difference between one test-taker and another in performance on the SAT is generally and consistently getting all of the various items correct, which certainly takes cognitive strengths.
I still think Stanovich's point is interesting that there are very strong correlations with IQ scores and SAT scores with some of what everyone regards as "smart" behavior (and which psychologists by convention call "general intelligence") while there are still other kinds of tests that plainly have indisputable right answers that high-IQ people are able to muff.
(Disclosure: I enjoy this kind of research discussion partly because I am acquainted with one large group of high-IQ young people
This is a wonderful comment. However, what psychologists know as g is the result of over-interpretation of a descriptive method known as factor analysis (which is very similar to principal components analysis).
Also, the SAT grew from IQ tests, so it really isn't surprising that the two measures are correlated, given that the questions for SAT were probably kept on the test because they correlated with IQ tests.
That being said, the work of Kahneman (And the new paper which I should probably read instead of commenting on HN) is pretty rock solid as far as it goes. It is worth noting that there is a converse position in this field, that of Gerd Gigerenzer who argues that these heuristics exist because they are useful, and only go wrong in artificial situations.
www.cogsci.msu.edu/DSS/2007-2008/Todd/environments_that_make_us_smart.pdf
Personally, I incline far more towards the views of Gigerenzer than those of Kahneman, especially given that Gierenzer and colleagues attempt to model the mind given their theories computationally, which is something that psychology could do with more of.
Full disclosure: I'm a psychologist who's very frustrated with the lack of statistical sophistication and interpretation in my field.
Isn't there also another argument that could be made?
Suppose Y is only weakly correlated to X, Y might still be used to show that Z and X correlated if we can show that Y and Z are not otherwise correlated.
If only some smart people score high on SAT and most who score high on SAT are more easily are easily fooled by certain questions, it might indicate most smart people are easily fooled by these question but you would want to engage in further investigation to be sure...
Despite a number of statements to the contrary in the various comments here, taking SAT scores as an informative correlate (proxy) of what psychologists call "general intelligence" is a procedure often found in the professional literature of psychology, with the warrant of studies specifically on that issue. Note that it is standard usage among psychologists to treat "general intelligence" as a term that basically equates with "scoring well on IQ tests and good proxies of IQ tests," which is why the submitted article has a point.
http://www.iapsych.com/iqmr/koening2008.pdf
"Frey and Detterman (2004) showed that the SAT was correlated with measures of general intelligence .82 (.87 when corrected for nonlinearity)"
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144549/
"Indeed, research suggests that SAT scores load highly on the first principal factor of a factor analysis of cognitive measures; a finding that strongly suggests that the SAT is g loaded (Frey & Detterman, 2004)."
http://www.nytimes.com/roomfordebate/2011/12/04/why-should-s...
"Furthermore, the SAT is largely a measure of general intelligence. Scores on the SAT correlate very highly with scores on standardized tests of intelligence, and like IQ scores, are stable across time and not easily increased through training, coaching or practice."
http://faculty.psy.ohio-state.edu/peters/lab/pubs/publicatio...
"Numeracy’s effects can be examined when controlling for other proxies of general intelligence (e.g., SAT scores; Stanovich & West, 2008)."
As I have heard the issue discussed in the local "journal club" I participate in with professors and graduate students of psychology who focus on human behavioral genetics (including the genetics of IQ), one thing that makes the SAT a very good proxy of general intelligence is that its item content is disclosed (in released previous tests that can be used as practice tests), so that almost the only difference between one test-taker and another in performance on the SAT is generally and consistently getting all of the various items correct, which certainly takes cognitive strengths.
I still think Stanovich's point is interesting that there are very strong correlations with IQ scores and SAT scores with some of what everyone regards as "smart" behavior (and which psychologists by convention call "general intelligence") while there are still other kinds of tests that plainly have indisputable right answers that high-IQ people are able to muff.
(Disclosure: I enjoy this kind of research discussion partly because I am acquainted with one large group of high-IQ young people
http://cty.jhu.edu/set/
and am interested in how such young people develop over the course of life.)