What reasoning predicts
Employers test reasoning because, across a century of studies, general reasoning ability has been one of the more reliable predictors of how well people learn and perform at work. How reliable is now genuinely disputed. This lesson gives you both sides with their numbers, so you can take the tests seriously without treating a score as a verdict on you.
The classic claim: .51
In 1998 Frank Schmidt and John Hunter published a summary of 85 years of personnel-selection research. Their headline was that general mental ability, the broad factor that reasoning tests tap most directly, predicted job performance with a validity of .51. That put it level with the best methods known and, crucially, it worked for almost any job and did not require the candidate to have experience. They also estimated that adding an integrity test lifted the combined validity to .65, and adding a structured interview to .63.
Those numbers shaped hiring for two decades. They are one reason high-volume graduate recruiters put an aptitude screen at the front of the funnel. A 2016 working paper by Schmidt, Oh and Shaffer went further, putting the validity of general mental ability at .65 after a newer correction method.
The 2022 revision: .31
Then Paul Sackett, Charlene Zhang, Christopher Berry and Filip Lievens re-examined how those figures had been produced. Their target was the correction for range restriction.
A validity study can only measure the job performance of people who were hired. If the hired group was selected partly on the test, its test scores span a narrower range than the applicant pool, and a narrow range shrinks any correlation. Meta-analysts correct for this. Sackett and colleagues argued that the standard corrections were applied to studies that had barely been affected by selection at all, mostly concurrent studies of existing employees, and so systematically inflated the estimates.
With corrections they considered defensible, the validity of cognitive ability tests fell from .51 to .31. Most other predictors fell too, by .10 to .20 points, and the ordering changed.
Read the revision carefully, because it is often summarised as "aptitude tests don't work". That is not what it says.
- Cognitive ability is still in the top five of roughly two dozen predictors reviewed. Sackett and colleagues describe the change as a reframing, not a dismissal: structured interviews might now be treated as the focal predictor, with others judged against them.
- The strongest predictors in the revision (structured interviews, job knowledge, biodata, work samples) are all job-specific. They need something to be specific about.
- The authors point out the practical asymmetry that matters most to you:
some predictors (e.g., work samples, knowledge tests) require a selection scenario in which candidates are expected to have prior training and/or experience, while others (e.g., cognitive ability tests) can be used with untrained individuals who will acquire job-specific KSAs on the job or in post-hire training.
That is precisely the campus-hiring situation. A company recruiting thousands of engineering graduates for roles they will be trained into cannot run a work sample on skills nobody has yet, and cannot interview everyone in depth. A reasoning screen is cheap, standardised, and still predictive. The debate changes how much weight a thoughtful employer should put on it. It does not change why the screen is there.
Learning and school
The associations with learning outcomes are larger than those with job performance.
- School grades. Roth and colleagues pooled 240 samples covering 105,185 students and found a corrected correlation of .54 between intelligence and school grades. The size varied by subject and school stage.
- National exams. Deary, Strand, Smith and Fernandes followed more than 70,000 English children from age 11 to their GCSE exams at 16. The correlation between a latent general-intelligence factor and a latent exam-achievement factor was .81. General ability accounted for 58.6 per cent of the variance in mathematics results and 18.1 per cent in art and design.
- Training at work. Schmidt, Oh and Shaffer's 2016 update reports an average validity of .67 for general mental ability in predicting performance in job training programmes. This is a working paper from the same school of thought that the 2022 revision challenged, so treat it as the upper end of the range.
Life outcomes, and the limits
Strenze meta-analysed longitudinal studies, where intelligence was measured years before the outcome. The corrected correlations were .56 with educational attainment, .43 with occupational level and .20 with income.
The same paper is a useful check on overclaiming. Strenze concluded that intelligence is a powerful predictor of success but, on the whole, not an overwhelmingly better one than parental socioeconomic status or school grades. Reasoning ability is one strong input among several.
And it is not fixed. Ritchie and Tucker-Drob meta-analysed natural experiments, such as changes in school-leaving age, covering more than 600,000 people. They found that each additional year of education raised intelligence test scores by about 1 to 5 IQ points, with effects that persisted into later life. Your reasoning score reflects your education and practice, not only a fixed endowment.
What the numbers mean for you
- Expect a reasoning screen early. Its job is to shortlist at scale. Clearing the cut-off matters more than topping it, because later stages (technical rounds, interviews) do the fine sorting.
- Structured interviews now rank highest. Many campus processes follow the screen with technical and HR interviews. Reasoning shows there too, when you explain how you would debug something or break down a problem out loud.
- The screen measures a trainable performance. Format familiarity, strategy and pacing all move your observed score, even though the underlying ability changes slowly. Lesson 4 covers what does and does not improve.
Key points
- Schmidt and Hunter put the validity of general mental ability for job performance at .51; Sackett and colleagues revised it to .31 after questioning range-restriction corrections.
- In the revision, structured interviews (.42) rank first, and cognitive ability is still in the top five.
- Cognitive tests remain the practical tool for hiring untrained candidates who will be trained on the job, which is the fresher's situation.
- Reasoning ability predicts school grades (.54) and national exam results (.81 at the latent level) more strongly than job performance.
- It predicts education, occupation and income, but not overwhelmingly better than family background or grades, and schooling raises it.
Next, where reasoning is tested: the exact sections and formats you will meet.
Sources
- Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. doi:10.1037/0033-2909.124.2.262
- Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. doi:10.1037/apl0000994
- Schmidt, F. L., Oh, I.-S., & Shaffer, J. A. (2016). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings (Working paper). doi:10.13140/RG.2.2.18843.26400
- Roth, B., Becker, N., Romeyke, S., Schäfer, S., Domnick, F., & Spinath, F. M. (2015). Intelligence and school grades: A meta-analysis. Intelligence, 53, 118–137. doi:10.1016/j.intell.2015.09.002
- Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement. Intelligence, 35(1), 13–21. doi:10.1016/j.intell.2006.02.001
- Strenze, T. (2007). Intelligence and socioeconomic success: A meta-analytic review of longitudinal research. Intelligence, 35(5), 401–426. doi:10.1016/j.intell.2006.09.004
- Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological Science, 29(8), 1358–1369. doi:10.1177/0956797618774253
- Society for Industrial and Organizational Psychology. Is cognitive ability the best predictor of job performance? New research says it's time to think again. TIP. siop.org