How to get better

What training does not do

The brain-training industry has promised that practising cognitive games makes you smarter in general. In 2016 a team of seven psychologists led by Daniel Simons reviewed the evidence. Their conclusion, as the Association for Psychological Science summarised it: brain-training tasks "seem to improve performance on the trained tasks themselves", there is "less evidence that cognitive training improves performance on closely related – but not identical – tasks", and "very little evidence that it improves performance on distantly related tasks or improves everyday cognitive performance".

Later meta-analyses point the same way.

• Melby-Lervåg, Redick and Hulme pooled 87 publications on working-memory training. Trained people improved on working-memory tasks, but for far-transfer measures, including nonverbal ability of the kind reasoning tests measure, there was "no convincing evidence of any reliable improvements" against properly treated control groups.
• Sala and colleagues ran a meta-analysis of 14 earlier meta-analyses, covering nearly 22,000 participants. The benefits of cognitive-training programmes "hardly go beyond the trained task and similar tasks".
• A dissenting meta-analysis by Au and colleagues, limited to n-back training, found a small but significant gain in fluid intelligence. It is worth knowing; it is also the minority view, and the effect is small.

So be clear about what our games are for. They are not a promise to raise your IQ. They are close relatives of the tasks you will actually face, which puts them in the near-transfer ring where practice does help.

What does move your score

Familiarity with the format. Hausknecht and colleagues meta-analysed 107 samples and 134,436 people who took cognitive ability tests more than once. Scores rose on retest by an adjusted effect of .26 standard deviations, and the gain was larger when practice came with coaching and when the same test form was reused. A quarter of a standard deviation is roughly the gap between the middle of a cohort and the 60th percentile. You get most of it simply by not meeting the format for the first time on the day.

Strategy. This is the most underrated lever. Gonthier and Thomassin gave 250 people half of Raven's Advanced Progressive Matrices. One group was told to study the matrix first, work out the rules, picture the missing piece, and only then look at the options, a strategy called constructive matching. The matrix was even shown alone for at least 15 seconds before the options appeared. That group averaged 12.77 of 18 correct against 10.45 for the uninstructed group. The gain was concentrated in people with lower working-memory capacity: for them, predicted scores rose from about 8.6 to about 12.1.

Explicit training in inductive comparison. Klauer and Phye reviewed 74 experiments with nearly 3,600 children taught a procedure for inductive problems: compare items attribute by attribute, and look for what is shared, what differs and what changes in order. Training improved both fluid-intelligence test scores and classroom learning, and the effect on intelligence tests grew over time. The studies were with children, so do not expect the same size in adults, but the procedure is exactly the one taught in the game lessons that follow.

Education. Over longer timescales, each year of education raises intelligence test scores by roughly 1 to 5 points. Reading, mathematics and technical study are the slow, real version of reasoning practice.

Six strategies that work on every reasoning item

1. Classify the item before solving it

   Is it deduction (the answer is forced) or induction (find the rule that fits every case)? The standard of proof tells you when you are done: deduction is done when nothing else is possible; induction is done when your rule fits every case shown and rules out every wrong option.

2. Build the answer before you look at the options

   Constructive matching. In a matrix, name every rule and picture the missing panel. In a sorting task, state the rule in words before you touch a card. Options are designed to look plausible; your own prediction is not.

3. Go one attribute at a time

   Shape, then count, then fill, then position. Carpenter and colleagues found that hard items are hard because several rules change at once, and the people who solve them handle one rule at a time and keep track of which ones they have checked.

4. Try the simplest rule first

   Feldman showed that the difficulty people have learning a logical concept is proportional to the length of its shortest description. Test designers start simple too. Check single-attribute rules before combinations, and only reach for "A or B" or "A but not B" when every simple rule has failed.

5. Hunt for the case that breaks your rule

   Wason's 2-4-6 finding is the warning: people test cases their rule predicts will fit, and miss that a broader rule fits too. Before committing, look for the example your rule gets wrong.

6. Decide your walk-away time in advance

   In a fixed-length section, one stubborn item can cost you three easy ones. Set a limit, roughly one and a half times the average time per question, then mark it and move on.

How our scoring rewards good method

The reasoning games are scored in a way that matches these strategies. A correct answer counts far more than a fast one: for most reasoning games, accuracy carries 85 per cent of the per-trial weight and speed relative to the expected time at that difficulty carries 15 per cent. Thinking three times longer than expected costs you at most about 15 per cent of a trial's value; getting it wrong costs all of it. Difficulty adapts: in practice sessions a correct answer moves the next trial up a level and a wrong answer or a skip moves it down, so you spend most of your time near the edge of what you can do. Your score of 50 means you perform at the level your working difficulty predicts.

A four-week plan

Twenty to thirty minutes a day, five days a week, is enough. Spread practice beats one long weekend session.

1. Week 1: learn the formats
   • Read the game lessons for Matrix Logic and Symbol Grid, then do one session of each daily
   • Add one session of Rule Learning or Rule Sorter on alternate days
   • After every session, write down the rule you missed most often
   • No timing pressure: aim for accuracy above 80 per cent
2. Week 2: install the method
   • Matrix Logic: say every rule out loud before looking at the candidates
   • Symbol Grid: always list the target's candidates before scanning anything else
   • Rule Sorter and Rule Learning: state the rule in words before sorting or declaring it
   • Add RuleMatch and ColorCode once each for variety
3. Week 3: breadth and pace
   • Rotate all four facets: Matrix Logic, Rule Learning or Rule Sorter, Symbol Grid, Numerosity
   • Keep one Object Sorting or SwapCode session for flexibility and deduction under change
   • Notice your level: if it has stalled, go back to the method lesson for that game
   • Do one session a day when tired, because test days rarely find you fresh
4. Week 4: simulate the day
   • Take a timed practice section in the format of your target test
   • Follow each timed section with one game session of the family you did worst on
   • Practise the walk-away rule: mark, skip, return
   • Stop two days before the test and rest

Key points

• Brain training does not reliably raise general intelligence; gains shrink with distance from the practised task.
• Retesting raises cognitive-test scores by about a quarter of a standard deviation, more with coaching and familiar formats.
• Building the answer before looking at the options (constructive matching) raised Raven scores from 10.45 to 12.77 of 18 in one experiment, most for people with lower working memory.
• Work one attribute at a time, try simple rules first, and look for the case that breaks your rule.
• Our reasoning scores weight accuracy far above speed, so careful method is rewarded.

Next, the checkpoint: test what you have learned across the subject.

Sources

1. Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016). Do "brain-training" programs work? Psychological Science in the Public Interest, 17(3), 103–186. doi:10.1177/1529100616661983
2. Association for Psychological Science. Do brain-training programs work? (summary). psychologicalscience.org
3. Melby-Lervåg, M., Redick, T. S., & Hulme, C. (2016). Working memory training does not improve performance on measures of intelligence or other measures of "far transfer". Perspectives on Psychological Science, 11(4), 512–534. doi:10.1177/1745691616635612
4. Sala, G., Aksayli, N. D., Tatlidil, K. S., Tatsumi, T., Gondo, Y., & Gobet, F. (2019). Near and far transfer in cognitive training: A second-order meta-analysis. Collabra: Psychology, 5(1), 18. doi:10.1525/collabra.203
5. Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: A meta-analysis. Psychonomic Bulletin & Review, 22(2), 366–377. doi:10.3758/s13423-014-0699-x
6. Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92(2), 373–385. doi:10.1037/0021-9010.92.2.373
7. Gonthier, C., & Thomassin, N. (2015). Strategy use fully mediates the relationship between working memory capacity and performance on Raven's matrices. Journal of Experimental Psychology: General, 144(5), 916–924. doi:10.1037/xge0000101
8. Klauer, K. J., & Phye, G. D. (2008). Inductive reasoning: A training approach. Review of Educational Research, 78(1), 85–123. doi:10.3102/0034654307313402
9. Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological Science, 29(8), 1358–1369. doi:10.1177/0956797618774253
10. Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature, 407, 630–633. doi:10.1038/35036586
11. Carpenter, P. A., Just, M. A., & Shell, P. (1990). What one intelligence test measures. Psychological Review, 97(3), 404–431. doi:10.1037/0033-295X.97.3.404
12. Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology, 12(3), 129–140. doi:10.1080/17470216008416717