Rule Sorter

What you see

Two panels, Group A and Group B, each hold three to six example cards. Every card in Group A obeys a hidden rule; every card in Group B breaks it. Below them, under To sort, are two to four new cards. The highlighted one is the card you are sorting: send it to Group A or Group B with the buttons under the board or the left and right arrow keys, or click another card to sort it first. A card you have sorted appears under Sorted in its group; click it to take it back. When every card is sorted, press Check answer (or Enter).

The rule is new on every trial. There is no feedback while you sort. You see the verdict once you check the trial, and the session report shows the hidden rule for each trial. Among the cards to sort there is always at least one that belongs in each group.

The seven features

Every card is described by seven features. Any of them can carry the rule, so you need to be able to read all seven quickly.

• Count: one to five shapes.
• Kind: circle, square, triangle or diamond.
• Fill: outline, striped or solid.
• Size: small, medium or large.
• Turn: 0, 90, 180 or 270 degrees, so a triangle points up, right, down or left. Only triangles are ever turned: a circle, square or diamond keeps its outline after a quarter turn, so those shapes are always drawn upright, and no rule depends on a turn you cannot see.
• Centre marked: a dot in the card's top-left corner.
• Symmetric: a small "sym" tag in the card's top-right corner. Treat it as an on-off label; it is not something you judge from the drawing.

The rules

A rule is built from terms. A term tests one feature: it equals a value ("fill is solid"), is at least a value ("count is at least 4", "size is at least medium"), or, for count, is odd or even. About a quarter of the time a term is negated ("not a triangle").

Every trial uses its level's settings. The generator chooses the example cards so that the guarantee below holds, instead of hoping that random examples happen to meet it. If it ever failed 200 times in a row it would fall back to a one-term rule with five examples per group, but in 3,000 generated trials at every level that never happened. It also never uses a rule that a shorter rule would match on every possible card, so a two-term rule needs both of its terms and a three-term rule needs all three. No two cards in a trial are exactly alike.

A three-term rule has the shape "A joined with (B joined with C)", for example "triangle OR (solid AND large)". XOR means exactly one of the two holds.

The guarantee that makes it manageable

Before a trial is accepted, the generator lists every rule of one or two terms that could be built from these features: several thousand of them. It keeps the ones that fit every example card, and rejects the trial if any of them would sort even one test card differently from the hidden rule.

That has a practical consequence you should rely on: any one- or two-term rule you find that fits every example card will sort the test cards correctly, whether or not it is the rule the generator used. At levels 1 to 7 the hidden rule is itself one of these. At levels 8 to 10 it has three terms, but the guarantee still holds for every simpler rule that fits.

So you do not need the "true" rule. You need a simple rule that fits all the evidence. Often dozens of rules fit; they all agree on the cards you have to sort.

What it measures

Rule Sorter measures inductive reasoning, the ability to infer a category rule from examples, with a smaller load on cognitive flexibility: every trial brings a new rule, and the rule from the last trial is a tempting wrong answer. It feeds the rule learning facet of your reasoning score and carries 12 per cent of the dimension.

Its closest ancestor is the Bongard problem. In 1967 the Soviet computer scientist Mikhail Bongard published 100 puzzles, each showing twelve diagrams in two groups of six, where one rule is true of every diagram on the left and of none on the right. They became famous through Douglas Hofstadter's Gödel, Escher, Bach.

the skill of solving Bongard problems lies very close to the core of 'pure' intelligence

The psychology of category rules explains the level structure. Shepard, Hovland and Jenkins found single-feature classifications easiest and an exclusive-or over two features markedly harder, and Feldman showed that difficulty tracks the length of the shortest description of a rule. XOR rules appear here from level 4.

A contrast is worth drawing. The Wisconsin Card Sorting Test also asks people to sort cards by an unstated rule, but it teaches the rule through right-or-wrong feedback after every card and changes it without warning after ten correct sorts in a row. Rule Sorter gives you all its evidence up front, in two labelled groups, and changes the rule between trials rather than within one. In this app, Object Sorting is the game in the Wisconsin family.

How it is scored here

Each card you sort is checked separately. Your trial score is the share of cards in the right group: sort three of four correctly and you earn 75 per cent. All correct counts as Correct, none correct as Incorrect, anything between as Partly correct.

Difficulty adapts on the trial result: a Correct trial moves the next one up a level, a trial with no correct cards moves it down, and a Partly correct trial holds the level. You start at level 3.

The measured difficulty of a trial rises with the number of terms in the rule (2 points each), the number of features it reads (1.5 points for each beyond the first), and the number of simple rules that fit the examples (1 point each, up to 3 points), because more surviving alternatives make the right split harder to see. It is measured from the rule the trial actually uses. Time is not part of the score, and 50 on your score means you are performing at the level your working difficulty predicts.

Your report shows Most conditions in a rule you sorted perfectly and Hardest rule type: the kind of rule (for example AND, OR or XOR) with your lowest average score, once you have at least two trials of two different kinds.

Where you will meet it

Two-group sorting by a hidden rule is a standard game-based inductive task.

In written tests the same skill appears as classification ("which figure belongs with these?") and odd-one-out items, which AMCAT lists among its inductive topics. Rule Sorter is the same family of task, not a copy of any vendor's version.

The method

1. Scan Group A for anything constant

   Go through the seven features in a fixed order: count, kind, fill, size, turn, dot, sym. For each, ask whether every card in Group A shares a value, or falls on one side of a line (all 3 or more, all odd).

2. Check each lead against Group B

   A lead only counts if no card in Group B has it. Group A all solid and Group B has a solid card? Not the rule. Keep going.

3. Scan Group B the same way

   Sometimes Group B is the uniform one (all outline), and the rule is a negation: "not outline". Look at both groups before combining terms.

4. Combine two terms only when single terms fail

   If Group A splits into two sub-groups, each with its own shared feature, try OR. If each feature in Group A also appears in Group B but never together, try AND. If Group A cards have exactly one of two features, try XOR.

5. Check the rule on every example, then sort

   One example that breaks your rule means the rule is wrong. When it fits every card in both groups, sort the new cards by it without second-guessing: the guarantee says any simple rule that fits will sort them correctly.

Worked example 1

Show the solution

Scan for constants. Kinds: Group A has a triangle, circles, squares and diamonds; so does Group B. Fills: all three appear on both sides. Sizes: mixed on both sides. The dot and the sym tag appear on both sides. None of those works.

Count. Group A has 1, 3, 5 and 3 shapes. Group B has 2, 4, 2 and 4. Every card in A has an odd count; every card in B has an even one.

Sort. Card 1 has five striped diamonds: odd, Group A. Card 2 has four solid triangles: even, Group B.

Why the guarantee helped. Other one- or two-term rules fit these examples too, such as "count is not 2 AND count is not 4". In a real trial the generator has checked that every rule like that sorts the cards the same way, so whichever one you find, your sort is right.

Worked example 2

Show the solution

Single features. Group A has circles, a triangle, squares, triangles and diamonds; no single kind. Counts in A are 4, 1, 5, 2 and 4; no single count or parity. Fills and sizes vary on both sides. No single term works.

Split Group A. Three of its cards have four or five shapes (circles, squares, diamonds). The other two are triangles (one and two of them). So every card in A has four or more shapes, or is a triangle.

Check Group B. Counts are 3, 1, 2, 3 and 2: none reaches four. Kinds are circles, a square, diamonds, squares and circles: no triangle. The rule holds: count at least 4 OR triangle.

Sort. Card 1, three triangles: triangle, Group A. Card 2, five circles: count at least 4, Group A. Card 3, three diamonds: neither, Group B.

Other simple rules may fit these examples as well. In a real trial the generator guarantees that all of them agree on the cards to sort.

Tips and traps

Levelling up

Level 4 is the first big step: two-term rules, including XOR, the hardest two-feature form to learn. At level 7 the rule still has two terms, but you get only three examples per group and four cards to sort, so each example carries more weight. From level 8 the hidden rule has three terms, but thanks to the guarantee your task does not get three times harder: you still only need a simple rule that fits.

Adapt in three ways as you climb:

1. Fix your feature order so you never skip one under time pressure.
2. Look for sub-groups. When Group A is mixed, split it into the smallest number of clusters that each share one feature. Two clusters means OR.
3. Use partial credit sensibly. If you are sure of some cards and not others, sort the sure ones by your rule and think hardest about the one card that separates your two best candidate rules.

1. Week 1
   • Three sessions of 10 trials at levels 1 to 4
   • Say the seven features aloud in the same order on every trial
   • Check each lead against the other group before sorting
2. Week 2
   • Work at levels 4 to 6: practise OR and XOR splits
   • After each miss, write down the rule from the report and the feature you skipped
   • Add one Rule Learning session to practise stating rules
3. Week 3
   • Push to levels 7 and 8
   • Rely on the guarantee: stop searching once a simple rule fits
   • Add one Object Sorting session to practise dropping an old rule
4. Week 4
   • Timed sets to match six- and twelve-minute game rounds
   • Review the Hardest rule type in your report and drill it
   • Rest before the real test

Check yourself

5 practice questions

1. Every card in Group A is striped, and one card in Group B is also striped. What follows?
2. At level 6 you find a two-term rule that fits every example in both groups, but you are not sure it is the rule the generator used. What should you do?
3. You sort four cards and three are right. What does the trial score, and what happens to the level?

and 2 more

Create a free account to answerSign in

Sources

1. Bongard problem. Wikipedia. en.wikipedia.org/wiki/Bongard_problem
2. Hofstadter, D. R. (1979). Gödel, Escher, Bach: An Eternal Golden Braid. Basic Books.
3. Shepard, R. N., Hovland, C. I., & Jenkins, H. M. (1961). Learning and memorization of classifications. Psychological Monographs: General and Applied, 75(13), 1–42. doi:10.1037/h0093825
4. Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature, 407, 630–633. doi:10.1038/35036586
5. Nyhus, E., & Barceló, F. (2009). The Wisconsin Card Sorting Test and the cognitive assessment of prefrontal executive functions: A critical update. Brain and Cognition, 71(3), 437–451. doi:10.1016/j.bandc.2009.03.005
6. Luchins, A. S. (1942). Mechanization in problem solving: The effect of Einstellung. Psychological Monographs, 54. doi:10.1037/h0093502
7. Aon. Prepare for your online assessment: logic tests. aon.com
8. GameAssessmentPrep. Capgemini game-based test: games and rounds (candidate reports). gameassessmentprep.com/employers/capgemini
9. AMCAT. AMCAT syllabus: Logical Ability. myamcat.com/amcat-syllabus

Put it into practice: Rule SorterSign in to try it