Object Sorting

What you see

A trial is one stream of objects, between 10 and 26 of them. Each object is one to three copies of a shape, with four properties:

• shape: circle, square, triangle or diamond
• fill: outline, striped or solid
• count: one, two or three copies
• size: small or large

The question above the board names the property to sort by, for example Sort each object into a bin by fill. Each bin is a card that lists the values it takes underneath its name (Bin 1: striped, Bin 2: outline, solid). The object to sort sits in a frame in the middle, and a Next row above it previews up to three objects that follow it. The line under the board shows your place in the stream, for example Object 7 of 16.

Choose a bin, or press its number key (1 to the number of bins), to drop the object in. The next object slides in straight away. Sorted objects cannot be taken back. When the whole stream is sorted, the line reads, for example, All 16 objects sorted; press Check answer to see how you did. There is no feedback on individual objects while you sort.

The stream is split into segments, each with its own rule. At a switch, the question changes to a different property, and for the first object under the new rule it reads, for example, The rule has changed: sort by fill. Consecutive segments never sort by the same property. Each rule splits that property's values across the bins in an arbitrary way, with at least one value per bin, so the mapping has to be read, not guessed. Square and diamond to Bin 2 is as likely as square alone to Bin 2.

The two objects straight after every switch are drawn to be bivalent: the old rule and the new rule put them in different bins. At higher levels, the next two objects are also made bivalent some of the time.

Size is never a sorting rule. It has only two values, so it cannot fill three bins, and it is not in the rule set at the two-bin levels. It is always there on the objects, though, as something to ignore. Switches are spaced roughly evenly through the stream, and the last segment always has at least four objects.

In practice mode there is no countdown. A clock counts up in the side rail, and a session starts at level 3.

What it measures

Object Sorting measures cognitive flexibility, also called set shifting: dropping one way of responding and adopting another when the task demands it. Internally the game is weighted 70 percent cognitive flexibility and 30 percent selective attention, since each object has four properties and only one matters at a time. On our profile it sits in the Rule learning facet of Reasoning and carries 7 percent of the Reasoning dimension score.

Card sorting has a long history as a test of flexibility.

1. 1948

   Grant and Berg publish the Wisconsin card-sorting procedure: cards vary in colour, number and form, the sorting principle is never stated, and it changes without warning once the participant has it.

2. 1995

   Rogers and Monsell publish an influential study of the cost of a predictable switch between two simple tasks.

3. 2000

   Miyake and colleagues show that shifting, updating and inhibition are separable executive functions, with Wisconsin card sorting loading most strongly on shifting.

4. 2006

   Zelazo publishes the standard protocol for the Dimensional Change Card Sort, in which children are told the new rule and must switch to it.

5. 2013

   A computerised Dimensional Change Card Sort becomes the flexibility measure in the NIH Toolbox Cognition Battery.

Our version is closer to the Dimensional Change Card Sort than to the Wisconsin test. In the Wisconsin test you must discover the new rule from feedback. In the Dimensional Change Card Sort, as in Object Sorting, you are told the new rule and the question is whether you can act on it. Zelazo's protocol makes the point vividly: most 3-year-olds, told to switch from sorting by colour to sorting by shape, keep sorting by colour, much as patients with prefrontal damage do. By age 5, most switch when instructed.

That is exactly the window Object Sorting targets. The first objects after a switch are where the old rule lingers, which is why the game places its bivalent objects there and counts errors in that window separately.

Miyake and colleagues' 2000 study of 137 students is the reason flexibility is treated as its own skill. Shifting, updating working memory and inhibiting prepotent responses were moderately correlated but clearly separable, and card-sorting performance was related most strongly to shifting. Being good at holding information in mind does not guarantee being good at letting go of a rule.

How it is scored here

Each object is right or wrong against the rule that was live when it arrived. The trial score is the share of objects sorted correctly.

• Every object right: Correct, and you move up a level.
• Some wrong: Partly correct, with credit in proportion (19 of 20 right is 95 percent), and you stay at the same level.
• None right: you move down a level.

Because only a perfect stream moves you up, a single slip at a switch is enough to hold you in place. That is by design: clean switching is the skill.

The evaluator also counts perseverations: wrong sorts within the first four objects after a switch that match where the previous rule would have sent the object. The session report's headline, Sticking with a rule that changed, relates your perseverations to the number of switches you faced (lower is better). It also shows Items sorted by the old rule and Items sorted correctly.

Your 0–100 Object Sorting score is centred so that 50 means what the difficulty you work at predicts. The difficulty credited to a stream rises with the number of switches, the number of bins and the length of the stream. Time is not part of the score. A score appears after at least four trials, and recent sessions count for more.

Where you will meet it

Rule-switching tasks are widespread in research and clinical testing, and a version of the same demand turns up in game-based hiring.

Card-sorting tasks themselves appear mainly in neuropsychology and research: the Wisconsin test in clinical assessment, and the Dimensional Change Card Sort in the NIH Toolbox Cognition Battery, a research battery. Hiring vendors rarely publish their task lists, so we cannot say which employer batteries include a sorting task with this exact design. What transfers is the skill: whenever a test changes its instructions mid-task, the first few responses after the change are where candidates lose marks. Our Arrows game trains a related demand at speed.

The method

1. Turn the rule into one question

   Read the property the question names and the values on each bin, then compress it. With two bins, name the smaller group: striped? Bin 1, otherwise Bin 2. With three bins, name each bin by its shortest description: square is 1, triangle is 3, anything else is 2.

2. Look at one property only

   Every object has four properties and three of them are noise. Size is always noise. Train your eye to go straight to the property the question names and nowhere else.

3. Use the preview

   The Next row shows what is coming. While you place the current object, you can already be classifying the next one.

4. At a switch, stop for one beat

   When the question reads The rule has changed, take your hand off the keys. Read the new bins, form the new question, and say it once. The first two objects under the new rule are built to be sorted wrongly by the old one.

5. Distrust fluency right after a switch

   If the first object after a switch feels easy and obvious, check which rule made it feel that way. The old rule is still fast and familiar; the new one is not yet.

6. Settle into a steady rhythm

   Between switches, sort at an even pace. There is no undo, so a steady rhythm beats bursts of speed followed by slips.

Worked example 1: two bins

The question reads Sort each object into a bin by shape. and the bins read Bin 1: circle, triangle and Bin 2: square, diamond. The first two objects are two solid circles, then one outline diamond. Then the question changes to The rule has changed: sort by fill. and the bins read Bin 1: striped and Bin 2: outline, solid. The next two objects are three large striped squares, then one solid triangle.

Show the solution

Under the shape rule, the question is circle or triangle? Bin 1, otherwise Bin 2.

• Two solid circles: Bin 1.
• One outline diamond: Bin 2.

The rule changes. New question: striped? Bin 1, otherwise Bin 2.

• Three large striped squares: Bin 1. The old rule would have said Bin 2 (square).
• One solid triangle: Bin 2. The old rule would have said Bin 1 (triangle).

Both post-switch objects are bivalent. Sorting either one by shape would count as a perseveration. Notice that the size of the squares makes no difference at all.

Worked example 2: three bins

Show the solution

Before the switch, the question is count: two is Bin 1, one is Bin 2, three is Bin 3.

• Object 1, three outline triangles: Bin 3.
• Object 2, one striped circle: Bin 2.

After the switch, the question is shape: square is 1, triangle is 3, circle or diamond is 2.

• Object 3, two large solid diamonds: Bin 2. Count would have said Bin 1.
• Object 4, three striped squares: Bin 1. Count would have said Bin 3.
• Object 5, one outline triangle: Bin 3. Count would have said Bin 2.

All three post-switch objects disagree between the rules. A candidate still thinking in counts would score zero on them, and all three wrong answers would be classed as perseverations, because each falls within the first four objects after the switch.

Tips and traps

Levelling up

Difficulty rises on five fronts: more switches (up to four), longer streams (up to 26 objects), three bins from level 5, more bivalent objects after each switch, and a wider set of rule properties. Three bins is the biggest step, because a three-way mapping takes longer to compress into one question and there are two wrong bins to fall into instead of one.

• With three bins and a shape rule, find the bin that holds two shapes and treat it as everything else. With fill or count, each of the three bins holds exactly one value, so the bins are a straight lookup.
• With four switches, keep each re-set identical. The same small routine every time (pause, read, say) matters more than raw speed.
• Review perseverations, not just accuracy. If most of your errors sit in the four objects after a switch, the fix is the pause. If they are spread through the segments, the fix is the question you form from the rule.

1. Week 1
   • Three sessions of 10 to 12 streams, saying the sorting question aloud at the start of every segment
   • After each session, compare Items sorted by the old rule with your total errors
   • Practise the one-beat pause at every rule change, even when it feels unnecessary
2. Week 2
   • Three sessions aiming for level 5, where three bins begin
   • Before each three-bin segment, name the everything-else bin first
   • Add one Arrows session to practise switching under time pressure
3. Week 3
   • Three sessions at level 7 and above
   • Aim for zero perseverations across a full session
   • Use the Next row actively: classify the next object while placing the current one

Check yourself

5 practice questions

1. The question says to sort by fill. Bin 1 lists striped and Bin 2 lists outline, solid. The object is three large striped squares. Which bin?
2. Which property is never used as a sorting rule in our Object Sorting?
3. You sort 19 of 20 objects correctly in a stream. What happens?

and 2 more

Create a free account to answerSign in

Sources

1. Grant, D. A., & Berg, E. A. (1948). A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigl-type card-sorting problem. Journal of Experimental Psychology, 38(4), 404–411. PhilPapers
2. Rogers, R. D., & Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124(2), 207–231. doi.org
3. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex "frontal lobe" tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. PubMed
4. Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7(3), 134–140. PubMed
5. Zelazo, P. D. (2006). The Dimensional Change Card Sort (DCCS): A method of assessing executive function in children. Nature Protocols, 1(1), 297–301. PubMed
6. Zelazo, P. D., Anderson, J. E., Richler, J., Wallner-Allen, K., Beaumont, J. L., & Weintraub, S. (2013). NIH Toolbox Cognition Battery (CB): Measuring executive function and attention. Monographs of the Society for Research in Child Development, 78(4), 16–33. Wiley
7. JobTestPrep. Pymetrics games guide: the Arrows game (candidate-reported). jobtestprep.co.uk

Put it into practice: Object SortingSign in to try it