The PISA study is usually used to measure educational achievement across countries.
However, as well as taking a two hour test covering reading, science and mathematics, young people are also asked to complete a background questionnaire.
This covers a wide range of topics, such as self-confidence, life satisfaction, expectations for the future and out-of-school activities. There is hence a whole host of things – outside of educational achievement – that PISA allows us to compare across the world.
This includes, for the first time ever, how big bullshitters teenagers are [PDF].
Let me explain…
How is bullshit measured in PISA?
In the 2012 edition of PISA, 15-year-olds across the world were asked how well they understood a set of 16 mathematical constructs. They were asked to indicate a response on a five-point scale, from 1 (“never heard of it”) through to 5 (“know it well, understand the concept”).
These constructs were:
- Exponential function
- Quadratic function
- Proper number
- Linear equation
- Complex number
- Rational number
- Subjunctive scaling
- Declarative fraction
- Congruent figure
- Arithmetic mean
There is, however, a catch. The three constructs in italics above are completely made up! There is no such thing as a ‘proper number’, ‘subjunctive scaling’ or a ‘declarative fraction’ – yet some teenagers said that they knew such constructs well’. I hence use how teenagers responded to these three fake constructs to create a ‘bullshit scale’.
The chart below compares average scores on this scale across English-speaking countries. Results have been standardised, so positive figures indicate being above the international average on the bullshit scale and negative figures below the international average. The magnitude of results are reported as effect sizes (standard deviation differences).
North American teenagers are found to be the biggest bullshitters in the English-speaking world – Canada and the United States take first and second place. In the middle of the pack are England, Australia and New Zealand. Whereas it is the Irish, Northern Irish and Scottish who are the most straight-talking.
The paper also explores differences between demographic groups, as illustrated in the chart below.
Young men were found to be much bigger bullshitters than young women, while rich kids were much more likely to tell porkies than the poor.
So, if on the way home tonight you meet a charming, posh, male Canadian, perhaps its best you don’t believe everything that they say…
This work was, of course, meant to be light-hearted and a bit of fun. But it does also have a serious side.
Everyone knows a bullshitter. Yet there has been surprising little research done into this widespread social phenomenon. Resources such as PISA, when used in imaginative ways, can tell us a lot more about the world in which we live than just educational achievement alone.
Want to stay up-to-date with the latest research from FFT Education Datalab? Sign up to Datalab’s mailing list to get notifications about new blogposts, or to receive the team’s half-termly newsletter.