What is it about?
When aggregating judgments, individuals’ subjective confidence is a useful cue for deciding which judgments to accept. However, can confidence in one task set predict performance not only in the same task set, but also in another? We examined this issue through computer simulations.
Featured Image
Photo by Jessica Lewis on Unsplash
Why is it important?
We performed computer simulations using behavioral data in a novel way. Specifically, inspired by a cross-validation in machine learning, we developed an approach in which we simulated group judgments in what we called a “training-test” approach: we regarded part of (one or a few) the question(s) used in behavioral experiments as training questions and part of the other questions as test questions. Using these two-split task sets, we assumed that individuals were ranked based on their confidence ratings in the training questions. This procedure reflected situations of identifying who had high confidence based on training questions in the real world. We then assumed that these hypothetical groups solved both the training and test questions using a majority rule, and calculated the accuracy of the group judgments. This procedure reflected situations in which individuals made group judgments to deal with both training and test questions in the real world. Main findings: Although groups comprising individuals with high confidence in the training question(s) generally showed better accuracy, their accuracy in the test questions sometimes worsened compared to that in the training question(s), especially when only one training question was available.
Perspectives
Today, people live in a highly uncertain world. For example, three years ago, no one could predict the outbreak of the COVID-19 pandemic we are currently experiencing; and today, no one can correctly predict the number of infected people three years later. In such uncertain situations, formulating policies based on the judgments of one or few individuals with high confidence to address today’s current problems may not work well for the future’s unknown problems. We believe that our arguments will be supported even in such a "wicked" environment (i.e., wherein individuals with high confidence sometimes make mistakes). This is because in a wicked environment, aggregating only individuals with high confidence will lead to irrational or inaccurate group judgments while mixing various individuals in terms of their levels of confidence is more likely to avoid decreasing group performance and may lead to accurate group judgments.
Masaru Shirasuna
Read the Original
This page is a summary of: Can individual subjective confidence in training questions predict group performance in test questions?, PLoS ONE, March 2023, PLOS,
DOI: 10.1371/journal.pone.0280984.
You can read the full text:
Contributors
The following have contributed to this page
