Bias in Science and Communication

 Bias in Science and Communication: A Field Guide - Matthew Welsh

chp 1 - quiz

chp 2 - anchors aweigh

- decision making theories springing from probability theory, game theory, mathematicians trying to help maximize decisions in games/gambling

-Laplace - "common sense reduced to calculus", expected value (value of outcomes multiplied by likelihood)

-Nocolas and Daniel Bernoulli, St Petersburg Paradox

-homo economicus, rules for rational decision making (Von Neumann and Morgenstern), transitivity (A>B and B>C, means A>C), independence (preferences don't change), completeness

-Herbert Simon, bounded rationality, scissors metaphor, cognitive abilities and structure of world (how much info is readily available)

-Tversky and Kahneman - heuristics and biases

-two systems theory

    -Stanovich and West - system 1 as TASS (the autonomous set of systems)

    -metacognitive process of switching systems, error recognition, etc?

chp 3 - on message: reasons for and types of communication

-cognitive biases vs motivational biases

-implicit vs explicit bias

-elicitation of uncertainty

-wisdom of crowds effect (Galton) - have to be independent

chp 4 - Improbable interpretations: misunderstanding statistics and probability

-risk - multiple definitions, 1- probability of negative outcome, 2- product of likelihood and magnitude, 3 - dangerous and uncertain behavior/event

-variability - measurements of same parameter vary

-uncertainty - multiple definitions, 1 unable to measure, 2 - not knowing whether true or not

    -common error is for people to assume that past variability is an adequate measure of possible future values, use variability as measure of uncertainty

-monty hall problem, Let's Make a Deal, Marilyn vos Savant, many people responded angrily to her

-common problems in probability

    -sample size invariance

    -base rate neglect

chp 5 - truth seeking? biases in search strategies

-how search for info and update beliefs

-Laplacian demon, so many options and parameters

-"secretary problem"...an optimal stopping problem

-Simon - satisficing (when optimisation is too demanding)

-Naturalistic decision making (Klein and Zsambok) study how experts make decisions

-recognition and other selection heuristics

    -confirmation bias, confirming rather than testing

Chp 6 - same but different: unexpected effects of format changes

- people interpret percentages and natural frequencies differently

    -base rate neglect

    -twice as bad, etc (when risks are super low)

-nudge (default decisions, organ donors in europe)

-framing effects

-prospect theory: gains and losses, loss aversion

-order effects

    -primacy effect (remember first items, anchoring?)

    -recency effect - remember more recent items

-comparisons and preferences - single evaluation vs joint evaluation

Chp 7 - I'm confident, you're biased: accuracy and calibration of predictions

-positive correlations in confidence across fields of activity (higher level of competence or trait of confidence?)

-can someone accurately recognize whether the confidence they feel is justified or not? no

-Dunning-Kreuger effect (ignorant of ignorance)

-hard-easy effect - overconfident in hard questions, underconfident in easy questions

-three different forms of overconfidence

    -overplacement (more than 80% rate themselves above the median)

    -overprecision (predict too precisely, especially experts)

    -overestimation (overestimate the likelihood of getting things right)

-underestimate uncertainty!!!!

-planning fallacy

-reducing overconfidence, awareness helps but not solves problem

    -format, process changes (elicitation tools)

    -taking the outside view (use data from other similar situations)

Chp 8. - sub-total recall: nature of memory processes, their limitations and resultant biases

-long-term vs short-term memory

-forgetting

-availability bias, strong even when probability described, planing fallacy

-faulty trees (unpacking effect); hard to account for implicit categories (unavailable) - probability predictions change when you unpack the categories

-anchoring searches, still strong even after been explained

    -break dependence on a single anchoring value

-hindsight bias, causal explanations, once formed, difficult to imagine things having turned out differently, or etc, changing hypotheses

Chp 9 - Angels and demons: biases from categorization and fluency

-heightism

-halo effect

-pretty/good

-Matthew effect

-stereotypical thinking

    -fuzzy boundaries

    -probabilistic membership, predictive power

    -making predictions with limited info

        -combination of stereotypes, halo effect, confirmation bias - view a group as good or bad

        -implicit bias

            -"people are faster at completing a categorisation task with items that are stereotypically related, and slower with less stereotypically related

-Groupthink

-Easy to believe: fluency leads to believability, less likely to be checked

Chp 10 - us and them: scientists vs lay-people and individual differences in decision bias

-situational awareness developed by certain experts in certain fields (regular feedback, regular environment)

    -meteorologists: good calibration

-intelligence only weakly correlated to bias susceptibility, how about personality? (big 5)

-high conscientiousness more susceptible to hindsight bias

-cognitive reflection test

-"need for cognition" (NFC)

    -less prone to system one biases (stereotyping and halo effect)

    -more prone to system 2 biases (hindsight, confirmation bias)

Chp 11 - Warp and weft: publication bias example to weave it all together

-publication bias - tendency to decide whether to pursue publication based on what was found and how interesting, far more likely to pursue publication with positive results

    -bias toward novelty, toward confirmation, less replication

    -HARKing effect - hypothesising after results are known (works its way into papers)

    -too much dependence on small sample sizes

    -less re-test and replication

    -editorial decisions, who the author is, who you know, gender bias,

Chp 12 - Felicitous elicitation: reducing biases through better elicitation processes

    -outside view, calculate, transparency, decision making procedures, elicitation toles (MOLE, more or less elicitation - better and prefer making relative judgments, reduce anchoring effect, wisdom of crowds effect)

Chp 13 - A river in Egypt: denial, scepticism, and debunking false beliefs

    -gullibility: makes sense socially and evoluntionarily, have to learn a lot of info, makes sense to trust others

    -echo: more often factoid repeated

    -trust, trust the messenger first, then the messenger

Chp 14 - spotters guide to bias