Skip to content
The Intellectual Framework
Open the Atlas
The Atlas · Cosmos · Discipline 46 of 58

Quantitative and Qualitative Research Methods

How we find out, properly.

Read it free10 lessons · about 0.7 hours · 9,210 words
The story

In 1747 a ship's surgeon took twelve sailors with scurvy and split them into six pairs. Each pair got a different remedy: cider, vinegar, seawater, and so on. The two who got oranges and lemons were on their feet in a week. James Lind had run one of the first controlled trials. The method mattered more than the fruit.

The question to carry

What is the difference between an anecdote and evidence?

Say it at the table

The plural of anecdote is not data.

The film

Watch it in four minutes.

The briefing

What it is, in plain words.

  • 10 lessons
  • 10 modules, each with a test
  • 0 people explained
  • 72 works, places and ideas explained
  • 12 question final exam

Research methods are the procedures by which reliable knowledge is produced. Quantitative methods measure and count. Qualitative methods observe, interview and interpret. Knowing how a finding was reached is the only way to know how much it is worth.

Every study begins with a question and a design matched to it. Experiments change one thing and compare the result with a control. Surveys ask many people the same questions. Case studies and interviews go deep on a few. Each has typical weaknesses, so strong conclusions rest on several methods agreeing. Two tests apply to any result. Is it reliable, meaning it would come out the same again, and is it valid, meaning it measures what it claims to?

Five ideas to hold

1Variable

Anything in a study that can change or be changed, such as age, dose or opinion.

A variable is anything that can take different values. In an experiment, the independent variable is what the researcher changes and the dependent variable is what is measured in response. To test whether fertilizer helps plants grow, fertilizer is the independent variable and height the dependent one. Everything else that might matter, such as light and water, is a confounding variable and must be held constant or evened out. Otherwise you cannot say what caused the result.

2Control group

The participants who do not receive the treatment, used as the baseline for comparison.

The control group receives no treatment, or a dummy treatment, and shows what would have happened anyway. Many patients improve on their own, or because they expect to, an effect called the placebo response. Without a control group a useless remedy looks effective. Assigning people to groups at random keeps the groups alike. In a double-blind study neither participants nor researchers know who is in which group, so hopes cannot tilt the result.

3Validity

Whether a study measures what it claims to measure.

Validity asks whether a study measures what it claims to. An intelligence test that really measures reading speed is not a valid test of intelligence. Researchers separate internal validity, which is whether the study shows what caused the result, from external validity, which is whether the finding holds for other people and settings. A tightly controlled laboratory experiment often scores high on the first and low on the second.

4Reliability

Whether a measurement gives consistent results when repeated.

A reliable measure gives the same answer when nothing has changed. A scale that shows a different weight each time you step on it is unreliable. Reliability is necessary for validity and does not guarantee it. A scale that always reads five kilograms too heavy is perfectly reliable and consistently wrong. In surveys, reliability is checked by repeating the test later, or by seeing whether related questions get consistent answers.

5Qualitative research

Inquiry that gathers words, observations and meanings instead of numbers.

Qualitative research works with words, images and observations. Its tools are in-depth interviews, focus groups, fieldwork and the close reading of documents. It answers how and why questions that numbers cannot reach. A survey can show that patients miss appointments. Interviews can reveal that the clinic's hours clash with their shifts. Samples are small and chosen for insight, so the findings give depth and not statistics. Many strong studies combine both approaches.

Five moments that made the field

  1. 1747James Lind runs a controlled trial of cures for scurvy.
  2. 1935Ronald Fisher publishes The Design of Experiments.
  3. 1948The first modern randomized trial tests a drug for tuberculosis.
  4. 1967Glaser and Strauss publish their method for building theory from interviews.
  5. 2005John Ioannidis argues that most published research findings are false.

Still argued about

Open question 1

Which gives the truer picture, numbers or words?

Open question 2

How can the pressure to publish striking results be fixed?

Open question 3

When is it ethical to study people without telling them everything?

Plates

Things to look at.

Randomized controlled trial
Randomized controlled trial Form of scientific experiment · Wikipedia
Scurvy
Scurvy Disease resulting from a lack of vitamin C · Wikipedia
Laboratory
Laboratory Workplace for scientific activity · Wikipedia
Ideas

Three facts, one myth, one dare.

Did you know · 1A large project that reran published psychology studies could reproduce well under half of them.
Did you know · 2Significant in statistics does not mean important.
Did you know · 3Blinding a trial, so nobody knows who got the treatment, goes back two centuries.
The myth

One study settles a question.

Tap to turn it over
What is true

Single studies are often wrong. Confidence comes from many, done differently, agreeing.

Try this

Next time you read "a study shows", find out how many people were in it.

Battling Bad ScienceBen Goldacre, TED · one of the films in this discipline
Inside

10 sections, 10 lessons.

  1. 1

    The briefing

    Variable · Control group · Validity · Reliability

  2. 2

    Introduction to Research Methods

    Understanding Research · The Research Process · Ethics in Research · Conclusion

  3. 3

    Quantitative Research Methods

    Fundamentals of Quantitative Research · Designing Quantitative Studies · Data Collection Techniques · Statistical Analysis

  4. 4

    Qualitative Research Methods

    Fundamentals of Qualitative Research · Designing Qualitative Studies · Data Collection Techniques · Data Analysis and Interpretation

  5. 5

    Mixed Methods Research

    Combining Quantitative and Qualitative Approaches · Designing Mixed Methods Studies · Challenges and Considerations

  6. 6

    Research Design and Planning

    Formulating Research Questions and Hypotheses · Literature Review · Sampling Strategies

  7. 7

    Tools and Technologies in Research

    Research Software and Applications · Online and Digital Research Methods · Data Visualization

  8. 8

    Critical Appraisal of Research

    Evaluating Research Quality · Interpreting Research Results · Keeping Up with Research Trends

  9. 9

    Applying Research Skills

    Research in Academic Settings · Research in Professional Contexts · Disseminating Research Findings

  10. 10

    Conclusion

    The Role of Research in Advancing Knowledge · Fostering a Culture of Inquiry

From the text

Every study begins with a question and a design matched to it. Experiments change one thing and compare the result with a control. Surveys ask many people the same questions. Case studies and interviews go deep on a few. Each has typical weaknesses, so strong conclusions rest on several methods agreeing. Two tests apply to any result. Is it reliable, meaning it would come out the same again, and is it valid, meaning it measures what it claims to?

Keep reading in the Atlas

Figures

People to know.

James Lind1716-1794

An early controlled trial

Ronald Fisher1890-1962

Designed the modern experiment

Francis Bacon1561-1626

Induction from observation

Barney Glaser1930-2022

Grounded theory

Karl Pearson1857-1936

Founded mathematical statistics

Canon

Books to read.

  • The Craft of ResearchBooth, Colomb, Williams
  • Bad ScienceBen Goldacre
  • Research DesignJohn Creswell
From the viva

A question from its final exam.

Quantitative research primarily deals with

Also worth knowing

What a full picture of the field includes.

Every chapter was checked against a list of the names and terms a student of the subject should meet. These are the most important ones the book itself does not name.

Operationalization

Turning an abstract idea, such as anxiety, into something that can be measured, such as a score on a questionnaire.

Case study

An in-depth study of one case, or a few, used to understand how something works in its real setting.

Content analysis

Systematically counting and categorizing what appears in texts, images or recordings.

Saturation

The point in qualitative research at which new interviews stop producing new themes.

Positivism

The position that the methods of natural science can be applied to society.

Interpretivism

The position that social research must grasp the meanings people give to their own actions.

Cronbach's alpha

A statistic that shows how consistently the items of a scale measure the same thing.

Likert scale

The familiar response format that runs from "strongly disagree" to "strongly agree".

Hawthorne effect

People change their behavior when they know they are being studied.

Belmont Report

The 1979 American statement of research ethics, built on respect for persons, beneficence and justice.

Meta-analysis

A statistical method that combines the results of many studies of the same question.

Natural experiment

A study that uses an event outside the researcher's control, such as a new law, as if it were an experiment.

More of the field

12 more people and ideas a student of this subject should meet. In the Atlas each one opens a card with facts, dates and a portrait.

Clifford GeertzGeorge GallupHermann CohenJerzy NeymanJohn TukeyKarl PopperLeo HubermanMax WeberPaul LazarsfeldRobert KeohaneStanley MilgramÉmile Durkheim

Read Quantitative and Qualitative Research Methods.

Free. The film, the briefing and the first section open without an account.