Questionnaires
- Open questions let people answer in any way. They give detail but are hard to analyse.
- Closed questions have a fixed set of answers, such as tick boxes. They are easy to analyse but limit answers.
- Good response boxes don't overlap, cover every possible answer, and include a time frame where needed (such as "per week").
- Avoid leading questions ("Don't you agree that…?"), vague words ("often"), and unnecessary personal questions.
- Questionnaires can be posted, handed out or put online. Response rates may be low, and people who respond may not be typical.
Sensitive questions
- For sensitive questions (such as "Have you ever cheated in a test?"), use the random response technique. Each person flips a coin: heads, they answer "yes"; tails, they answer truthfully. Nobody knows which, so people answer honestly.
- Example: 100 people, 60 say yes. About 50 got heads and said yes, so 10 of the 50 answering truthfully said yes. The estimate is 10 ÷ 50 = 20%.
Interviews, observation and experiments
- Interviews allow follow-up questions and a high response rate, but take time, and the interviewer may influence answers (interviewer bias).
- Observation records what people actually do, and data logging uses machines to record automatically.
- Laboratory experiments are carried out in controlled conditions. Field experiments happen in a natural setting with some control. Natural experiments look at changes that happen without the researcher controlling them.
- A control group doesn't receive the treatment, so results can be compared.
Cleaning data
- Before analysing, clean the data: remove errors and impossible values, make units consistent, and deal with missing answers.
Key terms
- Open question
- A question that lets people answer in any way.
- Closed question
- A question with a fixed set of answers.
- Leading question
- A question that pushes people towards a particular answer.
- Random response technique
- A method using chance to get honest answers to sensitive questions.
- Control group
- A group that doesn't receive the treatment, for comparison.
- Interviewer bias
- When an interviewer influences the answers given.
- Data cleaning
- Removing errors and making data consistent before analysis.
- Field experiment
- An experiment in a natural setting with some control.
Quick questions on Collecting data
- What is a closed question?
- A question with a fixed set of answers: Tick boxes are an example.
- What is a leading question?
- A question that pushes people towards an answer: "Don't you agree that…?"
- What is wrong with the boxes 0–5, 5–10, 10–15?
- They overlap: Where would 5 go?
- What is a control group?
- A group that doesn't receive the treatment, for comparison: It shows what happens without the treatment.
- Why use the random response technique?
- To get honest answers to sensitive questions: Nobody knows if a "yes" is truthful.
- What is an advantage of interviews over postal questionnaires?
- A higher response rate and follow-up questions: Someone is there to ask.
- What is interviewer bias?
- When an interviewer influences the answers: Tone of voice can affect answers.
- What is data cleaning?
- Removing errors and making data consistent: Such as removing impossible values.
Collecting data: questions and answers
- Which experiment happens in controlled conditions?
- A laboratory experiment. Variables can be tightly controlled.
- 200 people use the coin-flip random response method and 130 say yes. What percentage answered yes truthfully?
- 30%. About 100 said yes because of heads; 30 of the other 100 said yes.