Methods of Data Collection
How do I choose variables to investigate?
- Keep the number of variables you investigate to a minimum
- Too many variables at once can be overwhelming
- It can be time-consuming to process unnecessary data
- You should choose variables that are linked to what you are investigating
- If you are investigating the ability of adults to solve puzzles you might use the time it takes them as a variable
- Consider which variables are likely to have an effect on what you are investigating
- An adult’s reading speed will affect their time to solve a puzzle
- An adult’s height is unlikely to affect their time to solve a puzzle
What makes a good survey?
- A survey is a method of collecting data
- Consider whether the survey needs to be in-person
- A person might be less likely to answer questions truthfully in person
- You can quickly survey more people remotely or electronically
- Such as postal surveys, phone surveys, internet surveys
- Consider whether the interviewer could unintentionally influence participants’ responses
- If a headteacher is asking students whether they enjoy school then they are more likely to say yes as they think that is what the headteacher wants to hear
- This will introduce bias
What makes a good questionnaire?
- A questionnaire is a list of questions
- The questions should be unbiased
- Questions should not be leading
- For example: “You enjoy school, don’t you?”
- If options are given for the participant to choose from then they should cover all possible responses
- The questions should not be personal
- This means you should not ask for unnecessary personal information
- Such as date of birth, address, etc
- The questions should not reflect your personal opinions
- For example: “Do you enjoy watching the boring news on TV?”
- People can find it difficult to rate personal feeling/qualities
- For example: “How smart do you think you are?”
- Questions can be structured or unstructured
- Structured questions usually ask the participants to choose from options, give a rating or rank options
- These can be quick to analyse
- The answer choices should be consistent where appropriate
- Unstructured questions let the participants to express their views in their own words
- These tend to be more open-ended questions
- These can take longer to analyse but can give more in-depth views
- Questions should be precise and unambiguous
- They should be phrased in a way in which the participants understand exactly what you mean
- For example: “Do you study French or Spanish at school” is not precise
- Some people might reply with “Yes” or “No”
- Some people might reply with “French” or “Spanish”
Reliability & Validity
What is reliability & validity of a data collection method?
- Reliability measures how consistent a process is at measuring a variable
- A process is reliable if you would get the same results by repeating the process with the same sample using the same conditions
- Validity measures how accurate a process is at measuring a variable
- A process is valid if it is accurately measuring the variable you want it to measure
- If your process is found not to be reliable or valid then:
- Adjust the data collection process
- Change the sampling technique
- Use a larger sample
What are tests to check reliability?
- Test-retest
- This is where you use a data collection process with a sample and then repeat the same process with the same sample at a later time
- The results should show positive correlation if the process is reliable
- The results might not perfectly match due to external factors during the gap between the data collection
- Once the sample has been through the process once they will be familiar so this could lead to different results from the second process
- Parallel forms
- This is where you give the same sample a second set of questions (or second set of experiments) which are similar to the first set
- The results should show positive correlation if the process is reliable
- It can be difficult to make the two processes similar to each other
What are tests to check validity?
- Content-related validity checks
- This is where you check how well the process measures all aspects of the variable
- If the process is valid then it should cover all aspects of the variable
- These checks require knowledge of the variable so experts are often used
- An example of a process that is valid:
A teacher wants to assess how well students understand calculus so they set questions covering differentiation, integration and applications - An example of a process that is not valid:
A restaurant manager wants to assess how good a chef is at cooking steaks so asks the chef to make 10 medium steaks - Criterion-related validity checks
- This is where you check how well one variable predicts the outcome for another variable (called the criterion variable)
- If the process is valid then the variable should be a good predictor
- An example of a process that is valid:
Results from a mock exam being used to predict the results in the actual exam - An example of a process that is not valid:
Results from measuring the heights of meerkats being used to predict the heights of squirrels
Worked Example
Tomas is a dog trainer. Before he agrees to train a dog he assesses the dog’s obedience. To do this, he first visits the dog, asks it to perform 10 basic commands and records how many the dog successfully carries out. Two days later, Tomas visits the dog a second time and asks it to do the same 10 commands. Tomas assesses 8 dogs using this process and the table below shows the number of commands performed successfully by each dog on each visit.
|
Dog |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
|
First visit |
3 |
5 |
2 |
3 |
6 |
2 |
0 |
5 |
|
Second visit |
3 |
5 |
2 |
4 |
5 |
2 |
1 |
5 |
a)
State the reliability test that Tomas is using.
b)
Comment on the reliability of Tomas' process.
