In 2007 a newspaper article in the Danish newspaper Politiken described a study which was carried out in Denmark to investigate if young people’s attitudes towards health - for example, diet and exercise - would influence their behaviour. What would researchers need to do to carry out this type of study effectively? Researchers need to have a plan, people who are willing to participate in the study, and a method for collecting and analyzing data. These are all important parts of the research process that must be carefully considered by the researcher. The process of thinking critically about research is very similar; when we read the conclusions of a study, we need to ask appropriate questions in order to assess the significance of the research in the best way. In this chapter, we are going to look at research in psychology and learn some simple questions that we should consider when presented with a study.

Describing studies

When we describe a study we tend to outline its aim, procedure, results, and findings. The aim is the purpose of a study. An aim indicates which behaviour or cognitive process will be studied within a specific population.  For example, we may want to know if bilingual students are better able to recall items on a list than monolingual students. The population that we are interested in studying is "bilingual students."

After identifying the aim, the researcher will plan a procedure. This is the step-by-step process used by the researcher to carry out the study. In research articles, the procedure is always written in a way that makes it possible for others to understand how the data were collected. A good procedure is written so that it may be replicated by other researchers.

The results of a study are the numerical or descriptive data that is obtained and processed.  For example, the results may be that bilingual students recalled 17 out of 20 words on average, whereas monolingual students recalled only 12 out of 20 words.

The findings state how the researcher interpreted the data that were collected. Research findings are always open to discussion and debate. Maybe the researcher has interpreted the results in a way that is biased, or perhaps there were flaws in the procedure. If other research studies confirm the findings, the study is more credible. However, findings say something only about the population that was studied and may not be relevant to other groups or cultures. It is important to interpret findings in terms of the culture in which the research has been conducted, and always to be aware of potential cultural bias.

Participants - who should be in the study?

People who take part in a psychological study are called participants. Normally, psychologists define a target population - that is, a specific group of people whom they are interested in for their study. For example, this could be adolescents who live with one parent; women who have given birth to twins; or people who have moved from one country to another. The nature of the group of participants who are chosen from the target population to take part in the study, what psychologists call the sample, is very important in determining the usefulness of a piece of research.

The goal in sampling is to obtain a sample that is representative of the target population. If the researcher is interested in teenage drinking habits, for example, the population would be teenagers. In psychological research, it is not possible to test the whole population. Researchers often try to obtain a sample that represents a population - that is, a representative sample. As you can imagine, this is not always easy. For example, in the study of teenage drinking habits, it could be that the students that volunteer for the study are students who are high achievers who feel that drinking could actually hurt their academic performance.  If these students were the majority of the sample, then we may say that the sample is not representative of the target population and lacks population validity.

In addition, the size of a sample matters if it is to be representative. Small groups are more open to distortions than large ones. In a small group, each individual has quite a lot of influence on the overall result.

How does a psychologist get people to participate in a study? One of the simplest ways is to find a group that already exists.  For example, you study the members of a local church community, a retirement home, or even an IB psychology class at your school.  These are examples of opportunity sampling. It is an easy way for the researcher to get participants, but one has to question the nature of an opportunity sample. For example, if you are going to study a local retirement home, what type of people are there? Is there a gender, cultural or socioeconomic imbalance in who is living in this home? Would another retirement home down the street have a different population?  Opportunity sampling can lead to rather biased results, and it is problematic to generalize from studies that use opportunity sampling.

Sometimes researchers use what is called haphazard sampling.  This is a method that is neither random nor systematic. An example would be standing in front of the local mall and choosing people to interview on their thoughts about exercise and health. Haphazard sampling is a biased form of sampling and should be avoided.

 Teacher only box

1. This is an opportunity sample.  The participants are available at a local fitness center.

2. There are many potential answers here. The time of day may mean that a certain type of person is over-represented. The neighborhood in which the fitness center is located may mean that a certain demographic is over-represented.  The fact that this is most likely a private club also means that a certain social class will go there. It is also a question how people are selected to interview.  It may be that the interviewer is more comfortable approaching certain types of people and not others. In addition, if people are at the gym, they clearly have some motivation.  There are no interviews with the people who sign up for a gym and never show up. Finally, it may be that not all people that are motivated to exercise go to a gym.

3. It is helpful to look at the list above to make suggestions.  First, going at different times of day and on different days - e.g. including a Saturday.  In addition, going to different clubs around the city - or going to public sports areas (e.g. public basketball courts). Saying that you would obtain a "random sample" is not realistic.  In addition, a volunteer sample would also be problematic as most of the people that would reply may have a specific motivation to take part in the study.  Finally, it might be possible to find out from the gym if there are people that have memberships but do not come to the gym. It would be interesting to also interview people from that group.

Another commonly used sampling method is the self-selected sample, which is made up of volunteers. One advantage of the self-selected sample is that it is relatively easy to obtain, and it is almost guaranteed that the sample will be highly motivated. The problem is that volunteer samples rarely reflect the more general population, which means that it is difficult to make generalizations - that is, it is not clear whether these results can apply to people outside of the group that took part in the study.

When participants recruit other participants from among their friends and acquaintances, this is called snowball sampling - the sample grows like a snowball rolling downhill. This is often used when it may be difficult to access research participants.  This may be because they would not voluntarily respond to an ad in the paper  - for example, where the target population is drug users -  or because the researcher is looking for a very specific type of person - for example, foreigners who lived in communist Europe in the 1980s. Using snowball sampling not only may save time, but it also helps to establish trust with the researcher. If your friend trusted the researcher enough to recommend that you take part in the study, then the researcher must be trustworthy. Of course, you can see if only people that you knew were in the sample, this may not be highly representative of the larger population.

In order to obtain a representative sample, psychologists may use random sampling. A random sample is defined as one in which every member of the target population has an equal chance of being selected. If researchers were going to carry out a study at a school to find out how anxiety affects test performance, they could put all the names of the students into a hat, and then draw out 30 names. In some research, random sampling is considered to be the most desirable sampling method because it is assumed that if the sample is large enough, it is most likely to contain all the characteristics of the population. From a random sample, it is easier to generalize our findings to the larger population - that is, the behaviours observed in the random sample are assumed to be representative of those in the larger population.

A random sample aims at getting rid of bias, but it is not always successful. If the researcher conducted a study in a school with a very diverse student body, and one particular group was over-represented in the randomly chosen sample, the findings might not be representative. In order to have a sample that takes into consideration the diversity of a target population, a researcher may choose a stratified sample. A stratified sample attempts to overcome this problem by drawing random samples from each subpopulation within the target population. For example, if the school had 20 percent Indian students, then for a sample of 30 students the researcher would randomly select 6 students from the Indian population. In this way, the sample is a more accurate reflection of the actual distribution of the school population.

Sampling bias in psychology research

When psychologists use a non-random sampling technique, this may result in sampling bias -  causing some members of the population to be less likely to be included than others.  This means that participant variables in the sample may not be representative and may influence the outcome of the study.

One example of sampling bias was discussed by Sears (1986). He found that over two-thirds of research performed at universities exclusively used students as participants. This brings into question whether the researcher can make the claim that these findings can be generalized to the larger population. Sears argues that samples of opportunity made up of university students have the following problems.

●          Students have a strong need for peer approval.

●          They were pre-selected for competence in cognitive skills.

●          They are more egocentric than adults.

As you can see, the study by Sears is quite old, so you might think that the data is outdated.  Unfortunately, it is not.  Henrich, Heine, and Norenzayan (2010) carried out an analysis of psychological research and found that 67% of American psychological research uses undergraduate university students as participants.  More shockingly, they found that 96 percent of all psychology samples come from countries that make up only 12 percent of the world’s population.

The researchers coined the acronym WEIRD to describe the population studied by psychologists: Western Educated Industrialized Rich Democratic.  The researchers argue that this means that a significant amount of psychology research has studied outliers - that is, the least representative populations one could find for generalizing to a global population.

A recent study carried out by Hanel and Vione (2016) of Cardiff University showed that the issue of student samples may be more complex than we think.  The researchers collected data from over 6000 students from over 50 countries.  They found that there were significant differences between students from different countries.

For example, in New Zealand, students showed high respect for the elderly, but in Australia, the opposite was found. In China, students showed more confidence in political institutions than the general public, while in Germany, students showed less confidence. Students in the USA saw dishonest behaviours, such as stealing, as more justifiable than the public, but in India, students saw such acts as less justifiable than the public. Hanel and Vione said their results “further support the claim that generalizing from students to the general public within personal and social psychology is problematic, at least while we do not know what predicts those differences.”

If we know that sampling bias in psychological research exists, why do psychologists continue to use student samples?  First, it is convenient. A good amount of psychology research takes place at universities. Using student samples saves psychologists working at universities time and a lot of effort. Since the university can require that all psychology students take part in a certain number of studies each year, it also saves the researchers money; there is no need to pay for advertising.

 Teacher only box

1. Students have to make use of memory strategies on a daily basis to an extent that does not reflect the average person.

2. Younger people are more active on social media than many older people.  Also, although depression can occur at any age, many young people experience depression as a result of the stress of university and the search for employment.

3. Younger people may find some movies funnier than older people.  Also, testosterone levels drop as we get older.

4. As younger people, childhood trauma is more recent and may be more salient than for older people.  Therefore, it may be that it has a significant effect on one's problem-solving that is not evident in older people.

The discussion as to which study would be the most or least affected is up to the students to argue.

Checking for understanding

Ethics in research