Throughout this course, we will be looking at many different ways to discuss and evaluate research.  It is important that we look at the strengths and limitations of psychological research, but in order to do that, we have to understand how psychologists talk about their research.  When you read an original piece of research, you will see that it always ends with a "discussion."  In this discussion section of the report, the researcher reflects on the potential limitations of the study and makes suggestions for further research.

In order to prepare you for reading and discussing research in this course, this final part of the introduction to research outlines some of the key concepts and vocabulary that are important for understanding these discussions.

Procedures

When researchers are designing a procedure, there are some basic decisions that they need to make. These basic decisions may have a significant influence on the quality of their research findings.

First, the location of the study must be decided.  Psychologists may either study a behaviour in a controlled laboratory setting, or they may carry out their research in a naturalistic setting - what is known as a field study.  Laboratory studies have the advantage that the situation can be controlled.  This allows the researcher to make sure that the variable that is being studied is not influenced by outside factors.  Field studies have the advantage of observing behaviour in "real life."  Ideally, when testing a theory the findings will be similar whether tested in the lab or in the field.

A second important consideration is whether a study is retrospective or prospective in nature. If a study is retrospective, it means that the researcher is asking the participant about past behaviour.  When a researcher asks an athlete about his childhood, especially with regard to his parents' attitude about fitness, this is a retrospective approach.  This is a way that researchers attempt to find a relationship or correlation between one's childhood and their adult behaviour.  If a researcher wants to study brain damage, domestic violence or winning the lottery has affected one's behaviour, a retrospective approach is the only reasonable way to carry out the research.  However, you can probably see that there are limitations to such an approach.  First, we are reliant on the participants' memories.  We will see later in the course that human memory is not always reliable.  Also, it is often not possible to verify the information that the participant shares.

In a prospective study, the researcher measures a variable at the beginning of a study and then watches the effect of this variable over time. In a study of fitness, the researcher may measure middle school students' level of fitness and cardiovascular health - and then measure these variables again every five years for as long as they will stay in the study.  In this way, the researchers are not dependent on the participants' memories of their level of physical activity in their early teens. However, prospective studies take a lot longer to carry out and there is always the possibility that many of the participants will drop out of the study over time. 

When a study involves repeated observations of the same variables over long periods, it is referred to as longitudinal research. The value of longitudinal studies is that we are observing the change in individuals over time.  This is in contrast to cross-sectional research in which a researcher analyzes data collected from a population at a specific point in time.  In a study of fitness and exercise, this would mean measuring both the level of fitness activity and cardiovascular health in 20, 30, 40, 50, 60, and 70-year-olds - all in one study. Cross-sectional studies are much more efficient than longitudinal studies - they take less time to carry out and they have a lower risk of participants dropping out.  However, although the researchers may see a trend that those with higher levels of exercise demonstrated higher levels of cardiovascular fitness at all age levels, the researchers are comparing different people, so individual differences - including diet, family medical history, and job stress - may play a role in the findings.  Ideally, a theory is stronger if it is supported by both longitudinal and cross-sectional research.

Findings: validity and reliability

Interpreting findings is an essential skill for a psychologist. One way that a study can be evaluated is to assess whether it has any practical applications. An application is how a theory or empirical study is used. People often think of applications in terms of therapy, but applications can also be made in education, crime, the workplace, or sport. Some examples of applications are:

• The use of memory research to improve how we take evidence from eyewitness testimonies (an application of cognitive theory in forensic psychology)
• The use of research on the effect of light on mood in order to improve working conditions in office spaces in Denmark (an application of biological theory in occupational psychology)
• Applying findings from decision-making research to improve a football team’s performance on the pitch (an application of cognitive theory in sport psychology).
• Applying research on group behaviour in order to increase carpooling (an application of sociocultural theory in social psychology).

Another way to discuss findings is to consider whether the research does what it claims to do. This is the most basic definition of what we call validity. Psychologists discuss two major types of validity - internal and external.

Internal validity refers to how well an experiment is done, especially whether it avoids the influence of outside or extraneous variables on the outcome of the study. In order to achieve high internal validity, studies must be well controlled, and the variables must be carefully defined.  This will be discussed more in the next chapter.

When evaluating the internal validity of a study, it is important that we can agree on what is being measured.  For example, how can psychologists measure one's level of love for his partner, one's intelligence, or one's level of tolerance?  If the concept that is being studied has an agreed-upon definition and can be measured, then we can say that it has a high level of construct validity.  Often, however, there are problems with construct validity in psychological research. 

External validity is the extent to which the results of a study can be generalized to other situations and to other people. External validity is usually split into two distinct types: population validity and ecological validity. Population validity is a type of external validity that describes how well the sample used can be generalized to a population as a whole. Ecological validity is a type of external validity that looks at the experimental environment and determines how much it influences behavior.

In order to determine the level of ecological validity, two things must be considered. Firstly, the representativeness of the testing situation. This is often called “mundane realism” – or the level to which the situation represents a real-life situation. For example, memorization of a random list of words is low in mundane realism, but retelling a story that you have heard is relatively high in mundane realism. Ecological validity is not mundane realism. Representativeness of the testing situation can help ecological validity, but it is not a guarantee.

Secondly, ecological validity refers to the generalizability of the study to other settings or situations outside of the laboratory. If an experiment has been carried out in a laboratory and was so well controlled that normal influences on behaviour were eliminated, this may lead to low ecological validity. If a study lacks ecological validity, it means that what was observed in the laboratory does not necessarily predict what will happen outside the laboratory.

Finally, researchers also discuss the reliability of research. If a study is reliable, it means that the results can be replicated. Usually, reliability is used in reference to experimental study because the procedure is standardized and, theoretically, if another researcher uses exactly the same procedure, it should give the same results.

Checking for understanding

Quantitative research methods