Feedback
Creating instrumentation plans for research studies
This is an excerpt from Applied Research and Evaluation Methods in Recreation by Diane Blankenship.
The instrumentation plan is composed of a number of decisions that need to be made before beginning the study. These decisions are made to determine
what data are needed to answer the research questions,
- how to gather the data,
- when to gather the data,
- where to gather the data, and
- how to analyze the data.
These decisions must be made as part of the instrumentation plan for the study. They help guide the progress of the study to the ultimate goal of gathering data and formulating conclusions to answer the research question.
Selecting Data
The previous decisions help define what data need to be gathered and why the data are important. What is data? Data refers to the information that is gathered to answer the research question. Data can be numbers, words, or actual objects, such as photos, articles, or video. The instrumentation process defines what data need to be collected and the timing of the data collection process.
What data are needed if the researcher is examining the relationship between taking swimming lessons and purchasing a summer session pass? The researcher needs the records of swimming lesson participants and season pass holders. Examining any other type of data, such as instructor evaluations, is a waste of time if it does not focus on answering the research question. If the researcher is examining the factors that lead a person to purchase a summer pass for the pool, interview data are needed to fully understand the factors in the decision-making process of the customer. By using the interview process, the researcher can conduct a two-way conversation in order to explore the factors that lead to purchasing the season pass. This type of data is very comprehensive in exploring the decision-making process, and it is more effective than looking at only one or two variables. In the example of the fitness center evaluating the personal trainers, a written survey will provide the data needed to evaluate the performances of the trainers. By identifying the specific type of data needed to answer the research question, the researchers' efforts are properly focused. Table 8.3 summarizes what data are needed in relation to the previous decisions in the instrumentation plan.
Gathering Data
The next natural question is, how will the data be collected? To answer this question, the researcher needs to identify whether the study is an evaluation, a quantitative study, or a qualitative study. This helps determine how to collect data. What instrument will be used to collect data? The researcher has a variety of options that may be used as a data collection instrument, such as surveys, interviews, observations, or rating instruments. Table 8.4 summarizes the decision of how to gather data along with the previous pieces of the instrumentation plan.
Determining When to Collect Data
Once the researcher decides what data to collect and how to collect them, the researcher must determine when to collect the data, where to collect them, and who should collect them. The results of the study can be influenced by the time that data are gathered, where the data are gathered, and who gathers the data. Studies that only use literature or the records of the agency are not influenced by these decisions. This is the case with the first research question concerning the relationship between taking swimming lessons and purchasing a summer season pass. Once the majority of season passes have been sold, the data should be gathered at the beginning of the summer.
This is not the case when data are being gathered from human subjects. Timing considerations that must be addressed are the month, the day of the week, and the time of day that the data are collected. In the example of identifying what factors lead a person to purchase a season pass to the pool, the interview should be conducted as soon as possible after the purchase of the pass. If the researchers wait weeks or months after the purchase, the subjects may not recall what factors led them to make the purchase. This interview could be done over the phone or in a small group.
For the fitness center evaluating the personal trainers, a specific time frame is necessary in order to ensure a high response rate. The manager decides to give the clients an evaluation form to complete after their last session with their personal trainer (before they leave the facility). This is a better plan than mailing the surveys to the clients' home weeks after the clients' last session with their trainer. The timing of data collection can affect the quality of the data received from the subjects. Table 8.5 presents this information, along with the previous decisions in the instrumentation plan.
Determining Where to Collect Data
The place that data are collected and the person collecting data must be specifically defined in the instrumentation plan. The best plan is to standardize the place and person for the data collection. This standardization helps enhance the truthfulness and validity of the data. The subjects should be in an environment where they feel at ease so that they will answer questions honestly. These two considerations are not an issue for the first research question because the data are coming from the documents of the agency and not from individuals.
In the case of identifying the factors that lead to purchasing a season pass for the pool, the interviews will be conducted by telephone. The people conducting the interviews should be trained in how to conduct an interview and how to record the information accurately. Each person being interviewed will be at home while participating in the interview, which is a comfortable environment for that individual.
In the example of evaluating personal trainers, having the clients' trainer administer the survey in the gym will most likely provide invalid data. The ideal situation would be to have one staff person provide the clients with a quiet room to complete the survey. Then the subject should return the survey to the same staff member who gave the survey to the subject. This standardizes where the data are collected and by whom. Table 8.6 provides a summary of all the instrument pieces and provides examples of each decision that needs to be made in developing the instrumentation plan for a study.
Analyzing Data
The next planning decision to make is how to analyze the data and what to do with the information once the analysis is completed. The type of analysis used with data is determined by whether the data are quantitative or qualitative data. To analyze quantitative data, some type of statistical analysis is used to provide the results. The type of statistical analysis used with data must be thought out and documented in the instrumentation plan.
Some of the most frequently used options for statistical analysis will be covered later in this text. Qualitative data are analyzed through a coding process that identifies themes; these themes become the foundation for the conclusions of the study. This type of data analysis will also be covered later in this text. The results of the data analysis should be summarized and presented in a report to supervisors and other parties for review. Research and evaluation efforts yield a wealth of information that can be used to educate commissioners, city councils, customers, and other decision makers. Studies that document outcomes of the programs and benefits to the community serve as a powerful tool that allows the agency to document its benefits to the community through facts and data, not perceptions and speculations. In today's cost-conscious society, the agency must prove that it is contributing to the quality of life of the community and must document the outcomes from the programs. This type of documentation and evidence is also a requirement for CAPRA accredited agencies and represents an ongoing evaluation process within the agency. Table 8.7 summarizes the key decisions that need to be made in the development of the instrumentation plan.
This is an excerpt from Applied Research and Evaluation Methods in Recreation.
More Excerpts From Applied Research and Evaluation Methods in RecreationSHOP
Get the latest insights with regular newsletters, plus periodic product information and special insider offers.
JOIN NOW