Pesticide educators today are expected to evaluate Pesticide Applicator Training (PAT) programs in terms of how many applicators changed behaviors as a result of training. In 1991, Wintersteen and Poli stated "Documenting success gives interested parties an indication of the value of the PAT program... Conversely, an undocumented program is a fragile thing - virtually defenseless and by all appearances, unworthy of defense." Canerday (1989) said, "...it has become quite clear that no longer is the supporting public going to blindly continue to shovel money into the gasping mouths of every public service agency, without requiring that agency to stand and be judged in the court of accountability." Many professionals are reluctant to evaluate their programs. Ladewig (1994) pointed out reasons for the unpopularity of evaluations: we don't want others to know of our failures, we don't know how to establish value, we don't know how to collect evidence for evaluation and we don't have the time or money for evaluation. This manual attempts to counteract all four of these objections.

Don't want others to know of our failures

While it is certain that you will uncover some failures when you do an evaluation, knowing what the failures are will help you correct them. Then you will have successes to report and applicators will have better programs!

Don't know how to establish value. Don't know how to collect evidence for evaluation.

This manual takes you step by step through the evaluation process. It starts with designing a training program, describes data collection, explains statistical analysis and ends with applying the results of the evaluation to your next pesticide applicator education event.

Don't have time or money for evaluation.

None of us has unlimited time or funds. This manual covers three methods of evaluation that are cost and time effective.

EVALUATION
To evaluate is to determine the comparative worth of something. The word comparative is important here. Evaluation does not set an absolute worth. One must always compare the result of an educational program to something. That something may be a goal or the result of training at another time or place. The purpose of evaluation can be to learn how to make a program better or to justify the existence of the program or both.

VALIDITY
If the evaluation measures what we set out to measure, then the procedure is valid. If we wanted to know how many applicators wear respirators, our evaluation instrument would not be valid if it measured the number of applicators who knew about wearing respirators or who thought they should wear respirators. These instruments would measure attitude.

RELIABILITY
If the evaluation measures the same way each time it is performed, then it is reliable. A survey administered by untrained enumerators may not be conducted the same way by each person and, therefore, would not be reliable.

GOAL
The target of the educational program is the goal. It may be to change attitudes, knowledge, skills, behavior or impact on the community. This manual is concerned only with measuring behavior changes.

REPRESENTATIVE SAMPLE
When a sample is drawn at random, it is representative of the population. If the sample is not representative, then the evaluator must characterize the sample and characterize the population. The evaluator must point out differences and similarities and take these into consideration when drawing conclusions. Using a non-representative sample does not mean that the evaluation has to be thrown away. It just means that the researchers will have extra work to do and they must make conservative generalizations.

COVERAGE ERROR
A sampling frame is a list of persons from which a sample is drawn. If the sample frame does not include all elements of the population that the researchers wish to study, then there will be coverage error. The degree of coverage error is a function of how different the missing, ineligible or duplicate entries are from the target population.

SAMPLE SIZE AND SAMPLING ERROR
The number of individuals included in the evaluation must be large enough to yield meaningful results. Unfortunately, researchers sometimes don't find out that the sample is too small until the evaluation is completed and the statistical analysis gives unusable results. An evaluation with a sample size too small will have too much sampling error. This will not allow the researcher to confidently generalize from the sample to the population. Always use a sample size as large as time and funds will allow. And be ready to increase the sample next time if there is too much sampling error.

QUESTIONS AND MEASUREMENT ERROR
Questions on an evaluation have to be asked in an objective manner and in a format that will not encourage the respondent to give the answer he/she thinks you want to hear. Measurement error is the difference between a respondent's answer to a question and the real answer that the respondent should have given. This source of error may come from the way the questions are worded or the belief by the respondent that the evaluator wants a certain answer.

NON-RESPONSE ERROR
If a significant number of people do not respond to the evaluation and if these non-respondents are different from those who do respond, there will be non-response error. One can never assume that non-respondents would have answered in the same way as the respondents. This manual gives some standard methods to increase the response rate in a mail survey.

Salant and Dillman list four characteristics of a perfectly accurate survey, after stating that such a survey is seldom, if ever, conducted:

1. every member of the population would have an equal (or known) chance of being selected for the sample,
2. enough people would be sampled randomly to achieve the needed level of precision,
3. Clear, unambiguous questions would be asked so that respondents are both capable of and motivated to answer correctly, and
4. everyone in the sample would respond to the survey, or non-respondents would be similar to the respondents on characteristics of interest in the study.

EDUCATING TO INCREASE ADOPTION OF SAFETY PRACTICES

1. The new behavior holds some value to them.
2. The benefits are greater than the cost.
3. The person has the resources for making a behavior change:

	knowledge	labor
	time	equipment
	skills	money

4. They see the behavior as attainable.
5. They get support and reinforcement for making the behavior change from peers, family, society or from themselves.
6. The old behavior is no longer really an alternative, or it is not as desirable.

1. The new behavior has no value to them.
2. The costs are too high.
3. The person does not have the resources for making a behavior change:

	knowledge	labor
	time	equipment
	skills	money

4. They don't see the behavior as attainable.
5. They don't get support or reinforcement for making the behavior change from peers, family, society or from themselves.
6. The old behavior is still an alternative and probably very easy to fall back on.

Step I. PRE-CONTEMPLATION

At this time you must provide facts, create awareness and interest in the new behavior. You could tell the applicators how much contamination lands on the forearms during handling. You could demonstrate with florescent dye and blacklight how much exposure is avoided with long sleeved shirts or forearm protectors. Fact sheets and newsletter articles would reinforce this information.

Step II. CONTEMPLATION

At this time you need to de-market the old behavior -- eliminate competition from old behavior patterns. You should discourage the old behavior of wearing tank tops, and t-shirts. You also need to show how benefits exceed costs. Explain that the benefits of reducing exposure to pesticides (reduction in illness, doctor bills, and lost pay from sick days) are greater than the costs of wearing long sleeved shirts (heat stress and temporary discomfort).

Step III. PREPARATION

Step IV. ACTION

• Where to buy forearm protectors
• The proper use and maintenance of forearm protectors.
• How to properly launder long sleeved shirts.
• How little time it takes to put on a long sleeved shirt or forearm protection.

Step V. MAINTENANCE

• Put a quiz that surveys use of PPE in your newsletter. Have a scoring system that praises applicators who wear long sleeved shirts.
• Have an annual award for the most safety conscious applicator. Ask county agents to hold contests to identify winners.
• Ask the Secretary of Agriculture, Farm Bureau, grower groups or pest control associations to recognize individuals who practice safety.
• Keep telling applicators they're doing the right stuff!

PLANNING AND EVALUATING EDUCATIONAL PROGRAMS

METHOD I: PESTICIDE USE-OBSERVATION

PROGRAM PLANNING TO IMPROVE COMPLIANCE WITH A TARGETED PESTICIDE HANDLING ACTIVITY/POLICY IN THE STATE

INTRODUCTION

PROCEDURE

I. Target Behaviors

Staff from the SLA and state cooperative extension system (CES) should discuss patterns of pesticide handling compliance in the state. One pesticide use activity or policy should be chosen as a target. The targeted activity/policy may be the most common compliance problem, the most serious, the activity with the greatest chance of change, or a combination of these factors. Each state should establish its own criteria when deciding which compliance activity/policy to target.

 

 

Gather base line data before you start the study. State inspection check lists and/or photos from pesticide use-observations should be reviewed for the past pesticide use season. Records should be made of numbers of handlers who comply with the targeted activity and numbers who do not comply. While trying to gather the base line data, you may find that the inspector check list is not designed to measure the pesticide practice that you wish to target. At this time you could pick another behavior to target. Make sure it is one that can be measured from the current check list. Or you may decide to revise the inspector check list and schedule this evaluation for a latter date.

II. Plan Educational Events and Set Goals

SLA and CES staff should plan educational events that will encourage handlers/ applicators to adopt the targeted practices. The audience must be certified applicators. Educational events should be chosen that will reach as large an audience as possible. In most states, only a small proportion of certified applicators will be selected for use-observations by the SLA. Education efforts should be designed to ensure that the few individuals selected for observation have in fact received the educational programming. Thus, far reaching educational programing is essential. Examples are newsletters that are mailed to all certified applicators; direct mailings of fact sheets or brochures to all applicators; required recertification courses; or popular, well-attended grower meetings. Designing a combination of these educational events will increase the chance that those applicators selected for use-observation have been exposed to the educational efforts. Keep in mind that it is not legal to target attendees who attended a specific training program for use-observation by the state. In some states that would constitute harassment.

 

 

The next step is to set the goal. The organizers should decide how much of an increase in compliance they can expect given their educational efforts, the attitudes of the regulated community, and public sentiment. As a guideline in setting the goal, the organizers should look at the success rate of comparable educational events in the state or other states. If similar projects have not been conducted, then the organizers must look to the breadth and depth of their planned educational events. A short term event for few handlers will probably result in a small increase in compliance state wide. This increase may not show up in the season's use-observations. If the event is more extensive, then a larger goal can be set. However, organizers should not plan resource-consuming educational events out of fear that a measurable behavior change won't be obtained. Such negative results can be used as a guide to planning the next year's educational events. As a final guideline, organizers should look at their base line data. A pesticide practice with a low compliance rate should increase with education, while a practice that is already accepted by applicators may not show improvement even with directed education.

III. Prepare Evaluation Instrument and Validate

Usually when conducting an evaluation, the researcher must show that the instrument and procedures measure what is intended in a consistently reliable manner. In the context of this study, validation would document that each use-observation is done the same way as all other use-observations in the state. Validation would also show that the inspection instrument and procedure tells the SLA whether the applicator is complying with selected pesticide handling practices. EPA supports annual inspector training in each region. This training provides the consistency needed for EPA to document that use-observations are valid. As long as this evaluation in your state is conducted by state inspectors who have attended EPA supported annual training, no further validation is needed.

IV. Gather Data

It is important to show cause and effect between the educational efforts and compliance. This can be measured by asking applicators if they read the newsletter, brochure or fact sheet or if they attended the recertification program or growers meeting. Applicators who were not exposed to the educational efforts must be dropped from the study. Those who were exposed to the educational event may be retained in the study.

 

 

The study should be divided into three equal time periods: before, during and after the educational events. State inspection check lists and/or photos from pesticide use- observations should be reviewed for the period preceding the educational events. Records should be made of numbers of handlers in the study who comply with the targeted activity and numbers who do not comply. The proportion of compliance should be compared to compliance during and after the educational events.

V. Analyze and Interpret Data

Comparisons should be made between the goal and the observed rate of compliance. Comparisons can also be made between the base line data and compliance during the educational events and compliance after the educational events.

 

 

With sample sizes of 120 applicators or more, the Z test of pair-wise comparisons of proportions (Moore, 1993) can be used to test the hypotheses that the observed proportion is different from the goal:
 

Where:

Z = the test statistic

g = the goal -- the number of applicators you hope will comply with the targeted behavior.
o = the observed number of applicators who comply with the targeted behavior.
n = the number of applicators inspected by the state during the use-observation who were exposed to the educational effort.

If the Z value is greater than 1.96, then the difference between the goal and the observed value is statistically significant. If your goal is higher that your observed value, you have not reached your goal. If your goal is lower that your observed value, you have exceeded your goal. If the Z value is less than 1.96, then the difference between the goal and the observed value is not statistically significant. This means that you can show no difference between the goal and the rate of compliance and you can consider your educational efforts to be on target.

If you are working with sample populations of fewer than 120 applicators, you need to consult a Student's t-distribution table or your statistician.

The Z test (Salant and Dillman) can also be used to test the hypotheses that the base line value is different from rates of compliance observed later in the study:

 
 

Where:

Z = the test statistic

a = the number of applicators who comply with the targeted behavior during the first time phase.
b = the number of applicators who where exposed to the educational effort during the first time phase. For calculating base line data, b = the number of applicators inspected during the use-observation.
x = the number of applicators who comply with the targeted behavior during the next time phase.
y = the number of applicators who where exposed to the educational effort during the next time phase.

The examples given in the case study below show how to use both formulas.

VI. Apply Results to Pesticide Education

If the evaluation shows an increase in compliance with the targeted activity after educational events, a state may theorize that the educational events contributed to compliance. The results from this evaluation can be used to design more effective educational programs in the state. Program coordinators will learn how extensive a program must be to have an impact. Other compliance activities/ policies in the state can then be targeted in the same manner as the first.

ADVANTAGES OF THE USE-OBSERVATION TECHNIQUE

1. Targets one specific activity in the state.
2. Study sample randomly chosen; therefore, it is representative of all applicators in the state.
3. Uses existing data gathering forms -- the SLA state inspector check list.
4. Uses existing data gathering procedure -- the SLA use-observation.
5. Data gatherers (state inspectors) are already trained at annual regional meetings.
6. Validation of procedure already done by EPA.
7. Fosters cooperation between SLA, CES & other agencies.
8. No mailings

DISADVANTAGES OF THE USE-OBSERVATION TECHNIQUE

1. Educational programing must reach the entire population of pesticide applicators.
2. Extra question must be added to state inspector check list to establish cause and effect relationship.
3. An employee must to go through inspection records and extract data.
4. Small sample size of use-observations may result in conservative generalizations.

CASE STUDY: THE LONG-SLEEVED SHIRT CAMPAIGN

Records were made of the numbers of applicators wearing long-sleeved shirts for the year before the newsletter stories began, the year while the stories were running and one year after the stories appeared. After the study began, state inspectors asked the additional question, "Do you read Pesticide Briefs?" Records were made of numbers of handlers inspected in each time phase and numbers who read the newsletter. 

These results show that there was a significant difference between the rate of compliance before newsletter stories and during newsletter stories. No firm conclusion can be drawn about the lasting impact of newsletter stories. The Z test shows no significant difference between either "before education" and "after education" or between "during education" and "after education." With results like this, it is wise to draw conservative conclusions. 

No significant difference was found between the goal and rate of compliance during the run of newsletter stories. 

And in this case, no significant difference was found between the goal and rate of compliance after the run of newsletter stories. After analysis, the researchers concluded that CES/SLA newsletter stories increased compliance among pesticide handlers by at least 10%. If less than a 10% increase in compliance had been reached, educational efforts would have been expanded and the study repeated. Next the researchers can conduct training with a goal of increasing compliance by a higher percentage.

METHOD II: PESTICIDE APPLICATOR TRAINING EVALUATION FORM

PROGRAM PLANNING TO INCREASE ADOPTION OF SAFETY PRACTICES THAT REDUCE EXPOSURE DURING PESTICIDE HANDLING ACTIVITIES POSING THE GREATEST HAZARDS TO APPLICATORS IN THE STATE

INTRODUCTION

PROCEDURE

Plan Educational Events and set goals

Staff from the SLA and CES should plan an educational workshop that includes personal safety procedures for pesticide handlers/ applicators. This may be initial certification training, recertification training, or WPS training. Organizers should estimate the proportion of handlers/applicators who will adopt new safety practices as a result of the training. As a guideline in setting the goal, the organizers should look at the success rate of comparable training in other states. If similar projects have not been conducted, then the organizers must look to the breadth and depth of their training.

Print evaluation form and validate

Workshop organizers may duplicate the evaluation instrument, "PUT SAFETY INTO PRACTICE! PESTICIDE APPLICATOR TRAINING EVALUATION," (Appendix A) or they may design a form of their own. Using the no-carbon-required (NCR) duplication procedure simplifies the study. The validation conducted by the University of Delaware can be adopted or the organizers can conduct their own validation following the example given in the case study below.

Gather Data

After the workshop, trainees should complete the evaluation instrument, "PUT SAFETY INTO PRACTICE! PESTICIDE APPLICATOR TRAINING EVALUATION." They should identify the most hazardous activity in their work, then list three new safety practices that they can adopt to alleviate this hazard. The applicators should keep the bottom copy of the 3-copy NCR form and turn in the other two copies to the trainer. After 3-12 months of the pesticide use season, one copy of the form should be returned to handlers/ applicators with instructions to check those practices that they adopted. Confidentiality should be emphasized. Standard procedures for increasing the response rate should be followed as described in the case study below. The proportion of handlers/ applicators adopting new safety practices should be calculated.

Analyze and Interpret Data

The p-formula of Cocran (97) can be used to derive the 95% confidence interval on the proportion of applicators adopting new safety practices:

 

 

Where:

P = The proportion of applicators adopting the new behavior.
t = 1.96
f = n/N, the sampling fraction
n = The number of applicators returning the evaluation form after marking.
N = The number of forms sent to applicators for marking.
p = s/n, the fraction complying
s = The number of applicators who adopt new behaviors.
q = 1-s/n, the fraction not complying

Apply Results to PAT and WPS Training

The results from this evaluation can be used to design more effective educational programs in the state. If handlers/ applicators do not follow through with their pledge to adopt new practices, then program coordinators should expand training in those areas.

ADVANTAGES TO USING THE TRAINING EVALUATION FORM TECHNIQUE

1. Handler/ Applicator buy-in because they choose the behaviors to adopt.
2. Uses existing training events.
3. Tracks individuals as well as groups.
4. Cause and effect relationships can be correlated.
5. Good learning tool for handlers/ applicators.
6. Chance for second learning experience when form is returned to them and they are reminded of their commitments.

DISADVANTAGES TO USING THE TRAINING EVALUATION FORM TECHNIQUE

1. Requires mailing of post-treatment form.
2. Requires printing of "official" form.
3. Each state must either conduct an expensive and time consuming validation or use the validation conducted by the University of Delaware.

CASE STUDY: PUT SAFETY INTO PRACTICE!

After PAT, trainees complete the evaluation instrument, "PUT SAFETY INTO PRACTICE! PESTICIDE APPLICATOR TRAINING EVALUATION." They identify the most hazardous activity in their work, then list three new safety practices that they can adopt to alleviate this hazard. Time for completing the evaluation form is allotted on the agenda. Each applicator keeps the bottom copy of his/her NCR form and turns the other two copies into the trainer. Forms similar to the one in this manual have been used in Delaware since 1991.

From 12/94 to 9/95, 285 applicators attended PAT. However, as shown below, not all of them participated in the study:

	285	Attendees.
-	47	Did not complete the form properly.
-	66	Either did not obtained licensing, apply pesticides after training or reside in the state.
_________
	172	Participated in the evaluation by completing the evaluation form properly, obtaining licensing, applying pesticides after training and residing in the state.

In January 1996, the 172 completed forms were returned to the individuals. Applicators were asked to check the practices that they used at least once since training. They were given instructions to return the form to the University of Delaware. A business reply envelope was enclosed. The mailing sequence was: heads-up post card, letter with original evaluation form, post card reminder, second letter with photocopy of evaluation form. In addition, an attempt was made to reach non-respondents by phone. Of the 172 forms, 135 were returned to the University. Of these forms, 133 indicated that those applicators used at least one new safety practice at least once.

In March 1996, the evaluation instrument was validated through face-to-face interviews with a subsample of 101 of the 133 respondents who indicated that they had used a new practice. Enumerators hired by the Delaware Agricultural Statistician were contracted to conduct the interviews using a second instrument, "UNIVERSITY OF DELAWARE COOPERATIVE EXTENSION PESTICIDE APPLICATOR SAFETY INTERVIEW," (appendix B). The enumerator read each question to the applicator and recorded his/her answer on the interview form.

After the completed forms were submitted to the University of Delaware, they were paired with the first form, "PUT SAFETY INTO PRACTICE! PESTICIDE APPLICATOR TRAINING EVALUATION." Pairs of forms, with names removed, were given to a panel of judges. Each behavior that an applicator indicated as having practiced at least once since training on the first form was searched for on the second form. If responses on the second form indicated that the applicator did practice that behavior, "seldom, sometimes, often or always," then a positive score was recorded. If the majority of such scores were found to be positive, then the evaluation instrument, "PUT SAFETY INTO PRACTICE! PESTICIDE APPLICATOR TRAINING EVALUATION," was determined to be a valid instrument. It is important to keep in mind that the validation process is concerned only with validating instruments, not people. Finding the instrument "valid" means that the instrument is designed in such a way as to allow individuals to express themselves accurately.

The panel of judges found that each behavior that an applicator indicated as having practiced at least once on the first form could be found on the second form as either, "seldom, sometimes, often or always." Thus all scores were positive. The evaluation instrument, "PUT SAFETY INTO PRACTICE! PESTICIDE APPLICATOR TRAINING EVALUATION," was determined to be a valid instrument.

The results of this study show that between 77.29 and 77.31% of applicators (133/172) who participated in the evaluation process practiced at least one new safety behavior at least once after training. No conclusion can be drawn about the 37 individuals who were non-respondents. Nor can any conclusion be drawn about the 47 individuals who did not participate in the study because they incorrectly completed the form or resided out-of-state. No conclusion can be drawn about the 66 individuals who did not obtain certification or did not apply pesticides after training. The next phase of this study will be to determine the proportion of applicators adopting new safety practices as routine.

METHOD III: NEEDS ASSESSMENT

PROGRAM PLANNING TO INCREASE ADOPTION OF TARGETED SAFETY PRACTICES IN THE STATE

INTRODUCTION

PROCEDURES

1. Target Behaviors
Staff from the SLA and CES should discuss applicator safety practices in the state and target one or two broad areas for improvement. These areas could be calibration, equipment maintenance, drift control, environmental protection, pesticide handling safety, PPE, record keeping, WPS, or understanding the label.
2. Plan Educational Events and Set Goals
Staff from the SLA and CES should plan educational events throughout the year that address pesticide handling/ application issues. Topics could be: calibration, equipment maintenance, drift control, environmental protection, pesticide handling safety, PPE, record keeping, WPS, or understanding the label. Delivery methods could be workshops, seminars, newsletters, and fact sheets. Training could be conducted for initial certification, recertification, or WPS.

Trainers should set relative goals for compliance within the one or two targeted categories. Although complete compliance is always desirable, setting a lower goal will allow the trainers to focus on the next set of issues more quickly. In this manner the trainers will have opportunity to address most pesticide applicators issues in a few years. Then they can return to the first set of issues for further emphasis.

3. Design Needs Assessment Instrument
A needs assessment instrument (Appendix C) should be designed to measure the extent to which handlers/ applicators practice indicator safety behaviors within the targeted categories. It is important to keep the instrument short. Twenty questions may be too many. The questions should be designed in a manner so that the handler/ applicator does not give the answer that is "expected." However, since this form will be used to obtain a relative measure of compliance, as opposed to an absolute measure of compliance, the organizers should not place too much concern on this issue. The form must be confidential and anonymous.
4. Data Gathering Events
Staff from the SLA and CES should identify one or more meetings to be the annual data collection event. The needs assessment instrument should be administered at this meeting. The first data collection will provide the base line data. Subsequent data collections will provide comparative data.
5. Analyze and Interpret Data
Results can be reported as proportion of handlers/ applicators practicing indicator behaviors as a function of time. For example, "Applicators reading the pesticide label changed from x% to y% in two years," or "held at z% for two years."
6. Apply results to PAT
The results from this evaluation can be used to design more effective educational programs in the state. If the proportion of handlers/ applicators practicing the indicator behaviors is smaller than the goal, then program coordinators should expand training in those areas to meet the needs. If the proportion of handlers/ applicators practicing the indicator behaviors is equal to the goal, training has been successful. The instrument can be modified to measure other needs and the trainers may move on to another area of pesticide safety for study.

ADVANTAGES TO USING THE NEEDS ASSESSMENT TECHNIQUE

1. Targets specific behaviors in state.
2. Quick and easy to administer.
3. No mailings.
4. Uses existing training events.
5. No validation required for a needs assessment.

DISADVANTAGES TO USING THE NEEDS ASSESSMENT TECHNIQUE

1. Handler/ Applicator can not choose safety practices to measure.
2. Questions need to be designed in such a way that handlers/ applicators don't give the answer that you want to hear.
3. Form will need to be revised as target behaviors are reached.
4. Requires expensive printing of "official looking" form.
5. Correlation between cause and effect not as strong as other methods.
6. Tracks only groups not individuals.
7. Must be used for more than one year to show trends

DELAWARE CASE STUDY: READ THE LABEL AND WEAR PPE

Acknowledgments

Kerry Hoffman, Extension Pesticide Education Coordinator, Pennsylvania State University, University Park, PA.

John Impson, USDA/CSREES National Program Leader, Health, Environmental and Pesticide Safety Education (HELPS), Washington, DC.

Larry Olsen, Extension Pesticide Coordinator and Integrated Pest Management Coordinator, Michigan State University, East Lansing, MI.

Grier Stayton, Pesticide Compliance Administrator, Delaware Department of Agriculture, Dover, DE.

John Tacelosky, Certification & Training Coordinator, Pennsylvania Department of Agriculture, Harrisburg, PA.

Larry Towle, Agriculture Specialist, Delaware Department of Agriculture, Dover, DE.

And I would like to thank all the Pesticide Coordinators and State Lead Agency staff who attended the workshops that I conducted on evaluation over the last 4 years. Their suggestions transformed this manual from theory into practice.

Literature Cited

Canerday, T. Don. 1989. Evaluation systems. Abstract. In Proceedings, National Pesticide Applicator Training/Certification Workshop, March 28-30, 1989, Denver, Colorado.

Cochran, William G. 1977. Sampling Techniques, 3rd Edition. John Wiley & Sons, Inc., New York.

Ladewig, Howard. 1994. Evaluation Research and Cooperative Extension: Friends or Foes? Abstract. Presented at University of Delaware Cooperative Extension Professional Development Annual Conference, May 24-26, 1994.

Moore, David S. 1993. Introduction to the Practice of Statistics, 2nd Edition. W.H. Freeman & Co., New York.

Nowak, Peter. 1994. Evaluation: Theory and Practice. In Conference proceedings, Strategies to manage Pesticides, September 17-22, 1994, University of Wisconsin-Madison, Wisconsin.

Rogers, Everett M. 1983. Diffusion of innovations. 3rd edition. The Free Press, A Division of Macmillan Publishing Co., Inc., New York.

Salant, Priscilla and Don A. Dillman. 1994. How to Conduct your own Survey. John Wiley & Sons, Inc., New York.

Wintersteen, Wendy K. and Bonnie Poli. 1991. Documenting PAT Successes. In Proceedings National Pesticide Applicator Training and Certification Workshop, April 16-18, Arlington, Virginia. Iowa State University Extension Service. 

Appendix A

PUT SAFETY INTO PRACTICE !

We want to know if Delaware's Pesticide Applicator Training is meeting your needs. Your response on this form will help us design better educational programs. Thanks for your help.

In the space below:

1. Describe the pesticide handling activity in your work that is the most hazardous to you.
2. List 3 new safety practices you can use to prevent exposure during this hazardous activity.
3. Keep the pink copy for your files. Give the yellow and white copies to the trainer.

1. The most hazardous pesticide activity in my job is:

2. Three new safety practices that will prevent pesticide exposure during this hazardous activity are:

	(Do not mark in this space)
a. ____________________________________	[ ] Used at least once
	[ ] Adopted routinely
b. ____________________________________	[ ] Used at least once
	[ ] Adopted routinely
c. ____________________________________	[ ] Used at least once
	[ ] Adopted routinely

Name 

Applicator keep pink copy. Give yellow and white to the trainer. 5/96

APPENDIX B

INTERVIEWER'S NAME ___________________________________________

INTERVIEW DATE _____________

INTERVIEW CODE _____________

DO YOU HAVE ANY COMMENTS ON THE UNIVERSITY OF DELAWARE COOPERATIVE EXTENSION PESTICIDE APPLICATOR TRAINING PROGRAM?
 
 

WOULD YOU LIKE A COPY OF THE REPORT THAT DR. WHITNEY WILL WRITE ON THIS STUDY?
 
 
 

THANKS FOR YOUR HELP!

May, 1996

For each question, circle the number that applies:

 1 = Never
2 = Seldom
3 = Sometimes
4 = Often
5 = Always
N/A = N/A

APPLICATION EQUIPMENT, DRIFT CONTROL AND CALIBRATION

2. HOW OFTEN DO YOU MAKE SURE YOUR APPLICATION EQUIPMENT IS CLEAN & IN GOOD WORKING ORDER?

1

2

3

4

5

N/A

3. HOW OFTEN DO YOU MAKE SURE YOUR APPLICATION EQUIPMENT IS CALIBRATED?

1

2

3

4

5

N/A

4. HOW OFTEN DO YOU MAKE SURE YOUR APPLICATION EQUIPMENT IS USED PROPERLY?

1

2

3

4

5

N/A

PROTECTING THE ENVIRONMENT

6. HOW OFTEN DO YOU TAKE MEASURES TO PROTECT OFF- TARGET ORGANISMS (HONEY BEES, BIRDS, FISH) WHEN USING PESTICIDES?

1

2

3

4

5

N/A

SAFE TRANSPORTATION, STORAGE, HANDLING & DISPOSAL

8. WHEN YOU STORE, PESTICIDES HOW OFTEN DO YOU TAKE STEPS TO DO SO IN A SAFE MANNER?

1

2

3

4

5

N/A

9. WHEN YOU DISPOSE OF EXCESS PESTICIDES &/OR CONTAINERS HOW OFTEN DO YOU TAKE STEPS TO DO SO IN A SAFE MANNER?

1

2

3

4

5

N/A

10. WHEN YOU MIX & LOAD, OFTEN DO YOU TAKESTEPS TO DO SO IN A SAFE MANNER?

1

2

3

4

5

N/A

PESTS & PEST CONTROL

12. HOW OFTEN DO YOU USE THE LOWEST RATE OF PESTICIDE POSSIBLE?

1

2

3

4

5

N/A

13. HOW OFTEN DO YOU IDENTIFY THE PEST BEFORE CHOOSING YOUR CONTROL MEASURE?

1

2

3

4

5

N/A

RECORD KEEPING

UNDERSTANDING THE LABEL

PPE AND SAFETY

17. HOW OFTEN DO YOU MAKE SURE THAT YOU ARE PROTECTED FROM EXPOSURE TO PESTICIDES?

1

2

3

4

5

N/A

18. HOW OFTEN DO YOU MAKE SURE THAT YOU ARE NOT EXPOSING OTHERS TO PESTICIDES?

1

2

3

4

5

N/A

19. HOW OFTEN DO YOU USE THE RIGHT PESTICIDE TO DO THE JOB WITH THE LEAST TOXICITY TO HUMANS?

1

2

3

4

5

N/A

20. HOW OFTEN DO YOU WEAR THE CORRECT PPE (PERSONAL PROTECTIVE EQUIPMENT)?

1

2

3

4

5

N/A

21. HOW OFTEN DO YOU CLEAN, MAINTAIN & STORE YOUR PPE PROPERLY?

1

2

3

4

5

N/A

22. HOW OFTEN DO YOU KEEP UP YOUR EDUCATION ON PESTICIDE SAFETY?

1

2

3

4

5

N/A

APPENDIX C

PESTICIDE APPLICATOR TRAINING EVALUATION INVENTORY

This form is confidential -- don't sign your name.
 

Thank you for your help,

Susan P. Whitney, Pesticide Educator

C O N F I D E N T I A L I N F O R M A T I O N

THE PESTICIDE LABEL

1. When do you read the pesticide label?

• Before the product is bought
• Before mixing and loading
• Before the application
• Before disposing of the container
• Before entering the field after an application

• Signal Word
• Statement of Practical Treatment
• Precautionary Statements
• Personal Protective Equipment (PPE)
• User Safety Recommendations
• Environmental Hazards
• Physical or Chemical Hazards
• Directions for Use
• Agricultural Use Requirements
• Mixing Directions
• Storage and Disposal

• Signal Word
• Statement of Practical Treatment
• Precautionary Statements
• Personal Protective Equipment (PPE)
• User Safety Recommendations
• Environmental Hazards
• Physical or Chemical Hazards
• Directions for Use
• Agricultural Use Requirements
• Mixing Directions
• Storage and Disposal

PERSONAL PROTECTIVE EQUIPMENT (PPE)

1. How do you decide what PPE to wear?

• I wear what I have on hand.
• I wear what the label says.
• I wear what I see other applicators wearing.

• I check my PPE before each use.
• I check my PPE once a week.
• I check my PPE once a month.
• I check my PPE once a season.

• I wash them after each use.
• I wash them once a week.
• I wash them once a month.
• I wash them once a season.

• I throw them away when I see rips or tears.
• I throw them away when they look dirty.
• I throw them away after 3 or 4 uses.

• I have a respirator fit test when I buy a new respirator.
• I have a respirator fit test once a season.
• I have a respirator fit test every year or so.

• I keep them in the pesticide storage room.
• I hang them in a separate room from the pesticide storage room.
• I keep them in a sealed plastic bag outside the pesticide storage room.