Program Quality - AITEPYD
The Afterschool Initiative’s Toolkit for Evaluating Positive Youth Development (AITEPYD) contains a number of question sets that can be used to measure positive youth development outcomes and program quality. Positive youth development domains assessed include: Academic Success, Arts and Recreation, Community Involvement, Cultural Competency, Life Skills, Positive Core Values, Positive Life Choices and Sense of Self. Program quality domains assessed include: Supportive, Caring Climate (Positive Adult Relationships), Youth as Partners and Resources, Safety and Trusting Environment, and Attractive and Meaningful Activities.
- Bullying: To What Extent Does the Program Offer a Positive Developmental Setting?
- Bullying: To What Extent Are You Using Effective or Promising Practices?
- OST: To What Extent Does the Program Offer a Positive Developmental Setting?
- OST: To What Extent Are Participants Engaged in the Program?
- SB Substance Use: To What Extent Are Participants Engaged in the Program?
- SB Substance Use: To What Extent Is the Program Being Delivered by Competent Facilitators?
The Colorado Trust. After-School Initiative’s Toolkit for Evaluating Positive Youth Development. Denver, CO: The Colorado Trust; 2004.
Adaptation made/subset of questions selected:
Only questions from Section 2: Program Quality (p 29-32), were selected for this tool.
Complete measurement tool hyperlink: http://www.coloradotrust.org/attachments/0000/2849/ASIToolkitJun04.pdf
Before analyzing data, code answer options as follows:
Average scores for each participant can be created by summing responses and dividing by the total number of responses. Higher scores reflect higher levels of program quality.
There are no benchmarks available for this measure.
The tips below are quoted from The Colorado Trust (2004). We recommend consulting the publication in its entirety.
1.Keep the survey short and to the point: The simpler the survey, the more likely it will be understood and completed. To enhance simplicity, only question sets that pertain to a particular program should be used. In addition, program staff should only use question sets that are at literacy and comprehension levels appropriate for the youth those programs serve.
2.Guarantee anonymity or confidentiality: Programs may choose to have surveys administered anonymously or confidentially. Anonymity means that absolutely no identifying information will be collected from respondents. Confidentiality indicates that programs enforce clear rules prohibiting unauthorized staff access to any information that would identify a particular respondent. NRC recommends administering surveys anonymously using no personal identifiers when using toolkit questions in a “post-only” format. ASI program staff should refer to consent forms when determining whether to make the survey anonymous or merely confidential.
3.Make the survey friendly and attractive: Surveys that are well laid out and logical will ensure higher response rates. Use caution when adding unnecessary mood boosters like cartoon graphics that might bias results. Here are some suggestions to make the survey more appealing to youth:
-Use appropriate size fonts (i.e., 11 point or 12 point font).
-Use interesting, but easy-to-read fonts (i.e., Comic Sans MS, Kristen ITC or Pooh, rather than Jokerman).
-Make sure the order of questions is logical and easy to follow. For younger audiences, consider adding the leading stem of the question to each question instead of only listing it at the start of a question set.
-Avoid designing a survey that looks like a “test.”
-Print surveys on pleasingly colored paper.
-Avoid overcrowding of questions — allow enough “white space” on each page.
-If the number of questions makes the survey too long for one implementation, create more than one survey and administer it on different days.
In addition, ASI program staff may want to customize questions from this toolkit so individual participants can relate to them. For instance, many program quality questions use the word “staff.” Programs may want to change this to program leader, teacher or some other term used at the program to mean those individuals working with the youth. Other ideas for customization are provided throughout the toolkit as “helpful hints.” The wording of questions, however, should not be changed if changes would impact the nature of the question. NRC evaluation liaisons will help program staff make these determinations as surveys are drafted. ASI program staff is advised to consult with their NRC evaluation liaison to customize toolkit surveys electronically.
For more helpful tips, please refer to the toolkit: The Colorado Trust. After-School Initiative’s Toolkit for Evaluating Positive Youth Development. Denver, CO: The Colorado Trust; 2004.