After School Matters
Chicago, Illinois | afterschoolmatters.org
Statement provided by Jill Young, Ph.D. Senior Director of Research and Evaluation
What are you most proud of related to your work in afterschool research and evaluation?
We are most proud of the culture we’ve created around data utilization. Our colleagues and stakeholders recognize the value of data and are excited to use it to improve program quality and access for our teens, as well as to advocate for teens and youth development. It has taken time to create this culture, but now we serve as collaborative thought partners to other departments in identifying successes and challenges, brainstorming and testing solutions, and creating real change based on what we learn.
We are also proud of our role in giving teens a voice, especially through our evaluation process. In the past two years, we’ve increased our capacity to collect qualitative feedback from teens through focus groups and interviews. This gives teens more opportunities to share their experiences, provide feedback, raise issues and offer solutions.
What has the impact of this work been?
Through collaborative thought partnerships with other departments and our incorporation of teen voice in evaluation efforts, we are seeing real change in our organizational goals. An example of this is our increase in male enrollment. After reviewing programmatic data, relevant literature and local context, our team helped identify males as a population that was being underserved by the organization.
To ensure our teens are more representative of the public high school population in Chicago that we aim to serve, After School Matters created a cross-departmental working group called the Male Initiative. This group is dedicated to increasing recruitment and retention of males in our programs and improving their experiences with us.
By bridging research and practice to promote equity in our programs for males, After School Matters identified and implemented innovative programming and revamped systems, processes and practices. The efforts of the Male Initiative led to both organizational and teen impact. As an organization, After School Matters increased focus on monitoring quality indicators and demographics, capacity to connect research and practice, and collaboration among departments on organizational goals. We also increased male participation year over year alongside our organizational growth since the initiative began, with male participation at an all-time high this year.
American Institutes for Research (AIR)
Washington, D.C. | air.org
Statement provided by Deborah A. Moroney, Ph.D., Managing Director, and Jessica Newman, M.A. Senior Researcher, Youth Development and Supportive Learning Environments
What are you most proud of related to your work in afterschool research and evaluation?
We are proud of our collaborations across the afterschool field and our role in making research relevant. The American Institutes of Research (AIR) team celebrates our joint efforts to elevate practitioner voice in driving evaluation decisions and in helping to tell the afterschool story. Our team of researchers and capacity-builders work together to translate rigorous research and evaluation into practical tools, readily available information and data-driven improvement practices.
What has the impact of this work been?
We have seen the impact of our work through our partners who make real-time research to practice connections with us. We see the impact of our work when we collaborate with a partner to identify a research or evaluation strategy that helps them answer their questions, while ensuring their efforts are useful in everyday practice. By making research and evaluation relevant, we have seen our partners come together to ensure that afterschool programs are asking the right questions, at the right time, and measuring the right outcomes on behalf of youth and their families, programs and the field.
Carol McElvain, J.D.
Independent Consultant
What are you most proud of related to your work in afterschool research and evaluation?
Our work has been able to show people that there is a practical way to translate the best results we see in research and evaluation into tools that can be incorporated into every day afterschool practices. I am also glad that we have been able to help people build stronger awareness of how their daily interactions with young people can have such a positive impact on their lives if they just pay a bit more attention to it.
What has the impact of this work been?
I hope the impact of the work demonstrates that even small, positive changes a person working in a program can ripple into bigger effects. For example, we have seen over time that simply by greeting young people by name daily, and asking how they are doing, can result in a young person feeling like someone cares about them—which in turn can make a difference whether they choose to stay in school.
David P. Weikart Center for Youth Program Quality
Ypsilanti, Michigan | cypq.org
Statement provided by Karen Pittman, Co-Founder, President and CEO, the Forum for Youth Investment
What are you most proud of related to your work in afterschool research and evaluation?
A decade ago, the Forum for Youth Investment created the Weikart Center for Youth Program Quality as a joint venture with the High Scope Educational Research Foundation, known for their groundbreaking research on the lifetime value of investments in quality early childhood education. Our goal was to complete the work Dave Weikart started—to not only demonstrate that high-quality youth programming matters, but to demonstrate that program quality is measurable, malleable and marketable.
The Weikart Center, under the leadership of Dr. Charles Smith, its former executive director, has spent a decade designing, evaluating and marketing quality improvement systems and continuous improvement interventions. As a result, administrators and funders of afterschool systems, provider networks and national organizations have the confidence and capacities they need to create supportive climates in which program managers and staff willingly use data to assess, plan and improve program practice and, more recently, to assess related social and emotional skill growth.
What has the impact of this work been?
As the forum’s co-founder and CEO, I am most proud of three contributions the Weikart Center has made that build on the critical work done to create a validated assessment and improvement intervention.
First, as summarized in Moving the Needle: The Weikart Center’s research has demonstrated the value of systematizing and scaling staff development efforts by helping out-of-school time systems implement quality improvement interventions—assessments, data technologies, staff support—with fidelity, allowing staff the freedom they need to adapt proven practices.
Second: our work with the National Summer Learning Association to create a Summer Learning PQA that was then used to evaluate the impact that integrated high-quality summer learning programs can have on student gains in academic and social skills. This evaluation demonstrated the important role that both OST programs and youth development approaches play in efforts to close achievement gaps.
Third, and most recently: Preparing Youth to Thrive—our work with an incredible group of expert practitioners to distill staff wisdom on practices that support social and emotional skills, create practitioner-friendly observation tools to measure skill growth, and demonstrate that high-quality programs can not only improve skill growth across domains for all youth but also accelerate skill growth among youth with the weakest skills. Combined, these efforts to use research and evaluation to inform management and practice have benefited over 5,000 programs across the country that now have a common language and a set of quality standards, trainings and assessment tools that give them and the field the power to know.
Joseph Durlak, Ph.D.
Emeritus Professor of Psychology, Loyola University
What are you most proud of related to your work in afterschool research and evaluation?
The review that my colleagues and I performed, A Meta-Analysis of After-School Programs That Seek to Promote Personal and Social Skills in Children and Adolescents, was timed just right. There was great interest in the potential value of afterschool programs, but also some concern about their overall impact. Our review offered very positive conclusions about what afterschool programs could achieve if they were appropriately conceived and conducted.
What has the impact of this work been?
It seems like our review had impact both on research, policy and practice. Our work has been cited many times in the professional literature. Moreover, from a funding and practice perspective, many, many, many people read our review and were supportive of it. Our work seemed to bolster the efforts of funders, organizations, local programs and frontline providers in terms of saying to them, in effect: "Yes, work in afterschool is important and helpful to youth and their families.
Every Hour Counts
New York, New York | everyhourcounts.org
Statement provided by Jessica Donner, Executive Director
What are you most proud of related to your work in afterschool research and evaluation?
Every Hour Counts pioneered the development of its Every Hour Counts Measurement Framework that defines a concrete set of outcomes at three levels—system, program and youth—that the coalition hopes to achieve as a result of building expanded-learning systems.
The expanded learning field is rich in research that shows the impact on young people in participating in high-quality programs. Yet for too long, the field has struggled with the complex and elusive process of developing and adopting a common framework for measuring these youth outcomes and the program and system practices that may influence them. The measurement framework is designed to serve as a blueprint for understanding the impact of programs on youth outcomes, making improvements at the system and program level and influencing policy.
With support from The Wallace Foundation and the Charles Stewart Mott Foundation, and in partnership with three intermediaries—Boston After School and Beyond; Providence After School Alliance; and Sprockets, St. Paul and Youthprise, Minnesota—Every Hour Counts is in the midst of a three-year developmental evaluation led by Jennifer McCombs, Ana Auger Whitaker and Paul Yoo, to better understand how intermediaries use the framework to drive continuous improvement efforts in an effort to share promising approaches with the expanded-learning field.
What has the impact of this work been?
The measurement framework has the potential to inform policy and practice.
At the policy level, the framework underscores the importance of program quality as an outcome in and of itself and that quality is measurable and movable. The framework provides guidance to federal and local policymakers that afterschool supports a holistic set of youth outcomes, beyond test scores and grades.
For practice, the framework can help communities narrow down what may seem like an overwhelming array of youth outcomes to a reasonable core set of outcomes that reflect the afterschool programs and the aspirations we want for young people to thrive in their lives. This work has helped the three participating communities become more efficient and strategic in their data collection efforts. This has resulted in streamlined data approaches that minimize burden on providers and bolsters the ability of intermediaries to analyze more and better data for improvement.
This framework was developed with the support of several evaluators supporting expanded learning, and we are grateful for their thought leadership: Neil Nafzger and Deborah Moroney at American Institute for Research; Elizabeth Devaney; Nicole Yohalem; Elizabeth Reisner, formerly of Policy Studies Associates; and Charles Smith, formerly of the Weikart Center at the Forum for Youth Investment.
Every Hour Counts and the RAND Corporation will publish a toolkit in spring 2019 and that resource will meet communities where they are and assist them in their path to use data to improve systems, programs and student outcomes.
Reed Larson, Ph.D., Professor of Human Development, Department of Human Development and Family Studies
University of Illinois | Urbana, Illinois | hdfs.illinois.edu
What are you most proud of related to your work in afterschool research and evaluation?
Many adults in our society underestimate the capabilities of teenagers; as a result, teens are often not given the opportunities for agency, voice and experiential learning that help them develop as whole people. Afterschool and out-of-school time programs are one part of young people’s lives where these experiences can happen.
By interviewing youth in programs over time, our research team has been able to understand how they can develop powerful social-emotional competencies. Our W.T. Grant-funded studies have identified how program projects (e.g., arts, technology, leadership) provide rich experiences through which youth learn skills for planning, thinking strategically and creating change. They also develop abilities to manage their motivation and develop sustained investment in meaningful activities; they learn awareness of emotions and develop skills for regulating and using emotions like excitement, pride, worry and even anger. Our research papers have also illuminated how youth in programs develop responsibility, learn to work in high-functioning teams, and become committed to social justice.
Through this research I have met many really smart and wise frontline staff who facilitate youth’s developmental experiences. From interviewing them, we have been able to describe challenges in their work and some of their effective strategies. Kate Walker and I have illuminated diverse “dilemmas of practice” that staff encounter. The strategies staff use are highly-nuanced and not reducible to formulas. Nonetheless, we have published papers on how they approach dilemmas, respond constructively to race-related incidents, and balance their roles as friend and authority figure, among other things.
What has the impact of this work been?
Researchers are not always able to gauge the impact of their work. I believe I played a role in introducing the paradigms of positive youth development and SEL into the afterschool field. I know that many of our findings have made it into college courses and training for youth professionals. Our research findings provided the starting framework for the Susan Crown/Weikart initiative on SEL practices, and our dilemmas are being used for SEL training. I think many findings and ideas from our research have also contributed to dialogue within the afterschool and public policy fields.
My long-term hope is that all of us in the afterschool field will succeed in reducing the negative stereotypes of teenagers, helping society recognize the difficult and amazing work of programs and their staff, and increasing opportunities for youth to be empowered and realize their potentials.
Maine Mathematics and Science Alliance (MMSA)
Augusta, Maine | mmsa.org
Statement provided by Sue Allen, Ph.D., Senior Research Scientist
What are you most proud of related to your work in afterschool research and evaluation?
Our organization is lucky to have a number of experienced STEM researchers and evaluators on staff, with expertise in a range of approaches to assessing learning. Over the last year, we have been creating quick, inexpensive assessments of afterschool providers’ skills as they facilitate STEM learning experiences with youth. While we appreciate the rigor and comprehensiveness of tools such as YPQA or DoS for assessing program quality, our goal is to create a few simple, yet effective, evaluation tools to examine whether afterschool providers are gaining some concrete knowledge and skills in some in key areas.
What has the impact of this work been?
One of our evaluation instruments tests how well afterschool providers can recognize effective ways to make STEM activities relevant to youth lives, interests and future careers. This assessment tool is a bit unusual and creative: It uses the idea of asking afterschool providers to imagine themselves in a dialog with youth and to think about the specific choices they make about what to say next, in order to support learning. We call it a “script improvement” assessment, because it provides an imaginary script of the dialogue and asks afterschool providers to choose what they might say to deepen youth learning.
We are proud to announce that this instrument has been pilot-tested and will be part of the upcoming STEM credential being offered by NAA. This means that many afterschool providers will be using the assessment as they apply for their “establishing relevance in STEM” micro-credential, and we hope they will find the questions thought-provoking and fair.
This evaluation work is part of a larger ongoing project called ACRES—Afterschool Coaching for Reflective Educators in STEM—where we are offering workshops and coaching to groups of educators on skills such as how to ask youth purposeful questions, how to give youth voice and choice, and how to effectively guide youth through a science or engineering process. Many of these skills are taken from the University of Nebraska’s Click2Science framework, and we hope that our new assessments will allow us to document afterschool providers’ professional growth as they participate in professional development programs. For more information, please see mmsa.org/acres or email us at acres@mmsa.org.
Finally, we are very proud that researchers from our organization have served as advisors and collaborators with NAA throughout the development of the new NAA STEM Credential. We look forward to working with and supporting NAA and their STEM Credentialing system in the future.
National Institute on Out-of-School Time (NIOST)
Wellesley, Massachusetts | niost.org
Statement provided by Georgia Hall, Ph.D., Director and Senior Research Scientist
What are you most proud of related to your work in afterschool research and evaluation?
The National Institute on Out-of-School Time has been working in the out-of-school time field for over 35 years—and much of that work devoted to research and evaluation tasks. In many ways the national OST school field has been built on the solid foundation constructed through the early work of NIOST, along with contributions from many other valuable partners over the years.
One of NIOST’s accomplishments of which we are most proud was the seminal publication of School-Age Child Care: An Action Manual in 1983. In this book, NIOST’s research laid the groundwork for a greater national understanding of the vulnerability of young children in self-care and the critical need for quality school-age care.
We are also proud to be working diligently to solidify the connections between the worlds of research and practice. Extensive work as a research and evaluation partner to the Massachusetts Department of Elementary and Secondary Education, 21st CCLC Program has translated into the collaborative development of a suite of evaluation and assessment tools that connect practitioners and researchers in meaningful and informative ways. The Survey of Academic and Youth Outcomes (SAYO) and the Assessment of Program Practices Tool (APT) were incubated in Massachusetts and have been used in states, cities and local communities around the country.
These tools provide valuable ways to understand, measure and devote our time and energy to elevating the quality of out-of-school time program experiences for all youth.
What has the impact of this work been?
We believe the research and evaluation work of NIOST has had a monumental influence on the afterschool program field. NIOST researchers have worked with on-the-ground partners to help them:
- Measure quality outcomes for all.
- Assess the effectiveness of policies and practices.
- Benchmark against best practices.
- Develop in-house evaluation skills.
- Create and establish standards for program quality and staffing credentials.
- Inform the greater field.
We have empowered our partners—from individual programs to system-wide networks—to help ensure quality out-of-school time experiences for all. For more than 35 years, NIOST has been a leader in defining, shaping and promoting out-of-school time as a distinct professional field with evidence-based quality standards. We bridge the worlds of research and practice to provide out-of-school time program directors, staff, planners, school administrators, community leaders and others with research, training, evaluation and consultation to enhance and improve the quality of programs for all children and youth.
Policy Studies Associates (PSA)
Washington, D.C. | policystudies.com
Statement provided by Christina Russell, Managing Director
What are you most proud of related to your work in afterschool research and evaluation?
Our evaluation team has had the privilege to support the evolution of afterschool for more than two decades, working with programs, intermediaries, and policymakers at the national, state and local levels. We are proud to have played a role in contributing to the growth and quality of afterschool opportunities for young people.
As an organization, we are committed to helping our partners use information to improve education and supports for learning, strengthen communities, and build capacity. We continue to develop knowledge and insight from across the youth and education sectors, and we are proud to contribute our expertise in teaching and learning, leadership, technical assistance, civic engagement, and other areas to create bridges that inform the afterschool field.
We are also proud of our long-term partnerships with nonprofit organizations and government agencies across the United States. We have conducted formative evaluation at the beginning of a new program or strategy, examined implementation and provide recommendations to improve fidelity and quality, and designed evaluations to understand the impact on youth, families and communities and to unpack what contributes to that impact. Witnessing the evidence-informed changes and improvements at each of those stages of learning is exciting and motivating.
What has the impact of this work been?
We know that our work has an impact when our partner organizations can use the results to strengthen program policies and designs, staff capacity, and, ultimately, the experiences of young people. For example, we have seen our findings influence policies about participation to maximize program benefit; about the qualifications and experiences of program leaders; and about indicators to determine successful implementation and impact. We are proud when our evaluations generate conversation within and across programs and help disseminate concrete examples of promising practice that program leaders can put into action.
Importantly, we have also seen impact on the internal capacity of our partner organizations. Tools we have developed for external evaluation purposes—including the PSA OST Observation Instrument—have often been adopted as part of continuous improvement and learning efforts within programs. More generally, we have seen leaders at all levels of the afterschool system increasingly seek out evaluation data for improvement, not just reporting to funders. This pushes us as external evaluators to refine both our methods and reporting to be responsive to new audiences and goals. We are excited for the next phases of growth for afterschool!
PEAR Institute
Belmont, Massachusetts | thepearinstitute.org
Statement provided by Gil G. Noam, Ed.D. Director and Founder
What are you most proud of related to your work in afterschool research and evaluation?
Over the past several years, the PEAR Institute has been working to create a data system that is able to provide big-picture aggregate trends and individual-level insights for both youth and educators in afterschool and in-school settings. Seeing our tools and systems established and used widely in the field has been a very fulfilling experience. I’m also proud that our observation tools are being used to shape thinking around quality programming around youth development, social-emotional learning and STEM. Programs are also using our tools to plan, design, implement and evaluate impact on the youth they serve. We believe that our approach to research can help programs enhance the relationships in afterschool programs and support the special nature of this work.
What has the impact of this work been?
In medicine, there is a lot of talk about translational work connecting research to the bedside; for us, it’s about the bidirectional connection between research and the community—which is so important for innovation. The PEAR Institute was created to support the vision of a translational center steeped in scholarship, theory and research that simultaneously understands practice and is dedicated to the wellbeing of youth and families. One of our most important areas of impact is increasing the effectiveness and social-emotional competencies of educators. Our work puts quality first; in the afterschool space, we’re specifically focused on STEM and SEL. When we work with educators, we emphasize the importance of focusing on research-informed practices to create positive outcomes for youth. We strive to help afterschool programs and schools use data as part of their day-to-day operations. By treating research as living data—not just as information you put in a file and only review at the end of the program—we can help programs know every child and continuously improve.
Public Profit
Oakland, California | publicprofit.net
Statement provided by Corey Newhouse, MPP, Founder and Principal
What are you most proud of related to your work in afterschool research and evaluation?
Two things: our longstanding collaboration with citywide afterschool networks in three California cities, and the evolution of our practice to include evaluations of professional development and network building in out-of-school time.
A long, long time ago, I wrote my master’s thesis about program quality standards and standardized performance reporting options for a loose coalition of out-of-school time stakeholders in Oakland, California. A few years later, Oakland’s school-based afterschool programs increased from about 25 to 80, thanks to a voter initiative investing substantial state dollars in out-of-school time.
The Oakland school district reached out to me to help them measure the programs’ effectiveness beginning in 2007. We now serve as the independent evaluator for citywide out-of-school time networks in Oakland, San Francisco and Richmond California, which collectively operate more than 200 programs serving more than 30,000 students a year.
It’s been a joy to work alongside these terrific teams as they build robust systems that support continuous quality improvement, professional development and performance management. We’ve helped to support this process by encouraging our clients to really focus on program quality and the direct (or proximal) benefits to youth, including stronger peer relationships, sense of belonging at school and a sense of mastery and accomplishment.
Our clients are continuously innovating—exploring ways to improve the quality of afterschool programs through professional development, network building and advocacy. As they have evolved their practice, so have our evaluation approaches. We’ve been able to apply innovative evaluation techniques to a range of out-of-school time initiatives, including Ripple Effect Mapping, network analysis and policy advocacy analysis.
What has the impact of this work been?
The OST programs in Oakland, San Francisco and Richmond are measurably better today than when they started. Public Profit has helped to contribute to this evolution by continually providing high quality information about the quality of OST programs, and by assuring that the opinions of youth, parents and staff are incorporated into the larger planning and decision-making process. We’ve helped to create a common language among program staff about what high-quality out-of-school time programs look like and can accomplish, and we help our clients to tell their stories to policymakers, funders and families.
Professional development, network building and advocacy are much more common in the afterschool field today than a decade ago. We’ve been able to contribute to our clients’ ability to be strategic about how to plan and implement these complex initiatives, and how to know what’s working and what can be improved. And that’s really at the heart of it for me—Public Profit set out to help mission-driven organizations be their best. It feels great when our clients use our stuff to do just that.
Jason Spector, M.P.P., Research and Evaluation Director
After-School All-Stars | Los Angeles, California, and Washington, D.C. | afterschoolallstars.org
What are you most proud of related to your work in afterschool research and evaluation?
I’m proud of the collective effort that we undertook over this last year at After-School All-Stars. From our research and evaluation team to our programs, regions and executive decision-makers, we convened as an Impact Working Group to systematically examine how we can best serve our All-Stars youth. We were able to conduct classroom observations and interview staff across the country, while also digging into attendance, program quality, staff and program outcomes data.
From this information, our leadership was able to develop data-informed, multipronged strategies pushing All-Stars forward as we work toward achieving our “North Star” of programming impact, through increased staff retention and program quality. I’m proud to lead and collaborate with such a dedicated group, who are all steadily working toward finding the best way to serve youth. After-School All-Stars leadership is bold and forward thinking in our efforts to hold ourselves to higher standards, year over year. It can be challenging and vulnerable to open your work and programs to an evaluator’s eye, and it reflects the commitment of all involved to do so with passion and, ultimately, such efficacy.
What has the impact of this work been?
Over the last year, we convened a cross-functional Impact Working Group weekly to identify growth opportunities through evaluation efforts and develop recommendations, including delineating local pilots and national priorities. We then convened our national leadership to finalize strategic priorities and high-leverage strategies and pilots to meet these priorities. The North Star strategic programming plan developed through these meetings was brought to our national board, and we were able to gain consensus and move toward our new impact strategy launch.
Our staff, chapters and national board are all behind the “North Star” strategy, which ensures all aspects of the organization are working to optimize our programs’ effect on youth lives. Specifically, we are initiating pilots in curriculum and content, line staff turnover and class-level assessments, with a focus on youth 21st Century Readiness. We are also launching a new national data system to track progress toward our program goals in real time. We anticipate these collective investments will increase the quality and impact of our afterschool programs serving 75,000 All-Star youth and improve the experience of the 5,600 plus All-Star staff across the national network. Further, the lessons learned from these efforts promise to inform the field at large around scaling standards-aligned programs, deepening quality, and optimizing part-time staff retention strategies to benefit youth and afterschool professionals in and beyond After-School All-Stars.
Robert Tai, Ed.D., Associate Professor
Curry School of Education at University of Virginia | Charlottesville, Virginia | curry.virginia.edu
What are you most proud of related to your work in afterschool research and evaluation?
Afterschool plays an immense role in the lives of so many young people, and I deeply appreciate hearing from afterschool educators how my work has been helpful to them and their work. Learning that my research has been used to support grant applications for afterschool programs, cited by educators speaking with policymakers, or referred to by organizations supporting afterschool in their mission has both energized and humbled me. When I began my career in educational research, I was working on questions related to how students make the transition into being scientists. This line of inquiry and my background as a former physics teacher led me to consider when young people first became interested in science as something that they might want to have as part of their future careers and whether this early interest had a lasting impact on their lives.
What has the impact of this work been?
This work led to a short article, Planning Early for Careers in Science, that was eventually published in Science in 2006. Quite frankly, I had no idea what impact this study would have. It was one of about a two-dozen research studies I had written in the past 12 months, at that time. The first time I submitted this work to a peer-reviewed journal, it was rejected as not suitable for that journal. The second journal I sent it too rejected it as well, but the rejection letter was encouraging. Six months after the second rejection, I finally got back around to rewriting this study. It was on this third attempt that it was accepted for publication. While I was elated that this study would appear in Science, I was too focused on getting together my tenure case to notice that people were actually reading it and sharing it. After two friends told me that it was handed out at board meetings of national science education organizations they attended, and my graduate student discovered it was one of the most downloaded articles in June 2006, I thought to myself: "Wow, I might not get fired after all."
Going through the tenure process can be very stressful. I am profoundly grateful to hear how this article—and the nearly two-dozen research studies that have followed since—have been useful to afterschool educators. I am always happy to hear from afterschool educators about the research they believe will help them in their work.
Search Institute
Minneapolis, Minnesota | search-institute.org
Statement provided by Gene Roehlkepartain, Ph.D., Vice President of Research and Development
What are you most proud of related to your work in afterschool research and evaluation?
Search Institute is proud to have been a leader in giving language, measures and tools to build and broaden the field of positive youth development. This role began in the 1980s with our studies of early adolescence and continued through the groundbreaking research on developmental assets, beginning in the 1990s. Now, we are excited to be catalysts in a national research partnership focused on the vital role developmental relationships play in helping young people be and become their best selves.
We have been part of building the field by introducing language and frameworks that capture core principles and priorities of positive youth development. This is most evident in our work on developmental assets, a framework that has been widely used to shape and monitor programs and to train leaders for the field. Our newer work on developmental relationships also shows promise in giving fresh language to the specific ways adults and peers influence youth development through intentional relationships.
In each of these and other instances, Search Institute not only has brought new insights and research to the afterschool community, but we have worked hard to ensure that the insights strengthen practice and build the capacity of youth workers and other leaders to engage positively with young people—recognizing, tapping and building their strengths to be contributors to their communities both now and in the future.
What has the impact of this work been?
Search Institute-applied research has reached broadly across the afterschool community, serving as a resource across many types of programs and many settings across the United States and worldwide. Whether the focus is on sports, arts, mentoring, martial arts, camping, spiritual or cultural nurture, family strengthening, or youth leadership, programs have found that our research helps them focus on nurturing young people’s strengths and embedding them in webs of positive relationships. In the process, the research has provided common frameworks that allow the youth development field to learn from each other, collaborate and work together to ensure that each and every young person has the relationships, supports and strengths they need to thrive.
University of Utah Education Policy Center
Salt Lake City, Utah | uepc.utah.edu
Statement provided by Cori Groth, Ph.D. Associate Director
What are you most proud of related to your work in afterschool research and evaluation?
As a research-based center at the University of Utah in the College of Education—whose mission is to bridge research, policy and practice—we could not be prouder of our partnerships with the Utah Afterschool Network (UAN), Utah State Board of Education, Department of Workforce Services Office of Childcare, and the other schools, districts and community-based organizations we serve. The coordination between these groups have resulted in a focused and sustained effort to serve students and their families across Utah through high-quality afterschool programs that offer a balance of academic and developmental enrichments. We are honored to be a part of this work and we take seriously our role as researchers and evaluators within this network.
What has the impact of this work been?
Collaborating with the UAN, state agencies, state legislators and a variety of afterschool program providers throughout Utah has offered a unique opportunity to work toward our mission of bridging research policy and practice. We aim to synthesize and integrate recommendations from ongoing research and evaluation activities into our evaluation reports, presentations and policy recommendations, as well as the design of technical assistance and other professional learning opportunities that offer practical considerations and guidance for program improvement for leaders at all levels of the system. The UEPC works closely with stakeholders to ensure that they have the information required to make well-informed decisions that result in improved youth outcomes.
Working with funders who invest in multiyear evaluations has resulted in a body of knowledge that documents not only the impact of afterschool program implementation and outcomes, but the implications for ongoing improvement and policy solutions that leverage resources in new and creative ways. The consistent and iterative nature of systematic, ongoing afterschool program evaluations has established evidence-based programming as a critical statewide standard. Collectively, this work has resulted in more informed and effective technical assistance, increased state-level funding for afterschool programs, and improved statewide capacity to ensure that youth and families have access to high-quality afterschool programs throughout the state.
We are honored to be recognized among other researchers and evaluators across the country who are also contributing to an important body of evidence and research-based practices in support of high quality afterschool programs.
Maureen R. Weiss, Ph.D., Professor
School of Kinesiology at University of Minnesota | Minneapolis, Minnesota | twin-cities.umn.edu
What are you most proud of related to your work in afterschool research and evaluation?
In reflecting on my life’s work, I am proud of how involvement in afterschool programming has maintained a thread of continuity. My journey got its start when, as a university student in California, I served as a youth sports coach for the City of Newport Beach Parks and Recreation. This experience illuminated the impact that afterschool programs can have on youths’ social, psychological and physical development. Through experiences on the softball field and volleyball court, my kids developed friendships, gained confidence and acquired skills to be physically active for a lifetime. Being a youth sports coach was the driving force that inspired me to focus my academic studies on the benefits of sport and physical activity for children’s holistic development.
I followed through on this inspiration and pursued a career studying youth development through physical activity. Along the way, I am proud of the many students I mentored that continue this legacy. Now, 40 years since hanging up my coaching gear, I am grateful for the many opportunities to fulfill my dream of conducting research and evaluating the effectiveness of youth development programs such as Girls on the Run and The First Tee. I am thankful that the trust and respect afterschool programs showed me produced measurable outcomes that enabled improved curricula, coach training and developmental outcomes. I am especially proud to have become a “family member” of community programs and will cherish my relationships forever.
What has the impact of this work been?
First, many studies have shown that afterschool programs play a major role in promoting youth development by teaching skills, fostering relationships and instilling a sense of joy when being physically active. This consistent finding reinforces the importance of afterschool programing, and it also means a responsibility to ensure that curricular lessons and delivery are effective in achieving developmental goals.
Thus, a second area of impact is evaluating whether afterschool programs are delivering on the promise of effectively teaching skills and nurturing relationships within an enjoyable learning environment. These studies uncovered positive aspects of curricula and coach training, and thus why programs are effective in promoting youths’ social, emotional and physical development. Results also identified what aspects of the program can be added or improved to be even more effective. Collectively, the impact of our research translates to reinforcing the significance of afterschool programs for youths’ holistic development, currently as children and in the future as adults.
Roger Weissberg, Ph.D., UIC/LAS Distinguished Professor Psychology/NoVo Foundation Endowed Chair in Social and Emotional Learning, University of Illinois at Chicago; Chief Knowledge Officer, Collaborative for Academic, Social and Emotional Learning (CASEL)
University of Illinois at Chicago | Chicago, Illinois | uic.edu
Collaborative for Academic, Social and Emotional Learning (CASEL) | Chicago, Illinois | casel.org
What are you most proud of related to your work in afterschool research and evaluation?
I have focused my professional career on one key question: How do schools, families and communities enhance the social, emotional and academic learning of children and adolescents?
For 25 years, my colleagues and I at the Collaborative for Academic, Social and Emotional Learning (CASEL) have partnered with researchers and practitioners to advance social and emotional learning research, practice and policy.
I am most proud of three accomplishments related to my work in afterschool research and evaluation. First, Joseph Durlak, a team of graduate students from the University of Illinois at Chicago and Loyola University, and I have conducted meta-analytic reviews of afterschool SEL programs. We documented that they can significantly improve young people’s self-perceptions, bonding to school, positive social behaviors, school grades and academic achievement, while significantly reducing problem behaviors.
Second, my colleagues and I have identified four beneficial practices associated with effective afterschool programming that we call SAFE:
- Sequenced step-by-step training
- Active forms of learning
- Focus on social and emotional skill development
- Explicit SEL goals.
Third, my colleagues and I at CASEL have worked with our collaborating school districts to establish systemic SEL practices that combine programming during the school day with OST programming. Currently, we are working with six communities and many field leaders on the design, implementation, evaluation and documentation of these approaches through the Wallace Foundation’s Partnerships for Social and Emotional Learning Initiative (PSELI).
What has the impact of this work been?
Today, CASEL is working with 25 collaborating states and 20 collaborating school districts that provide evidence-based SEL programming to several million children and adolescents. Also, our research has helped to support policies that encourage quality afterschool programming at the federal, state and local levels. My hope is that our research and evaluation efforts will improve the learning opportunities and lives of many more young people across the United States over the next decade.
Heng-Chieh Jamie Wu, Ph.D., State Evaluator, Michigan 21st Century Community Learning Centers Afterschool Programs & Great Start Readiness Programs; Community Evaluation and Research Collaborative, University Outreach and Engagement
Michigan State University| East Lansing, Michigan | msu.edu
What are you most proud of related to your work in afterschool research and evaluation?
We have a strong afterschool network in Michigan and I’m very proud to be a part of it. Through research and evaluation, my team and I work closely with state agencies, frontline staff and administrators to support a continuous improvement system, so we can ensure Michigan’s youth are getting high-quality program experiences during OST.
Each year, our work shows that more than 200 federally funded 21st Century Community Learning Centers—as well as many other community-based programs—operate year-round to enrich the lives of young people. Parents and teachers advocate for afterschool programs because they’ve seen improved homework completion rates and school behaviors. Youth have access to healthy snacks and meals, and engage in sports, arts, music, community affairs, and hands-on projects that help them unwind from the school day and form meaningful relationships outside of home and school.
Our Michigan afterschool network keeps up with the latest research and evaluation. We offer innovative modules and professional development opportunities toward STEM, SEL, youth-driven space and family engagement programs around the state, especially in resource-poor communities. Working together, the data speaks for itself showing our afterschool programs go beyond providing latchkey services to promote equity and quality learning experiences for all.
What has the impact of this work been?
Because research and evaluation has been an integral part to the Michigan afterschool network, the impact of our work has been multifaceted. At the local level, we return all the data to youth programs in a way that’s relatable to them, so the information can be used for continuous program improvement and marketing purposes. At the state level, we demonstrate the offerings and impacts of afterschool programming and how it is aligned with Michigan Department of Education’s goal in making Michigan a Top 10 State in 10 Years. We also help ensure the funded programs are compliant with quality standards and grant requirements. At the national level, we use Michigan’s data to help enrich the knowledge and best practices of quality afterschool programming and call the nation’s attention to the benefits of afterschool programs in strengthening families and transforming youth into well-rounded, engaged and better-prepared citizens.
Youth-Nex
Charlottesville, Virginia | bit.ly/youthnextnaa
Statement provided by Nancy L. Deutsch, Ph.D., Director
What are you most proud of related to your work in afterschool research and evaluation?
Founded in 2009, Youth-Nex has been dedicated to expanding and applying the science of positive youth development to address fundamental challenges facing societies around the world. A key aspect of our work has been focused in afterschool, which is currently one of our three focal areas. We are particularly proud of the ways in which our work has contributed to local and regional efforts to improve afterschool systems as well as the field nationally by partnering practitioners, researchers and policymakers in knowledge-building efforts.
The first example of these activity types is represented by our work with NextUp, an organization that works to ensure access to high-quality afterschool programs for middle school youth in Richmond, Virginia, through building communication and collaboration across youth-serving agencies and families, supporting more efficient funding models, and providing professional development and quality improvement systems for programs. Youth-Nex worked with NextUp to conduct a developmental evaluation to help consolidate the collaboration’s innovative intent. The second effort is represented by our 2014 conference, Let’s Talk After-School, which brought together 150 scholars, youth workers, educators, policymakers and youth to discuss key issues and emerging best practices in the field. The conference resulted in a two-volume special issue of Advances in Child and Family Policy and Practice about afterschool programs and positive youth development. The first volume focuses on key components of effective programs; issues for evaluation and measurement of quality in programs; and discussion of how researchers, practitioners and policymakers can maximize the potential of afterschool programming for all youth. The second volume presents an overview of summer learning, discussion of best practices in different types of targeted programs (e.g., STEM, sports, etc.) and case studies of a series of specialized programs.
What has the impact of this work been?
Youth-Nex has not only put research in the hands of policymakers and practitioners, but helped to create a two-way street, where practitioners are also speaking to scholars about the knowledge they require from the field. This type of research-practice partnership that focuses on researchers and practitioners each learning from one another is central to our future work in afterschool and represents the kind of impact we hope to continue to impart on the field.
