. 2020 Feb 17;24(Suppl 2):132–140. doi: 10.1007/s10995-020-02890-x

Table 1.

Overview of the Adolescent Family Life Program (AFLP) Positive Youth Development (PYD) Model development activities and lessons learned

Opportunity and program development components	Key points and activities	Links to the lessons learned
Opportunity: Pregnancy Assistance Fund Grant	• CDPH/MCAH was awarded a PAF grant, which provided an opportunity to develop, implement and evaluate a program model specifically for expectant and parenting youth • In 2010–2011, CDPH/MCAH hired two full time equivalent program staff to lead program development and contracted with 11 local agencies and a University of California, San Francisco research team to support program development and external formative evaluation • In 2013, MCAH added an additional .75 FTE state staff to the project with expertise in developing, monitoring, and evaluating evidence-informed program models • The pre-existing AFLP infrastructure and the state leadership were key to the project	Lesson 1: Ensure Adequate Staffing to Support the Multiple Components of Program Development, Implementation and Monitoring When developing a new program model, consider the diverse expertise needed from the beginning. At a minimum, it is essential to have people with the following expertise: topical, program development and implementation, monitoring and evaluation, and training. This can either be through governmental staff or contractors. Careful thought should be given to all components of program development and mechanisms should be in place to hire internal staff or contract staff, based on the needs and goals of the project. Given the relatively lengthy process of program development, efforts to hire staff with a sustained commitment to the project can help to ensure continuity in development.
Early program development (2010–2012)	The CDPH/MCAH project team: • Developed the vision and program strategies (case management, positive youth development, motivational interviewing, and life planning) • Conducted literature reviews on positive youth development, reproductive life planning, goal setting, motivational interviewing, and evidence-based program models for EPY and other youth populations • Selected 11 of the 34 AFLP sites to participate in the program development process as pilot sites • Learned about best practices among pilot sites through environmental scans and site profiles • Provided foundational training	Lesson 2: Identify the Guiding Theory and Framework Early and Link them with Clearly Articulated Core Components Identifying the theory and framework early provides the foundation for developing all program model elements and supports the process of identifying areas where training and technical support are needed. Having a strong foundation to link all parts of program development helps support and justify the many changes that arise from a participatory and iterative development process. Articulating clear and specific core components leads to a program structure that can be consistently implemented and evaluated.
Iterative Pilot Implementation and Formative Evaluation (2011–2013)	The CDPH/MCAH project team: • Developed/adapted tools and processes for a standardized program model • Selected a foundational PYD framework: Resiliency in Action (Bernard 2004) • Worked to take the theory and operationalize it in case management practice • Produced guides and trainings for implementers to help with the paradigm shift • Led an iterative development process with the pilot sites • Sent tools/processes out into the field (with training/technical assistance/written guidance) to be implemented with youth in AFLP • Gathered feedback (through interviews, surveys, site visits, focus groups, group calls, etc.) and produced recommendations • Made revisions (further engaging local agencies through work groups, site visits and calls, when needed) and then sent revised materials out into the field for further piloting. The process was repeated through multiple cycles	Lesson 3: Engage Implementing Staff on the Ground and Focus Early, and Often, on Processes for Supporting People through Change The AFLP PYD Model program development occurred through pilot implementation in an existing program. The model was a paradigm shift for some agencies and the iterative feedback/implementation cycle led to revisions and frequent changes for implementers. Lesson 3 speaks to the importance of intentionally supporting the change process as the new program model elements and supporting systems are developed and refined.
Formalizing Model Structure for Expanded Implementation and Evaluation (2013–2014)	CDPH/MCAH further synthesized formative evaluation results and produced an evaluation-ready version of the AFLP PYD Model. This included: • A formalized model structure, program tools and materials, guiding documents, trainings, fidelity and outcome monitoring system, and a quality improvement process • CDPH/MCAH launched AFLP PYD implementation with 14 additional AFLP sites • December 2014, Mathematica launched the Positive Adolescent Futures study. Results are anticipated in 2020	Lesson 1: Ensure Adequate Staffing to Support the Multiple Components of Program Development and Implementation When developing a new program model diverse expertise is needed from the beginning to help ensure that the final product can be implemented and evaluated across settings. Lesson 2: Identify the Guiding Theory and Framework Early and Link them with Clearly Articulated Core Components The guiding theory and framework are foundational for the development work. Specific core components that articulate the content, logistics and pedagogy are necessary to support implementation that can be monitored and evaluated across multiple agencies/settings.

Opportunity and program development components

Key points and activities

Links to the lessons learned

Opportunity: Pregnancy Assistance Fund Grant

• CDPH/MCAH was awarded a PAF grant, which provided an opportunity to develop, implement and evaluate a program model specifically for expectant and parenting youth

• In 2010–2011, CDPH/MCAH hired two full time equivalent program staff to lead program development and contracted with 11 local agencies and a University of California, San Francisco research team to support program development and external formative evaluation

• In 2013, MCAH added an additional .75 FTE state staff to the project with expertise in developing, monitoring, and evaluating evidence-informed program models

• The pre-existing AFLP infrastructure and the state leadership were key to the project

Lesson 1: Ensure Adequate Staffing to Support the Multiple Components of Program Development, Implementation and Monitoring

When developing a new program model, consider the diverse expertise needed from the beginning. At a minimum, it is essential to have people with the following expertise: topical, program development and implementation, monitoring and evaluation, and training. This can either be through governmental staff or contractors. Careful thought should be given to all components of program development and mechanisms should be in place to hire internal staff or contract staff, based on the needs and goals of the project. Given the relatively lengthy process of program development, efforts to hire staff with a sustained commitment to the project can help to ensure continuity in development.

Early program development (2010–2012)

The CDPH/MCAH project team:

• Developed the vision and program strategies (case management, positive youth development, motivational interviewing, and life planning)

• Conducted literature reviews on positive youth development, reproductive life planning, goal setting, motivational interviewing, and evidence-based program models for EPY and other youth populations

• Selected 11 of the 34 AFLP sites to participate in the program development process as pilot sites

• Learned about best practices among pilot sites through environmental scans and site profiles

• Provided foundational training

Lesson 2: Identify the Guiding Theory and Framework Early and Link them with Clearly Articulated Core Components

Identifying the theory and framework early provides the foundation for developing all program model elements and supports the process of identifying areas where training and technical support are needed. Having a strong foundation to link all parts of program development helps support and justify the many changes that arise from a participatory and iterative development process. Articulating clear and specific core components leads to a program structure that can be consistently implemented and evaluated.

Iterative Pilot Implementation and Formative Evaluation (2011–2013)

The CDPH/MCAH project team:

• Developed/adapted tools and processes for a standardized program model

• Selected a foundational PYD framework: Resiliency in Action (Bernard 2004)

• Worked to take the theory and operationalize it in case management practice

• Produced guides and trainings for implementers to help with the paradigm shift

• Led an iterative development process with the pilot sites

• Sent tools/processes out into the field (with training/technical assistance/written guidance) to be implemented with youth in AFLP

• Gathered feedback (through interviews, surveys, site visits, focus groups, group calls, etc.) and produced recommendations

• Made revisions (further engaging local agencies through work groups, site visits and calls, when needed) and then sent revised materials out into the field for further piloting. The process was repeated through multiple cycles

Lesson 3: Engage Implementing Staff on the Ground and Focus Early, and Often, on Processes for Supporting People through Change

The AFLP PYD Model program development occurred through pilot implementation in an existing program. The model was a paradigm shift for some agencies and the iterative feedback/implementation cycle led to revisions and frequent changes for implementers. Lesson 3 speaks to the importance of intentionally supporting the change process as the new program model elements and supporting systems are developed and refined.

Formalizing Model Structure for Expanded Implementation and Evaluation

(2013–2014)

CDPH/MCAH further synthesized formative evaluation results and produced an evaluation-ready version of the AFLP PYD Model. This included:

• A formalized model structure, program tools and materials, guiding documents, trainings, fidelity and outcome monitoring system, and a quality improvement process

• CDPH/MCAH launched AFLP PYD implementation with 14 additional AFLP sites

• December 2014, Mathematica launched the Positive Adolescent Futures study. Results are anticipated in 2020

Lesson 1: Ensure Adequate Staffing to Support the Multiple Components of Program Development and Implementation

When developing a new program model diverse expertise is needed from the beginning to help ensure that the final product can be implemented and evaluated across settings.

Lesson 2: Identify the Guiding Theory and Framework Early and Link them with Clearly Articulated Core Components

The guiding theory and framework are foundational for the development work. Specific core components that articulate the content, logistics and pedagogy are necessary to support implementation that can be monitored and evaluated across multiple agencies/settings.