Standard 4: Program Impact

Click the arrow for more information

4.1 Completer Impact on P-12 Student Learning and Development

Kentucky Collaboration for Quality Data (KCQD) state agency-EPP partners are building a data dashboard linking program completers to students' achievement results. The EPP is partnering with the Kentucky Center for Education and Workforce Statistics (KCEWS) to pilot a system for gathering student growth percentile data (see KCEWS Program Impact). This plan relies upon district-level student achievement generated by "K-PREP" (Kentucky Performance Rating for Educational Progress) tests. To analyze impact on students at this time, the EPP used the District Placement Map to identify districts who employed the most program completers. Student achievement in eight of 17 districts who employed our completers ranked in the 90th percentile or above. Students in three districts ranked in the 47-63 percentiles. Because of aggregated results and lack of completer-student correlation, the provider can infer but not definitively state Murray State completers caused the high rankings. A definite correlation will be possible once the new KCEWS system is in place.

Kentucky public school districts recently developed customized Certified Evaluation Plans (CEP) to evaluate certified educators' efficacy and to increase student academic achievement. Districts' CEPs use multiple measures to document certified educators' impact on student achievement, such as student learning (p. 11) and student growth (p. 20). Final CEPs are approved by local school boards and the Kentucky Department of Education. Once CEPs are actualized, the KCQD group will add student learning and student growth data to the statewide data dashboard.

Kentucky is not an EdTPA state; EPPs design their own TPAs. Student teachers' capstone efforts are placed in their TPA Eligibility Portfolios. Aggregated Student Teacher Impact on Student Learning data indicate students' mean growth percentiles increased as a result of student teachers' instructional efforts. Disaggregated program data document large gains in student achievement. Student Teacher Evaluations indicate candidates impact student learning as well. From F14-F15, 89-100% of student teachers received the two highest ratings on 'helps all students learn.' Once state-provided completer impact on student learning data are available, the EPP will use CEP teacher performance data, student teacher's TPA assessments, and Student Teacher Evaluation items as predictive/comparative data points to inform program improvement.

4.2 Completer Effectiveness via Observation and/or Student Surveys

Districts' CEPs included evaluative measures to document teachers' proficiency and professional growth needs. Trained district administrators evaluate educators' proficiency based upon performance criteria aligned with Charlotte Danielson's Framework for Teaching model. CEPs also include an opportunity for students' voices to be heard (Certified Evaluation Plan, pp. 18-19). Once CEPS are in place, the KCQD team will provide completers' performance data and student voice data to analyze for program improvement.

Kentucky EPSB mandates EPPs align all educator preparation efforts with the Kentucky Teacher Standards (KTS). The EPP's common course syllabi, key assignments, and clinical evaluation instruments are KTS-aligned. The InTASC/KTS Crosswalk Table demonstrates how the national and state teacher standards align. EPP faculty are using this crosswalk table to facilitate their efforts in aligning all EPP syllabi, assignments, and assessments with state/national standards. Data gleaned from two KTS-aligned assessments, the TPA Eligibility Portfolio and Student Teaching Evaluations document candidates' teaching proficiency (see section 4.1).

Beginning Kentucky teachers participate in the Kentucky Teacher Internship Program (KTIP). Performance criteria are aligned with the Kentucky Teacher Standards. EPPs do not receive individual intern scores. However, EPSB does provide overall KTIP pass rates upon request. The EPP's higher pass rate for program completers evidences the high level of preparedness.

2011-2012 Program Completers: 100% of the 164 interns passed KTIP
2012-2013 Program Completers: 99.3% of the 156 interns passed KTIP
2013-2014 Program Completers: 100% of the 98 interns passed KTIP

The Kentucky EPSB disseminates a KTS-aligned New Teacher Survey to interns and resource teachers to ascertain responders' perceptions about interns' preparation programs. NTS data are aggregated at the EPP level and shared online. Because of the two-year survey cycle, the provider cannot compare student teachers' and cooperating teachers' perceptions of preparedness with respondents' perceptions of the same candidates' preparedness during the intern year. The provider plans to include those data as a point of comparison once the EPSB administers the survey annually.

New Teacher Survey data indicate 2009-2010 interns and resource teachers consistently rated interns' abilities within the "good" level; resource teachers' responses ranged from 3.22-3.51 and interns' responses ranged from 3.16-3.49. Resource teachers rated 2011-2012 interns' abilities from 2.93-3.39; interns rated themselves from 2.78-3.33. Fall 2013 interns rated themselves from 2.76-3.18 on the KTS. Interns' ratings were consistently below the state means. Resource teachers rated interns from 3.25-3.53. With the exception of working with ELLs, resource teachers' ratings were consistently at or above the state mean. Recurring perceived needs included assessment, collaboration, plus teaching English Language learners and children with special needs.

In response to these concerns, the provider divided the existing K-12 assessment course, EDU 405, into two courses - EDU 405 (middle school/high school) and ELE 383 (elementary). The provider also hired an assessment expert who previously worked for ETS and a former elementary principal who is well versed in best assessment practices. These experts have introduced more rigor and currency to the delivery of the assessment and evaluation courses. The COE-TPA Lesson Plan template includes differentiated instruction, formative/summative assessment, and accommodating special needs. Kentucky adopted a state-wide co-teaching student teacher experience model fall 2013 (see Co-Teacher Training); co-teaching strategies are being integrated throughout existing course work to enhance candidates' collaborative skills. Provider faculty are seeking ways to integrate culturally relevant teaching strategies across the curriculum to better equip candidates to instruct children with special needs and/or limited English proficiency.

4.3 Employer Satisfaction

KCEWS provides annual District Placement Maps depicting where completers are employed. EPP used this map and completer-provided information to identify employers to participate in a survey about perceptions of completers' preparedness. Historically minimal return rates have not yielded useful data. Therefore, the EPP surveyed superintendents in districts who employed completers and also conducted a focus group section (Employer Perceptions). Analyses of these results indicated employers perceived that completers were well-prepared. Employers also share their perceptions of the quality of teacher preparation programs during advisory council meetings (WKEC Directory Agenda; Meeting Minutes). Program coordinators work with faculty to review and actualize these suggestions to inform program improvement.

Through the KCQD initiative, state agencies and EPPs are cooperatively creating a new employer survey that reflects the KTS, Framework for Teaching criteria, and milestones/retention information. EPPs will be able to customize the state-wide template. KCEWS will provide contact information so EPPs can survey employers of specific program candidates who are teaching in Kentucky and neighboring states. Furthermore, the EPP plans to continue to conduct focus group interviews of major employers to seek their input about the preparedness of program completers. These measures will greatly enhance the response rate and relevance of employer survey efforts. This initiative will become part of the EPP's SIP.

4.4 Completer Satisfaction

To generate predictive/comparative data, the EPP administers Student Teaching Surveys to student teachers during their final seminar session before graduation. Student teachers rate the extent to which their programs prepared them to acquire professional knowledge, skills, and dispositions using a 5-point Likert scale. Student teachers perceived teacher preparation course work gave them extensive or very extensive opportunities to hone their professional knowledge, skills, and dispositions. Areas of perceived need included assessment, classroom climate, differentiating instruction, and collaboration. These are the same areas of concern interns cited in the New Teacher Survey. The EPP addressed these concerns using the measures described in section 4.2. These needs will be targeted in the EPP's SIP as well. In compliance with the Kentucky Council of Post-Secondary Education requirements, national accreditation requirements, and best practices, the provider works with various constituents to survey program completers (Completers Perceptions). Response rates have been historically low and current survey items do not focus upon completers' perceptions of their preparedness. Completers have an opportunity to share their perceptions of the quality and currency of teacher preparation programs during advisory council sessions (Student Advisory Council; Meeting Minutes). Program coordinators work with faculty to review and actualize these suggestions to inform program improvement.

Therefore, the EPP conducted an additional survey in November 2015 and a focus group session in March 2016 to gather more Completer Perceptions. November survey results document the majority of completers perceived the preparation program as relevant to the responsibilities they confront on the job, with 88.8% of respondents agreeing or strongly agreeing to the applicable question. Seven of the survey questions asked respondents to judge the preparation program in terms of effectiveness. Survey results ranged from 60.9% to 97.9% of respondents agreeing or strongly agreeing that the program was effective in preparing them to teach. Areas of perceived need for respondents in both settings included classroom management, working with parents, and the need for more co-teaching experiences. EPP administrators and program coordinators will review and extend the Murray State-provided survey template to garner additional information for program improvement. The Kentucky Collaboration for Quality Data initiative will provide contact information so providers can survey completers who are teaching in Kentucky and neighboring states. The provider will share this contact information with the Murray State Assessment team as well. Furthermore, the provider plans to continue to conduct annual focus group interviews of completers to seek their input about the quality of preparedness for their current teaching positions. The provider anticipates these measures will greatly enhance the response rate and relevance of completer survey efforts.

Conclusion

The provider uses multiple measures to document completers' impact on P-12 student learning and completers'/employers' satisfaction. Although some EPP-generated and state provided data are available, the EPP is actively participating in the Kentucky Collaboration for Quality Data and partnering with KCEWS to seek ways all EPPs can gather more consistent survey and student impact data to inform program improvement. Because producing competent, proficient completers is paramount for student success, element 4.4 is the focus of the EPP's Selected Improvement Plan.