CAEP Annual Reporting Measures

The information on this page presents the annual reporting measures required by the Council for Accreditation of Education Preparation (CAEP).

CAEP Measure 1: Completer Effectiveness and Impact on P-12 Learning and Development (R4.1)

UMF’s College of Education utilizes multiple measures to ascertain the impacts program completers have on P-12 student learning and development and gauge completer satisfaction with the effectiveness of their preparation.  End of Program Questionnaires and Student Focus Groups query students nearing graduation about their experiences in their programs to determine which courses, activities, and assignments had the greatest effect on their preparation as well as what concerns they may have as they prepare to enter the profession. The Panorama Survey instrument is utilized in student teaching placements to determine student perceptions of teacher candidates and collect information regarding their impacts on student learning in the clinical setting. Biennial Alumni Surveys seek feedback regarding program strengths and areas for improvement from recent program completers (1-3 years since graduation). Finally, the Teacher Education Unit (TEU) implements a Case Study to examine cohorts of recent graduates across all programs to investigate and ascertain their impact on student learning. Summaries from each of these instruments are included below.

 

End of Program Questionnaire

Assessment Description: The End of Program Questionnaire is administered to student teachers across all TEU programs at the end of their placements. Three open-response questions address program strengths and areas for improvement, and allow respondents to identify the most impactful courses or experiences on their growth as educators. These are followed by 29 questions gauging the effectiveness their teacher education program on a Likert-scale (Effective; Partially effective; Not effective) across the five Common Core Teaching Standards (CCTS) categories – Learner and Learning; Content Knowledge; Instructional Practices; Professional Responsibilities; and, Technology – and their respective indicators. The survey is analyzed for unit-wide trends as well as disaggregated by program to help determine and adapt program goals.

2021-22 End of Program Questionnaire

  • 96% of respondents across all TEU programs indicated they were Very Satisfied or Satisfied with their UMF teacher preparation program.
  • Skills where more than 90% of respondents rated their preparation as Effective are found primarily in the Learner and Learning, Instructional Practices, and Professional Responsibilities categories, and include:
    • Supporting and encouraging successful learning for all students (Learner and Learning)
    • Creating positive classroom cultures (Learner and Learning)
    • Designing, planning, and implementing instruction linked to appropriate standards (Instructional Practices)
    • Solving problems creatively and constructively (Instructional Practices)
    • Collaborating effectively with colleagues (Professional Responsibilities)
    • Practicing reflective and self-directed, life-long learning (Professional Responsibilities)

 

 

Student Focus Groups

Assessment Description: Focus Groups are short (30 min) discussions with groups of upper-level students to solicit student perspectives on their experiences, coursework, and preparation. Focus groups are held annually with students nearing the end of their program and additional focus groups are facilitated as needed. All focus groups are facilitated by the Associate Provost & Dean of Education and all sessions address the same series of basic questions. To ensure that responses are representative of the group as a whole, the facilitator will often ask follow up questions to determine agreement with identified topics and seek clarification when necessary .

2021-22 Focus Group Feedback

 

Panorama Survey

Assessment Description: The Panorama Student Survey instrument was developed collaboratively by researchers at the Harvard Graduate School of Education and Panorama Education, a Boston-based survey tool design company. It is a valid and reliable instrument that appraises student perceptions of teaching and learning. The original survey included nine scales, each containing multiple related statements elaborating on student experiences in the classroom. Students evaluate their teachers by rating their level of agreement with these statements. To keep the length of the survey manageable the UMF Field Services office selected three of the scales to assess student experiences with UMF student teachers – Pedagogical Effectiveness (PE), Expectations and Rigor (ER), and Relationships and Support (RS). Three versions of the survey are employed by the TEU: Grades K-2, Grades 3-5, and Grades 6-12. While Panorama does not have a specific survey for K-2 students it was felt that young students could also provide important information. Field Supervisors adjusted the questions to make them developmentally appropriate and modified the scoring of the assessment to a 3-point scale. All UMF student teachers utilize the instrument during their clinical placements (student teaching or internship) at the end of their programs.

2021-22 Panorama Survey Summary

 

Alumni Survey

Assessment Description: UMF consistently seeks input from recent completers regarding their satisfaction with both the effectiveness and relevance of their preparation, how that preparation has impacted their current career, and their educational goals and plans. The alumni survey is sent out to recent program completers (one to three years since graduation) biennially in a digital format. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect feedback about new initiatives and community needs. Alumni responses are anonymous and all questions are optional, providing alumni the opportunity to opt out of any questions they do not feel comfortable answering. Questions identifying the alumni’s program, license, and graduation year allow for disaggregation and comparison. Likert-style scaled questions are identical to questions included in both the employer survey and the end-of-program survey which is completed by student teachers, and evaluates individual indicators related to different aspects of preparation. Short-answer questions inquire about program strengths, weaknesses, and feedback regarding additional preparation needs.

2022 Alumni Survey

  • More than 91% of respondents (n = 105) are currently employed in the field, with two-thirds in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.).
  • 78% of respondents were hired within 4 months of graduation, and nearly 87% had been hired within a year of graduation.
  • In open response questions, alumni highlighted field experiences, faculty and staff, and skills regarding lesson planning and instructional methods aligned with state teaching standards as strengths of their preparation at UMF.
  • UMF seeks to graduate caring teachers, competent educators, and collaborative professional leaders. In this regard, alumni felt TEU’s teacher preparation programs were quite successful with 88% of graduates indicating that UMF was Very Effective or Effective in preparing them to be caring, competent, and collaborative professionals. Areas of strength include:
    • building respectful relationships;
    • supporting and encouraging successful learning in their classrooms;
    • honoring and responding to student differences;
    • effective communication skills;
    • collaborating with colleagues;
    • and, demonstrating a commitment to the ethical and legal responsibilities of the profession.
  • A comment from an alum of the Elementary Education program sums up well the overall sentiment conveyed by respondents to open response questions:

“After leaving UMF, I left feeling confident and ready to enter a school
system and begin teaching. UMF prepared me with incredible classroom
management skills, communication [and] collaboration skills with
colleagues, students, and families, and content knowledge before
graduating. I felt confident in creating student centered lessons and
activities that would meet the needs of each of my future students as
individuals.”

 

 

CAEP Measure 2: Satisfaction of Employers and Stakeholder Involvement (R4.2, R5.3)

Stakeholder Involvement

The University of Maine at Farmington’s Teacher Education Unit (TEU) regularly engages with practicing educators and school administrators through a variety of venues to ascertain local and state areas of concern as well as emerging hiring and professional development needs. UMF’s Office of Graduate and Continuing Education’s strong Outreach Office engages in ongoing communication with regional school districts and other stakeholders throughout the state to develop relevant and timely course content and degree programs. The TEU’s Field Services Director regularly attends regional superintendent association meetings to remain abreast of local and state needs. Field Services also collects survey data from mentor teachers every semester to improve and strengthen clinical experiences for both the students and the cooperating schools. An Education Advisory Council facilitated by the Dean of the College of Education and populated with TEU faculty and staff, teachers across various grade levels and content areas, and regional administrators gathers each semester to discuss ongoing concerns and trends within their schools and districts to map out potential areas for collaboration and support. Additionally, the TEU and its faculty regularly apply for and participate in grants that allow for collaboration with school partners to address recruitment, mentoring, and retention of new teachers. Recent examples include a grant to recruit and support adult learners interested in becoming educators and another to study the teacher shortage in rural Maine school districts to determine the factors impacting hiring and retention.

 

Satisfaction of Employers

A biennial survey of employers who have hired recent TEU graduates provides additional opportunities for the College of Education to connect with districts across the state. Findings are summarized and shared with TEU administration and faculty to determine the strengths of its preparation programs and clarify potential areas for improvement.

Assessment Description: The Employer Survey is sent every two years to employers of recent alumni (one to three years since graduation) requesting feedback about their satisfaction with both the effectiveness and relevance of the preparation their new hires have received from UMF. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect employer feedback about new initiatives and community needs. Employer responses are anonymous and are not linked to specific alumni; accordingly, data from the survey cannot be directly linked to or disaggregated by a specific TEU program or licensure group. However, the data can be disaggregated by grade level: Early (Infants and toddlers, PK, and K); Elementary (Grades 1 to 5); Middle (Grades 6 to 8); and Secondary (Grades 9 to 12). The first section of the survey includes Likert-style scaled questions that are identical to questions included in both the Alumni Survey and the end-of-program survey. These questions evaluate individual indicators related to different aspects of educator preparation. The second section includes open response questions about perceived program strengths, areas for improvement, and identification of additional preparation needs.

2022 Employer Survey

The Employer Survey summarizes unit-wide findings but is also disaggregated by grade level: Early, Elementary, Middle, and Secondary grade levels. Some unit-wide findings include:

  • Just over 90% of employers were Very Satisfied or Satisfied with UMF graduates’ abilities to carry out the assigned responsibilities of their grade level.
  • More than 96% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be caring teachers, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • create positive classroom communities
    • support and encourage successful learning
    • honor and respond to learner differences
    • utilize knowledge of human development
  • More than 86% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be competent educators, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • assess students in order to monitor proficiency and/or growth
    • effectively use instructional technology and facilitate student use of available technology.
  • More than 90% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be collaborative professional leaders, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • collaborate effectively with colleagues
    • demonstrate a commitment to ethical and legal responsibilities of the profession
  • Finally open response questions highlighted that recent completers are strong in content knowledge, lesson planning, technology skills, inclusive practices, collaboration skills, and their overall preparedness.

 

 

CAEP Measure 3: Candidate Competency at Program Completion (R3.3)

UMF’s College of Education ensures its teacher candidates possess the competencies they will need to teach effectively and to positively impact student learning and development. The Essential Areas of Teaching assessment measures candidate growth across semester-long field experiences in key areas such as planning, instruction, classroom management, assessment, and technology. Candidates provide evidence of their understanding of, and proficiency, with Maine’s Common Core Teaching Standards and the ISTE technology standards through the completion of Rationales and Artifacts for each of the standards. The Teacher Work Sample and Contextual Factors Analysis assignments evaluate candidate proficiency with unit-planning and design, implementation, assessment, and instructional decision-making. Teacher candidate dispositions are assessed at multiple times throughout their program as part of the the Classroom Dispositions and Professional Expectations assessment. Finally, the Classroom Management Observation Checklist provides feedback to teacher candidates to strengthen their skills to create effective and safe learning environments. Summaries of each of these assessments are found below. Program graduation requirements and the TEU’s four-year and six-year graduation rates are provided for comparison as well as completer GPA summaries.

 

Essential Areas of Teaching

Assessment Description: The Essential Areas of Teaching is a unit-wide assessment conducted by mentor teachers and field supervisors during selected field experiences (Advanced Practicum and Student Teaching). The assessment is conducted at mid-term and end-of-term. The Essential Areas instrument assesses growth over the course of semester-long field experiences on indicators within five key categories: Planning, Instruction, Management, Assessment, and Technology.

2021-22 Essential Areas of Teaching Summary

Strengths of TEU student teachers include:

Instruction

  • Demonstrating an enthusiastic approach and using a variety of approaches and strategies
  • Demonstrating strong subject matter knowledge and building on prior knowledge
  • Providing appropriate feedback
  • Modeling what is to be learned
  • Monitoring and reteaching as necessary

Management

  • Positively reinforcing appropriate student behaviors and demonstrating personal regard for each student

Assessment

  • Giving meaningful feedback to students and checking for understanding throughout the lesson

Technology

  • Demonstrating ethical and legal use of technology
  • Using available technology to design and plan instruction

 

Rationales and Artifacts

Assessment Description: The purpose of the CCTS Rationales and Artifacts assessment is to ensure student familiarity and proficiency with each of the Common Core Teacher Standards (CCTS) and the International Society for Technology in Education (ISTE) standards. Candidates select artifacts that they believe demonstrate proficiency with at least one specific indicator for a given standard and then articulate how the artifact serves as evidence of the given standard and indicator. Candidates are evaluated on their ability to (1) communicate the meaning of the standard, (2) align evidence with indicators, (3) justify and articulate the relationship between evidence and proficiency with the standard, (4) clearly communicate through writing, and (5) demonstrate proficiency through evidence.

2021-22 Rationale and Artifacts Summary

 

Teacher Work Sample and Contextual Factors Analysis

Assessment Description: The Teacher Work Sample (TWS) and Contextual Factors Analysis (CFA) is a unit-analysis project designed and vetted by the Renaissance Project. This project is a performance-based assessment tool for evaluating student teaching field experiences and evaluates proficiency at unit planning, implementation, assessment, and instructional decision-making. Field supervisors utilize rubrics closely aligned with the CCTS/InTASC Standards. The TWS rubric contains eight categories encompassing 28 indicators while the CFA rubric consists of nine indicators. Summaries are disaggregate by program.

2021-22 TWS & CFA Summaries

 

Teacher Candidate Dispositions and Professional Expectations

Assessment Description: The TEU assesses teacher candidate dispositions multiple times throughout a student’s program, both in the college classroom and in the field. The purpose of the dispositions assessment is to provide early and frequent feedback to students regarding the professional expectations that are embedded in the education field. Two versions of the assessment exist. A classroom version is utilized in methods and non-field-based courses and is completed by course faculty. A field version is utilized in practicum, advanced practicum, internships, and student teaching placements and is completed by both mentor teachers and field supervisors. Students in practicum and advanced practicum placements also complete self-evaluations using the instrument. One goal of the assessment is to help students understand the professional expectations of the field, to provide a tool for providing objective, actionable feedback to students regarding those expectations, to serve as a trigger for providing support to struggling students, and to add a level of selectivity to ensure teacher candidate quality. The instrument is a key component required for both program candidacy and successful completion of student teaching.

2021-22 Teacher Candidate Dispositions Summary

 

Classroom Management Observation Checklist

Assessment Description: The Classroom Management Observation Checklist (CMOC) is an EPP-developed observation instrument utilized during a teacher candidate’s student teaching placement. The tool includes indicators of both teacher and student actions in the school setting that are integral to an effective and safe classroom community. Indicators are intended to be actionable to provide useful feedback to candidates about strategies to improve their classroom management. The rubric consists of 23 indicators within three categories: Student Behavior Observed (referring to students in the student teachers’ classroom), Teacher Behavior Observed (referring to the student teachers themselves), and Misbehavior. A broad assortment of validated indicators are included which vary in complexity and difficulty in order to provide a comprehensive view of candidate proficiency. Summaries are disaggregated by program.

2021-22 Classroom Management Observation Checklist

 

 

Program Graduation Requirements

Teacher Certification programs at the University of Maine at Farmington are rigorous and include a comprehensive general education curriculum and well-designed academic majors. All programs include early field-based experiences as well as a full semester of full-time student teaching or internship. Teacher candidates seeking certification must maintain at least a 2.75 grade point average and course grades of B- or higher in their professional education courses, including successfully completing required clinical placements. Finally, teacher candidates must successfully complete a 16-week student teaching/internship placement.

Candidates who complete programs in a teacher education major with a certification option are fully certified to teach in the state of Maine and in many other states across the U.S.

Early Childhood Education and Early Childhood Special Education also have program tracks that lead to dual certification at the B-5 and K-3 levels. Additionally, many education candidates, and especially Secondary/Middle Education majors, choose to complete a double major in a subject area concentration.

 

2014–2018 TEU Four- and Six-Year Graduation Rates for Entering First-Time, Full-Time Students

The University of Maine at Farmington Teacher Education Unit (TEU) has a mean four-year graduation rate of 47.5% for students enrolled as first-time, full-time students in Education programs (2014 to 2018 entering classes). A “first-time, full-time” enrollee is defined as a student who enrolls at a college as a first-year/freshman and as a full-time student. Students who enroll in at least 12 credits (for example, three four-credit courses) are considered full-time at UMF. Transfer students are not included in these summaries.

The TEU’s four-year graduation rate for first-time, full-time students entering in Fall 2014 is higher than the national average for the 2014 entering cohort of all public four-year institutions (51.0% to 42.4%, respectively). The TEU’s six-year graduation rate for the entering 2014 cohort (68.3%) also exceeds the national average for all public four-year institutions (64.0%). Graduation rates for first-time, full-time undergraduate students at 4-year degree-granting institutions can also vary significantly according to institutional admissions selectivity. In 2014, UMF was positioned within the 75% to 89.9% acceptance rate band. Compared to other four-year public institutions within the same acceptance rate band, the six-year TEU graduation rate of 68.3% exceeds other similar public universities (a six-year graduation rate of 59.0%)*. The TEU’s mean four-year and six-year rates (47.5% and 65.0%, respectively) also compare favorably to overall UMF’s mean four-year and six-year graduation rates (34.6% and 49.5%, respectively).

The accompanying table shows the number of first-time, full-time students TEU students enrolled at UMF for each academic year starting in Fall 2014. It includes the number of students who graduate within four years and six years of their Fall enrollment year and the respective graduation rate. The data does not include the graduation of students who transferred into the TEU within a given four-year or six-year span.

 

Entering Term Enrollment 4-Year Graduates 4-Year Rate 6-Year Graduates 6-Year Rate
Fall 2014 208 106 51.0% 142 68.3%
Fall 2015 186 91 48.9% 119 64.0%
Fall 2016 152 70 46.1% 95 62.5%
Fall 2017 170 80 47.1%
Fall 2018 148 66 44.6%
  Average 47.5% Average 65.0%

*2014 is the most recent national entry cohort year available for comparison purposes for four-year and six-year graduation rates. NCES Digest of Education Statistics. (2022). Retrieved on January 25, 2023 from https://nces.ed.gov/fastfacts/display.asp?id=40.

 

Completer GPA

 

UMF Program

2021-22 Number of Completers Mean GPA Median GPA
All UMF Undergraduate Completers 339 3.51 3.60
Teacher Education Unit Completers 126 3.64 3.72
Non-TEU Completers 213 3.43 3.49

 

TEU Program

2021-22 Number of Completers Mean GPA Median GPA
Early Childhood Education 30 3.70 3.77
Early Childhood Special Education 10 3.54 3.57
Elementary Education 50 3.63 3.72
Secondary Education 29 3.68 3.76
Special Education 5 3.51 3.57
School Health Education 1 3.65
World Languages K-12 1 3.60

 

CAEP Measure 4: Ability of Completers to be Hired

One pathway to licensure in Maine is graduation from a state-approved educator preparation program. As seen above, nearly half of teacher candidates graduate with a degree within four years of entering the TEU as a first-time, full-time students and over two-thirds graduate within six years. The TEU endeavors to attract, enroll, support, and graduate high-quality teacher candidates through the maintenance of high standards and expectations from admissions through program completion. Students who graduate with an Education degree in a certification track meet State of Maine licensing requirements.

Candidate Selectivity

Education is a demanding profession, and TEU faculty are committed to ensuring that UMF graduates are of the highest caliber, ready to teach, and meet state certification requirements. Students interested in majoring in one of UMF’s teacher education programs are initially accepted directly into education programs, but must meet a series of benchmarks in order to advance. These benchmarks are designed to monitor student progress through their programs and ensure that students are meeting Maine’s certification requirements. In order to enroll in upper division professional education courses, a student must apply for and attain formal Candidacy in their programs and meet the requirements outlined below:

  • Completion of the Candidacy application
  • Cumulative GPA of 2.75 or higher
  • Grades of B- or better in all professional education courses. ( A student may have one exemption, although the student must earn a grade of C- or higher in that particular course. Practicum courses are not eligible for exemption.)
  • A grade of C or better in English 100 (AP credit is accepted, as is an equivalent English course transferred with a grade of C or higher.)
  • Successful completion of practicum/practica with a grade of B- or higher
  • Successful completion of the Teacher Candidate Dispositions and Professional Expectations Assessment

*Prior to June 2021, State of Maine certification requirements necessitated passing scores on both the Praxis Core Skills for Educators exam (Praxis Core) and program-relevant Praxis Subject Assessment exams (Praxis II). These Praxis requirements were also criteria for Candidacy. As a result of changes in state policy and the removal of the Praxis as a certification requirement, the TEU amended its Candidacy requirements to match the state’s stance on these tests.

 

 

Title II Reports

Title II is a federal policy tasked with assisting states in “preparing, training, and recruiting high quality teachers and principals” to improve educator and administrator quality. For more information about Title II, please visit their site at https://title2.ed.gov/Public/Home.aspx

UMF’s Title II reports for several academic years are included below:

2020-2021 Title II Report

 

Where UMF Graduates Teach

Becoming a graduate of a UMF Education program opens the door for employment around Maine and New England, across the United States, and even internationally. While the State of Maine does not collect data tracking teacher candidate hiring, findings from the TEU’s 2022 biennial survey of recent graduates found more than 91% of respondents are currently employed in the field, with two-thirds in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.). More than three quarters (78%) of respondents were hired within 4 months of graduation, and nearly 87% had been hired within a year of program completion.