CAEP Annual Reporting Measures

The information on this page presents the annual reporting measures required by the Council for Accreditation of Education Preparation (CAEP).

CAEP Measure 1: Completer Effectiveness and Impact on P-12 Learning and Development (R4.1)

UMF’s College of Education utilizes multiple measures to ascertain the impacts program completers have on P-12 student learning and development and gauge completer satisfaction with the effectiveness of their preparation.  End of Program Questionnaires and Student Focus Groups query students nearing graduation about their experiences in their programs to determine which courses, activities, and assignments had the greatest effect on their preparation as well as what concerns they may have as they prepare to enter the profession. The Panorama Survey instrument is utilized in student teaching placements to determine student perceptions of teacher candidates and collect information regarding candidates’ impacts on student learning in the clinical setting. Biennial Alumni Surveys seek feedback regarding program strengths and areas for improvement from recent program completers. Finally, because the State of Maine does not provide data related to teacher impacts of P-12 student learning and development, the Teacher Education Unit (TEU) implements a Case Study to examine cohorts of recent graduates and interrogate their impacts on student learning. Summaries from each of these instruments are included below.

End of Program Questionnaire

The End of Program Questionnaire is administered to student teachers across all TEU programs at the end of their placements. Three open-response questions address program strengths and areas for improvement, and allow respondents to identify the most impactful courses or experiences on their growth as educators. These are followed by 29 questions gauging the effectiveness their teacher education program on a Likert-scale (Effective; Partially effective; Not effective) across the five Common Core Teaching Standards (CCTS) categories – Learner and Learning; Content Knowledge; Instructional Practices; Professional Responsibilities; and, Technology – and their respective indicators. The survey is analyzed for unit-wide trends as well as disaggregated by education program to help determine and adapt program goals.

2021-22 End of Program Questionnaire

  • 96% of respondents across all TEU programs indicated they were Very Satisfied or Satisfied with their UMF teacher preparation program.
  • Skills where more than 90% of respondents rated their preparation as Effective are found primarily in the Learner and Learning, Instructional Practices, and Professional Responsibilities categories, and include:
    • Supporting and encouraging successful learning for all students (Learner and Learning)
    • Creating positive classroom cultures (Learner and Learning)
    • Designing, planning, and implementing instruction linked to appropriate standards (Instructional Practices)
    • Solving problems creatively and constructively (Instructional Practices)
    • Collaborating effectively with colleagues (Professional Responsibilities)
    • Practicing reflective and self-directed, life-long learning (Professional Responsibilities)

 

Student Focus Groups

Focus Groups are short (30 min) discussions with groups of upper-level students with the goal of soliciting student perspectives on their experience, coursework, and preparation. Focus groups are held annually with students in each program (undergraduate and graduate) near the end of their program. Additional focus groups are facilitated as needed. All focus groups are facilitated by the Associate Provost & Dean of Education, which ensures all conversations are facilitated in the same manner. During focus group conversations, when students name a topic (strength, benefit, area for improvement, etc), the facilitator follows up with the topic to see if other students feel the same way. Notably, because the focus groups are qualitative in nature, trends are not quantified according to number of participants/comments; however, they can be monitored and quantified longitudinally for number of years in which a topic is named.

2021-22 Focus Group Feedback

 

Panorama Survey

The Panorama Student Survey instrument was developed collaboratively by researchers at the Harvard Graduate School of Education and Panorama Education, a Boston-based survey tool design company. It is a valid and reliable instrument that appraises student perceptions of teaching and learning. The original survey included nine scales, each containing multiple related statements elaborating on student experiences in the classroom. Students evaluate their teachers by rating their level of agreement with these statements. To keep the length of the survey manageable the UMF Field Services office selected three of the scales to assess student experiences with UMF student teachers – Pedagogical Effectiveness (PE), Expectations and Rigor (ER), and Relationships and Support (RS). Three versions of the survey are employed by the TEU: Grades K-2, Grades 3-5, and Grades 6-12. While Panorama does not have a specific survey for K-2 students it was felt that young students could also provide important information. Field Supervisors adjusted the questions to make them developmentally appropriate and modified the scoring of the assessment to a 3-point scale. All UMF student teachers utilize the instrument during their clinical placements (student teaching or internship) at the end of their programs.

2021-22 Panorama Survey Summary

 

Alumni Survey

UMF consistently seeks input from recent completers regarding their satisfaction with both the effectiveness and relevance of their preparation/program, how that preparation has impacted their current career, and their future plans. The alumni survey is sent out to recent alumni (one to three years since graduation) biennially in a digital format. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect feedback about new initiatives and community needs. Alumni responses are anonymous and all questions are optional, providing alumni the opportunity to opt out of any questions they do not feel comfortable answering. Questions identifying the alumni’s program, license, and graduation year allow for disaggregation and comparison. Likert-style scaled questions are identical to questions included in both the employer survey and the end-of-program survey completed by student teachers, and evaluates individual indicators related to different aspects of preparation. The short answer questions are about program strengths, weaknesses, and additional preparation needs.

2022 Alumni Survey

  • More than 91% of respondents (n = 105) are currently employed in the field, with two-thirds in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.).
  • 78% of respondents were hired within 4 months of graduation, and nearly 87% had been hired within a year of graduation.
  • In open response questions, alumni highlighted field experiences, faculty and staff, and skills regarding lesson planning and instructional methods aligned with state teaching standards as strengths of their preparation at UMF.
  • UMF seeks to graduate caring teachers, competent educators, and collaborative professional leaders. In this regard, alumni felt TEU’s teacher preparation programs were quite successful with 88% of graduates indicating that UMF was Very Effective or Effective in preparing them to be caring, competent, and collaborative professionals. Areas of strength include:
    • building respectful relationships;
    • supporting and encouraging successful learning in their classrooms;
    • honoring and responding to student differences;
    • effective communication skills;
    • collaborating with colleagues;
    • and, demonstrating a commitment to the ethical and legal responsibilities of the profession.
  • A comment from an alum of the Elementary Education program sums up well the overall sentiment conveyed by respondents to open response questions:

“After leaving UMF, I left feeling confident and ready to enter a school
system and begin teaching. UMF prepared me with incredible classroom
management skills, communication [and] collaboration skills with
colleagues, students, and families, and content knowledge before
graduating. I felt confident in creating student centered lessons and
activities that would meet the needs of each of my future students as
individuals.”

 

Case Study

The State of Maine does not provide data measuring teacher impacts on P-12 student learning and development. To obtain data related to completer impacts, UMF designs and implements case studies in collaboration with area school districts that employ recent EPP completers. The purpose of these case studies is to determine if the University of Maine at Farmington’s teacher preparation program leads to appropriate student academic growth in P-12 classrooms. According to the findings of the “Measures of Effective Teaching Project” (Spring 2013, Bill and Melinda Gates Foundation), great teaching can be identified by combining three types of measures: classroom observations, student surveys, and student achievement gains. The TEU’s case studies utilize these three measures to examine cohorts of recent graduates and their impact on student learning.

For the last two academic years (2019-20 and 2020-21), the TEU has been unable to complete case studies due to restrictions on entry into schools as a result of the coronavirus pandemic. The case study has resumed for the 2021-22 academic year.

2021-22 Case Study

 

CAEP Measure 2: Satisfaction of Employers and Stakeholder Involvement (R4.2, R5.3)

The TEU values input from the employers of program completers and its school and community partners. A biennial Employer Survey seeks feedback from employers of recent completers regarding the scope and effectiveness of UMF candidate preparation. Each semester, a select group of regional teachers and school and district administrators meet with TEU faculty and administration on an Education Advisory Committee to discuss ways the College of Education can continually improve program design, address local needs, and strengthen and broaden clinical placements for teacher candidates.

 

Employer Survey

The Employer Survey is survey sent every two years to employers of recent alumni (one to three years since graduation) requesting feedback about their satisfaction with both the effectiveness and relevance of the preparation/program UMF alumni have received. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect employer feedback about new initiatives and community needs. Employer responses are anonymous and are not linked to specific alumni; accordingly, data from the survey cannot be directly linked to or disaggregated by a specific TEU program or licensure group, but can be disaggregated by grade level: Early (Infants and toddlers, PK, and K); Elementary (Grades 1 to 5); Middle (Grades 6 to 8); and Secondary (Grades 9 to 12). The first section of the survey includes Likert-style scaled questions that are identical to questions included in both the alumni survey and the end-of-program survey, which evaluate individual indicators related to different aspects of preparation. The second section includes open response questions about perceived program strengths, weaknesses, and identifying additional preparation needs.

2022 Employer Survey

The Employer Survey summarizes unit-wide findings but is also disaggregated by grade level: Early, Elementary, Middle, and Secondary grade levels. Some unit-wide findings include:

  • Just over 90% of employers were Very Satisfied or Satisfied with UMF graduates’ abilities to carry out the assigned responsibilities of their grade level.
  • More than 96% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be caring teachers, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • create positive classroom communities
    • support and encourage successful learning
    • honor and respond to learner differences
    • utilize knowledge of human development
  • More than 86% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be competent educations, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • assess students in order to monitor proficiency and/or growth
    • effectively use instructional technology and facilitate student use of available technology.
  • More than 90% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be collaborative professional leaders, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • collaborate effectively with colleagues
    • demonstrate a commitment to ethical and legal responsibilities of the profession
  • Finally open response questions identified content knowledge, lesson planning, technology skills, inclusive practices, collaboration skills, and overall preparedness as strengths of recent TEU program completers.

 

 

 

CAEP Measure 3: Candidate Competency at Program Completion (R3.3)

UMF’s College of Education ensures its teacher candidates possess the competencies they will to teach effectively with positive impacts for all P-12 student learning and development through the application of content and standards knowledge, foundational pedagogical skills, and the integration of technology. The Essential Areas of Teaching assessment measures candidate growth across semester-long field experiences in key areas such as planning, instruction, classroom management, assessment and technology. Candidates provide evidence of their understanding of and proficiency with Maine’s Common Core Teaching Standards and the ISTE technology standards through the completion of Rationales and Artifacts for each of the standards. The Teacher Work Sample and Contextual Factors Analysis assignments evaluates candidates’ proficiency with unit-planning and design, implementation, assessment, and instructional decision-making. Teacher candidate dispositions are assessed a multiple times throughout their program as part of the the Dispositions and Professional Expectations assessment. Finally, the Classroom Management Observation Checklist provides opportunities to help candidates become proficient with the skills to create effective and safe learning environments. Summaries of each of these assessments are found below. Program graduation requirements and the TEU’s four-year and six-year graduation rates are provided for comparison as well as completer GPA summaries.

 

Essential Areas of Teaching

The Essential Areas of Teaching is a unit-wide assessment conducted by mentor teachers and field supervisors during selected field experiences (Advanced Practicum and Student Teaching). The assessment is conducted at mid-term and end-of-term. The Essential Areas instrument seeks to assess growth over the course of semester-long field experiences on indicators within several key categories: Planning, Instruction, Management, Assessment, and Technology.

2021-22 Essential Areas of Teaching Summary

Strengths of TEU student teachers include:

Instruction

  • Demonstrating an enthusiastic approach and using a variety of approaches and strategies
  • Demonstrating strong subject matter knowledge and building on prior knowledge
  • Providing appropriate feedback
  • Modeling what is to be learned
  • Monitoring and reteaching as necessary

Management

  • Positively reinforcing appropriate student behaviors and demonstrating personal regard for each student

Assessment

  • Giving meaningful feedback to students and checking for understanding throughout the lesson

Technology

  • Demonstrating ethical and legal use of technology
  • Using available technology to design and plan instruction

 

Rationales and Artifacts

The purpose of the CCTS Rationales and Artifacts assessment is to ensure student proficiency in and familiarity with each of the Common Core Teacher Standards (CCTS) and the International Society for Technology in Education (ISTE) standards. Candidates select artifacts that they believe demonstrate proficiency with at least one specific indicator for a given standard, and then write a rationale indicative of their level of proficiency with that standard, justifying how the artifact serves as evidence of that proficiency level. Candidates are assessed on ability to (1) communicate the meaning of the standard, (2) align evidence with indicators, (3) justify and articulate the relationship between evidence and proficiency with the standard, (4) clearly communicate through writing, and (5) demonstrate proficiency through evidence.

2021-22 Rationale and Artifacts Summary

 

Teacher Work Sample and Contextual Factors Analysis

The Teacher Work Sample (TWS) and Contextual Factors Analysis (CFA) is a unit-analysis project designed and vetted by the Renaissance Project. This project is a performance-based assessment tool for evaluating student teaching field experiences and evaluates proficiency at unit planning, implementation, assessment, and instructional decision-making. Field supervisors utilize rubrics closely aligned with the CCTS/InTASC Standards. The TWS rubric contains eight categories encompassing 28 indicators while the CFA rubric consists of nine indicators. Summaries are disaggregate by program.

2021-22 TWS & CFA Summaries

 

Teacher Candidate Dispositions and Professional Expectations

The TEU assesses teacher candidate dispositions multiple times throughout a student’s program, both in the college classroom and in the field. The purpose of the dispositions assessment is to provide early and frequent feedback to students regarding the professional expectations that are embedded in the education field. Two versions of the assessment exist. A classroom version is utilized in methods and non-field-based courses and is completed by course faculty. A field version is utilized in practica, advanced practica, internships, and student teaching placements and is completed by both mentor teachers and field supervisors. Students in practicum and advanced practicum placements also complete self-evaluations using the instrument. One goal of the assessment is to help students understand the professional expectations of the field, to provide a tool for providing objective, actionable feedback to students regarding those expectations, to serve as a trigger for providing support to struggling students, and to add a level of selectivity to ensure teacher candidate quality. The instrument is a key component required for both program candidacy and successful completion of student teaching.

2021-22 Teacher Candidate Dispositions Summary

 

Classroom Management Observation Checklist

The Classroom Management Observation Checklist (CMOC) is an EPP-developed observation instrument utilized during a teacher candidate’s student teaching placement. The tool includes indicators of both teacher and student actions in the school setting that are integral to an effective, safe classroom community. Indicators are intended to be actionable to provide useful feedback to candidates about strategies to improve their classroom management. The rubric consists of 23 indicators within three categories: Student Behavior Observed (referring to students in the student teachers’ classroom), Teacher Behavior Observed (referring to the student teachers themselves), and Misbehavior. A broad assortment of validated indicators are included which vary in complexity and difficulty in order to provide a comprehensive view of candidate proficiency. Summaries are disaggregated by program.

2021-22 Classroom Management Observation Checklist

 

 

Program Graduation Requirements

Teacher Certification programs at the University of Maine at Farmington are rigorous and include a comprehensive general education curriculum and well-designed academic majors. All programs include early field-based experiences as well as a full semester of full-time student teaching or internship. Teacher candidates seeking certification must maintain at least a 2.75 grade point average and course grades of B- or higher in their professional education courses, including successfully completing required clinical placements. Finally, teacher candidates must successfully complete a 16-week student teaching/internship placement.

Candidates who complete programs in a teacher education major with a certification option are fully certified to teach in the state of Maine and in many other states across the U.S.

Early Childhood Education and Early Childhood Special Education also have program tracks that lead to dual certification at the B-5 and K-3 levels. Additionally, many education candidates, and especially Secondary/Middle Education majors, choose to complete a double major in a subject area concentration.

Note: Until June 2021, teacher candidates seeking licensure were required to pass the Praxis Core Academic Skills for Educators (Praxis I) exam and the related Praxis Subject area assessments (Praxis II) for their particular education program.

 

2014–2018 TEU Four- and Six-Year Graduation Rates for Entering First-Time, Full-Time Students

The University of Maine at Farmington Teacher Education Unit (TEU) has a mean four-year graduation rate of 47.5% for students enrolled as first-time, full-time students in Education programs (2014 to 2018 entering classes). A “First-time, full-time” enrollee is defined as a student who enrolls at a college as a first-year/freshman and as a full-time student. Students who enroll in at least 12 credits (three four-credit courses) are considered full-time at UMF. Transfer students are not included in these summaries.

For comparison purposes, the TEU’s four-year graduation rate for first-time, full-time students entering in Fall 2014 is higher than the national average for the 2014 entering cohort of all public four-year institutions (51.0% to 42.4%, respectively). The TEU’s six-year graduation rate for the entering 2014 cohort (68.3%) also exceeds the national average for all public four-year institutions (64.0%). Graduation rates for first-time, full-time undergraduate students at 4-year degree-granting institutions can also vary significantly according to institutional selectivity. In 2014, UMF was positioned within the 75% to 89.9% acceptance rate band. Compared to other four-year public institutions within the same acceptance rate band, the six-year TEU graduation rate of 68.3% also exceeds other similar public universities (a six-year graduation rate of 59.0%)*. The TEU’s mean four-year and six-year rates (47.5% and 65.0%, respectively) also compare favorably to overall UMF’s mean four-year and six-year graduation rates (34.6% and 49.5%, respectively).

The accompanying table shows the number of first-time, full-time students TEU students enrolled at UMF for each academic year starting in Fall 2014. It includes the number of students who graduate within four years and six years of their Fall enrollment year and the respective graduation rate. The data does not include the graduation of students who transfer into the TEU within a given four-year or six-year span.

 

Entering Term Enrollment 4-Year Graduates 4-Year Rate 6-Year Graduates 6-Year Rate
Fall 2014 208 106 51.0% 142 68.3%
Fall 2015 186 91 48.9% 119 64.0%
Fall 2016 152 70 46.1% 95 62.5%
Fall 2017 170 80 47.1%
Fall 2018 148 66 44.6%
  Average 47.5% Average 65.0%

*2014 is the most recent entry cohort year available for comparison purposes for four-year and six-year graduation rates. NCES Digest of Education Statistics. (2022). Retrieved on January 25, 2023 from https://nces.ed.gov/fastfacts/display.asp?id=40.

Completer GPA

2021-22 Program Completer GPA

Includes GPA data disaggregated by TEU program; three-year comparisons for each TEU program; and three-year comparisons of TEU and non-TEU completer GPA.

 

 

CAEP Measure 4: Ability of Completers to be Hired

One pathway to licensure in Maine is graduation from a state-approved educator preparation program. As seen above, nearly half of teacher candidates graduate with a degree within four years of entering the TEU as a first-time, full-time students and over two-thirds graduate within six years. The TEU endeavors to attract, enroll, support, and graduate high-quality teacher candidates through the maintenance of high standards and expectations from admissions through program completion. Students who graduate with an Education degree in a certification track meet State of Maine licensing requirements.

Candidate Selectivity

Education is a demanding profession, and TEU faculty are committed to ensuring that UMF graduates are of the highest caliber, ready to teach, and meet state certification requirements. Students interested in majoring in one of UMF’s teacher education programs are initially accepted directly into education programs, but must meet a series of benchmarks in order to advance. These benchmarks are designed to monitor student progress through their programs and ensure that students are meeting state of Maine’s licensure requirements. In order to enroll in upper division professional education courses, a student must apply for and attain formal Candidacy in their programs and meet the requirements outlined below:

  • Completion of the Candidacy application
  • Cumulative GPA of 2.75 or higher
  • Grades of B- or better in all professional education courses. ( A student may have one exemption, although the student must earn a grade of C- or higher in that particular course. Practicum courses are not eligible for exemption.)
  • A grade of C or better in English 100 (AP credit is accepted, as is an equivalent English course transferred with a grade of C or higher.)
  • Successful completion of practicum/practica with a grade of B- or higher
  • Successful completion of the Teacher Candidate Dispositions and Professional Expectations Assessment

*Prior to June 2021, State of Maine certification requirements necessitated passing scores on both the Praxis Core Skills for Educators exam (Praxis Core) and program-relevant Praxis Subject Assessment exams (Praxis II). These Praxis requirements were also criteria for Candidacy. As a result of changes in state policy and the removal of the Praxis as a certification requirement, the TEU amended its Candidacy requirements to match the state’s stance on these tests.

 

Title II Reports

Title II is a federal policy tasked with assisting states in “preparing, training, and recruiting high quality teachers and principals” to improve educator and administrator quality. For more information about Title II, please visit their site at https://title2.ed.gov/Public/Home.aspx

UMF’s Title II reports for several academic years are included below:

2019-2020 Title II Report

2018-2019 Title II Report

 

Where UMF Graduates Teach

Becoming a graduate of a UMF Education program opens the door for employment around Maine and New England, across the United States, and even internationally. While the State of Maine does not collect data tracking teacher candidate hiring, findings from the TEU’s biennial alumni survey of recent graduates completed in 2020 found nearly three quarters (73%) of recent graduates were hired within four months of graduation, with 85% hired within one year of graduation. Close to 80% graduates are employed as classroom teachers, with 70% working in their preferred career path and desired position.