CAEP Annual Reporting Measures

The information on this page presents the annual reporting measures required by the Council for Accreditation of Education Preparation (CAEP).

CAEP Measure 1: Completer Effectiveness and Impact on P-12 Learning and Development (R4.1)

UMF’s College of Education utilizes multiple measures to ascertain the impacts program completers have on P-12 student learning and development and gauge completer satisfaction with the effectiveness of their preparation.  End of Program Questionnaires and Student Focus Groups query students nearing graduation about their experiences in their programs to determine which courses, activities, and assignments had the greatest effect on their preparation as well as what concerns they may have as they prepare to enter the profession. The Panorama Survey instrument is utilized in student teaching placements to determine student perceptions of teacher candidates and collect information regarding candidates’ impacts on student learning in the clinical setting. Biennial Alumni Surveys seek feedback regarding program strengths and areas for improvement from recent program completers. Finally, because the State of Maine does not provide data related to teacher impacts of P-12 student learning and development, the Teacher Education Unit (TEU) implements a Case Study to examine cohorts of recent graduates and interrogate their impacts on student learning. Summaries from each of these instruments are included below.

End of Program Questionnaire

The End of Program Questionnaire is administered to student teachers across all TEU programs at the end of their placements in the fall and spring semesters. Three open-response questions address program strengths, areas for improvement, and courses or experiences program completers found to be the most influential in their growth as educators. These are followed by 29 questions gauging the effectiveness their teacher education program on a Likert-scale (Effective; Partially effective; Not effective) across the six Common Core Teaching Standards (CCTS) categories – Learner and Learning; Content Knowledge; Assessment; Instructional Practices; Professional Responsibilities; and, Technology – and their respective indicators. The survey is analyzed for unit-wide trends as well as disaggregated by education program to help determine and adapt program goals.

2020-21 End of Program Questionnaire

For the 81 respondents who completed student teaching in Fall 2020 and Spring 2021, the following indicators were considered strengths of UMF’s teacher education programs:

  • Building respectful relationships
  • Creating communities of learners
  • Creating a positive classroom culture
  • Designing instruction linked to appropriate standards
  • Solving problems creatively and constructively
  • And, practicing reflective, life-long learning

In open response questions, respondents noted UMF’s dedicated, supportive and engaging faculty; opportunities to create and implement lessons in practical settings and the number of opportunities to be in the field; meaningful and purposeful education courses scaffolded within their programs.

2019-20 End of Program Questionnaire

 

Student Focus Groups

Focus Groups are short (30 min) discussions with groups of upper-level students with the goal of soliciting student perspectives on their experience, coursework, and preparation. Focus groups are held annually with students in each program (undergraduate and graduate) near the end of their program. Additional focus groups are facilitated as needed. All focus groups are facilitated by the Associate Provost & Dean of Education, which ensures all conversations are facilitated in the same manner. During focus group conversations, when students name a topic (strength, benefit, area for improvement, etc), the facilitator follows up with the topic to see if other students feel the same way. Notably, because the focus group participants’ responses are not quantified, trends cannot be quantified according to number of participants/comments; however, they can be monitored and quantified for number of programs with participants identifying specific trends, and longitudinally for number of years in which a topic is named.

2020-21 Focus Group Feedback

2019-2020 Focus Group Feedback

In 2019-20 the TEU was unable to schedule focus group discussions due to the coronavirus pandemic. Focus groups will be resumed in the 2020-21 academic year.

2018-2019 Focus Group Feedback

 

Panorama Survey

The Panorama Student Survey instrument was developed collaboratively by researchers at the Harvard Graduate School of Education and Panorama Education, a Boston-based survey tool design company. It is a valid and reliable instrument that appraises student perceptions of teaching and learning. All UMF student teachers utilize the instrument during their clinical placements at the end of their programs. Three grade-level appropriate versions of the Panorama Survey are utilized by the TEU: Grades K-2, Grades 3-5, and Grades 6-12.

2020-21 Panorama Survey Summary

2019-20 Panorama Survey Summary

 

Alumni Survey

UMF consistently seeks input from recent completers requesting their satisfaction with both the effectiveness and relevance of their preparation/program, their current career, and their future plans. The alumni survey is sent out in the spring every two years in a digital format to all recent alumni (0-3 years since graduation). In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect alumni feedback about new initiatives and community needs. Alumni responses are anonymous and all questions are optional, providing alumni the opportunity to opt out of any questions s/he does not feel comfortable answering. Questions identify the alumni’s program, license, and graduation year in order to better disaggregate and analyze responses. Likert-style scaled questions are identical to questions included in both the employer survey and the end-of-program survey completed by student teachers, and evaluates individual indicators related to different aspects of preparation. The short answer questions are about program strengths, weaknesses, and additional preparation needs.

2020 Alumni Survey

Highlights from the 2020 Alumni Survey include:

  • Nearly three quarters (73%) of recent graduates were hired within four months of graduation, with 85% hired within one year of graduation. Close to 80% are employed as a teacher, with nearly 70% working in their preferred career path and desired position.
  • UMF’s faculty, staff, mentors, supervisors, and advisors and the preparation and support they provide were highlighted as a strength by alumni.
  • UMF’s reputation as a excellent teacher preparation program opened doors for our graduates.
  • UMF seeks to graduate caring teachers, competent educators, and collaborative professional leaders. In this regard, alumni felt the teacher preparation programs were incredibly successful with 90% of graduates indicating that UMF was very effective or effective in preparing them for their careers in education. Areas of strength include preparation in:
    • building respectful relationships;
    • creating positive classroom communities;
    • honoring and responding to student differences;
    • using the best instructional practices;
    • collaborating with colleagues;
    • and, demonstrating a commitment to the ethical and legal responsibilities of the profession.

2018 Alumni Survey

2016 – 2020 Alumni Survey: Three Year Comparisons

2012 – 2016 Alumni Survey: Three Year Comparisons

 

Case Study

The State of Maine does not provide data measuring teacher impacts on P-12 student learning and development. To obtain data related to completer impacts, UMF designs and implements case studies in collaboration with area school districts that employ recent EPP completers. The purpose of these case studies is to determine if the University of Maine at Farmington’s teacher preparation program leads to appropriate student academic growth in P-12 classrooms. According to the findings of the “Measures of Effective Teaching Project” (Spring 2013, Bill and Melinda Gates Foundation), great teaching can be identified by combining three types of measures: classroom observations, student surveys, and student achievement gains. The TEU’s case studies utilize these three measures to examine cohorts of recent graduates and their impact on student learning.

For the last two academic years (2019-20 and 2020-21), the TEU has been unable to complete case studies due to restrictions on entry into schools as a result of the coronavirus pandemic. The case study has resumed for the 2021-22 academic year.

2017-18 Case Study

 

CAEP Measure 2: Satisfaction of Employers and Stakeholder Involvement (R4.2, R4.3)

The TEU values input from the employers of program completers and its school and community partners. A biennial Employer Survey seeks feedback from employers of recent completers regarding the scope and effectiveness of UMF candidate preparation. Each semester, a select group of regional teachers and school and district administrators meet with TEU faculty and administration on an Education Advisory Committee to discuss ways the College of Education can continually improve program design, address local needs, and strengthen and broaden clinical placements for teacher candidates.

 

Employer Survey

The Employer Survey is survey sent every two years to employers of recent alumni (0-3 years since graduation) requesting feedback about their satisfaction with both the effectiveness and relevance of the preparation/program UMF alumni have received.  In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect employer feedback about new initiatives and community needs. A faculty work group revised the survey this year, 2018, to remove questions that overlapped in content and sought feedback from the Education Advisory Council. Employer responses are anonymous and are not linked to specific alumni; accordingly, data from the survey cannot be directly linked to or disaggregated by a specific program or licensure group.  The first section of the survey includes Likert-style scaled questions that are identical to questions included in both the alumni survey and the end-of-program survey, which evaluate individual indicators related to different aspects of preparation. The second section includes short answer questions about program strengths, weaknesses and additional preparation needs.

2020 Employer Survey

The Employer Survey summarizes unit-wide findings but is also disaggregated by grade level: Early, Elementary, Middle, and Secondary grade levels. Some unit-wide findings include:

  •  Employers identified building successful relationships, collaboration with colleagues, and the effective use of instructional technology as strengths of UMF’s teacher preparation programs.
  • Just under 90% of employers were very satisfied or satisfied with UMF graduates ability to carry out the assigned responsibilities of their grade level.
  • Short answer responses identified the following strengths of recent UMF graduates, including:
    • Classroom management
    • Collaboration and relationship building
    • Content knowledge, instructional technology, and understanding of standards-based instruction
    • Professionalism
    • An attitude of care towards their students

2018 Employer Survey

2016 Employer Survey

 

 

 

CAEP Measure 3: Candidate Competency at Program Completion (R3.3)

UMF’s College of Education ensures its teacher candidates possess the competencies they will to teach effectively with positive impacts for all P-12 student learning and development through the application of content and standards knowledge, foundational pedagogical skills, and the integration of technology. The Essential Areas of Teaching assessment measures candidate growth across semester-long field experiences in key areas such as planning, instruction, classroom management, assessment and technology. Candidates provide evidence of their understanding of and proficiency with Maine’s Common Core Teaching Standards and the ISTE technology standards through the completion of Rationales and Artifacts for each of the standards. The Teacher Work Sample and Contextual Factors Analysis assignments evaluates candidates’ proficiency with unit-planning and design, implementation, assessment, and instructional decision-making. Teacher candidate dispositions are assessed a multiple times throughout their program as part of the the Dispositions and Professional Expectations assessment. Finally, the Classroom Management Observation Checklist provides opportunities to help candidates become proficient with the skills to create effective and safe learning environments. Summaries of each of these assessments are found below. Program graduation requirements and the TEU’s four-year and six-year graduation rates are provided for comparison as well as completer GPA summaries.

 

Essential Areas of Teaching

The Essential Areas of Teaching is a unit-wide assessment conducted by mentor teachers and field supervisors during selected field experiences (Advanced Practicum and Student Teaching). The assessment is conducted at mid-term and end-of-term. The Essential Areas instrument seeks to assess growth over the course of semester-long field experiences on indicators within several key categories: Planning, Instruction, Management, Assessment, and Technology.

2021-22 Essential Areas of Teaching Summary

Strengths of TEU student teachers include:

Instruction

  • Demonstrating an enthusiastic approach and using a variety of approaches and strategies
  • Demonstrating strong subject matter knowledge and building on prior knowledge
  • Providing appropriate feedback
  • Modeling what is to be learned
  • Monitoring and reteaching as necessary

Management

  • Positively reinforcing appropriate student behaviors and demonstrating personal regard for each student

Assessment

  • Giving meaningful feedback to students and checking for understanding throughout the lesson

Technology

  • Demonstrating ethical and legal use of technology
  • Using available technology to design and plan instruction

2020-21 Essential Areas of Teaching Summary

 

Rationales and Artifacts

The purpose of the CCTS Rationales and Artifacts assessment is to ensure student proficiency in and familiarity with each of the Common Core Teacher Standards (CCTS) and the International Society for Technology in Education (ISTE) standards. Candidates select artifacts that they believe demonstrate proficiency with at least one specific indicator for a given standard, and then write a rationale indicative of their level of proficiency with that standard and justifying how the artifact serves as evidence of that proficiency level. Candidates are assessed on ability to (1) communicate the meaning of the standard, (2) align evidence with indicators, (3) justify and
articulate the relationship between evidence and proficiency with the standard, (4) clearly communicate through writing, and (5) demonstrate proficiency through evidence.

2020-21 Rationale and Artifacts Summary

2019-20 Rationale and Artifacts Summary

 

Teacher Work Sample and Contextual Factors Analysis

The Teacher Work Sample (TWS) and Contextual Factors Analysis (CFA) is a unit-analysis project designed and vetted by the Renaissance Project as a performance-based assessment tool for evaluating students’ field experience and proficiency at unit planning, implementation, assessment, and instructional decision-making. The rubric for evaluating TWS/CFA is closely aligned with the CCTS/InTASC Teacher Standards. CFA/TWS is assigned and evaluated during the student teaching field experience by field supervisors. The rubric is divided into two sections. The TWS contains eight categories encompassing 28 indicators, while the CFA consists of nine indicators.

2020-21 TWS & CFA Summaries

2019-20 TWS & CFA Summaries

 

Teacher Candidate Dispositions and Professional Expectations

The Teacher Education Unit at the University of Maine Farmington assesses teacher candidate dispositions multiple times throughout a student’s program, both in the classroom and in the field. The purpose of the dispositions assessment is to provide early and frequent feedback to students regarding the professional expectations that are embedded in the education field. Two versions of the assessment exist. A classroom version is utilized in methods and non-field-based courses and is completed by course faculty. A field version is applied in practica, advanced practica, internships, and student teaching placements and is completed by both mentor
teachers and field supervisors. The goal of the dispositions assessment is to help students understand the professional expectations of the field, to provide a tool for providing objective, actionable feedback to students regarding those expectations, to serve as a trigger for providing support to struggling students, and to add a level of selectivity to ensure teacher candidate quality. Additionally, the instrument is a key component required for both program candidacy and successful completion of student teaching.

2020-21 Teacher Candidate Dispositions Summary

2019-20 Teacher Candidate Dispositions Summary

 

Classroom Management Observation Checklist

The Classroom Management Observation Checklist (CMOC) is an EPP-developed observation instrument utilized during a teacher candidate’s student teaching placement. The tool includes indicators of both teacher and student actions in the school setting that are integral to an effective, safe classroom community. Indicators are intended to be actionable to provide useful feedback to candidates about strategies to improve their classroom management. The rubric consists of 23 indicators within three categories: Student Behavior Observed (referring to students in the student teachers’ classroom), Teacher Behavior Observed (referring to the student teachers themselves), and Misbehavior. A broad assortment of validated indicators are included which vary in complexity and difficulty in order to provide a comprehensive view of candidate proficiency. Summaries are disaggregated by program.

2021-22 Classroom Management Observation Checklist

2020-21 Classroom Management Observation Checklist

 

 

Program Graduation Requirements

Teacher Certification programs at the University of Maine at Farmington are rigorous and include a comprehensive general education curriculum and well-designed academic majors. All programs include early field-based experiences as well as a full semester of full-time student teaching or internship. Teacher candidates seeking certification must maintain at least a 2.75 grade point average and course grades of B- or higher in their professional education courses, including successfully completing required practicum/practica. Finally, teacher candidates must successfully complete a 16-week student teaching/internship placement.

Candidates who complete programs in a teacher education major with a certification option are fully certified to teach in the state of Maine and in many other states across the U.S.

Early Childhood Education and Early Childhood Special Education also have program tracks that lead to dual certification at the B-5 and K-3 levels. Additionally, many education candidates, and especially Secondary/Middle Education majors, choose to complete a double major in a subject area concentration.

Note: Until June 2021, teacher candidates seeking licensure were required to pass the Praxis Core Academic Skills for Educators (Praxis I) exam and the related Praxis Subject area assessments (Praxis II) for their particular education program.

 

2013–2021 TEU Four- and Six-Year Graduation Rates 

The University of Maine at Farmington Teacher Education Unit (TEU) has a mean four-year graduation rate of 49.6% for students enrolled as an Education major as a first-time, full-time student (2013 to 2017 entering classes). A “First-time, full-time” enrollee is defined as a student who enrolls at a college as a first-year/freshman and as a full-time student. Students who enroll in at least 12 credits (three four-credit courses) are considered full-time at UMF. This summary excludes students who transfer into an education program.

The TEU’s mean four-year graduation rate of 49.6% surpasses national averages for public four-year colleges (40.7% within four years)*. Graduation rates for first-time, full-time undergraduate students who begin seeking a bachelor’s degree at 4-year degree-granting institutions can vary according to institutional selectivity. UMF has a 75% to 89.9% acceptance rate. Compared to other four-year public institutions within the same acceptance rate band, the mean six-year TEU graduation rate of 67.2% greatly exceeds other public universities in the same acceptance rate band (a six-year graduation rate of 57.6%)*. The TEU’s four-year and six-year rates also compare quite favorably to overall UMF graduation rates (42% and 55%, respectively).

The accompanying table shows the number of first-time, full-time students TEU students enrolled at UMF for each academic year starting in Fall 2013. It includes the number of students who graduate within four years and six years of their Fall enrollment year and the respective graduation rate. The data does not include the graduation of students who transfer into the TEU within a given four-year or six-year span.

 

Entering Term Enrollment 4-Year Graduates 4-Year Rate 6-Year Graduates 6-Year Rate
Fall 2013 191 108 56.5% 132 69.1%
Fall 2014 221 114 51.6% 152 68.8%
Fall 2015 192 93 48.4% 122 63.5%
Fall 2016 160 70 43.8%
Fall 2017 170 78 45.9%
  Average 49.6% Average 67.2%

*NCES Digest of Education Statistics. (2022). Retrieved on January 12, 2022 from https://nces.ed.gov/fastfacts/display.asp?id=40.

 

Completer GPA

2020-21 Program Completer GPA

Includes GPA data disaggregated by TEU program; three-year comparisons for each TEU program; and three-year comparisons of TEU and non-TEU completer GPA.

2019-2020 Program Completer GPA

 

 

 

CAEP Measure 4: Ability of Completers to be Hired

One pathway to licensure in Maine is graduation from a state-approved educator preparation program. As seen above, nearly half of teacher candidates graduate with a degree within four years of entering the TEU as a first-time, full-time students and over two-thirds graduate within six years. The TEU endeavors to attract, enroll, support, and graduate high-quality teacher candidates through the maintenance of high standards and expectations from admissions through program completion. Students who graduate with an Education degree in a certification track meet State of Maine licensing requirements.

Candidate Selectivity

Education is a demanding profession, and TEU faculty are committed to ensuring that UMF graduates are of the highest caliber, ready to teach, and meet state certification requirements. Students interested in majoring in one of UMF’s teacher education programs are initially accepted directly into education programs, but must meet a series of benchmarks in order to advance. These benchmarks are designed to monitor student progress through their programs and ensure that students are meeting state of Maine’s licensure requirements. In order to be allowed to enroll in upper division professional education courses, a student must apply for and attain formal Candidacy in their programs and meet the requirements outlined below:

  • Completion of the Candidacy application
  • Cumulative GPA of 2.75 or higher
  • Grade of B- or better in all professional education courses. ( A student may have one exemption, although the student must earn a grade of C- or higher in that particular course. Practicum courses are not eligible for exemption.)
  • Grade of C or better in English 100 (AP credit is accepted, as is an equivalent English course transferred with a grade of C or higher.)
  • Successful completion of practicum/practica with a grade of B- or higher
  • Successful completion of the Teacher Candidate Dispositions and Professional Expectations Assessment

*Prior to June 2021, State of Maine certification requirements necessitated passing scores on both the Praxis Core Skills for Educators exam (Praxis Core) and program-relevant Praxis Subject Assessment exams (Praxis II). These Praxis requirements were also criteria for Candidacy. As a result of changes in state policy and the removal of the Praxis as a certification requirement, the TEU amended its Candidacy requirements to match the state’s stance on these tests.

 

Title II Reports

Title II is a federal policy tasked with assisting states in “preparing, training, and recruiting high quality teachers and principals” to improve educator and administrator quality. For more information about Title II, please visit their site at https://title2.ed.gov/Public/Home.aspx

UMF’s Title II reports for several academic years are included below:

2019-2020 Title II Report

2018-2019 Title II Report

Where UMF Graduates Teach

Becoming a graduate of a UMF Education program opens the door for employment around Maine and New England, across the United States, and even internationally. While the State of Maine does not collect data tracking teacher candidate hiring, findings from the TEU’s biennial alumni survey of recent graduates completed in 2020 found nearly three quarters (73%) of recent graduates were hired within four months of graduation, with 85% hired within one year of graduation. Close to 80% graduates are employed as classroom teachers, with 70% working in their preferred career path and desired position.