DCPS gets all its teacher evaluators on the same page

In an effort to build confidence in its teacher evaluation system, DCPS has launched a program that uses videos to ensure that its various evaluators are scoring teachers consistently.

As part of the IMPACT evaluation system introduced by former Chancellor Michelle Rhee, teachers are observed in their classrooms up to 5 times a year. Sometimes a principal or assistant principal observes the teachers, while other times it’s one of a team of outside evaluators called Master Educators.

As detailed in the guidebooks that explain the evaluation system, a teacher’s classroom performance is measured against the district’s “Teaching and Learning Framework,” and observation scores can make up as much as 75% of a teacher’s overall IMPACT score. (For some teachers it’s 40%; the remainder of the score is largely determined by student test scores, which is a whole other source of controversy.)

A teacher’s IMPACT score has significant consequences. teachers rated “highly effective” are eligible for bonuses and salary increases, while those at the opposite end of the spectrum can lose their jobs. And teachers have had complaints about the IMPACT system since its inception.

One problem has been that two evaluators observing the same lesson may come up with two significantly divergent scores. For the past year and a half, DCPS has been working to rectify that situation through a project called Align, funded by a $1.5 million grant from the Bill and Melinda Gates Foundation. (In a recent Washington Post op-ed, Bill Gates argued that an evaluation system that uses multiple measures, including classroom observations, is fairer than a system that relies solely on students’ test scores.)

Classroom teachers received cameras and microphones and tips on how to film themselves teaching. Then, groups of Master Educators and principals watched those videos together, discussed their reactions, and built consensus on “anchor ratings.”

It’s been a long and painstaking process. According to Michelle Hudacsko, Deputy Chief of IMPACT, it takes many hours of filming to produce one usable video. And getting multiple viewers to agree on ratings across the 9 different “teaching standards” in the Teaching and Learning Framework rubric can take many more hours.

Among the 9 teaching standards are “Provide Students Multiple Ways to Move Toward Mastery” and “Develop Higher-Level Understanding Through Effective Questioning.” Although the criteria for evaluating a teacher’s performance against the standards are spelled out in detail, there’s still a lot of room for subjectivity.

In the process of aligning their scores with one another, evaluators say they begin to recognize the different approaches and orientations each of them bring to their observations. The idea is that once evaluators are aware of where they tend to diverge from the norm, they’ll be better able to control for their own particular biases.

Now that the anchor ratings are in place, DCPS will soon unveil an online platform that it can use to train other evaluators. An evaluator will be able to watch a 3- to 4-minute video clip that illustrates a particular teaching standard, come up with a score, and then compare it to the anchor score. DCPS will then provide in-person training for evaluators whose scores in certain areas vary significantly from the anchor ratings. The goal is to have all Master Educators and principals trained in the Align system by the beginning of the next school year.

DCPS hopes that standardizing evaluations will lead teachers to place more trust in the evaluation process. “Observations used to be a threatening thing in my building,” says Tenia Pritchard, the principal at Whittier Education Campus, who was a pilot participant in Align training. “Now the teachers know we’re using the framework to help them improve their instruction.”

Under the new system the emphasis in post-evaluation conferences will shift from providing a score to providing feedback based on evidence, says Eric Reese of the DC Public Education Fund, an organization that funnels private philanthropic funds to DCPS. “The big change is that we’re using the master educators as coaches,” he says, “especially in the lowest-performing 40 schools.”

It remains to be seen how many DCPS teachers will embrace IMPACT even after the Align system is in place. Teachers have had other complaints about observations aside from the lack of consensus in scoring. For example, while one “informal” observation is announced in advance, the others (the ones that count) are unannounced. The surprise nature of those visits can be a source of stress. And some teachers feel under pressure to demonstrate proficiency across all 9 teaching standards in the space of 30 minutes.

Beyond that, there’s a basic question about whether it’s fair to compare a teacher who has a class of high-performing students at School Without Walls or Wilson against a teacher at a non-selective neighborhood school in Ward 7 or 8. If a class has a high proportion of students with learning disabilities or behavior problems, some teachers say, the teacher is far more likely to be rated poorly.

But since DC adopted IMPACT, teaching quality (at least as measured by the system itself) has gone up: according to DCPS, the proportion of “minimally effective” teachers whose rating improved went up to 73% last year, as compared to 58% the year before.

IMPACT isn’t perfect, but some kind of systematic evaluation system for teachers is necessary. We can’t go back to the old, pre-Rhee days when principals had a lot of leeway in doing evaluations and nearly all teachers got high ratings, even though student performance was dismal. It may be impossible to design a comprehensive evaluation system that pleases everyone, but bringing consistency to evaluators’ ratings is certainly a step in the right direction.