Comparing Apples to Apples: Striving for Consistency in the Equitable Distribution Conversation

When teacher performance is assessed differently, can we ensure equitable access to effective teachers?

Pressure is building in the nation’s schools to get teacher evaluation right. States want to ensure that all students, regardless of ZIP code, have access to effective teachers. Leaders are making important decisions about compensation and personnel based on teacher evaluation results. But how do we ensure the equitable distribution of high-quality teachers when states, districts, and schools are using different systems and data sources to assess teacher effectiveness? Could a teacher be labeled as “effective” in District A but ”ineffective” in District B?

Answers to these questions are important—teachers’ jobs and student learning are at stake—so one critical need is for accurate and consistent data from all schools and districts. Unfortunately, the data sources used now haven’t been examined, and we don’t yet know if:

• We are far enough along in teacher evaluation reforms to get the data we need to understand how effective teachers are distributed across a state.
• Teacher evaluation systems that use different measures of performance yield data that can be compared across districts.
• Concentrations of high-performing teachers in a specific set of schools or districts truly signal inequitable distribution or simply reflects differences in evaluation measures and processes.

During the past eight years, a large number of states, districts, and charter schools have been designing and implementing new educator evaluation and performance-based compensation systems to weigh both teacher practice and student academic growth, sometimes along with other factors. Most Teacher Incentive Fund (TIF) grantees formally observe teachers, often adopting such research-based frameworks as those developed by Danielson, Marzano, and others. Besides observations of teacher classroom practice, teacher evaluation systems often measure student growth in the subjects and grades for which state assessment scores are available.

But one-size-fits-all professional practice frameworks (which help ensure fairness) may not work well for those who teach certain subjects or non-tested subjects, or who mainly teach special education or English learner students. As indicated in the GTL Center’s database on state teacher evaluation policies, most states also give districts significant flexibility in measures used to assess teacher practice in local evaluation systems. Further, reliably assessing student growth in the grades and subjects that are not part of state testing programs remains a challenge. All told, these caveats apply to about 70 percent of all teachers!

In response, some school systems are exploring such measures as (1) comparing pre- and post-test data on tests, including end-of-course examinations, whether custom or off-the-shelf; or (2) creating student learning objectives (SLOs), which use data to create measurable targets for student performance and then track student progress toward those goals. But it’s clear from commonly reported implementation challenges that we’re still far from getting these teacher- evaluation measures “right.” So for at least the next few years, we may not be able to say that the data yielded from teacher evaluation systems across states and districts are “comparable”.

The results of most new educator evaluation systems—for now anyway—are likely to be unpredictably influenced by the methodologies used to assess teacher practice and student academic growth. That means that teacher effectiveness ratings, performance-based compensation incentives tied to them, and the district and state measures of equitable distribution of effective teachers must be monitored hawkishly while we strive to understand which measures are more accurate, consistent, and rigorous. Meanwhile, we must ensure fairness and still honor state and district needs and priorities.

The ultimate goal is to create evidence-based educator evaluation systems that are transparent and fair to all teachers while improving teaching and student outcomes. Expect that to take several years of patient revisions and sometimes painful lessons. In the meantime, we still need to be focused on equity issues and using the data as best we can.

Now it’s your turn. What challenges are you facing in making sense of diverse evaluation system data when thinking about equitable distribution?