Evaluating Teacher Effectiveness: The What, How and Why of Educator Evaluation
TQ Center Webcast
Monday, March 30, 2009, 3 p.m.–4:30 p.m. (ET)
Summary
The National Comprehensive Center for Teacher Quality (TQ Center) and the Regional Educational Laboratory (REL) Midwest cohosted a webcast titled Evaluating Teacher Effectiveness: The What, How, and Why of Educator Evaluation on Monday, March 30, 2009, at 3 p.m. (ET). Douglas Harris, Ph.D., of the University of Wisconsin–Madison, and Laura Goe, Ph.D., of ETS and the TQ Center, provided practitioners and policymakers with an overview of the existing and emerging research on teacher evaluation methods, including value-added models and observation protocols. The presentations included a discussion on the quality of the evidence supporting these methods and the applications for each method that are consistent with the evidence supporting it.
Presentations
Laura Goe, Ph.D.
Research Scientist, ETS
Measuring Teacher Effectiveness (PDF 233 KB)
Douglas N. Harris, Ph.D.
University of Wisconsin at Madison
Teacher Value-Added Credentials as Tools for School Improvement (PDF 5 MB)
Presenter Bios
Laura Goe, Ph.D.
Laura Goe, Ph.D., is a research scientist at ETS in Princeton, New Jersey, and principal investigator for research and dissemination for the National Comprehensive Center for Teacher Quality. Her research interests include teacher qualifications, measuring teacher quality, teacher effectiveness, and the equitable distribution of teachers as well as school finance and resource use. Dr. Goe currently is a coeditor of the American Educational Research Association journal Educational Evaluation and Policy Analysis. She was the research director for the Bay Area Consortium for Urban Education while at the University of California, Berkeley, working to improve teacher quality and supply in urban school districts. Previously, Dr. Goe taught at-risk middle school students (special education and language arts) in Mississippi and Memphis, Tennessee. She earned a master's degree in educational policy and leadership from the University of Memphis and a doctorate from the University of California, Berkeley, in the Policy, Organizations, Measurement, and Evaluation program.
Douglas Harris, Ph.D.
Douglas Harris, Ph.D., is an economist whose research explores how the level and equity of student educational outcomes are influenced by education policies such as desegregation, standards, teacher certification, test-based accountability, school choice, privatization, and school finance. His work also focuses on the educational role of factors such as families and neighborhoods and the way in which educational outcomes affect the long-term labor market success of individual students and the overall competitiveness of national economies. In studying these topics, Dr. Harris develops and utilizes innovative research methods, including value-added modeling, mixed methods, and cost analysis. His research frequently is cited in current policy debates, and he consults widely on policy matters with organizations such as the National Academy of Sciences, RAND, the U.S. Department of Education, and state education agencies.
Moderator
Tricia Coulter, Ph.D.
Tricia Coulter, Ph.D., is deputy director of the National Comprehensive Center for Teacher Quality (TQ Center) at Learning Point Associates. She coordinates the TQ Center's work to build the capacity of regional comprehensive centers and states to implement the highly qualified teacher requirements of the No Child Left Behind Act. Previously, Dr. Coulter was director of the Teaching Quality and Leadership Institute at the Education Commission of the States. In this position, she created and managed work related to the preparation, support, and compensation of quality teachers and leaders. Dr. Coulter also has worked as a senior research analyst at the State Higher Education Executive Officers organization, where she developed experience and expertise in issues of teacher preparation and professional development. She has extensive experience analyzing and using policy and research to help states create quality policy and innovative practice that meet their needs and challenges related to teacher quality and leadership. Dr. Coulter also has worked directly with states and districts on federal reporting requirements and their efforts to ensure all their students are served by highly qualified teachers. She earned a doctorate in counseling and educational psychology from the University of Nevada–Reno.
Resources
Policy
Brandt, C., Mathers, C., Oliva, M., Brown-Sims, M., & Hess, J. (2007). Examining district guidance to schools on teacher evaluation policies in the Midwest region (Issues & Answers Report, REL 2007-No. 030). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, REL Midwest. Retrieved February 19, 2009, from http://ies.ed.gov/ncee/edlabs/regions/midwest/pdf/REL_2007030.pdf
Heneman, H. G., III, Kimball, S., & Milanowski, A. (2006). The teacher sense of efficacy scale: Validation evidence and behavioral prediction (WCER Working Paper No. 2006-7). Madison: Wisconsin Center for Education Research. Retrieved February 19, 2009, from http://www.wceruw.org/publications/workingPapers/Working_Paper_No_2006_07.pdf
Heneman, H. G., III, Milanowski, A., Kimball, S. M., & Odden, A. (2006). Standards-based teacher evaluation as a foundation for knowledge- and skill-based pay (CPRE Policy Brief No. RB-45). Philadelphia: Consortium for Policy Research in Education. Retrieved February 19, 2009, from http://www.cpre.org/images/stories/cpre_pdfs/RB45.pdf
McCaffrey, D. F., Koretz, D. M., Lockwood, J. R., & Hamilton, L. S. (2004). The promise and peril of using value-added modeling to measure teacher effectiveness (Research Brief No. RB-9050-EDU). Santa Monica, CA: RAND Corporation. Retrieved February 19, 2009, from http://www.rand.org/pubs/research_briefs/2005/RAND_RB9050.pdf
McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: RAND Corporation. Retrieved February 19, 2009, from http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf
Murnane, R. J., & Steele, J. L. (2007). What is the problem? The challenge of providing effective teachers for all children. The Future of Children, 17(1), 15-43. Retrieved February 19, 2009, from http://futureofchildren.org/futureofchildren/publications/docs/17_01_02.pdf
Office of Postsecondary Education. (2003). Meeting the highly qualified teachers challenge: The secretary's second annual report on teacher quality. Washington, DC: U.S. Department of Education. Retrieved February 19, 2009, from http://www.ed.gov/about/reports/annual/teachprep/2003title-ii-report.pdf
Office of Postsecondary Education. (2005). A highly qualified teacher in every classroom: The secretary's fourth annual report on teacher quality. Washington, DC: U.S. Department of Education. Retrieved February 19, 2009, from http://www.ed.gov/about/reports/annual/teachprep/2005Title2-Report.pdf
Porter, A. C., Kirst, M. W., Osthoff, E. J., Smithson, J. L., & Schneider, S. A. (1993). Reform up close: An analysis of high school mathematics and science classrooms (Final report to the National Science Foundation). Madison, WI: Consortium for Policy Research in Education. (ERIC Document Reproduction Service No. ED364429). Retrieved February 19, 2009, from http://eric.ed.gov/ERICDocs/data/ericdocs2sql/
content_storage_01/0000019b/80/13/39/3c.pdf
Rice, J. K. (2003). Executive summary and Introduction. In J. K. Rice, Teacher quality: Understanding the effectiveness of teacher attributes (pp. v-vii and 1-7). Washington, DC: Economic Policy Institute. Retrieved February 19, 2009, from http://www.epi.org/page/-/old/books/teacher_quality_exec_summary.pdf
The Teaching Commission. (2004). Teaching at risk: A call to action. New York: Author. Retrieved February 19, 2009, from http://ftp.ets.org/pub/corp/ttcreport.pdf
Research
Amrein-Beardsley, A. (2008). Methodological concerns about the education value-added assessment system. Educational Researcher, 37(2), 65-75.
Bracey, G. W. (2004). Value-added assessment findings: Poor kids get poor teachers. Phi Delta Kappan, 86, 331-333.
Braun, H. I. (2005). Using student progress to evaluate teachers: A primer on value-added models. Princeton, NJ: Educational Testing Service, Policy Information Center. Retrieved February 19, 2009, from http://www.ets.org/Media/Research/pdf/PICVAM.pdf
Cavalluzzo, L. C. (2004). Is National Board certification an effective signal of teacher quality? (Report No. IPR 11204). Alexandria, VA: The CNA Corporation. Retrieved February 19, 2009, from http://www.nbpts.org/sites/default/files/documents/research/Cavalluzzo_IsNBCAnEffectiveSignalofTeachingQuality.pdf
Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2006). Teacher-student matching and the assessment of teacher effectiveness (NBER Working Paper No. 11936). Cambridge, MA: National Bureau of Economic Research.
Goldhaber, D., & Anthony, E. (2004). Can teacher quality be effectively assessed? Washington, DC: Urban Institute. Retrieved February 21, 2014, from http://www.urban.org/UploadedPDF/410958_NBPTSOutcomes.pdf
Hakel, M. D., Koenig, J. A., & Elliott, S. W. (Eds.). (2008). Assessing accomplished teaching: Advanced-level certification programs. Washington, DC: National Academies Press.
Harris, D. N., & Rutledge, S. A. (2007). Models and predictors of teacher effectiveness: A review of the literature with lessons from (and for) other occupations. Madison, WI: Teacher Quality Research. Retrieved February 19, 2009, from http://www.teacherqualityresearch.org/models.pdf
Harris, D. N., & Sass, T. R. (2008). Teacher training, teacher quality, and student achievement (CALDER Working Paper No. 3). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. Retrieved February 19, 2009, from http://www.caldercenter.org/PDF/1001059_Teacher_Training.pdf
Kane, T. J., Rockoff, J. E., & Staiger, D. O. (2006). What does certification tell us about teacher effectiveness? Evidence from New York City (NBER Working Paper No. W12155). Cambridge, MA: National Bureau of Economic Research. Retrieved February 19, 2009, from http://gseweb.harvard.edu/news/features/kane/nycfellowsmarch2006.pdf
Kupermintz, H. (2002). Teacher effects as a measure of teacher effectiveness: Construct validity considerations in TVAAS (Tennessee Value-Added Assessment System) (CSE Technical Report No. 563). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Lockwood, J. R., McCaffrey, D. F., Hamilton, L. S., Stecher, B. M., Le, V.-N., & Martinez, J. F. (2007). The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, 44(1), 47-67.
McColskey, W., Stronge, J. H., Ward, T. J., Tucker, P. D., Howard, B., Lewis, K., et al. (2007). National Board Certified Teachers and Non-National Board Certified Teachers: Is there a Difference in Teacher Effectiveness and Student Achievement? Journal of Personnel Evaluation in Education, 20(3-4), 185-210.
National Association of State Boards of Education. (2005). Evaluating value-added: Findings and recommendations from the NASBE study group on value-added assessments. Alexandria, VA: Author. [A synopsis appears at http://www.naesp.org/resources/2/Research_Roundup/2006/RR2006v22n4a1.pdf ]
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417-458. Retrieved February 19, 2009, from http://www.nber.org/papers/w6691.pdf?new_window=1
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247-252.
Rothstein, J. (2009). Student sorting and bias in value added estimation: Selection on observables and unobservables (NBER Working Paper No. 14666). Cambridge, MA: National Bureau of Economic Research.
Sanders, W. L., Ashton, J. J., & Wright, S. P. (2005). Comparison of the effects of NBPTS certified teachers with other teachers on the rate of student academic progress. Arlington, VA: National Board for Professional Teaching Standards. Retrieved February 19, 2009, from http://files.eric.ed.gov/fulltext/ED491846.pdf
Stone, J. E. (2002). The value-added achievement gains of NBPTS-certified teachers in Tennessee: A brief report (ECF Education Brief). Arlington, VA: Education Consumers Foundation. Retrieved February 19, 2009, from http://www.education-consumers.org/briefpdfs/2.5-certified_teachers.pdf
Vandevoort, L. G., Amrein-Beardsley, A., & Berliner, D. C. (2004). National Board certified teachers and their students' achievement. Education Policy Analysis Archives, 12(46). Retrieved February 19, 2009, from http://epaa.asu.edu/ojs/article/view/201/327.
White, B. (2004). The relationship between teacher evaluation scores and student achievement: Evidence from Coventry, RI (CPRE-UW Working Paper No. TC-04-04). Madison, WI: Consortium for Policy Research in Education. Retrieved February 19, 2009, from http://cpre.wceruw.org/papers/CoventryAERA04.pdf
Practice
Betebenner, D. (2004). An analysis of school district data using value-added methodology (CSE Technical Report No. 622). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Retrieved February 19, 2009, from http://www.cse.ucla.edu/products/reports/r622.pdf
Coggshall, J. (2007) Communication framework for measuring teacher quality and effectiveness: Bringing coherence to the conversation. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/NCCTQCommFramework.pdf
Goe, L., Bell, C., Little, O. (2008). Approaches to evaluating teacher effectiveness: A research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality. http://www.tqsource.org/publications/EvaluatingTeachEffectiveness.pdf
Gordon, R., Kane, T. J., & Staiger, D. O. (2006). Identifying effective teachers using performance on the job (Hamilton Project Discussion Paper 2006-01). Washington, DC: The Brookings Institution. Retrieved February 19, 2009, from http://www.brookings.edu/~/media/Files/rc/papers/2006/04education_gordon/200604hamilton_1.pdf
Junker, B., Weisberg, Y., Matsumura, L. C., Crosson, A., Wolf, M. K., Levison, A., et al. (2006). Overview of the instructional quality assessment (CSE Technical Report No. 671). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Retrieved February 19, 2009, from http://www.cse.ucla.edu/products/reports/r671.pdf
Matsumura, L. C., & Pascal, J. (2003). Teachers' assignments and student work: Opening a window on classroom practice (CSE Technical Report No. 602). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Retrieved February 19, 2009, from http://www.cse.ucla.edu/products/reports/r602.pdf
McCaffrey, D. F., & Hamilton, L. S. (2007). Value-added assessment in practice: Lessons from the Pennsylvania Value-Added Assessment System Pilot Project. Santa Monica, CA: RAND Corporation. Retrieved February 19, 2009, from http://www.rand.org/pubs/technical_reports/2007/RAND_TR506.sum.pdf
Millett, C. M., Stickler, L. M., Payne, D. G., & Dwyer, C. A. (2007). A culture of evidence: Critical features of assessments for postsecondary student learning. Princeton, NJ: Educational Testing Service. Retrieved February 19, 2009, from http://www.ets.org/Media/Resources_For/Higher_Education/pdf/4418_COEII.pdf
New Mexico 3-Tier Licensure Implementation Teacher Training Work Group. (2005). A handbook on annual evaluation for New Mexico teachers holding a level 1 license. Retrieved February 19, 2009, from http://nctq.org/docs/New_Mexicos_3-Tiered_Licensure_System.pdf
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2006a). Classroom assessment scoring system (CLASS) manual, pre-K. Baltimore: Brookes.
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2006b). Classroom assessment scoring system (CLASS) manual, K-3. Baltimore: Brookes.
Tucker, P. D., & Stronge, J. H. (2005). Linking teacher evaluation and student learning. Alexandria, VA: Association for Supervision and Curriculum Development.
Watson, A., & De Geest, E. (2005). Principled teaching for deep progress: Improving mathematical learning beyond methods and materials. Educational Studies in Mathematics, 58(2), 209-234.
Webster, W. J. (2005). The Dallas school-level accountability model: The marriage of status and value-added approaches. In R. W. Lissitz (Ed.), Value added models in education: Theory and applications (pp. 233-271). Maple Grove, MN: JAM Press.
