[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-602343-105":3,"detail-sidebar-cat-0-en-105":80,"doc-detail-602343-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":73,"head_meta":75,"extra_data":77,"updated_unix":79},105,"en","should-teacher-observation-systems-be-used-for-making-high-stakes-decisions-602343","Should Teacher Observation Systems be Used for Making High-Stakes Decisions","","This study questions the suitability of teaching observation data for making high-stakes decisions that affect teachers. The concept of suitability is defined as advancing intended purposes without causing undue harm, and it is argued to depend on technical properties built into observation-system data. An experiment compared three systems with different attributes, including rubrics with varying inference load, assessed by trained, experienced raters using lesson videos.",{"@graph":14,"@context":72},[15,34,55],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/should-teacher-observation-systems-be-used-for-making-high-stakes-decisions-602343/602343/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/should-teacher-observation-systems-be-used-for-making-high-stakes-decisions-602343/602343.png","ImageObject",300,407,{"name":42,"@type":43},"dawugda","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-30","2026-10-15",true,{"@type":52,"interactionType":53,"userInteractionCount":30},"InteractionCounter",{"@type":54},"ViewAction",{"@type":56,"mainEntity":57},"FAQPage",[58,64,68],{"name":59,"@type":60,"acceptedAnswer":61},"How does the study define suitability for high-stakes decisions?","Question",{"text":62,"@type":63},"Suitability is defined as the extent to which intended purposes are advanced without causing undue harm.","Answer",{"name":65,"@type":60,"acceptedAnswer":66},"What was compared in the experiment?",{"text":67,"@type":63},"Three teaching observation systems with different attributes, including rubrics that impose varying inference load on raters.",{"name":69,"@type":60,"acceptedAnswer":70},"Which criteria were used to judge the resulting data?",{"text":71,"@type":63},"The study assessed predictive power for teachers’ contribution to student learning, score correlations across systems, and rater agreement within each system.","https://schema.org",{"og:url":32,"og:type":74,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":76,"canonical":32},"index,follow",{"doc_id":78,"site_id":7},602343,1792437496,{"code":4,"msg":81,"data":82},"success",[83,87,91,95,100,105,110,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":84,"show_sort_weight":85,"slug":86},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":88,"show_sort_weight":89,"slug":90},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":92,"show_sort_weight":93,"slug":94},"Exam",70,"exam",{"id":96,"doc_module":4,"doc_module_name":25,"category_name":97,"show_sort_weight":98,"slug":99},5,"Comic",60,"comic",{"id":101,"doc_module":4,"doc_module_name":25,"category_name":102,"show_sort_weight":103,"slug":104},6,"Technology",50,"technology",{"id":106,"doc_module":4,"doc_module_name":25,"category_name":107,"show_sort_weight":108,"slug":109},7,"Healthcare",40,"healthcare",{"id":111,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},8,30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":96,"slug":129},19,"General","general",{"code":4,"msg":81,"data":131},{"doc_id":78,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":111,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":30,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145},3985747870947,"https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2","| education policy analysis archives\u003Cbr>A peer-reviewed, independent, open access, multilingual journal | |\n| --- | --- |\n| Arizona State University |  |\n| Volume 34 Number 33 April 7, 2026 ISSN 1068-2341 |  |\n\nShould Teacher Observation Systems be Used for Making  \nHigh-Stakes Decisions?  \nMichael Strong  \nJaehoon Lee  \nTexas Tech University  \nJohn Gargani  \nGargani + Co.  \nMinju Yi  \nHyunjin Shim  \nTexas Tech University  \n&  \nHyunchang (Henry) Moon  \nAugusta University  \nUnited States  \nCitation: Strong, M., Lee, J., Gargani, J., Yi, M., Shim, H., & Moon, H. (2026) . Should teacher observation systems be used for making high-stakes decisions? Education Policy Analysis Archives, 34(33). [https://doi.org/10.14507/epaa.34.9841](https://doi.org/10.14507/epaa.34.9841)  \nAbstract: This study questions the suitability of teaching observation data for making high-stakes decisions that affect teachers. We define suitability (the extent to which intended purposes are advanced without causing undue harm) and argue that it fundamentally depends on the technical properties of the data produced by an observation system, which, in turn, depend on the attributes  \ndesigned into the system. We conducted an experiment to understand better the relationship between the attributes of teaching observation systems and the suitability of their data. We compared three systems with different attributes, including rubrics that impose varying inference loads on raters. Experienced raters were randomly assigned to a system and properly trained. Then, they evaluated the instruction of advanced teacher candidates by viewing videos of their lessons. We considered three criteria when judging the resulting data: the power to predict a teacher’s contribution to student learning, the correlation of scores across systems, and rater agreement within systems. We found that a system with a low inference load (along with other attributes) outperformed systems with higher inference loads, but it may still be insufficient for making confident, high-stakes decisions. We maintain that few, if any, widely used observation systems are.  \nKeywords: teacher observation; high-stakes decisions; inference  \n¿Deben utilizarse los sistemas de observación docente para la toma de decisiones dealto impacto?  \nResumen: Este estudio cuestiona la idoneidad de los datos provenientes de la observación dela enseñanza para la toma de decisiones de alto impacto que afectan al profesorado. Definimosla idoneidad como el grado en que los propósitos previstos se logran sin causar dañosindebidos y sostenemos que esta depende fundamentalmente de las propiedades técnicas delos datos producidos por un sistema de observación, las cuales, a su vez, dependen de los atributos diseñados en dicho sistema. Realizamos un experimento para comprender mejor larelación entre los atributos de los sistemas de observación docente y la idoneidad de los datos que generan. Comparamos tres sistemas con atributos diferentes, incluidos instrumentos derúbricas que imponen distintas cargas inferenciales a los evaluadores. Evaluadores conexperiencia fueron asignados aleatoriamente a un sistema y recibieron capacitación adecuada. Posteriormente, evaluaron la enseñanza de candidatos avanzados a docentes mediante la visualización de videos de sus clases. Consideramos tres criterios para valorar los datos resultantes: la capacidad predictiva de la contribución del docente al aprendizaje estudiantil, la correlación de las puntuaciones entre sistemas y el grado de acuerdo entre evaluadores dentrode cada sistema. Encontramos que un sistema con baja carga inferencial (junto con otrosatributos) superó a los sistemas con mayores cargas inferenciales; sin embargo, aun así podríaresultar insuficiente para tomar decisiones de alto impacto con plena confianza. Sostenemosque pocos, si es que alguno, de los sistemas de observación ampliamente utilizados cumplen con este estándar.  \nPalabras-clave: observación docente; decisiones de alt","cbCaiaUSA1o9FULN","https://ap.wps.com/l/cbCaiaUSA1o9FULN","pdf",484175,21,"English","# Abstract\n## Suitability and technical properties\n## Experiment design and systems compared\n## Criteria for judging results\n## Findings and implications","[{\"question\":\"How does the study define suitability for high-stakes decisions?\",\"answer\":\"Suitability is defined as the extent to which intended purposes are advanced without causing undue harm.\"},{\"question\":\"What was compared in the experiment?\",\"answer\":\"Three teaching observation systems with different attributes, including rubrics that impose varying inference load on raters.\"},{\"question\":\"Which criteria were used to judge the resulting data?\",\"answer\":\"The study assessed predictive power for teachers’ contribution to student learning, score correlations across systems, and rater agreement within each system.\"}]","Should Teacher Observation Systems be Used for Making High-Stakes Decisions | PDF",1792094505,53]