HumRRO has conducted validity and reliability studies for the testing programs we serve.

Studies in this area include alignment studies, DIF, standard setting, and estimating different types of reliability. In addition, to test reliability issues, HumRRO has also conducted validity studies that examine relationships between test scores. In addition, we have conducted convergent validity studies with Pennsylvania’s assessment and college placement exams. Currently, we are conducting research on College Career Preparedness for the National Assessment Governing Board, on the effect of omitting NAEP items for the National Center on Education Statistics.

Alignment Studies verify links among test items, test coverage, and achievement expectations or between curriculum and instruction. HumRRO has developed criteria for use with static test forms and with computer adaptive tests. We conduct alignment studies for many states and are currently developing the alignment plan for the Smarter Balanced Assessment Consortia which was conducted in the spring of 2014.

HumRRO’s expertise in student classification accuracy can help policymakers understand that important decisions, such as passing a high school exit exam, can be adjusted to reduce damaging errors. We conduct analyses that estimate the probability of students’ true scores being in the same performance category as a set of observed scores for a set of test results. These estimates will result in better and more informed evaluations of standards-based tests and their implications for classifying students into performance categories.

HumRRO has extensive experience in setting standards for numerous high-stakes exams both in the education and credentialing arenas. We have used a variety of methods including the modified Angoff technique and both the Body of Work and Bookmark methods. In addition, we have served as third-party observers for standard setting workshops. In addition, since Performance Level Descriptors (PLD) are an important aspect of the standard setting, we reviewed state PLDs to be able to provide guidance to states and to assist with consistent interpretation of PLD levels.