Mary Roduta Roberts, PhD, BScOT

Contact

Associate Professor, Faculty of Rehabilitation Medicine - Occupational Therapy

Email: mroberts@ualberta.ca
Phone: (780) 492-7274
Address: 3-48P Corbett (E.A.) Hall
8205 - 114 St NW

Edmonton AB

T6G 2G4

Associate Dean Grad Studies, Faculty of Rehabilitation Medicine - Rehabilitation Medicine

Email: mroberts@ualberta.ca

Overview

Area of Study / Keywords

Education Assessment Applied Measurement Evaluation

About

Credentials

PhD (Measurement, Evaluation, and Cognition), University of Alberta, 2012
BScOT (Occupational Therapy), University of Alberta, 2000

Clinical Experience

Home Care/Home Living Program, Edmonton Zone, Alberta Health Services
Geriatric Psychiatry Admissions Unit, Alberta Hospital Ponoka, Alberta Mental Health Board

Professional Licensure and Affiliations

Alberta College of Occupational Therapists (ACOT)
Canadian Association of Occupational Therapists (CAOT)
World Federation of Occupational Therapists (WFOT)
American Educational Research Association (AERA), Division D: Measurement and Research Methodology
American Educational Research Association (AERA), Division I: Education in the Professions
National Council on Measurement in Education (NCME)

Current Committee Memberships and Working Groups

Association of Canadian Occupational Therapy University Programs (ACOTUP) Research on Education Committee
ACOTUP National OT Fieldwork Evaluation Tool Development Working Group
ACOTUP Ad Hoc Committee - Canadian OT Simulation Guideline Development
AERA Division D Equity and Inclusion Committee (EIC) - Black Lives Matter Working Group
AERA Division I Committee on Diversity and Equity (CoDE)

Background Information

Dr. Roduta Roberts is a registered Occupational Therapist (OT) in Alberta. She has clinical experience in geriatric psychiatry and community-based case management/clinical experience working with people living with disabilities and complex needs. Dr. Roduta Roberts earned a PhD in Measurement, Evaluation, and Cognition and maintains an active research program focusing on educational assessment methods primarily within health professions education. Professionally, she has provided consulting services on quantitative research methods and design, applied measurement, statistical analysis, and the design and reporting of professional certification test scores. Dr. Roduta Roberts joined the Department of Occupational Therapy as an instructor in August 2011 and began her faculty appointment in July 2013. She served as the Associate Chair of the occupational therapy department from 2021-2022 and was appointed Associate Dean, Graduate Studies in July 2022.

Research

Dr. Roduta Roberts draws from her diverse academic backgrounds to improve research and practice in health professions education and educational measurement. Her research aims to contribute to better assessment practices, support and promote evidence-informed decision-making, and demonstrate the potential of assessment for enabling positive educational and social outcomes. As an education researcher, Dr. Roduta Roberts works to support the development of capable and competent health professionals who continue to be active learners and contribute positively to the lives of their clients and communities. Her research is collaborative, working with instructors, curriculum groups, and other educational and health researchers locally, nationally, and internationally. Dr. Roduta Roberts advises students on projects involving quantitative methods, research design, and applied measurement. Her education research has been funded by the Faculty of Rehabilitation Medicine, University of Alberta's Teaching and Learning Enhancement Fund (TLEF), Social Sciences and Humanities Research Council of Canada (SSHRC), and the Institute of Educational Sciences (IES; US Department of Education).

Areas of research interest include educational assessment, assessment validation, communication of assessment results/score reporting, applied measurement, and diversifying the health professions.

Teaching

Dr. Roduta Roberts has taught and coordinated multiple courses within the occupational therapy curriculum since Fall 2011, including Human Systems (musculoskeletal (MSK) anatomy and common physical health conditions) and practice-oriented seminars within the Integration and Application curriculum theme. Her current teaching area is the assessment and evaluation of occupational performance.

Scholarly Activities

Research - Assessment of practice-based competencies in health professions education

Performance-based assessments are a critical assessment approach in health professions education. Structured opportunities for the demonstration of competencies required for practice may be provided through simulation or practical/objective structured clinical examinations (OSCEs). Performance-based assessments sample and reflect valued disciplinary tasks that students will likely experience in fieldwork and in clinical practice.

Dr. Roduta Roberts' studies have centered on gathering and integrating technical and non-technical sources of validity evidence to support appropriate interpretations and uses of performance assessment in health professions education. For example, in the context of summative assessment, she conducted a study using quantitative measures and think-aloud interviews to investigate assessor cognition as a source of score variability from an objective structured clinical examination (OSCE). Studies such as this address the paucity of research on the reliability and validity of performance assessments conducted within realistic operational contexts using nested data collection designs and small sample sizes. Her research has also examined associations between OSCE performance and fieldwork, how assessors evaluate and score observed performance, and the impact of participation in OSCEs on student learning.

Funding: Faculty of Rehabilitation Medicine New Investigators' Grant

Research - Communication of assessment results

Dr. Roduta Roberts explores educational test score reporting as a social activity and not just the end product of the test development process. For example, she used conversation analysis/discourse analysis to understand the implicit communication afforded by the language choices present in large-scale testing score reports to students and their families. Reframing score reporting as a social activity (i.e., communication of assessment results) may fruitfully inform the study of use and consequences, which is an understudied area of scholarship in educational measurement relative to other, more technical domains.

Currently, with Dr. Gotch at Washington State University Pullman, Dr. Roduta Roberts is developing and exploring the use of narratives or vignettes as an adjunct to traditional score reports from the Washington Assessment of Risks and Needs of Students (WARNS). While current reports help identify heightened areas of risk and need, they present an atomized view of the student. Vignettes reassemble students ’responses to the WARNS into a coherent picture of how profiles of risk and need manifest in everyday settings. The intent is for counselors and students to explore vignette content that resonates with students’ lived experiences, thereby supporting more effective intervention planning.

Funding: US Department of Education - Institute of Educational Sciences (IES)

Research - Pathways and admissions to health professions programs

Health professions programs aim to select applicants who will succeed within the program and beyond as a practicing clinician. Emphasis is often placed on an applicant’s academic performance, typically measured by GPA. However, strong noncognitive or interpersonal skills are required to succeed in health professions programs and clinical practice. Noncognitive skills are commonly assessed during admissions using personal statements, reference letters, and interviews. But, there are limitations with these sources of information, including the potential for bias in data collection, interpretation, and use. Another measure of noncognitive attributes is the situational judgment test (SJT). SJTs, such as Casper, are being increasingly used as part of the admissions process to health professions programs in North America.

Dr. Roduta Roberts is exploring the use of Casper, an online SJT, to inform selection decisions within an occupational therapy program. She is also exploring the potential of pathway programs and their integration with admissions practices in occupational therapy from the perspective of aspiring health professions students with diverse lived experiences. Early studies have consistently demonstrated that Casper is a better predictor than GPA of future performance in fieldwork, comprising activities closely aligned with professional practice. Her recent study demonstrated that assigning greater weight to Casper versus GPA when making selection decisions optimized the prediction of performance in fieldwork and led to small changes in the diversity, or background characteristics, of a selected cohort. Together, these studies suggest that the historic emphasis on GPA is an insufficient metric and may pose an unnecessary barrier to students. Admissions strategies such as assigning optimal weights to measures of achievement and non-cognitive skill alone cannot diversify the applicant pool if diverse students are not making applications in the first place.

Funding: Social Sciences and Humanities Research Council of Canada (SSHRC) and the Alo Grant

Research - Virtual reality simulations to enhance student performance and well-being

2019-07-01 to 2022-05-30

Immersive and accessible virtual reality (VR) technology is fast becoming mainstream. The use of VR is emerging within a variety of disciplines in higher education ranging from the arts and humanities, to health sciences, to science and engineering. Although numerous studies describe and evaluate specific uses of VR within an educational program, these uses are often standalone events not fully integrated into the curriculum. Dr. Roduta Roberts' project explores the development, integration, and impact of Artificial Intelligence-enhanced VR within a professional education program involving performance-based assessments. Research on the formative use of VR to assist in learning and performance of complex competencies, such as communication, is needed given the potential of VR to improve outcomes for skill development and engagement.

Funding: Social Sciences and Humanities Research Council of Canada (SSHRC)

Featured Publications

Post-COVID-19 Pandemic Restrictions: Follow-Up on Changes Within Canadian Academic and Fieldwork Curricula

MacKenzie, D., Roduta Roberts, M., Martini, R., Ausman, C., & Schmitz, C.

Canadian Journal of Occupational Therapy. 2025 January; 10.1177/00084174241310076

Associations between an open-response situational judgment test and performance on OSCEs and fieldwork: Implications for admissions decisions and matriculant diversity in an occupational therapy program

Roduta Roberts, M., Alves, C. B., Chen, F., & St. Jean, C. R.

BMC Medical Education. 2024 October; 24 (1146):1-18 10.1186/s12909-024-06071-0

Identifying factors influencing program selection in health sciences by underrepresented minority students—a scoping review

Mohammed, S. A. S., & Roduta Roberts, M.

Frontiers in Education. 2024 July; 10.3389/feduc.2024.1369230

View additional publications