Mary Roduta Roberts, PhD, BScOT

Contact

Associate Professor, Faculty of Rehabilitation Medicine - Occupational Therapy

Email: mroberts@ualberta.ca
Phone: (780) 492-7274
Address: 3-48P Corbett (E.A.) Hall
8205 - 114 St NW

Edmonton AB

T6G 2G4

Associate Dean Grad Studies, Faculty of Rehabilitation Medicine - Rehabilitation Medicine

Email: mroberts@ualberta.ca

Overview

Area of Study / Keywords

Education Assessment Applied Measurement Evaluation

About

Credentials

PhD (Measurement, Evaluation, and Cognition), University of Alberta, 2012
BScOT (Occupational Therapy), University of Alberta, 2000

Clinical Experience

Home Care/Home Living Program, Edmonton Zone, Alberta Health Services
Geriatric Psychiatry Admissions Unit, Alberta Hospital Ponoka, Alberta Mental Health Board

Professional Affiliations

Alberta College of Occupational Therapists (ACOT)
Canadian Association of Occupational Therapists (CAOT)
World Federation of Occupational Therapists (WFOT)
American Educational Research Association (AERA), Division D: Measurement and Research Methodology
American Educational Research Association (AERA), Division I: Education in the Professions
National Council on Measurement in Education (NCME)

Current Committee Memberships and Working Groups

Association of Canadian Occupational Therapy University Programs (ACOTUP) Research on Education Committee
ACOTUP National OT Fieldwork Evaluation Tool Development Working Group
ACOTUP Ad Hoc Committee - Canadian OT Simulation Guideline Development
AERA Division D Equity and Inclusion Committee (EIC) - Black Lives Matter Working Group
AERA Division I Committee on Diversity and Equity (CoDE)

Background Information

Dr. Roduta Roberts is a registered Occupational Therapist (OT) in Alberta. She has clinical experience in geriatric psychiatry and community-based case management/clinical experience working with people living with disabilities and complex needs. Dr. Roduta Roberts holds a PhD in Measurement, Evaluation, and Cognition and maintains an active research program focusing on educational assessment and evaluation methods within health professions education. Professionally, she has provided consulting services on quantitative research methods and design, applied measurement, statistical analysis, and the design and reporting of certification test scores. Dr. Roduta Roberts joined the Department of Occupational Therapy as an instructor in August 2011 and began her faculty appointment in July 2013. She served as the Associate Chair of the OT department from 2021-2022 and was appointed Associate Dean - Graduate Studies in July 2022.

Research

Dr. Roduta Roberts draws from diverse academic backgrounds to improve research and practice in health professions education and educational measurement. Her research aims to contribute to better practices in assessment, support and promote evidence-informed decision-making, and demonstrate the potential of assessment for enabling educational and social outcomes. As an education researcher, Dr. Roduta Roberts works to support the development of capable and competent health professionals who continue to be active learners and contribute positively to the lives of their clients and communities. Her research is collaborative, working with instructors, curriculum groups, and other educational and health researchers locally, nationally, and internationally. Dr. Roduta Roberts advises students on projects involving quantitative methods, research design, and applied measurement. Her education research has been funded by the Faculty of Rehabilitation Medicine, University of Alberta's Teaching and Learning Enhancement Fund (TLEF), Social Sciences and Humanities Research Council of Canada (SSHRC), and the Institute of Educational Sciences (IES; US Department of Education).

Areas of research interest include educational assessment, assessment validation, communication of assessment results/score reporting, applied measurement, and diversifying the health professions.

Teaching

Dr. Roduta Roberts has taught and coordinated multiple courses within the occupational therapy curriculum since Fall 2011, including Human Systems (MSK anatomy and common physical health conditions) and practice-oriented seminars within the Integration and Application curriculum theme. Her current teaching area is the assessment and evaluation of occupational performance.

Scholarly Activities

Research - Communication of assessment results

Dr. Roduta Roberts explores educational test score reporting as a social activity and not just the end product of the test development process. For example, she used conversation analysis/discourse analysis to understand the implicit communication afforded by the language choices present in large-scale testing score reports to students and their families.

Currently, with Dr. Gotch at Washington State University, Dr. Roduta Roberts is developing and exploring the use of narrative reports (vignettes) as an adjunct to traditional score reports from the Washington Assessment of Risks and Needs of Students (WARNS). Score reports have the potential to impact educational and social outcomes of students and programs beyond a proximal outcome of improving student test scores and grades.

Funding: US Department of Education - Institute of Educational Sciences (IES)

Research - Pathways and admissions to health professions programs

Health professions programs aim to select applicants who will succeed within the program and beyond as a practicing clinician. Emphasis is often placed on an applicant’s academic performance, typically measured by GPA. However, strong noncognitive or interpersonal skills are required to succeed in health professions programs and clinical practice. Noncognitive skills are commonly assessed during admissions using personal statements, reference letters, and interviews. But, there are limitations with these sources of information, including the potential for bias in data collection, interpretation, and use. Another measure of noncognitive attributes is the situational judgment test (SJT). SJTs, such as Casper, are being increasingly used as part of the admissions process to health professions programs in North America.

Dr. Roduta Roberts is exploring the use of Casper, an online SJT, to inform admissions decisions within an occupational therapy program. Using several analytic techniques, she has conducted a series of studies examining the incremental predictive validity of Casper over GPA and its potential to inform selection decisions that increase the diversity (in terms of demographic characteristics) of admitted applicants.

Funding: Alo Grant and Social Sciences and Humanities Research Council of Canada (SSHRC)

Research - Performance assessment in health professions education

Dr. Roduta Roberts studies the measurement properties of practical (i.e., performance-based) exams assessing professional competencies. Her research has examined OSCE score reliability, associations between OSCE performance and fieldwork, how assessors judge and score observed performance, and the impact of participation in OSCEs on student learning.

Dr. Roduta Roberts' studies have centered on collecting and integrating technical and non-technical sources of validity evidence to support appropriate interpretations and uses of performance assessment in health professions education. For example, she conducted a study using quantitative measures and think-aloud interviews to investigate assessor cognition as a source of score variability from an objective structured clinical examination (OSCE). Studies such as this address the paucity of research on the reliability and validity of performance assessments conducted within realistic operational contexts using nested data collection designs and small sample sizes.

Funding: Faculty of Rehabilitation Medicine New Investigators' Grant

Research - Virtual reality simulations to enhance student performance and well-being

2019-07-01 to 2022-05-30

Immersive and accessible virtual reality (VR) technology is fast becoming mainstream. The use of VR is emerging within a variety of disciplines in higher education ranging from the arts and humanities, to health sciences, to science and engineering. Although numerous studies describe and evaluate specific uses of VR within an educational program, these uses are often standalone events not fully integrated into the curriculum. Dr. Roduta Roberts' project explores the development, integration, and impact of Artificial Intelligence-enhanced VR within a professional education program involving performance-based assessments. Research on the formative use of VR to assist in learning and performance of complex competencies, such as communication, is needed given the potential of VR to improve outcomes for skill development and engagement.

Funding: Social Sciences and Humanities Research Council of Canada (SSHRC)

Featured Publications

Associations between an open-response situational judgment test and performance on OSCEs and fieldwork: Implications for admissions decisions and matriculant diversity in an occupational therapy program

Roduta Roberts, M., Alves, C. B., Chen, F., & St. Jean, C. R.

BMC Medical Education. 2024 October; 24 (1146):1-18 10.1186/s12909-024-06071-0

Identifying factors influencing program selection in health sciences by underrepresented minority students—a scoping review

Mohammed, S. A. S., & Roduta Roberts, M.

Frontiers in Education. 2024 July; 10.3389/feduc.2024.1369230

View additional publications