Developing an Evaluation Framework for a Custom-designed Course Management System

Jason T. Abbitt, Ph. D.
Miami University

Abstract

This study evaluated a custom-designed course management system in use at a mid-sized public institution in the Pacific Northwest region of the United States. While focusing primarily on gathering information relevant to the continued development of the course management system, this study sought to identify the relationship among user characteristics, usability factors, and acceptance of the course management system. Additionally, this study sought to identify common usability problems with specific course features encountered by the users. Participants in this study were 308 undergraduate and graduate students who were currently using the online course management system to take an Internet-enhanced or fully Internet-delivered course. The data collected via an online survey instrument indicated that three usability factors, ease of navigation, ease of learning, and visual perception were significant predictors of acceptance of the course management system. The usability of specific course management system features was examined using both Likert-scale items and content analysis of open-ended survey questions. The content analysis of the open-ended items revealed several specific problems that were commonly encountered while using specific course features. Based on these findings, several recommendations are made toward the improvement of the custom-designed course management system and also toward the improvement of the survey instrument for future use.


About the Author(s)...

Jason T. Abbitt is an Assistant Professor of Instructional Design and Technology at Miami University in Oxford, Ohio. Dr. Abbitt’s research interest include the development and evaluation of Internet-based learning systems as well as the investigating factors influencing technology integration into K-12 and higher education teaching practices. He has also been active in several grant-funded initiatives focusing on the professional development of educators in technology integration and Internet-based course design. He may be contacted at This email address is being protected from spambots. You need JavaScript enabled to view it..

Go to top