Review and Validation of the Benchmarking of Technology-Enabled Learning

Abstract

The benchmarking toolkit for Technology-Enhanced Learning (TEL) developed by the Commonwealth of Learning (COL) is designed to assess TEL practices in higher education institutions. This study evaluated the content validity, internal consistency, and inter-domain relationships of the toolkit using a survey of 355 practitioners across 21 institutions, yielding 90 usable responses. The survey also included open-ended questions for each domain and also for general feedback. Content validity was assessed with the Content Validity Index (CVI), which showed an overall score of 0.83, indicating high validity. Further, internal consistency was examined using Cronbach’s α, average inter-item correlation, and signal-to-noise ratio, with Cronbach’s α values ranging from 0.85 to 0.93 and inter-item correlations between 0.60 and 0.76. Domain-level mean scores were calculated, and inter-domain Pearson correlations revealed moderate to strong relationships (0.44 to 0.79), suggesting that while domains are related, they measure distinct aspects of TEL implementation. Institutions receiving structured support from COL generally achieved higher TEL scores, highlighting the positive impact of TEL practices. In addition, qualitative analysis of the open-ended questions showed that the benchmarking toolkit was useful in identifying gaps and work on those gaps using an action plan to support digital transformation in higher education institutions. Overall, the findings demonstrate that the toolkit is a reliable and valid instrument for assessing TEL practices, providing a strong foundation for institutional improvement and further validation in larger samples.