Reflections on Usability Testing
Reflect on who you were able to have conduct the usability testing? Were you able to get the right people? Why or why not? What can you do to better improve this in the future?
Due to EDLD 5318 occurring in the summer, very few teachers were responsive to emails about reviewing the Professional Learning course. Fortunately, I was able to have at least one teacher from my campus to complete the usability test. I also had a peer from this course complete the usability test, which allows me to test the course with a teacher who is unfamiliar with our campus like our new-hires will be. I believe the course should be able to be usable for teachers who have prior experience in Round Rock as well as those who are new to our campus. Both of these testers were from secondary mathematics backgrounds. In the future I would conduct a usability test on professional learning while my campus is open to get a larger number of participants from a variety of disciplines.
​
What impact did your platform (LMS, Google Docs, or other digital sharing) have on the testing and results?
I chose to create the course using the Google Classroom LMS on my personal account. This prevented teachers from using their school accounts to join. However, this also enabled teachers from outside the district to use the course and give feedback. The usability test submission was done on Google Forms. I had an optional space where users could submit a video of their experience using the course. However, none of my participants were willing to record their experiences. I anticipated this as a potential problem, so I made sure to ask specific questions about the criteria I wanted to measure. These questions included multiple Likert Scales and free responses. By using a variety of questions I was able to measure the criteria I had planned while also allowing my testers to give feedback on areas of their choice. Using a variety of question types during user testing can give instructional designers a better understanding of students’ perspectives (Bellard, 2023). You can view the questions on my Usability Test Google Form.
​
What were the lessons you learned from the usability testing feedback?
This was not my first time conducting a usability test for an online course, as I used them during the 2020-21 academic year with online learning. However, it was my first time doing a usability test for a professional learning course. I had an audience of teachers who are adults with a background in instructional design, so I felt more comfortable asking questions about their perspectives on the alignment of activities. Overall the usability test made me feel more confident in my instructional design decisions for this course.
I was inspired to ask about my participants’ backgrounds by the Usability Test Demo by Steve Krug (2020). The stakeholders I have for this course have a wider variety of backgrounds than my typical student, so I made sure to ask about their prior experience with the LMS and the topic of the course. This gave me a better understanding of my stakeholders’ perspectives. I believe collecting the background information on a larger group of stakeholders would help me identify potential gaps in foundational knowledge that could be addressed in the course.
​
​
What have you done to your design to address the usability issues revealed in the testing? What have you added or taken away?
There were no large usability issues identified during this process. The lack of issues may be attributed to the use of the OSCQR Course Design Scorecard to ensure all information and materials provided were accessible (SUNY, 2021). One user did mention they had a preference for YouTube videos over the Loom video. However, during a follow-up discussion, they agreed that it was beneficial for the students of the course to see the modeling of the variety of screen-recording options teachers can use. Therefore there were no changes made to the infrastructure of the course.
​
How has this process improved your course and your learner's experience
The only issue identified by my testers was the need to improve the Module 1 Quiz on Identifying Language Proficiency Levels. They recommended shifting from descriptions of student performance to examples of student performance as the base for the questions. To remedy this, I will be seeking assistance from the Emergent Bilingual department chair when we return in August to add examples of student performance with each proficiency level.
​
How has the testing impacted your alignment of outcomes activities and assessment?
This process has affirmed that all the materials in the course are accessible and aligned with the course objectives. By using backward design to develop a course the assessments are designed to be aligned to the learning activities. When I developed the questions for the usability test I reflected on what my response would be as a student. This encouraged me to look over the course multiple times to ensure the usability testers would have as seamless of an experience as possible. Using the feedback from the usability test I will be making changes to the Module 1 Quiz to better align with teachers’ experience looking at actual student work rather than descriptions of the work. While the initial assignment was aligned to the course learning goal, the updated version of the quiz will also be aligned with teachers' classroom experiences.
​
How will you address the infrastructure, system, and support needs and issues the learner may face?
My usability testers did not identify any infrastructure issues within the course. However future students may encounter usability issues. Therefore I have included a help form in the Welcome Module. This form enables students to ask questions about both the course content and structure. The form is programmed to notify me about all submissions so that I can promptly address them. I have also included a link to the Google Classroom for Students Help Page if they need support with the LMS. Students can also directly contact the instructor of the course using the contact information from the course outline, course overview video, and on the help page.
References
​
Bellard, Q. (Summer 2023) EDLD 5318 week 7 web conference. [Webinar]. Lamar University. Retrieved from
​
Krug, S. (2020, May 6). Usability test demo. [Video] YouTube. Retrieved from
​
SUNY Online (2021) OSCQR course design review scorecard. [PDF.] Retrieved from
https://onlinelearningconsortium.org/consult/oscqr-course-design-review/