Table 1.

The 36 items from the final Medical E-Learning Evaluation Survey with a short explanation of their meaning from the creators’ points of view.

Domain and original item		Short explanation
Motivators
	I felt this module was important.	Creating a feeling of importance is very important for the user. The challenge is to convey to your users that the learning aims are important for their work and personal development.
	I felt it was my responsibility to undertake this module.	Along with importance, your user needs to feel responsible for the learning aim as well. This can be done by emphasis on the importance, but also by, for example, rewarding or giving responsibility for an outcome.
	I had enough time to complete the module.	Proving time to do the module seems contradictory to “anytime, anywhere” learning, but it does give the learning the feeling of priority from a management level.
	I had a good understanding of the general purpose of the module.	The general purpose is the learning aim: knowledge, skills, or attitude/behavior. It should be very clear to the user what they gain from finishing the module.
	The e-learning objectives (for each educational section) were clear to me.	When a module is separated into different sections/chapters, make sure you communicate what the learning objectives are for each section.
	There was a clear overview of all content.	Providing an overview of all lessons, objectives, and options gives the user the possibility to manage expectations and, if possible, create their own learning process.
	I knew how to navigate to the content.	Navigation is an important part of the user interface and should be very clear for the user so they can find content easily and go back and forward through the content.
	I felt comfortable with the quality and truthfulness of the content.	Trust is important when learning. If the user has doubts about the truthfulness or quality, it will limit the working memory used for learning. Trust can be gained by the transparency of the creators, referring to recent literature, etc.
	I was able to undertake this module without being forced.	Forcing a user to undertake a module is the opposite of motivating them. If force or even blackmail is needed, the user will feel resentment, which kills motivation.
	I felt taken seriously as an adult learner.	Taking the learning seriously means avoiding childish illustrations or examples and aiming at the level of experience means that you take into account what the user already knows to prevent repetition of basic knowledge.
	The module was aimed at my level of experience.	Making your module too easy will decrease motivation, and making it too complicated will make users learn less. This is why knowing the background knowledge of the target audience is of great importance.
Barriers
	I was not able to create my own learning path to my own needs.	This questions the difference between synchronized and asynchronized learning paths. Creating your own learning path means the option to test and skip already known sections or to go from A to C and then to B.
	The module was not easily accessible at my location or with my device.	Accessing the module should ideally be possible from every device and location, so consider, for example, internet speeds in foreign countries. If access is not possible, consider helping your users get the right device.
	The navigation did not make sense to me.	Good navigation is helpful but poor navigation will not only limit a module but make it impossible to finish. Make sure your users can follow all steps without using their cognitive load for navigation.
	The layout of the module was too complicated.	Navigation and layout are both important aspects of the user interface. The less cognitive energy is used for the learning environment, the more can be used for the learning itself.
	There was no instrument to help me navigate the module (eg, a sitemap).	Even if the navigation is of a high standard, it is still very helpful to have an instrument that gives an overview of all content and helps direct users where they want to be.
	I had concerns about the security and safety of the module regarding my personal information.	Worries about security and privacy are relevant in many countries and may even have a legal aspect.
	The module was slow and took too long to load.	Fast and logical use of the module is also an important aspect of the user experience. Waiting on affordances or loading frustrates and distracts and should be minimized.
	I did not know which devices the module was compatible with, and I might have used the wrong one.	If your module has specific needs (eg, a specific operating system such as iOS) you need to clearly state that at the beginning. Try to prevent users from experiencing your module in a wholly different way than planned because they use the wrong device.
	The module was too long.	The duration should have been specified. Duration of videos, sections, and the module overall are taken together as one item. If there are, for example, longer videos, their duration can be added as a separate item.
	The module did not divide the content into proper sections.	Learning and memory theories suggest that learning has a limited time span. Sectioning or chunking is a very effective way to help users through a bigger module.
Learning enhancers
	I could personalize the module (eg, by saving and continuing, filling out questionnaires, and getting my personal score).	Personalizing a learning experience allows the user to know how they are doing and follow a preferred method and path. The more personal and specific such things as feedback are, the more the user will gain. This is a very important motivator as well.
	I could create my own learning path and was not forced to follow the directed path (eg, by skipping parts or returning to previous sections if needed).	This questioned the difference between synchronized and asynchronized learning paths. Creating your own learning path means the option to test and skip already known sections or to go from A to C and then to B.
	I had an idea of the progress I had made and what was left to do (eg, by a progress bar).	When learning, it’s important to manage expectations. Knowing what is already done and what is left to do is an important affordance of, for example, a book, and should preferably be available in a module as well.
	I had access to technical support if needed.	To minimize the effort spent on technical aspects rather than learning, providing support as fast as possible will prevent users from stopping learning.
	The module provided summaries where needed.	Learning theory suggests that summaries support learning by offering repetition of content in a new format and allowing chunking of the bigger picture.
	The module provided feedback on my answers.	Learning theory also suggests that learning is more effective when based on previous experience and knowledge, and providing feedback helps the user to make connections between new knowledge and their mistaken or correct assumptions.
	There were exercises and/or assignments in the module.	Learning theory suggests that actively using new knowledge will help it to go from working memory to long-term memory. Therefore, exercises or assignments help the transfer of the learning aim to long term memory.
	I could interact with the content of the module (eg, questions, exercises, or other interactivities).	Interaction is another example of actively using the content, helping users learn more efficiently.
Learning discouragers
	I got stressed or frustrated by the module for whatever reason.	Stress can be caused by many things but will always distract from learning. Stress can come from failing hardware, deadlines, the consequences of failing, etc.
	The content was not able to adapt to my device when needed (eg, the module should work on a mobile device, but the icons were much too small for that).	Nonadaptable content can cause frustration and degrade the user experience, again moving energy away from learning and toward technical aspects.
	The e-learning design and visuals were too distracting for me.	Multimedia learning provides a theory and guidelines for how to use the combination of visuals and auditory stimuli effectively. Distraction should always be prevented.
Real-world translators
	The e-learning content and examples are translatable to my daily real-world work.	Adult learning theory suggests that adults prefer learning in a professional environment, if they can use the lessons learned in daily practice. Providing content and examples that are relatable will help.
	The module seems up to date and properly maintained.	When a user thinks they are learning old material, it might not seem applicable to their daily work anymore. This will kill motivation and minimize the effort the user is willing to put in.
	The module provided sources for the information that were also accessible after finishing it.	Health care professionals in particular might want to undertake further reading in a relevant topic or refresh their memory after finishing the module. Providing this and letting the user know this is possible will increase motivation.
	Besides this questionnaire, the module was evaluated on topics like user experience, effectiveness, usability, and/or costs.	The literature suggests that evaluation is an important step. This question is an oxymoron because by asking it, you are already evaluating. Therefore, the question is: Are OTHER evaluation instruments ADDED to this evaluation—for example, focus group discussions?