“How-to” Guide to Creating Online Tests

Send to a Colleague

Text Size

By Dana Fine, Senior Instructional Designer at SyberWorks, Inc.

Whether you are an experienced test writer or have no experience writing online tests, this “How-to” Guide to Creating Online Tests will give you a quick but useful overview of how to maximize the quality of online tests and test questions.

To start, what are the main reasons for giving online tests? Online tests usually follow up on recertifications, just-in-time training, or other courses that present “must know” information to student employees. Typically, online tests are required for courses that have either no instructors or low-levels of instructor participation. Online tests allow the LMS system to track and evaluate student answers to questions, and grade tests, with little or no instructor participation. (More about test question types in a moment.)

Throughout the test creation process, you must keep asking yourself several questions:

“What are the main objectives of this course?” (Very important!)
How hard should the test be?
How do I ensure that students don't cheat?
How many times should students be able to take the test?
How do I know if I've created a good test?

We will look at each question in turn. But, with these always in mind, you will be able to create questions that support the objectives, and you will be well on your way to helping students capture and retain the must-know course information.

What are the main objectives of this course?

A course's design (and its online tests) fall out of its objectives. These usually come in two forms:

What knowledge must students acquire in the course?
Without a clear, well-thought-out outline of a course's important topics and subtopics, a course can quickly enter detours that confuse students or dilute the training message. Depending on the complexity of the main course subject, it should be divided into the most important topics that students must learn, and these topics must similarly be divided into the most important sub-topics that students must learn. This outline usually translates directly into the course's lesson structure and its online test questions.
What must students learn to do with this knowledge?
Is it enough that they only learn facts, or must they also learn specific operations and functions that depend on their new knowledge? Online tests can help to reinforce both.

How hard should the test be?

Many resources describe philosophies of testing, but in general, online tests should be easy, as long as they capture all of the important concepts and key terms in the course. Often, it is practical to offer at least a few “gimmies” to start off the test, which will increase a student employee's confidence in their ability to complete the test successfully.

In terms of setting a passing grade, there are also varying philosophies, depending on the industry, the company, and the required competency. Sometimes, a 70% passing rate is fine. An example of this might be in a course entitled The Complete Sales Method, which may have a lot of information, with the practical application being much more individual. However, a company that requires its employees to take Work Security and Procedures might require a 100% grade to pass, especially if there are key terms and phone numbers to remember.

How do I ensure that students don't cheat?

Critics of online tests have long complained that getting honest test results is difficult, even for mandatory adult training. Commercially available training systems may employ two methods to limit the possibility of cheating: randomization and question banks. Test question randomization has become a common feature in many LMS and test-creation systems. Randomized test generators place questions in a different order each time the test is generated or brought up in a web browser. For example, in a 10-question test, question number 4 in the first instance of the test might become question number 7 in the second instance.

While randomization limits exact copying, it only slows down the determined cheater because the overall set of questions remains essentially the same. However, a second technique (question banks) can make it much harder to cheat. Every time a new instance of a test is created, the test question generator selects one question from each bank for the test. For example, say that a test contains 10 question slots, and a “bank” of 4 different but related questions has been created for each slot. Simple algebra tells us that there are 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 = 1,048,576 different randomly generated tests that any one student might receive. This makes it extremely difficult for students to copy (or even anticipate) the questions that will appear on their tests.

How many times should students be able to take the test?

The answer to this question also depends on the purpose of the test. As a creator, do you have a particularly stringent passing grade, such as 100%? In this case, you might want to allow the student employee to take the test as many times as possible, so that they can retain the most information. Other test creators believe that a student employee should only be allowed three tries to take a test. This may make the student focus more on course content than if they had unlimited chances. But remember that there is no right or wrong answer. The bottom line is always that the student actually understand and retain the course information!

How do I know if I've created a good test?

Entire books and websites are dedicated to the art and science of writing test questions. While a more thorough reading and exploration is encouraged, the basics are:

Write clear and logical questions and answers.
Use terminology and grammar that your test population will understand.
Make sure that there is only one way to interpret and answer each question.
Be brief and to-the-point in both questions and answers.
Limit multiple-choice answer selections to at most 4 or 5 options (more might overwhelm students).

Do these things and it is likely that you are on the right track!

You may also be able to apply some quantitatively measures to your test design, including:

Average times needed to take the test.
Student success in answering each question.
Students' overall test status.

Now, let's look more specifically at the types of questions that may be used in online tests.

Common Question Types

It is wise to use a variety of question types in online tests. Types that may be available to you include Multiple Correct, Single Correct, Graphical Choice, True/False, Fill-In-the-Blank, Item Match, and Drag-and-Drop.

A Multiple Correct Question contains several correct responses among its answer choices. These questions are best used when the test creator wants to emphasize a point. For example, if a semiconductor producer wants to emphasize why a particular processor is better than its competitors, two potential correct answers might be “cheaper” and “faster.” As mentioned previously, one should offer only 4 or 5 answer options. And be sure that the question itself states that it is Multiple Select! A common way of doing this is: “...Select all answers that apply.” It is also a great idea to state the number of correct answers (such as: “...Select the 3 answers that are correct.”). Remember, the goal is not to trick students; we want them to understand and retain the information presented.
A Single Correct Question gives the student a set of choices, only one of which is correct. A good Single Correct answer should stand out among the options. However, keep the other (incorrect) answers close in subject or nature to the correct answer, to help ensure that the student actually grasped the course content well enough to recognize the correct answer.
Graphical Questions are designed to help students recognize something visually. Students answer by clicking on hotspots associated with an image. These questions are particularly useful for reinforcing knowledge where color is associated with real-world applications or processes. For example, many companies have security codes, wherein different colors mean different security threats and responses. In another example, chemical companies may require that students understand the meaning and use of different colored solutions in chemical processes. Graphical questions also provide a good “change of pace” that engages the test taker's thoughts in different (and more interesting) ways.
A True/False Question is a statement that is either true or false, and the student must respond correctly. True/False questions are great “gimmie” questions, and can also help to ensure that student employees know the correct answers in either/or scenarios.
A Fill-In-the-Blank Question offers a question or statement to which the student responds by entering a small amount of text. These questions are great for testing student retention of specific terminology. Fill-In-the-Blank questions should be very focused and short, to emphasize the topic being tested, and to keep the student from getting confused.
An Item Match Question presents two columns of text, with the items in the second column numbered sequentially. For each item in the first column, the student is asked to enter the number of the single correct “matching” item in the second column. When creating Item Match questions, be careful about the order of the items in the columns. Especially, be sure that the columns the student sees do not place items in the left column beside their correct answers in the right column. Item Match Questions are also a great way to help students remember terms and definitions. Again, a student can easily become overwhelmed by a large Item Match question, so limit the item pairs to at most 6.
A Drag-and-Drop Question is a form of Item Match question, but the student drags item to the “correct” items they match. This provides variety for the student and helps to hold their attention and interest.

Conclusion

We hope that this “How-to” Guide to Creating Online Tests has been informative but not overwhelming. By following the advice in this guide, you will be writing effective online tests in no time!

About the Author:

Dana Fine is a Senior Instructional Designer at SyberWorks, Inc. SyberWorks is a custom e-Learning solutions company that specializes in Learning Management Systems, e-Learning solutions, and custom online course development. Dana is also a frequent contributor to the Online Training Content Journal

About SyberWorks

SyberWorks, Inc. is a leader in the custom e-Learning Solutions and Learning Management System industries for Fortune 1000 corporations, higher education, and other organizations. Located in Waltham, Massachusetts, the company serves the multi-billion-dollar e-Learning market. Since 1995, SyberWorks has developed and delivered unique and economical solutions to create, manage, measure, and improve e-Learning programs at companies and organizations in the United States, Canada, Europe, and other countries.

The Online Training Content Journal

The Online Training Content Journal blog looks at best practices, techniques, and trends in online training development and e-Learning instructional design.