Which question type should I use?

TestGorilla has several question types you can use in your tests. In this article, we explain when it's best to use each type.

There are four question types available to testpreneurs. These allow for automated scoring since you provide the correct answer(s). The question types are:

  1. Multiple-choice
  2. Multiple-response
  3. True or false
  4. Short text (requiring an exact match)

Our customers have a few additional types available to them when adding their own custom questions: video response, file upload, and essay. The answers to these questions need to be scored manually by the customers themselves.

In this article, we'll only focus on the four question types available to testpreneurs.

Important: Each test-taker receives a subset of your total question set. The questions are delivered in random order. For that reason, all questions must work on a stand-alone basis.

Question type 1: Multiple-choice

In a multiple-choice question, a test-taker has to choose one right answer from a list of choices (often four). If designed well, multiple-choice questions have very good psychometric properties, meaning they can clearly distinguish good candidates from poor ones. This makes them the best question type for your tests in general. The majority of your tests should be composed of multiple-choice questions.

Here's a simple example of a multiple-choice question. If "Brasilia" is selected as the answer below, 1 point is awarded.

What is the capital of Brazil?

 Sao Paolo

Brasilia

 Rio de Janeiro

 Salvador

 

In the stem (the question itself), you can:

  • Use rich text (bold, italics, underlined)
  • Use bullets and numbering
  • Insert a picture (e.g. diagram, graph, figure)
  • Insert a math formula (LaTeX)
  • Insert a video (not URL), 30 seconds maximum
  • Insert audio (not URL), 30 seconds maximum

For each question, you can provide 2 to 10 answer options. By default, the answer options are shuffled when presented to test-takers. You can disable the shuffle if there is a logical or fixed order to these options.

For example, if your answer options are all numbers, you may want to improve readability by ordering them from smallest to largest. Or, if you have duplicate answer sets across multiple questions, you may want to have them appear in the same order every time, making it easier for candidates to select their answer.

If there is only one correct answer for your question, assign it 1 point.

If you have answer options that are correct to some extent, you can assign 0-5 points to each one. This is helpful for "situational judgment" tests, in which the answers are not necessarily fully right or fully wrong. Instead, one answer may be better than the other. Each answer will be normalized by our system to 0-1 points, so all questions will have equal weight.

For example: if a partially correct answer is worth 3 points within your question, it will receive a score of 0.6 in the final score, since 3 out of 5 possible points is equal to 60% of the total.

Below is an example of a situational judgment question that awards different points based on the selected answer.

It’s Monday morning, and you’ve just returned to the office after a week’s vacation. The colleague that covered for you while you were out hasn’t sent you a handover message. You notice that the executive you work for has an unusually busy week ahead with several overbookings and conflicting meetings.

How are you going to make this week run smoothly for the executive?

0 points: You call up your colleague and ask her to fix the mess she created. If you fix it yourself, she won't learn from the situation and you can't let that happen.

1 point: You call the executive on Monday morning to align on his priorities and trust your ability to make last-minute changes throughout the week as needed.

5 points: You call your colleague to ask if she has already discussed priorities with the executive and note what those are. You adjust the plan for the week accordingly before sharing it with the executive.

3 points: You email a proposal of changes to the executive on Monday morning and send him a Whatsapp message to raise awareness of the situation since you feel it’s critical to get his input ASAP.

    Question type 2: Multiple-response

    Multiple-response questions offer a creative approach for testing certain skills. However, it can be more challenging for you to come up with the right balance of correct answers and wrong ones (distractor options) in your choice list. These questions are often less reliable than standard multiple-choice questions and should be used sparingly, though we do encourage you to include a few.


    Here is an example of this type of question:

    Which of these is a fruit? Select all that apply.

     Potato

     Eggplant

     Avocado

     Carrot

    Pear

     Broccoli

     Asparagus

    If "avocado" and "pear" are both selected, 1 point is awarded. In other combinations that include one or both of the correct answers along with incorrect answers, a score between 0 and 1 points is awarded.

    For the stem (the question itself), the same style and media options (as in a multiple-choice question) are available to you.

    For answers, you can enter 2-10 options. Make sure to vary the ratio of right to wrong answers every time you use this question type so that no two questions have the same number of right and/or wrong options.

    Note that every question will be normalized by our system to 1 point for a fully correct answer, so all questions have equal weight.

    Question type 3: True/false

    True/false questions are a special case of multiple-choice questions. They allow for a faster response from the test-taker. You can use this type if you feel that some of your other questions take longer and you'd like to reduce the average time necessary to answer.

    As test-takers have a 50% chance of guessing the right answer, you should also use this question type sparingly.


    Example:

    True or false: The capital of Brazil is Sao Paolo.
     True
     False

    False is awarded 1 point.

    For the stem (the question itself), the same style and media options (as in the other question types) are available to you.

    The answer options for this question type are simply "True" or "False," where 1 point is awarded to the right answer and 0 points to the wrong answer.

    Question type 4: Short text

    This question type should only be used for short answers expressed in specific words (and only those words). This question type works well for testing vocabulary and spelling in a language test, checking terminology (such as the name of a function or method in coding), and for questions that have only one specific answer.

    Example:

    What is the largest country in the world by surface area?

    __________________________

    "Russia", "Russian Federation"  and "The Russian Federation" all are awarded 1 point.

    For the stem (the question itself), the same style and media options (as in the other question types) are available to you.

    For answers to this type of question, you can provide up to 10 right answers, each of which can be up to 300 characters in length. To be awarded a point, the test-taker's entered text must match exactly with one of the right answers. The answer, however, is not case-sensitive. Test-takers can only enter plain text.