How do I develop an effective screening test?

In this article, we take you through the development process in four steps. These guidelines will help you meet our key quality standards.

Ready to create your first screening test as a testpreneur? Great! You'll be applying your expertise and experience in a new and exciting way, and we're here to help you through the entire process! Follow this step-by-step guide to craft the most effective screening tests.

Step 1: Define the skill areas for a test

General idea: Every test is meant to test candidates for something specific. It’s important to take time to clarify what exactly you're trying to measure. This will form the foundation of your test.

What this means in practice: Clearly define the skill areas within your test. These areas break the test down into different domains of knowledge or skill. Each test should contain 3 or 4 skill areas; they should be mutually exclusive and collectively exhaustive.

Depending on the job role you're testing for, you may be able to refer to official standards. It's also helpful to research and review current job descriptions to see what employers expect.

For example: You intend to develop a "Customer Service" test at the intermediate level. First, review current industry standards (like the ones from the NOS). You can also find job descriptions for customer service representatives elsewhere on the web (for example here or here). Based on these standards, you could define skill areas that look like this:

  1. Ability to understand customer concern or request, and uncover the root cause
  2. Ability to interact with customers in a positive, constructive way
  3. Ability to accurately and efficiently determine appropriate solutions
  4. Ability to avoid negative or unintended consequences

        Essentially, defining the skill areas for a test is about crafting a high-level structure for the knowledge and skills that the test will assess.

        Step 2: Define the categories (skills to test) within each skill area

        General idea: Each skill area tests candidates in a specific section of the overall scope of the test. It's important to define the specific skills or categories that each skill area covers. This will give you a framework for writing questions in the next step.

        What this means in practice: Clearly define the categories (or specific skills) within each skill area. These categories break your skill areas down into different skills. Each skill area should have 5 or 6 categories; they should be mutually exclusive and collectively exhaustive.

        For example: In the "Customer Service" test from above, you would further define the key categories covered in each of the main skill areas. For the first skill area, it might look something like this:

        1. Ability to understand customer concern or request, and uncover the root cause:

          • Understanding the main issue in a stated problem
          • Uncovering the root cause of an issue
          • Ability to empathize with customers
          • Understanding the potential impact of an issue for the customer
          • Understanding the potential impact of an issue to the company

        The skill areas and categories together test form a comprehensive outline of your test. That way, you can create questions in an easy and goal-oriented way.

        Step 3: Write questions based on the defined skill areas

        General idea: Your test's questions should only relate to the skill areas that you've defined in the previous step. Each question should address only one skill area. Another expert in your field should be able to read your questions and immediately understand which skill areas they relate to.

        What this means in practice: To come up with relevant and effective questions, first determine what a test-taker should be able to demonstrate in each skill area. Ask yourself what knowledge or skill would be good evidence of the test-taker's abilities.

        You should create a total of 100 questions, evenly spread across skill areas. Not all of these questions will be used in a single test. In every assessment, TestGorilla uses a different subset of your questions. This helps protect the integrity of your test, as it reduces the chance of questions being leaked online.

        If you have 4 skill areas, there should be 25 questions per area. The difficulty of the questions should vary somewhat within each skill area. That way, the test can accurately distinguish a range of ability levels.

        For example: In the Customer Service test, a question in the fourth skill area ("Ability to avoid negative or unintended consequences") could be:

        Which action can have a negative impact while dealing with an angry customer?

        image-95  Showing your compassion about the inconvenience experienced.

        image-94  Allowing the customer to vent and calm down.

        image-94  Asking the customer to come back when they have calmed down.

        image-94  Explaining what the best solution is for this moment.


        Important: Do not copy questions from an existing source, either online or offline. Your questions should be your own original content. TestGorilla verifies all submitted tests to make sure they're original.

        Step 4: Write for clarity—don't confuse test-takers

        General idea: The goal of each question is to measure the skill, ability, or knowledge defined in each skill area. Therefore, you should make sure your questions are straightforward and easy to understand. Candidates shouldn't have to spend time figuring out what exactly a question is trying to ask, nor should they be purposely confused by overly ambiguous answers.

        What this means in practice:

        1. Be clear and concise in your wording. If not, you're measuring someone's language skills, vocabulary, or reading ability rather than the skill area to which the question belongs.
        2. Do not use negatives. Avoid words like NOT, FALSE, LEAST, WORST, and EXCEPT.
        3. Do not try to trick test-takers. In multiple-choice questions, for example, have one correct answer and several distractor choices. The distractors should be somewhat tempting, but not overly tricky.
        4. Do not give your answer away. Don't make one answer much longer or shorter or very different from the others, and don't give the answer in another question.
        5. Limit questions that have multiple correct responses and "all of the above" or "none of the above" options. While including some are okay, avoid using them whenever possible.

        Step 5: Perform a thorough final check

        General idea: Ultimately, TestGorilla is only as good as the tests we provide. Every test should be a good predictor of job performance and should also provide a good experience for job candidates. Thoroughly reviewing your finished test questions is critical for quality control.

        What this means in practice: Make sure you can say "Yes" to each of the following questions:

        1. Do your questions cover the skill areas appropriately and completely?
        2. Are the questions sufficiently varied (to avoid duplication of knowledge tested)?
        3. Is every question and answer option crystal clear?
        4. Is all provided information relevant to the answer (to avoid confusion)?
        5. Are the image, audio, and/or video materials you're using clear and relevant? Are graphs and diagrams properly and correctly labeled?
        6. Did you check grammar and spelling (U.S. English) for all questions?
        7. Did you write a clear and compelling description of the test? Does it clarify to customers and test-takers when to use the test and what to expect from it, respectively?