1. TestGorilla Help center
  2. Creating your own tests

Writing situational judgement questions

Article 3 of 5 in a series on creating your own tests.

This is the third article in a series of five, to help you create your own tests using the TestGorilla platform. The full series includes:

  1. Developing an effective screening test
  2. Choosing a question type
  3. Writing situational judgment questions
  4. How to create your own test
  5. Creating a coding question or test (optional)

The option for adding your own tests is available to account owners, admins, and recruiters who are on our Scale and Business plans.

When creating your own test, we recommend outlining it before putting it into the TestGorilla platform. This will help ensure that everything is organized and laid out exactly as you want.

Approx. reading time 8 minutes

 

In this article

  1. Intro to situational judgment tests — or SJTs
    1. Elements and scoring
    2. Challenges
  2. Template for creating good SJTs
    1. Template for scenarios
    2. Creating a scenario using the template
  3. Template for response options
    1. Checking response options using the template
  4. Common questions
  5. Next steps

    Introduction to situational judgment tests — or SJTs

    Situational Judgment Tests — or SJTs — assess a candidate’s ability to respond appropriately to specific situations in the workplace. How well a candidate responds in each situation depends on previous training, experience with similar situations, and behavioral tendencies.

    When the questions clearly define the situations to be considered by the candidate, SJTs can be used across a wide range of professions.

    For example, a leadership test doesn’t need to be specifically tailored to a marketing or design leader, as long as the leadership situations are clearly defined. This makes SJTs some of the most widely used tests in pre-employment assessments.

    The elements and scoring of SJTs

    Situational judgment test items consist of two elements:

    1. The scenario which provides the situation to be solved 
    2. The possible response options that a candidate can choose from.

    In the same way that real-world situations are never entirely black or white, SJT scenarios don’t always have just one right answer. The response options of an SJT item can contain:

    • One action that’s the most appropriate for the question asked in that situation, which earns full points. 
    • One or two actions that are somewhat appropriate, and earn partial points 
    • One or two actions that would be inappropriate for the question asked in that situation, earning no points.

    When creating a test, you can distribute points based on the correctness of the response. Assigning 5 points to the correct option and fewer points — usually between 1 and 3 — to somewhat correct options. Those points will then be equalized so that the question is only worth one point for the total test.

      back-to-top

      The challenge with writing good SJTs

      Creating scenarios for a timed test presents a unique challenge. You need to provide candidates with all the necessary context of a specific situation — with no extraneous information — so they can choose a response.

      Extraneous information not only adds reading time and stress for the candidate, it can also introduce doubts about which information is most relevant for solving the scenario.

      In a 10-minute test of 15 questions, candidates have 40 seconds to dedicate to each question. This means scenarios and answer option need to be succinct and to the point.

      We recommend that:

      • Scenarios are 80 words or less and
      • Each response option is 25 words or less.

      Most scenarios and response options should be significantly shorter than the max.

      To help you achieve that, we’ve developed a template for writing good SJT items that helps you zero in on the specific skill being assessed in each scenario, and the appropriate response actions — without going off-topic or going overboard with extra details.

      back-to-top

      Template for creating good SJT's

      Situational judgment test items can vary widely in context and details depending on the ability they aim to test. What remains the same in good scenarios is the structure and logistics of introducing complexity and making decisions.

      Our template takes care of the structure and logistics for you so you can focus on the most important part of each item that requires your expertise — the situation and its context.

      Template for situational judgment scenarios

      The most important thing in creating a situational judgment scenario is identifying the goal of your question. Each question should assess only one specific skill, and the scenario you develop should be directly linked to that. You can see our article on developing an effective screening test for help with this.

      Once you identify the skill to test, you’ll come up with a general scenario related to that skill and refine it using the following four steps:

      blue-1 Clarify the position of the subject in your scenario. Clarifying the position at the start of your scenario helps candidates understand the information correctly.
      blue-2 Explain what situation the subject is in. Explaining the situation helps candidates determine the kind of action that’s appropriate. The situation should be directly related to the skill you’re testing.
      blue-3 Introduce the complication. The complication is the gist of your situational judgment question. It gives the specific situation the candidate must respond to with one of the available response actions.
      blue-4 Pose the question and goal. Often the question in situational judgment scenarios is simply, “What should you do?” because the complication already clarifies the situation to be solved. Adding a goal to your question clarifies the intended outcome that the correct answer should have. 

      back-to-top

       

      Creating a scenario using the template

      Let’s see the template in action by going through the process step by step for one of the sample items from our negotiation test

      The final question reads:

      You work for a company that makes plastic bottle tops. You are negotiating with supplier X to sell you dye so you can color your plastics. You know that company X has the best dyes. You know company Y also sells dyes. Y’s dyes are of inferior quality to X’s dyes, but are nevertheless acceptable for you as a Plan B. 

      Which of the following is the best negotiation tactic to take with supplier X?

      We’ll break down how we can get to a brief scenario for what is a complex situation by applying the template.


       

      You’ll start by selecting the skill area and specific category of skill you want to work on from the ones you have defined for your test. Our question above corresponds to the following skill area and skill.

      What skill area does this scenario belong to?

        • The ability to control and drive discussion.

      What specific skill does this scenario test?

        • Negotiating for the best option when alternatives are available.

      From this, we’ll come up with a rough scenario that’s relevant to the skill area and category.

      You are in negotiations with a vendor for a product. This vendor has a high-quality product and you’d like to buy from them, but their asking price is higher than what you can pay. Another vendor also sells a similar product of lower quality for less. The second vendor is a good Plan B, but ideally, you want to get a better price from Vendor A. How do you accomplish that?

      We’ll put our rough scenario through the four steps of the template to define its structure and logistics and turn it into a good situational judgment question:

      blue-1 Clarify the position of the subject. (Yellow hightlight below)
      • You work for a company that makes plastic bottle tops. It’s your job to secure raw material for production.
      blue-2

      Explain the situation. It myst be relevant to the skill you're testing. (Green highlight)

      • You are negotiating with a supplier to sell you dye so you can color your plastics. You want to secure the best price possible for your company.
      blue-3

       

      Explain the complication. (Blue highlight)
      • You know that the company you’re talking to has the best dyes. But you also know that another company sells similar dyes. The other company’s dyes are of lesser quality but are nevertheless acceptable as a Plan B. 
      blue-4 Pose the question and goal. (Pink highlight)
      We’re not using a general question here but looking for a specific goal — get the best price — which helps us define specific response options.
      • What negotiation tactic will get you the best results with the first supplier?

      We’ll then put together the sentences from each of the steps, and edit any overlap between sentences so they flow together in a natural and logical way. 

      Here’s our resulting scenario from the steps above, with each part of the question highlighted with the color of the part of the template it belongs to.

      back-to-top

      Template for response options

      The response options of situational judgment scenarios should follow the same rules as all response options for multiple-choice questions. Specifically, response options should:

      • Follow the same grammatical structure.
      • Be related in terms of content. They should be different, but different in the same way
      • Be internally consistent. They should logically flow from the scenario and question asked.
      • Not contain additional context that’s not given in your scenario.

      In addition to the above rules, response options for situational judgment questions must be actions a test taker can take to resolve the situation, not the results of actions taken. Even wrong answers should be logically possible for the specific scenario. 

      Checking response options using the template

      The response options to the sample question we reviewed above are as follows: 

      Discuss tentatively with Y to see what price is possible before starting your discussions with X. 5 points

      Initiate discussions with X and Y in parallel.

      2 points

      See how far you get with X, and if you get stuck, start a negotiation with Y as your next best alternative.

      0 points

      Tell X upfront that you are talking to Y, in order to put pressure on X. 

      1 point

       

      Notice how all the response options adhere to the rules discussed above by:

      • Following the same grammatical structure. They all start with action verbs that the candidate will take or not take.
      • Being related in content. All options include the same actions to take — talk with X, talk with Y — but differ in the order in which the actions are taken. 
      • Being internally consistent by answering the question directly. 
      • Not containing additional context that’s not given in the original scenario. Additional context here could be an option like “Get your manager to negotiate with X.” The additional context of the manager generates confusion. Are you even authorized to have these negotiations or is the manager the best option?

      Finally, notice that all answers options are actions to take with suppliers X and Y. Our scenario already defines the result — negotiate with supplier X for the best price — so our response options can focus on actions that could lead to that result.

      back-to-top

      Common questions

      Why should I use situational judgment tests?
      It’s one thing to know the skills that a candidate has, but to see how they would actually use them is another thing entirely. SJTs are designed to assess how a candidate would react in hypothetical situations based on real-life work-based scenarios. The questions require your candidates to call upon the skills and experience they have gained in previous roles and put them into practice to identify the best possible solution to the scenario. A candidate’s answers to these questions can help shed some light on their potential job performance.

      Do I have to use the template to create my questions?
      Of course not! You are welcome to create your own questions your own way. We eat our own dog food here at TestGorilla — so we follow this template when creating content for our tests as we believe it is a great way to structure and create your questions. It has worked very well for us, so we thought we would share it with you.

      back-to-top

      Next steps

      As mentioned above, this is the first article in a series of five, to help you create your own tests using the TestGorilla platform.

      The full series includes:

      1. Developing an effective screening test
      2. Choosing a question type
      3. Writing situational judgment questions
      4. How to create your own test
      5. Creating a coding question or test

      We recommend you next read article four: How to create your own test so you can begin the process of putting your test into TestGorilla.