How do I write good situational judgment questions?

Good situational judgment questions consist of two things: a concise scenario and actionable response options. We show you how to create both.

This article is relevant for TestGorilla subject matter experts.

Situational judgment tests (SJTs for short) assess a test-taker’s ability to respond appropriately to specific situations in the workplace. How well a test-taker responds in each situation depends on previous training, experience with similar situations, and behavioral tendencies.

Rather than test specific profession-related skills (such as the use of a software language for developers or knowledge of SEO for marketing professionals), SJTs measure a test-taker’s aptitude in wider professional abilities like leadership, negotiation, communication, time management, etc.

When the questions in situational judgment tests define the situations to be considered by the test taker clearly, SJTs can be used across a wide range of professions. A leadership test, for example, doesn’t need to be specifically tailored to a marketing or a design leader, as long as the leadership situations (such as “you have a direct report that’s underperforming” or “your team needs training in communications” etc.) are clearly defined. This makes SJTs some of the most widely used tests in pre-employment assessments.

In this article, we'll show you how to create good situational judgment questions and responses for your tests by going through: 

Let's get to it!

The elements and scoring of situational judgment test items

Situational judgment test items consist of two elements: 

  • The scenario which gives the situation to be solved and 
  • The possible actions (also called response options) that a test-taker can choose from.

Just as real-world situations are never entirely black or white, so SJT scenarios sometimes don’t have just one right answer. The response options of an SJT item can contain one action that’s the most appropriate for the question asked in that situation (which earns full points), one or two actions that are somewhat appropriate (and earn partial points), and one or two actions that would be inappropriate for the question asked in that situation (earning no points).

(Conversely, if the question asks you test-takers to select the most inappropriate or least appropriate action for a situation, then the most inappropriate action will earn full points while appropriate actions will earn no points.)

The algorithm of the TestGorilla platform can distinguish between fully and somewhat correct response options by assigning 5 points to the correct option and fewer points (usually between 1 and 3) to somewhat correct options, depending on their appropriateness. 

The challenge with writing good situational judgment test items

Creating scenarios for situational judgment items for a timed test presents a unique challenge. You need to present test-takers with all the necessary context of a specific situation so they can decide on a response option and only the necessary context with no extraneous information. 

Extraneous information not only adds reading time (and therefore stress) for the test-taker, but it can also introduce doubts around which is the most important or pertinent information for solving a specific scenario.

When it comes to answers, the options available must be clear actions to take (or ways to behave) rather than results of actions. Each action must be logically possible for the specific scenario (even wrong ones).

In a 10-minute test of 15 questions, test-takers have 40 seconds to dedicate to each item, which means scenarios and answer options need to be succinct and to the point. We recommend that scenarios be no longer than 80 words max and each response option no longer than 25 words max, with most scenarios and response options being significantly shorter than the max. 

But how exactly do you achieve all that?

At TestGorilla, we’ve developed a template for writing good SJT items that helps you zero in on the specific skill being assessed in each scenario and appropriate response actions without going off-topic or going overboard with extraneous details.

The TestGorilla template for creating good situational judgment test items

Situational judgment test items can vary widely in context and details depending on the ability they aim to test. What remains the same in good scenarios, however, is the structure and logistics of introducing complexity and making decisions.

The TestGorilla template takes care of the structure and logistics for you so you can focus on the most important part of each item that requires your expertise: the situation and its context. 

Template for situational judgment scenarios

The most important thing in creating a situational judgment scenario is identifying the goal of your question. Each question should assess only one specific skill, and the scenario you develop should be directly linked to that. The best way to zero in on the skills you’re testing is to divide your test into skills areas (as we explain here) that ensure you’re covering the full spectrum of skills, and then break down each skill area into the specific skill categories. You’ll usually have between 4-6 skills per skill area and create 3 - 5 questions per skill. 

Once you identify the skill to test, you’ll come up with a general scenario related to that skill and refine it using the following four steps:

    1. Clarify the position of the subject in your scenario. Is the subject of the scenario acting from the position of a team leader towards direct reports or as a customer success representative addressing a customer? Clarifying the position at the start of your scenario helps test-takers read the information to follow in the right light.
    2. Explain the situation the subject is in. The situation must be directly related to the skill you’re testing with your scenario. Is the team leader, for example, handling a situation with their entire team in a team meeting, or are they talking to a direct report privately in a one-to-one meeting? Explaining the situation helps test-takers determine the kind of action that’s appropriate.
    3. Introduce the complication. The complication is the gist of your situational judgment question: it gives the specific detail situation the test-taker must respond to. The complication is the part of the scenario that one of the available actions or responses must solve.
    4. Pose the question (and goal). Often the question in situational judgment scenarios is simply “what should you do?” because the complication already clarifies the situation to be solved. Sometimes, however, you’ll want to specify a goal to achieve in your final questions, such as “how should you start the conversation?” Adding a goal to your question clarifies the intended outcome that the correct answer should have. If you find that you’re using results rather than actions in your response options (see below for examples), you should consider defining a specific goal in your question to narrow its scope and focus on actions.

    Creating a situational judgment scenario with the template step by step

    Let’s see the template in action by going through the process step by step for one of the sample items from our negotiation test

    The final question reads: 

    “You work for a company that makes plastic bottle tops. You are negotiating with supplier X to sell you dye so you can color your plastics. You know that company X has the best dyes. You know company Y also sells dyes. Y’s dyes are of inferior quality to X’s dyes, but are nevertheless acceptable for you as a Plan B. 

    Which of the following is the best negotiation tactic to take with supplier X?”

    We’ll break down how we can get to a succinct scenario like the above for what is admittedly a complex situation by applying the TestGorilla template. 

    First, you start by selecting the skill area and specific category of skill you want to work on from the ones you have defined for your test. Our question above corresponds to the following skill area and skill.

    What skill area does this scenario belong to?

    • The ability to control and drive discussion.

    What specific skill (category) does this scenario test?

    • Negotiating for the best option when alternatives are available.

    From this, we’ll come up with a rough scenario that’s relevant to the skill area and category.

    “You are in negotiations with a vendor for a product. This vendor has a high-quality product and you’d like to buy from them, but their asking price is higher than what you can pay. Another vendor also sells a similar product of lower quality for less. The second vendor is a good Plan B, but ideally, you want to get a better price from Vendor A. How do you accomplish that”?

    Then we’ll put our rough scenario through the four steps of the template to define its structure and logistics and turn it into a good situational judgment question.

    (Note: The highlight colors in parentheses below refer to the colors used in scenarios below to help you deconstruct each part of the template.)

    1. Clarify the position of the subject. (Yellow highlight)
      • You work for a company that makes plastic bottle tops. It’s your job to secure raw material for production.
    2. Explain the situation. It must be relevant to the skill you’re testing. (Green highlight)
      • You are negotiating with a supplier to sell you dye so you can color your plastics. You want to secure the best (cheaper) price possible for your company.
    3. Explain the complication. (Blue highlight)
      • You know that the company you’re talking to has the best dyes. But you also know that another company sells similar dyes. The other company’s dyes are of lesser quality than the company you’re negotiating with but are nevertheless acceptable as a Plan B. 
    4. Pose the question (and goal). (Pink highlight)
      Note that we’re not using a general question here (“what should you do?”) but looking for a specific goal (get the best price) which (as we’ll see later) helps us define specific response options.
      • What negotiation tactic will get you the best results with the first supplier?

      We’ll then put together the sentences from each of the steps of the template to build our scenario. We’ll edit any overlap between sentences so they flow together in a natural and logical way. 

      Here’s our resulting scenario from the steps above, with each part of the question highlighted with the color of the part of the template it belongs to.

      Screenshot 2020-06-18 19.24.14

      You can find more sample questions from our situational judgment tests below, with each section of the templated highlighted in a different color so you can understand how it’s been constructed. But before we look at that, let’s see how we can create appropriate response options for our scenario. 

      Template for situational judgment response options

      The response options of situational judgment scenarios must follow the same rules as all response options for multiple choice questions. More specifically, response options should:

      • Follow the same grammatical structure
      • Be homogeneous in terms of content (i.e. they should be different, but different in the same way)
      • Be internally consistent (i.e logically follow from the scenario and question asked)
      • Not contain additional context that’s not given in your scenario. 

      In addition to the above rules, response options for situational judgment questions must be actions a test taker can take to resolve the situation, not the results of actions taken. That’s the key to creating clear response options that don’t introduce confusion for test-takers and check test-takers' practical skills in a given scenario rather than their theoretical understanding of it. 

      Let’s look at an example to understand what we mean when we say that response options should be actions, not results for action. Let’s say we have a scenario asking us to deal with an angry customer. 

      If the scenario ends on the question “how should you appease this customer?” we can have the following response options:

      • Offer the customer a discount
      • Answer the customer’s questions

        Both of the above are actions that the test-taker can take to achieve the desired outcome. But the option. But if our question ended on the general question “what should you do?” we might be tempted to give the following as a response option:

        • Calm the customer down before proceeding

          This phrase, however, describes the desired result of an (unknown) action. How did we calm the customer down? Anyone is likely to know (in theory) that you want to calm the customer down. The successful test-taker is the one that knows what action to take to achieve the desired result in the specific situation. 

          As mentioned above in step 4 of the scenarios template, if you find yourself writing response options that describe results rather than actions, re-read your scenario to check if you need to add a specific goal to your question to help the test-taker (and yourself) move towards specific actions to take, not results.

          Checking response options using the template

          The response options to the sample question we reviewed above are as follows: 

          5 points

          Discuss tentatively with Y to see what price is possible before starting your discussions with X.

          2 points

          Initiate discussions with X and Y in parallel.

          0 points

          See how far you get with X, and if you get stuck, start a negotiation with Y as your next best alternative.

          1 point

          Tell X upfront that you are talking to Y, in order to put pressure on X. 

          Note how all the response options adhere to the rules discussed above by: 

          • Starting with action verbs that the test-taker will take (or not take)
          • Being homogeneous in content. All options include the same actions to take (talk with X, talk with Y) but differ in the order in which the actions are taken. (An example of non-homogeneous options would be having one option that included a different action, such as “send a written proposal to Y.” It’s unclear in the context of this scenario how a written proposal would be better or worse than the verbal negotiations of the other answers. And it’s, therefore, not an appropriate difference to introduce in this case.)
          • Being internally consistent by answering the question directly. 
          • Not containing additional context that’s not given in the original scenario. (Additional context here could be an option that said “Get your manager to negotiate with X.” The additional context of the manager generates confusion. Are you (as the subject of the question) authorized to have these negotiations or is the manager the best option?)

          Finally, notice that all answers options are actions to take with suppliers X and Y. Had our question not included that goal of choosing the best negotiation tactic, but only asked “what should you do?” we could have ended up with descriptions of results in our answer options such as: 

          • Try to get the best price from X after finding out about Y’s prices
          • Get the best prices that both X and Y can give you and decide on the cheapest option.
          • Talk to X and Y in parallel to pressure them into lowering their prices

          The above examples aren’t appropriate because they describe different results for the same situation. Our scenario should already define the result (negotiate with supplier X for the best price) so our response options can focus on actions that could lead to that result.

          Samples of good situational judgment scenarios and response options

          Below you’ll find examples of good situational judgment test items. We’ll start with the example we analyzed above to review both the scenario and answer options together, and then move to other items from the sample questions of our negotiation, leadership and people management, and business ethics and compliance tests.

          We’ve highlighted each section of the scenarios with its corresponding template color from above so you can understand how each one was constructed and how you can construct your own scenarios. 

          Remember that the steps of the template are: 

          • Identifying the skill area the scenario belongs to
          • Identifying specific skill (category) the scenario tests
          • Clarifying the position of the subject (yellow highlight)
          • Explaining the situation. It must be relevant to the skill you’re testing (green highlight)
          • Explaining the complication (blue highlight)
          • Posing the question (and goal) (pink highlight)

            Sample questions from the negotiation test

            situational judgment test question - negotiationScreenshot 2020-06-18 19.26.03situational judgment test question - negotiation 2Sample question from the leadership and people management test

            Screenshot 202situational judgment test question - leadership and people management

            Sample questions from the business ethics and compliance test

            situational judgment test question - business ethics

            situational judgment test question - business ethics 2

            Note that in the above questions the position isn’t given explicitly. It’s subsumed in the situation because it’s inseparable from it. The question relates to anyone involved with the company that received the information.