How do I interpret a coding test report?

Even non-technical recruiters can evaluate a candidate's coding test by using the information below. You can also share it with your developers to get their expert opinion.

The overall result of a coding test shows up as a percentage score. This score appears among the other test scores from an assessment on the candidate results page:

Clicking on the link "(Report)" will take you to the following screen:

    • Email this report: This allows you to send the coding report to one of your developers for review. They’ll be able to access the report for the next 14 days.
    • Test score: The coding test is scored on the basis of test cases. These test cases check if a given input in the candidate's code returns the required output. There are test cases that the candidate can use during the test to see if they're headed in the right direction. There are also validation test cases. These are not available to the candidate. The validation test cases determine the test score. The test score represents the % of the validation test cases that are passed. The test score breaks down into a correctness score and a performance score, though most coding tests will only have a correctness score. 
    • Correctness score: The correctness score covers all the validation test cases that determine if a given input returns the required output.
    • Performance score: Performance test cases make up the performance score. Those test cases not only require the right output, but the output also needs to be returned within a given timeframe (in milliseconds). Performance test cases are used if there is an efficiency requirement in the code.
    • Question: On the left-hand side, you'll see the coding question that the candidate received. Every coding test in our library has a database of possible coding questions. Only one of these is given to the candidate during the assessment. The question typically gives some context, specifies the requirements, and gives a few examples to illustrate how the code should work.
    • Solution: On the right-hand side, you'll see the code that the candidate wrote or edited (in the case of a debugging test).
    • Timeline: You can play the code back from start to finish using the timeline. This allows you to see how it came together. On the timeline, we label when candidates copy & pasted elements of the code. The pasted parts are marked yellow in the editor. It also shows the instances when the candidate ran the code (showing “FAIL” and “PASS”).
    • User result: Underneath the code, you can find the results of test cases. For every code execution by the candidate, there is a result of the test cases in the “User result.” You can access these by clicking a "PASS" or "FAIL" label on the timeline. Here is an example of a user test result:
    • Validation result: The evaluated code determining the test score is marked on the timeline by a blue label. It shows the correctness score (100% in the example below):
      Below the code, you can find the validation result, showing the result of each of the validation test cases. Here is an example of one such test case:

    Typical error messages you may encounter:

    • "Task finished with errors": An exception occurred during the execution of at least one test case. In this case, the score is 0%, even if some test cases may have passed.
    • "Task reached unknown status": The code could not be compiled. This could happen if the candidate made changes to the function signature, for example. In this case, none of the test cases pass, and the score on the test is 0%.
    • "Task timed out": The code could not be executed in the given runtime limit. This leads to a 0% score on the test.