Applicants’ reactions to selection procedures is a topic that many organizations monitor and value. Research has shown that when applicants have positive reactions to employee assessments, they will be more likely to accept job offers, refer the organization to friends, purchase goods and services from the organization (even if not offered the job), and less likely to take legal action against the organization.
Clearly, the benefits mentioned here of positive applicant reactions have the potential to greatly impact the organization. As such, organizations have started to implement tools that they believe will be perceived positively by individuals applying to the position(s) of interest. One hypothesis that has been greatly discussed, but not examined extensively, is the notion that assessment length is related to applicant reactions. Specifically, it is believed that the longer an assessment is, the more likely it is to elicit negative reactions from applicants.
While this idea makes logical sense when thinking about potential burnout and fatigue, there are many factors that this hypothesis does not seem to capture. For example, longer assessments provide the applicant greater opportunity to demonstrate their knowledge and skills. This likely translates into feelings of fairness, which would ultimately be a desirable reaction.
Further, longer assessments are more likely to contain engaging and interactive components, such as simulations or game-like features. These types of simulations tend to elicit positive reactions from candidates especially when they are also face-valid and job-related. Therefore, there may be reason to believe that longer assessments do not necessarily lead to negative reactions.
To test whether or not test length negatively impacts applicant reactions, researchers from Select International conducted a study comparing applicant reactions across six assessments of varying lengths. The assessments that were examined were a Manufacturing Screen, Manufacturing In-Depth, Service Screen, Extended Service Screen, Professional Screen and Professional In-Depth. The typical time it takes to complete each assessment is provided in the table below:
As you can see, the assessments that were included in the study ranged in length from 15 to 105 minutes. These tests have been administered to approximately 2,000 (Professional In-Depth) to 58,000 (Service Extended Screen) job candidates across two job families and levels.
The Figure below depicts the average applicant reaction score for each assessment in order of test length (i.e., shortest test is to the left and longest to the right). You can see that applicant reactions are relatively the same except for the Professional In-Depth assessment, which was the longest assessment coming in at around 100 minutes.
Our results revealed a small effect indicating that test length does not have a large impact on applicant reactions. In fact, when exploring this finding deeper by conducting individual comparisons between assessments of different lengths in the same job family results revealed that test length had no practical effect on reactions except when comparing the Professional Screen with the Professional In-Depth. In this case, a medium effect was found.
Therefore, only when comparing the shortest with the longest assessment was an effect revealed. Therefore, test length may have an impact on applicant reactions, but only once tests reach the extreme levels of length (i.e., over 100 minutes).
Our results shed light for those practitioners who have been stuck between a rock and hard place when it comes to the applicant reaction by test length debate. The results of this study demonstrate that practitioners do not need to worry about longer assessments having a negative impact on applicant reactions at least until test length exceeds 100 minutes, at which point we do see a slight dip in reactions.
Overall, this should ease the minds of practitioners who want to implement longer assessments for the gain in validity and to obtain more data to get a better overall picture of the applicant.