Role of Aptitude, Personality and Domain Tests in Predicting Employee Performance
We have all taken aptitude tests at some point in our life, either as an applicant to an educational institute or in a company’s hiring process. It has been established beyond doubt that aptitude tests (cognitive ability tests) are good predictors of both academic performance and job success. Since the early 1990’s, with the emergence of the big five model of personality, personality assessments have made their way into the applicant selection process. Multiple meta-analytic studies have shown that personality measures such as conscientiousness, a measure of a person’s strive for achievement and self-discipline, is a favorable trait for success in almost any job. Other personality traits such as agreeableness and extraversion provide a measure of interpersonal skills of a person and have been established as useful traits in multiple job profiles such as that of sales and customer service .
The predictive power of these tests is established by doing criterion validity studies. In these studies, correlation between scores and the actual performance of an employee at his/her job is evaluated. Stated simply, we need to determine whether a person with a higher score in these tests is also a better performer and vice-versa. Multiple studies show that aptitude tests (together with domain knowledge tests) have a criterion validity ranging anywhere between 0.35 to 0.5, whereas personality traits show a correlation between 0.25 to 0.35 with performance. These correlation figures make these tests useful as a significant input in the hiring decision process. However it does leave a lot of variance unexplained. For instance, if an instrument combining aptitude and personality assessment provides a correlation of 0.55, 70% of the variance in the output, read performance, is still unexplained. This implies there is considerable room to improve the prediction.
Situation Judgment Test: Measuring the Unexplained Traits
Situational Judgment Tests have emerged as a strong contender beyond aptitude, domain and personality tests for improving job performance prediction. Before we understand what SJTs are, let us take a quick peek into their power. SJTs typically provide a correlation of 0.34 , as adjudged from 102 different studies. What makes them so interesting is that this correlation is over and above that of aptitude and personality tests . What does that mean? It means that if aptitude and personality tests give a correlation of 0.55 with job performance, inclusion of an SJT test can take the correlation upto 0.89, which in effect means 80% of the variance of the output explained. Simply stated, SJT is able to measure traits that exist in high performers but not in low performers, which an aptitude or personality instrument is unable to measure. This makes SJTs an extremely attractive tool to be included besides a personality, aptitude or domain instrument for the purpose of hiring.
So what are these situational judgment tests? Think of them as a case interview where you describe a situation to an applicant and ask them how they shall manage the situation. You may ask several questions about how the interviewee shall respond to the situation: what will they do, will they delegate, escalate or act themselves, how will they prioritize, will they take a risk, etc. SJTs are very similar to this, where a situation is posed to the candidate and we probe what he/she shall do. Situations are carefully chosen to mirror the work place, situations which are challenging, critical and related to the candidate’s key responsibility areas in the job. In contrast to case interviews, the candidate has to choose an answer from a given set of options (See figure 1 for a retail situation and two questions based on the same). The candidate may (1) rate each option as the right or the wrong way to act, (2) rank the options or (3) just choose the best and worst response to the situation. In this way, SJTs combine the practicality and open-endedness of a case study with the objectivity of a multiple-choice test. In one line, it can be seen as an objective test to measure how one handles critical and challenging situations at work place.
You are manager at a supermarket. The cashier comes to you with a case where the aisle displayed a lower price of the item, whereas the actual price is higher. When the cashier told the customer about the higher price, he politely declined to buy the item.
i. Which of the following actions will you do first?
ii. Who should manage the situation?
SJT : Directly Measuring and Predicting Candidate’s On-job Performance
But what do these SJT tests measure? They are clearly simulation based tests which rather than testing the hidden traits required for a job, simulate job scenarios and directly test whether the candidate will respond in the correct manner. This direct relationship with the job, as opposed to an indirect one (high quantitative ability is needed to do programming; extraversion is needed for sales), helps in providing a context to the test-taker as well as better correlation to job outcome. There is considerable debate on what these tests measure . Whereas some have argued that they measure a construct separate from personality, domain and cognitive skills, others claim that they are just a different, more effective, way to uncover these traits which current tests are unable to do.
One of the most plausible views is that they measure something called practical intelligence, as proposed by Sternberg, et.al . Practical intelligence comprises both explicit and tacit knowledge gained by a person. It also comprises of the correct behavior required to address a situation. Even though one may have the knowledge, one may not be able to act the right way due to behavioral likes and dislikes. Whereas SJTs measure the knowledge aspect, both tacit and explicit, they also make a strong attempt to indirectly measure behavioral tendencies . Given that practical intelligence is generally believed (such as that termed as ‘street-smartness’) to be apart from one’s mathematical prowess or his/her personality traits, such a hypothesis seems plausible.
This practical intelligence can be further broken down into various components strongly connected to business outcomes: measuring client communication, team management, operations management, etc. This helps provide a score on such competencies for a candidate, which is of substantial use to an interviewer. Apart from this, SJTs have other favorable properties: they provide a realistic experience to the candidate who faces questions related to the kind of job he/she is applying to; they are less susceptible to faking and show low sub-group difference among races, genders, etc.
At Aspiring Minds, we carefully developed a situational judgment test for hiring vice presidents at a large call center. A matrix of situations was constructed after critical situations were identified, covering all aspects of the day-to-day functioning of the incumbents. The test comprised of situations with respect to both internal and external interaction which the incumbents faced. Possible response options were built after inviting open-ended answers from several subject matter experts. In a criterion validity study, the test provided a correlation of 0.34 with job performance, as measured at the time of appraisal. In contrast, a big five inventory administered to the same sample showed no significant correlation to performance. This clearly showed the merit of the situational judgment tests.
If you are thinking about improving your hiring process, SJT is the solution for you. It is hard to construct, requiring considerable attention, but very simple to administer, scale and interpret.