On June 24th, 2015, we held a safety webinar: What Were You Thinking? Getting to the Real Root Cause of Risks at a Global Company. By the end of the webinar, we had plenty of great questions and comments. Unfortunately, we ran out of time before being able to answer all of those questions. But there is some good news, we took note of all of the questions we didn’t get to, and decided to answer them after the fact. The questions and answers are below. Even if you didn’t attend the webinar, you will find some interesting data relating to safety in the answers below.
1) In reference to your assessments, what are your validity and reliability coefficients?
As we stated in the webinar, Select International’s validity coefficients are high by industry standards. Our assessments have overall coefficients that typically range between 0.45 and 0.55, and in some cases as high as 0.60. This indicates that the assessments are very accurate in predicting outcomes. In the case of our safety-specific outcomes, our assessments show correlations between -0.48 and -0.56 with safety incidents and injuries, respectively. They are also significantly correlated with supervisor ratings of safety behavior with coefficients ranging from 0.35 to 0.45. It is also important to mention that we have been able to achieve these high validity coefficients while having very low sub-group differences, which result in no adverse impact, particularly with our safety-based assessments. This table below can help put the magnitude of these coefficients into perspective:
And in terms of our reliability coefficients, our assessments consistently meet recommended guidelines, with reliability coefficients above 0.70 and mostly in the 0.80's
2) What is the adverse impact of your selection tool?
We routinely conduct analyses on all of our assessments to continuously minimize any potential adverse impact on protected minority groups in the U.S. The safety assessments we discussed in our webinar, including SecureFit, which is used for selection purposes, have never been found to have any adverse impact on any minority group. Furthermore, it shows near-zero (minority vs. majority) group differences in average scores and pass/fail rates, even in datasets of thousands of applicants across the variety of industries. In other words, the likelihood of adverse impact occurring is negligible because minority and majority group members tend to have very similar test results. We accomplish this by using test content that is always highly job-relevant, incorporating assessment methods that are shown by research to minimize adverse impact, and continuously improving our test content to increase accuracy and fairness.
3) We saw the statistics for Wabtec using your assessments and programs. Are your statistics with other companies similar?
On the selection (pre-hire) side, yes we have similar results from other companies where we have helped reduce injuries and workers compensation claims over time through our safety pre-hire assessments. Our safety programs for development are newer than our selection tools, therefore we have somewhat limited client results for those tools as we are still collecting data on clients currently using the programs. However, of the companies who have used our full programs for existing employees for at least 6 months, over half of them have seen significant reduction in injury metrics such as TRIR, number of recordables, or lost days due to injury. In contrast, none of them have seen any significant increases in injury rates since implementing the programs. These companies are from various types of manufacturing sectors, as well as the petrochemical, transportation, and heavy equipment maintenance.
4) What is the implementation process like for the safety training programs for front-line leaders?
Our implementation process can vary depending on the needs, situation, and structure of the company we are working with. All solutions are client-focused based on the situation, but critical aspects that we utilized in most projects are: measuring Safety Leadership profiles through validated assessments, providing training and skills development to leaders, and coaching them on their unique strengths and blind spots in order to reduce the risk to their team members.
5) Does Select International offer ebooks or whitepapers on the topic of SafetyDNA?
Interested in watching the webinar? We’ve got it available on demand, just click the image below: