Four key questions you should ask any AI vendor
aiGuest blog by James McGill VP International Customer Success, EMEA &APAC at HireVue
The amount of media attention, content and opinions on the subject of AI has grown exponentially over the past few years. While sentiment towards this latest stage of technology evolution varies, most agree that AI is here to stay. The potential applications are seemingly endless across multiple business sectors, with corporate recruitment being no exception.
In the current war for talent, many companies are beginning to view AI-driven recruiting technology as a key competitive advantage. However, if you have made the decision to incorporate AI into your hiring process, it is important to remember that the space is still relatively new, and it’s becoming ever easier for companies to claim to be developing ‘AI’ solutions. With the marketplace increasingly cluttered, organisations may find themselves struggling to choose the correct AI technology and vendor for their needs. But how do you know what to look for?
We’ve laid out a series of questions you should ask any potential AI vendor before signing on the dotted line.
What data does the vendor use to identify and evaluate candidates?
The primary role of AI in recruitment is to match the most suitable candidate to a role. However, there are two key differences that affect the efficacy of AI tools – where in the hiring process the evaluation takes place, and the accuracy with which the evaluation is made.
When it comes to location, the predictive accuracy and potential for bias of the data set depends very much on where it falls in the hiring process. Most AI used at the beginning of the recruitment process, for example, takes place in the form of CV screening – analysing a person’s demographic information, education, and job history. Because the data analysed at this stage is quite generic, it is not necessarily an indication of suitability for the role, and therefore has low predictive accuracy. On the other hand, if the vendor is feeding the AI algorithm data created by the candidate as a proof of ability to do a specific job (i.e. via a pre-hire assessment) then you’ll have a higher predictive accuracy. For this reason, solutions which employ AI-driven assessments are far greater predictors of future performance.
Does the vendor have a history of success with organisations of your size?
AI-enabled recruitment uses technology that’s constantly evolving and learning. It’s important to understand at what point of the vendor’s journey your organisation is joining and what that may mean, in terms of results. If the vendor has a library of documented and referenceable success stories, from organisations similar to yours, that indicates that you’ll be using mature technology, adapted for purpose and will be viewed as a partner, not a beta tester.
While early adoption of a vendor’s technology does not mean it will categorically not work, you should be aware that there will likely be teething problems. As the technology learns and adapts, there will be delays, bugs and inconsistencies as the vendor works to scale to the needs of your organisation.
Is the vendor experienced in recruiting?
Recently, there has been a significant boom in the number of companies creating AI-enabled software, including within recruitment. Many of these companies have teams who are highly experienced in data science and the creation of algorithms. However, some may not have the corresponding expertise within the hiring sector and may not be well-versed in people science (i.e. Occupational Psychology). The direct impact of this was seen last year, when a large tech company created an AI solution that would sort through CVs and identify candidates. Though experts in machine learning, the team building the algorithm failed to identify the flaws in the data it was using, or conduct any adverse impact measurement, resulting in a solution that showed clear bias against female applicants.
When choosing a vendor, look to see that it has a balanced team of both data and people scientists, and that it understands the laws, guidelines and best practices of your field.
Does the vendor audit for algorithmic bias?
While it is useful and potentially transformative, AI is a powerful technology that must be responsibly developed and monitored. AI, which uses a set of algorithms trained on a biased data set, can result in the exclusion of particular demographics from selection. However, if an algorithm is thoroughly tested to prevent adverse impact then it can become the most objective hiring tool in your arsenal. Any vendor you select needs to be able to provide evidence of its process for mitigating bias – how it identifies bias within its algorithms and how it is able to remove the factors creating it.
With so many different companies creating AI solutions, it can be hard to cut through the noise, hype and misinformation. In order to find AI solutions which will truly transform your hiring process, you have to take the time to ask the right questions and really understand the vendor’s prediction process before signing up. Now that you’re equipped with the knowledge and tools, go out there and find the right recruitment solution partner for you.