Unlike humans, “a machine isn’t affected by whether or not it’s had its coffee yet,” jokes Rikus Combrinck, Machine Learning Engineer at Aephoria. “With humans, there’s an enormous number of influences that can slightly nudge human marking one way or another.” 

His words may be light-hearted, but the sentiment hints at a key advantage of using artificial intelligence (AI) and machine learning to enhance the marking of maturity assessments. 

Pioneered by Aephoria, coaches and their clients can now enjoy one of the world’s first AI-powered maturity assessments based on a sentence completion assessment methodology. It’s a tool that promises to transform the landscape of personal development. However, like any technological advancement, it raises certain concerns. The good news is that Aephoria’s experts have brought ethical considerations to the forefront of its development and application. 

“If we don’t talk about ethics, we’re enabling an unboundaried space between us and the possibility of technology.” – Lucille Greeff, co-founder, and Chief Product Officer at Aephoria.

What We Discuss:

• What Is Artificial Intelligence in Assessments?
• The Path to Pioneering an AI-Powered Maturity Assessment
• Bias: An Inevitable Ingredient in AI Systems?
• How AI-Powered Maturity Assessments Support Coaches
• The Future of AI-Powered Assessments
• Coaches: Your Role in a World of AI

What is Artificial Intelligence in Assessments?

Movies and media have distorted our perception of AI. Hear the word “artificial intelligence”, and you may conjure up images of some sinister, static entity originating from the realm of science fiction. But the truth is far more mundane. In the context of an AI-powered maturity assessment, AI is simply a sophisticated machine learning model that learns from massive datasets over time – and never stops learning. 

“AI is constantly evolving and learning. It’s not a robot. It’s algorithms that can hold more data points than humans.” – Lucille Greeff.

Unlike humans, who might be overwhelmed by vast datasets, AI leverages its computing power to identify patterns we can’t even begin to discern. As we feed these models with more information, their proficiency in recognising and building upon these patterns only increases. This is hardly the stuff of science fiction; it’s rooted in hard-won technology gains.

The Path to Pioneering an AI-Powered Maturity Assessment

Traditionally the marking of qualitative, text-based data is an area that requires deep expertise. You need multiple markers, marking blindly within the same shared paradigm to build a valid measure. This is time-consuming and costly. Some sentence completion assessments take weeks to evaluate and are incredibly costly which inhibits accessibility. That was the problem Rikus and the Aephoria team aimed to address when they began building an AI model to evaluate maturity assessments.

“It’s difficult and labour-intensive to train people. It takes months, even years before people can mark those assessments proficiently. If you’re entering a company with 500 employees, and you have to mark 500 assessments, it also becomes very difficult.” – Rikus Combrinck. 

The Development Process

Developing this AI system wasn’t your average tech project. The AI model had to learn from a database of marked assessments, each response carefully marked by Aephoria’s experts. The AI then sifted through these assessments. But, as Rikus explains, the team didn’t have a ‘correct’ answer sheet. They relied on assessments conducted by human markers and drawing from a relatively limited data pool, given the amount of data typically needed for training. 

“I would have loved 10,000 assessments to start with, as we would have moved much faster. However, the initial lack of data made this project particularly challenging. There were only about 700 marked assessments to work with at the start.” – Rikus Combrinck. 

It was ultimately a trial-and-error process. The development process involved working through, evaluating, and adapting a range of relevant machine-learning approaches.

The technology landscape also changed over the four and a half years Rikus and the team spent developing the model. Tools and capabilities that were once out of reach became available. Rikus acknowledges the difference this technology made for the project: “Today’s technology wasn’t around when I started this project,” he says. “If we were stuck technology-wise where we were two or three years ago, we would not have cracked this task.”

Bias: An Inevitable Ingredient in AI Systems?  

In reality, machines, like the humans who build them, can have inherent biases. Data bias is one area of concern. “Can we trust the algorithm to be fair and equitable in a world that is not fair and equitable?” asks Lucille Greeff. In other words, can we trust the algorithm to be fair and equitable when it draws from data that is not fair and equitable?

Drawing parallels to early AI models in skin technology in the beauty industry, Lucille acknowledges how initial algorithms disproportionately misjudged darker skin as older. This was a reflection of the data they were fed. Similar biases sneak into systems used by banks for loan qualifications, again highlighting how AI can inadvertently perpetuate existing societal inequalities. 

The good news is that both machines and humans can learn to overcome biases. Surprisingly, AI systems can also be corrected much more efficiently. Once a bias is identified in one AI system (due to a human marker’s meticulous intervention), fixing it can instantly benefit all similar systems. This quick fix isn’t something we can replicate with human biases, which take much more time and effort to change.

“In some ways, it’s much easier to eliminate bias in AI systems than with people.” Rikus Combrinck.  

Aephoria has also taken ethical considerations a step further by prioritising transparency. The AIM for Maturity assessment will always ask users to opt into having the algorithm evaluate their responses. This process respects the individual’s right to self-determination. At this point the algorithm has not replaced all human markers, it simply marks alongside a human which ensures that the machine and human are aligning. 

How AI-Powered Maturity Assessments Support Coaches

As a coach, you understand the value of time – both yours and your clients. That’s why an AI-powered maturity assessment is a game-changer. Thanks to the AI model, assessment marking times plummet dramatically, from the typical 20 minutes per marker down to five seconds per individual. 

Here are several benefits of the AI-powered maturity assessment for coaches:

Scalable

AI-assisted marking for maturity assessments revolutionises scalability for you as a coach, enabling you to evaluate more clients without losing quality. Beyond expanding capacity, AI addresses a critical pain point in the industry: the prolonged turnaround times of traditional assessments, which can stretch up to three to four weeks due to limited marking resources. Aephoria’s approach, enhanced by AI yet still enriched by human insight, slashes this wait to a service-level agreement of just three to five days, often delivering results within a mere day or two. 

This swift turnaround not only makes our assessments more accessible to coaches and practitioners but also ensures timely feedback and actionable insights for clients.

Consistent

Using AI for marking assessments introduces a level of precision that cannot be found in traditional methods. This technology meticulously analyses responses with consistency and attention to detail beyond human capability. Human markers, even at their best, are not as consistent in evaluating qualitative text against the norms used in this methodology, as the machine is. 

Non-Judgemental 

Rikus Combrinck recalls that in Japan, a country grappling with a demographic skew towards an older population, robotics has been embraced in healthcare. A study compared patients’ comfort levels with robotic nursing assistants versus human staff. Surprisingly, the majority of participants expressed a preference for the robots, citing a lack of judgment and embarrassment in their interactions. 

This sense of ease extended beyond physical assistance to emotional support, with early tests on therapeutic chatbots revealing similar sentiments. Clients felt more open to discussing their feelings with AI, perceiving it as non-judgmental. Could the same be true of AI-powered assessments? Certainly. 

The Future of AI-Powered Assessments

Finding the sweet spot between AI’s efficiency and the irreplaceable value of human insight is a collective goal. While AI can handle the heavy lifting, a human touch is still relevant. It takes time for the AI model to be trained and biases identified and corrected; this is why human markers are needed, working alongside the AI.

This partnership promises a future where technology and humanity work hand in hand for the greater good. Here’s what Rikus Combrinck and Lucille Greeff are seeing from the AI evolution:

Democratisation at the Forefront

Democratisation is at the heart of AI-powered assessments, with a vision to make these tools universally accessible, breaking down barriers to inclusivity. Aephoria’s goal is to bring these assessments into widespread use, moving beyond the traditional limitation where only executives benefited due to high costs. By leveraging AI, we’re making these assessments more affordable at a time when the world needs people to develop their maturity and consciousness. 

Additionally, AI’s capacity to handle data across languages makes the assessments more accessible by supporting multiple mother tongues. This eliminates the necessity for a consensus language and the way in which this can be biased against those who speak the consensus language as a first language. This approach not only democratises access but also customises the experience to reflect diverse linguistic and cultural contexts while building greater reliability and validity. 

An Ever-Evolving Journey

The path of AI in assessments is one of continuous evolution. Like a skilled artist refining their masterpiece, feedback loops play a critical role in honing the AI model, ensuring it remains accurate, relevant, and aligned with the highest standards. The game changer is where the AI starts outperforming human markers, rather than just being able to mark on par with them. This is the space Aephoria is investing in at the moment.   

Navigating with Care and Responsibility

Amidst the excitement, there’s a grounded approach to the ethical implications of AI. Privacy, bias, and ethical practices are at the heart of the conversation, reminding us of the responsibility that comes with innovation. Vigilance in these areas ensures that we do so with integrity as we advance, safeguarding the trust and well-being of those we aim to serve.

Coaches: Your Role in The World of AI

Coaches, if you’re hesitant about embracing AI-powered maturity assessments, the key is to investigate the ethics and inclusivity of the algorithms and data behind them. Your responsibility is to verify the ethical foundation of these tools. 

“Don’t try to be an intermediary between your client’s agency – their choices – and the technology,” warns Lucille Greeff. Your clients should have the option to opt in or opt out of AI-powered marking, and make their own choices.

Are you still on the fence about AI? Lucille Greef poses the question: “Do we inherently trust a human’s judgment more than a machine’s?” You don’t have to make that choice. Aephoria’s approach blends technology with human expertise; we’ve developed an AI model that’s been trained by human experts to ultimately deliver consistent results. 

By integrating AI into the assessment process, you can deliver insights with unprecedented speed and accuracy, facilitating deeper, more impactful coaching conversations.

Contact Aephoria to experience one of the world’s first AI-powered sentence completion maturity assessments.