Skeptical about AI? It’s normal (and healthy)

Less frightened. More fatigued. That’s where many of us reside with AI. Yet, I am in awe of AI. Despite the plethora and platitudes of AI promising to reshape industry, intellect and how we live, it’s vital to approach the noise and hope with a fresh excitement that embraces complexity. One that encourages argument and sustains a healthy dose of skepticism. Operating with a skeptical mindset is liberating, pragmatic, challenges convention and nourishes what seems to be a frequently missing sense of sanity, especially if you’re restless with endless assumptions and rumor.

We seem to be caught in a chasm or battle of ‘hurry up and wait’ as we monitor the realities and benefits of AI.  We know there’s an advertised glowing future and the market size of global AI is estimated to be more than $454 billion by the end of 2024, which is larger than the individual GDPs of 180 countries, including Finland, Portugal and New Zealand. 

Conversely, though, a recent study predicts that by the end of 2025, at least 30% of generative AI projects will be abandoned after the proof-of-concept stage, and in another report “by some estimates more than 80% of AI projects fail — twice the rate of IT projects that do not involve AI”. 

Blossom or boom?

While skepticism and pessimism are often conflated descriptions, they are fundamentally different in approach.

Skepticism involves inquiry, questioning claims, a desire for evidence and is typically constructive laden with a critical focus. Pessimism tends to limit possibility, includes doubt (and maybe alarm), perhaps anticipating a negative outcome. It may be seen as an unproductive, unappealing and unmotivating state or behavior — although if you believe fear sells, well, it’s not going away.

Skepticism, rooted in philosophical inquiry, involves questioning the validity of claims and seeking evidence before accepting them as truth. The Greek word “skepsis” means investigation. For modern-day skeptics, a commitment to AI inquiry serves as an ideal, truth-seeking tool for evaluating risks and benefits, ensuring that innovation is safe, effective and, yes, responsible. 

We have a sound, historical understanding how critical inquiry has benefited society, despite some very shaky starts:

Vaccinations faced heavy scrutiny and resistance due to safety and ethical issues, yet ongoing research led to vaccines that have saved millions of lives.

Credit cards led to concerns about privacy, fraud and the encouragement of irresponsible spending. The banking industry improved the experience broadly via user-driven testing, updated infrastructure and healthy competition.

Television was initially criticized for being a distraction and a potential cause of moral decline. Critics doubted its newsworthiness and educational value, seeing it as a luxury rather than a necessity. 

ATMs faced concerns including machines making errors or people’s distrust of technology controlling their money. 

Smartphones were doubtful given they lacked a keyboard, had limited features, battery life and more, yet were alleviated by interface and network improvements, government alliances and new forms of monetization.

Thankfully, we have evolving, modern protocols that — when used diligently (versus not at all) — provide a balanced approach that neither blindly accepts nor outright rejects AI utility. In addition to frameworks that aid upstream demand versus risk decision-making, we do have a proven set of tools to evaluate accuracy, bias, and ensure ethical use.  

To be less resistant, more discerning and perhaps a hopeful and happy skepsis, a sampling of these less visible tools include: 

Evaluation MethodWhat it does…ExamplesWhat it’s seeking as ‘truth’…Hallucination detectionIdentifies factual inaccuracies in AI outputDetecting when an AI incorrectly states historical dates or scientific factsSeeks to ensure AI-generated content is factually accurateRetrieval- augmented generation (RAG)Combining results from trained models with additional sources to include the most relevant informationAn AI assistant using current news articles to answer questions about recent eventsCurrent and contextually relevant information from multiple inputsPrecision, recall, F1 scoringMeasures the accuracy and completeness of AI outputsEvaluating a medical diagnosis AI’s ability to correctly identify diseasesBalance between accuracy, completeness and overall AI model performanceCross-validationTests model performance on different subsets of dataTraining a sentiment analysis model on movie reviews and testing it on product reviewsSeeks to ensure the model performs consistently well across different datasets indicating reliabilityFairness evaluationChecks for bias in AI decisions across different groupsAssessing loan approval rates for various ethnic groups in a financial AIEquitable treatment and absence of discriminatory patterns and does not perpetuate biasesA/B testingRunning experiments to compare the performance of a new AI feature against an existing standardTesting an AI chatbot against human customer service representativesValidation, improvements or changes from compared performance metricsAnomaly detection checksUsing statistical models or machine learning algorithms to spot deviations from expected patterns. Flagging unusual financial transactions in fraud detection systemsConsistency and adherence to expected standards, rubrics and/or protocolsSelf-consistency checksEnsures AI responses are internally consistentChecking that an AI’s answers to related questions don’t contradict each otherLogical coherence and reliability; results are not erratic or randomData augmentationExpands training datasets with modified versions of existing dataEnhancing speech recognition models with varied accents and speech patternsImproved model generalization and robustnessPrompt engineering methodsRefining prompts to get the best performance out of AI models like GPT Structuring questions in a way that yields the most accurate responsesOptimal communication between humans and AIUser experience testingAssesses how end-users interact with and perceive AI systemsTesting the usability of an AI-powered virtual assistantUser satisfaction and effective human-AI interaction

4 recommendations for staying constructive and skeptical when exploring AI solutions

As we continue to navigate this age of AI fear and excitement, embracing skepticism-based approaches will be key to ensuring that innovations serve the best interests of humanity. Here are four recommendations to stay mindful of and practice broadly.

Demand transparency: Insist on clear technology explanations with referenceable users or customers. In addition to external vendors and industry/academic contacts, have the same level of expectation setting with internal teams beyond Legal and IT, such as procurement, HR and sales.

Encourage people-first, grassroots participation: Many top-down initiatives fail as goals may exclude the impacts to colleagues and perhaps the broader community. Ask first: As non-hierarchical teammates, what is our approach to understand AI’s impact, versus immediately assigning a task force listing and ranking the top five use cases.

Rigorously track (and embrace?) regulation, safety, ethics and privacy rulings: While the European Union is deploying its AI ACT, and states such as California attempt to initiate controversial AI regulation bills, regardless of your position, these regulations will impact your decisions. Regularly evaluate the ethical implications of these AI advancements prioritizing human and societal impacts over scale, profit and promotion.

Validate performance claims: Request evidence and conduct independent testing when possible.  Ask about the evaluation methods listed above. This is especially true when working with new ‘AI-first’ companies and vendors.

Skepticism is nourishing.  We need methods to move beyond everyday chatter and commotion. Whether you’re in malnourished doubt, or discerning awe, this is not a zero sum competition. A cynic or pessimist’s gain does not lead to an equivalent loss in others’ optimism. I am in awe of AI. I believe it will help us win and our rules for success are grounded in humble judgment.

In a way, albeit with provocation, skepticism is a sexy vulnerability. It’s a discerning choice that should be in every employee manual to ensure new technologies are vetted responsibly without unattractive alarm. 

Marc Steven Ramos is chief learning officer at Cornerstone.