Why 95% of AI Pilots Fail
Picture this: you're handed the keys to a multimillion-dollar Formula One car, but all you get is a quick video on adjusting the rearview mirrors, and then you're told to go win the Monaco Grand Prix. Sounds absurd, right?
Well, that’s pretty much the situation 95% of companies find themselves in with generative AI!
In our chat today, we're diving into the staggering disconnect between the massive investments in AI and the actual value many organisations are squeezing out of it.
Spoiler alert: it's not about the tech but rather about the people and how they communicate.
We’ll explore critical insights from Heather at Start With AI and unpack how a simple tweak in our language skills could mean the difference between AI success and a costly crash. So, grab a cuppa, sit back, and let's get into it!
This Deep Dive podcast is AI generated from the Start With AI Newsletter on LinkedIn - linkedin.com/newsletter/start-with-ai
Chapters:
- 00:01 - Handing Over Responsibility
- 02:24 - The Human Element in AI Success
- 06:58 - The Impact of Language on AI Utilization
- 09:04 - Understanding AI Communication
- 15:40 - The Impact of AI on Communication
- 19:27 - The New Dividing Line: Language Mastery in AI
Takeaways:
- 95% of companies are mishandling generative AI by not training their users properly, akin to giving them an F1 car without driving lessons.
- A recent study found that 70% of AI success relies on people and culture, rather than just technology and algorithms.
- Investing heavily in AI technology without improving communication skills results in wasted resources and failed projects.
- Effective AI usage requires clear prompting, much like managing an intern – the clearer the request, the better the output.
Links referenced in this episode:
Companies mentioned in this episode:
- Start with AI
- Microsoft
- Dcebo
- Rand Corporation
- BCG
Transcript
So imagine.
Speaker A:Imagine handing an employee the keys to a multimillion dollar Formula one car.
Speaker B:Okay.
Speaker A:Right.
Speaker A:You just hand them the keys and you give them like maybe a 10 minute video on how to adjust the rear view mirrors.
Speaker B:Oh, boy.
Speaker A:Right.
Speaker A:You pat them on the back and you just tell them, go win the Monaco Grand Prix.
Speaker B:Which is.
Speaker B:Yeah, that's not going to end well.
Speaker A:No.
Speaker A:When they inevitably crash into the wall on the first turn, or, you know, honestly, if they just leave the car parked in the garage out of sheer terror, you stand there and you're completely baffled.
Speaker B:Right.
Speaker A:And it sounds absur, but that exact scenario is basically what 95% of companies are doing right now with generative AI.
Speaker A:And they are.
Speaker A:I mean, they're losing millions of dollars in the process.
Speaker B:It's really a remarkable failure of implementation.
Speaker B:I mean, we are watching organizations pour just staggering amounts of capital into acquiring these immensely powerful cognitive engines.
Speaker B:Right.
Speaker B:But they are completely neglecting the driver.
Speaker A:Yeah.
Speaker A:And I was reading this fascinating piece by Heather from Start with AI, and it really, it diagnoses this disconnect so perfectly.
Speaker B:Yeah, she really nailed it.
Speaker A:She did.
Speaker A:So the mission for our deep dive today is, well, okay, let's unpack this.
Speaker A:We need to explore this massive gulf between the money being spent on artificial intelligence and the actual, like, measurable business value or the complete lack thereof that organizations are getting back.
Speaker B:Right.
Speaker A:And Heather had this conversation with a friend of hers, a former IT training business owner, who basically watched this entire AI rollout frenzy from the sidelines.
Speaker A:And this person summarized it perfectly.
Speaker A:And it's where I got the F1 idea.
Speaker A:Really?
Speaker A:They said they're giving people a car and not teaching them to drive, which.
Speaker B:Highlights just a fundamental misunderstanding of what this tool actually is.
Speaker A:Yeah.
Speaker B:You know, a large language model.
Speaker B:An LLM is not a traditional software application.
Speaker A:Right.
Speaker B:You don't just, you know, click a dropdown menu and get a deterministic result every single time.
Speaker A:Yeah.
Speaker B:It.
Speaker B:It requires the operator to actually steer their own cognition.
Speaker A:Right.
Speaker A:Like they have to think.
Speaker B:Exactly.
Speaker A:Yeah.
Speaker B:The primary gap in AI adoption.
Speaker B:It isn't technological, really.
Speaker B:No, it has.
Speaker B:It has nothing to do with like server capacity or neural network latency or any of that.
Speaker A:Okay.
Speaker B:It is entirely an inability on the human side to think clearly, to communicate effectively and evaluate critically.
Speaker A:Wow.
Speaker A:Okay, so let's quantify that F1 car crash.
Speaker A:Right.
Speaker A:Because the data we have now is just staggering.
Speaker B:It is.
Speaker A: cent BCG study, they analyzed: Speaker B:10%?
Speaker A:Yeah, just 10%.
Speaker A:And another 20% is the technology and the data infrastructure, leaving a massive 70% of AI success coming down to people, processes and culture.
Speaker B:And that ratio, I mean, that flips the traditional IT procurement model completely upside down.
Speaker A:It totally does.
Speaker A: tions that actually nail that: Speaker A:But, and this is the crazy part, most enterprise budgets are poured almost entirely into that 30%.
Speaker B:Yeah, the tech, right.
Speaker A:They obsess over licenses, platforms, technical integration.
Speaker A:Meanwhile, that critical 70%, you know, the human element that actually determines if the whole thing even works, it gets either zero funding or they get like a generic half day tutorial that just shows them where the chat box is located.
Speaker B:Yeah, it's just interface training.
Speaker A:Right.
Speaker B:It teaches them how to log in, not how to actually think, which leads.
Speaker A:To just catastrophic results.
Speaker A:I mean, MIT's Project Nanda found that 95% of generative AI pilots produce absolutely zero measurable return.
Speaker B:Wow.
Speaker A:Zero.
Speaker A: the Rand Corporation analyzed: Speaker B:It's.
Speaker B:It's grim.
Speaker A:It is.
Speaker A:So let me just try to break down why this is happening, because I struggle to believe that executives are just, you know, willfully burning money.
Speaker B:Right.
Speaker A:I suspect that throwing money at the 30%, like buying a license, upgrading servers, it's a tangible, immediate action.
Speaker A:You can point to a budget line and tell the board, hey, we are innovating.
Speaker B:Yeah.
Speaker B:What's fascinating here is you are hitting on a major psychological hurdle in corporate leadership.
Speaker A:Really?
Speaker B:Yeah.
Speaker B:Buying a platform is a neat transactional event.
Speaker A:Sure.
Speaker B:It provides this illusion of progress.
Speaker B:Upgrading human culture, however, that is abstract, it's complex, and it is notoriously difficult to measure in a single fiscal quarter.
Speaker A:Right.
Speaker A:You can't just buy a box of culture.
Speaker B:Exactly.
Speaker B:But the cost of avoiding that messy human work is severe.
Speaker A:Yeah.
Speaker B: Look at the Dcebo: Speaker A:Oh, I saw that.
Speaker B:Yeah.
Speaker B:85% Of employees state they cannot apply the AI training they receive to their day to day jobs.
Speaker A:85%, That's.
Speaker A:That's almost everybody.
Speaker B:Right.
Speaker B:That tells us the training isn't just inadequate, it is fundamentally teaching the wrong paradigm.
Speaker B:It treats an interactive intelligence like a static spreadsheet.
Speaker A:Okay.
Speaker A:So if this elusive 70% is defined as people and culture, I feel like we need to bring that down to earth.
Speaker B:We do.
Speaker A:Because, you know, culture is one of those boardroom buzzwords that usually Just means like a ping pong table in the break room.
Speaker B:Right.
Speaker B:Or casual Fridays.
Speaker A:Exactly.
Speaker A:But Heather makes this brilliant distillation in her piece.
Speaker A:That entire 70% really just boils down to one highly specific, deeply human trait.
Speaker B:Language.
Speaker A:Yes, language.
Speaker B:That is the core of the entire issue.
Speaker B:It is the interface through which we translate our internal intent into external execution.
Speaker A:And it really changes how you view the tool entirely.
Speaker A:I mean, these AI systems are large language models.
Speaker A:They are literally designed to model human speech.
Speaker A:So that means they act as a mirror reflecting our own communication patterns back to us, right?
Speaker B:Yes, exactly.
Speaker A:So if our habitual communication within an organization is, I don't know, vague, full of generalization, intellectually lazy, the AI is going to mirror that exact vagueness right back.
Speaker B:Right.
Speaker B:And we had to look at the underlying mechanics to understand why that happens.
Speaker B:An LLM is at its core a highly advanced probabilistic engine.
Speaker A:Right.
Speaker B:It is predicting the next most likely word based on the context it is given.
Speaker A:Okay, so just guessing what comes next.
Speaker B:Basically.
Speaker B:Yeah.
Speaker B:So if you give it a vague generic prompt, you are giving it a massive unrestricted probability space.
Speaker B:And the most likely prediction in a wide open space is going to be average, generic and totally unhelpful because it's pulling from everything.
Speaker B:Right.
Speaker B:You are essentially asking it to generate the mathematical mean of human mediocrity.
Speaker A:That is.
Speaker A:Wow.
Speaker A:Mathematical mean of human mediocrity.
Speaker A:That's a great way to put it.
Speaker B:Yeah.
Speaker A:There is actually this fantastic personal anecdote from Heather that really brings this to life.
Speaker B:Oh, the charity one?
Speaker A:Yes.
Speaker A:So she spent a year working with Microsoft Copilot inside this small charity, and whenever she needed it to create a policy document or like a governance process or a handbook, she didn't just type a basic command like write a handbook,.
Speaker B:Which is what 99% of users do.
Speaker A:Oh, absolutely.
Speaker A:I would probably be guilty of that too.
Speaker B:Right.
Speaker A:If I need a trustee appointment policy, my immediate instinct is to type, you know, draft a policy for appointing new board trustees and just hit enter and.
Speaker B:You'd probably get a boring five paragraph document that looks like it was copied straight from Wikipedia.
Speaker A:Exactly.
Speaker A:But Heather provided incredibly deep context.
Speaker A:She told the AI exactly who the organization was.
Speaker A:She fed it specific regulatory frameworks.
Speaker B:Yes.
Speaker A:She told it what professional standards the output needed to meet.
Speaker A:And because she gave it that linguistic framing, the AI actually went to Companies House and the Charities Commission on its own.
Speaker B:Right.
Speaker B:It searched for it.
Speaker A:Yeah.
Speaker A:It pulled the relevant legal data to build a highly customized, rigorous document.
Speaker B:See, by providing that dense context, she radically narrowed the AI's probability space.
Speaker B:She forced the model to only pull from the semantic neighborhoods associated with high level corporate governance and those specific regulatory bodies.
Speaker A:Right.
Speaker A:She boxed it in.
Speaker B:Exactly.
Speaker B:She gave the machine the exact boundaries and coordinates it needed to operate effectively.
Speaker A:But the crazy part is, her colleagues were using the exact same software license, same tool, and they were failing because.
Speaker B:They lacked that linguistic framing.
Speaker A:Right.
Speaker A:They were typing in my hypothetical basic prompt, getting generic garbage back, and blaming the AI.
Speaker B:Yep.
Speaker A:They thought the technology was broken.
Speaker B:The irony being that the AI was functioning flawlessly.
Speaker A:It was.
Speaker B:It was doing exactly what it was designed to do.
Speaker B:It modeled Heather's precision, and it modeled her colleague's vagueness.
Speaker A:Wow.
Speaker B:The machine was not broken.
Speaker B:The human communication was.
Speaker A:So what does this all mean for the listener?
Speaker A:I mean, it sounds like we need to stop treating AI like a traditional search engine.
Speaker B:Yes, absolutely.
Speaker A:Because with a Google search bar, you just type in three keywords, you hit enter, and you let the algorithm do the heavy lifting.
Speaker A:Right, Right.
Speaker A:It's very passive, but we need to start treating it.
Speaker A:I don't know, it's like a highly capable but incredibly literal intern who needs extreme context to do their job.
Speaker B:That's a perfect analogy.
Speaker A:Like, if you tell an intern, fix the marketing, they'll fail.
Speaker A:They will fail.
Speaker A:But if you tell them, hey, analyze the Q3 marketing budget against these three specific competitors, focus only on digital ad spend, and format the findings in a bulleted summary.
Speaker B:You get brilliance.
Speaker A:Right.
Speaker B:And that intern metaphor is incredibly apt because it highlights the necessity of managing the intelligence.
Speaker A:Managing it.
Speaker A:Yeah.
Speaker B:And the crucial takeaway from that charity experience is that nobody taught Heather how to manage that intelligence in a standard corporate onboarding session.
Speaker A:Right.
Speaker A:They just gave her the login.
Speaker B:Exactly.
Speaker B:She achieved those results because she inherently understands how language shapes output.
Speaker B:The bottleneck is entirely upstream of the prompt box.
Speaker A:Upstream.
Speaker A:I like that.
Speaker B:We hear endless chatter about prompt engineering.
Speaker A:Oh, all the time.
Speaker B:Right.
Speaker B:But the real heavy lifting happens in the quality of the human thinking before the fingers even touch the keyboard.
Speaker A:So it's.
Speaker A:It's in your head first.
Speaker B:Yes.
Speaker B:It is about organizing your own thoughts, interrogating your own assumptions, and structuring your logic before you ever initiate contact with the machine.
Speaker A:But that fundamental misunderstanding of how to communicate, I mean, that leads directly to a massive breakdown in trust, doesn't it, between the user and the system.
Speaker A:Absolutely.
Speaker A:Because we have users typing in vague requests, the machine handing back a generic hallucination, and the user simply walking away.
Speaker B:They just give up.
Speaker A:Yeah.
Speaker A:And the data on Microsoft Copilot specifically highlights this abandonment crisis.
Speaker A:Good.
Speaker B:Bad.
Speaker A:Currently only 20 to 30% of paid co pilot seats are actually being used on a weekly basis across enterprise deployments.
Speaker B:That is wild.
Speaker A:Companies are paying full price for 100% of the seats and the vast majority are just gathering digital dust.
Speaker B:It represents a staggering misallocation of capital.
Speaker A:But.
Speaker B:But it also points to a deep psychological friction.
Speaker A:Yeah, because Recon analytics tracked this user friction and they found that 44% of people who stop using Copilot cite a distrust of its answers as their primary reason for leaving.
Speaker A:They tried it, they got an answer they felt was wrong or, you know, weird and they just quit the platform entirely.
Speaker A:See, they quit because they don't possess the mental frameworks to evaluate what they are interacting with.
Speaker B:What do you mean?
Speaker A:Well, they don't understand how the tool processed their initial output, so they have no methodology for correcting it when the output is flawed.
Speaker A:Ah.
Speaker B:And Heather points out that this dynamic has accidentally created a massive, like desperate demand for neuro linguistic programming practitioners in the corporate world.
Speaker A:NLP is deeply relevant here.
Speaker A:Yeah, yeah.
Speaker A:In the traditional sense.
Speaker A:It is the study of how language patterns shape human behavior, perception and outcomes.
Speaker A:Okay.
Speaker A:It is a discipline of recognizing generalizations, challenging vague assertions and understanding that the structural quality of a question dictates the utility of the response.
Speaker B:Here's where it gets really interesting to me though.
Speaker B:We are actively blaming the machine for our own inability to communicate clearly.
Speaker A:Yes.
Speaker B:And there is this concept from NLP mentioned in the piece called T O T. Right?
Speaker A:T O T. Yeah.
Speaker B:Tote testing, experimenting, adjusting.
Speaker B:It's this iterative loop used to reach a desired outcome.
Speaker A:But looking at it closely, that just sounds like the scientific method applied to everyday conversation.
Speaker B:Basically, yes.
Speaker A:So are we really that bad at iterating our own thoughts that we need a formal acronym to force us to do it?
Speaker B:Yes, because human conversation relies heavily on assumed shared context and that allows us to be inherently lazy.
Speaker A:Oh, that's true.
Speaker B:Think about it.
Speaker B:When you speak to a coworker, you don't have to define your terms from scratch every single time.
Speaker A:Right.
Speaker A:They know what I mean.
Speaker B:Exactly.
Speaker B:You share a physical environment, corporate history, cultural norms.
Speaker B:We use linguistic shortcuts constantly.
Speaker B:But an AI shares none of that implicit context.
Speaker B:Yet we expect the AI to possess human intuition, to read between the lines.
Speaker A:And when it doesn't, when it fails.
Speaker B:To read our minds, we get frustrated rather than realizing we failed to provide the context.
Speaker B:The todo remodel, test, operate, test exit forces us out of that lazy intuition.
Speaker A:Okay, break down how that todo cycle actually works.
Speaker A:Like if I'm sitting at my keyboard trying to get a usable document out of an LLM.
Speaker B:Sure.
Speaker B:It's a practical protocol for active engagement.
Speaker B:So you start with the test.
Speaker A:Okay.
Speaker A:The first T. Right.
Speaker B:You give the AI your initial prompt and you review the output against your specific goal.
Speaker A:Okay.
Speaker B:You will almost certainly find a gap between what you wanted and what you got.
Speaker A:Right.
Speaker A:It's never perfect on the first try.
Speaker B:Exactly.
Speaker B:And this is where most people quit.
Speaker B:Yeah, but in the tutorial model, you move to operate the O.
Speaker B:Yes.
Speaker B:This means actively tweaking the parameters of your language so the document is too formal.
Speaker B:You don't just say, make it better.
Speaker A:Right.
Speaker B:You say, rewrite this using a conversational tone appropriate for seventh grade reading level.
Speaker A:Much more specific.
Speaker B:You clarify constraints.
Speaker B:Then you test again, the second T by running the new prompt.
Speaker A:Okay.
Speaker B:And you repeat this loop, testing and operating until the output perfectly matches your goal.
Speaker B:Only then do you exit the loop.
Speaker A:That's the E. Exactly.
Speaker A:Wow.
Speaker A:It requires a level of active, ongoing dialogue that most people just aren't used to having with their software.
Speaker B:No, they're not.
Speaker B:Furthermore, it requires a healthy skepticism of the software's capabilities.
Speaker A:Right.
Speaker B:Because these models can and do hallucinate.
Speaker A:Oh, yeah.
Speaker A:They make things up constantly.
Speaker B:And they will produce incredibly confident sounding inaccuracies because they are designed to be persuasive, not necessarily truthful.
Speaker A:That's a scary combination.
Speaker B:It is.
Speaker B:Critical thinking is the only defense mechanism against information overload.
Speaker B:And these confident hallucinations.
Speaker A:Yeah.
Speaker B:If you lack the ability to evaluate the output critically, you'll either blindly trust a dangerous hallucination, or you'll blindly reject a genuinely brilliant insight just because it looks unfamiliar.
Speaker A:Wow.
Speaker B:The TOT model forces the human to remain the active cognitive driver, evaluating and refining at every step, rather than passively waiting for the machine to do the thinking.
Speaker A:Man, this.
Speaker A:This shifts our whole discussion from the theoretical mechanics of how we talk to computers right into the immediate real world consequences for the listener's career and their business.
Speaker B:Yes, it does.
Speaker A:I mean, the stakes here are existential for organizations right now.
Speaker B:Without a doubt, the technological transition we are undergoing is going to brutally separate.
Speaker A:The market, because the reality outlined by Heather is really stark.
Speaker A:The organizations that figure out how to get the 70% right, the people, the precision of language, the culture of critical thinking, they are the ones that will survive and thrive.
Speaker B:Yes.
Speaker A:The rest of the companies, the ones that keep blindly throwing money at the 30% right, they are just going to burn tokens, watch Their adoption rates flatline and wonder why their multimillion dollar investments just evaporated.
Speaker B:And we should.
Speaker B:We should really define that for clarity.
Speaker A:Oh, burning tokens.
Speaker B:Yeah, when we say burn tokens, we are talking about the literal cost of poor communication.
Speaker A:Right.
Speaker B:AI models process text in chunks called tokens.
Speaker A:Okay.
Speaker B:And companies pay for the computing power required to generate those tokens.
Speaker A:Right.
Speaker A:It's not free.
Speaker B:Exactly.
Speaker B:So if an employee uses a vague prompt and generates a useless 20 page document, they aren't just wasting their time.
Speaker B:They are actively burning company money on computing power that yields zero return.
Speaker A:Oh, man.
Speaker A:And multiply that over thousands of employees.
Speaker B:The financial drain is immense.
Speaker A:They are essentially revving the engine of that Formula One car in neutral until it just runs out of gas.
Speaker B:That's exactly it.
Speaker A:Which brings us to a significant challenge.
Speaker A:Heather is practically issuing a call to action for practitioners to step into this widening gap.
Speaker A:Yes, because almost nobody in the corporate AI training space is making the connection between human language skills and AI literacy.
Speaker B:No, because they simply don't have the background to see it.
Speaker A:Right, but Heather is actively trying to bridge this.
Speaker A:Yes, she is releasing an AI for NLP Practitioner's Guide this September.
Speaker B:That's right.
Speaker A: ,: Speaker A:Yep.
Speaker A:It's called Future Proof youf Thinking with AI and NLP.
Speaker A:And it's going to explore exactly how to build these specific, specific linguistic frameworks.
Speaker B:It is a vital initiative because it finally addresses the root cause of the failure rates rather than just, you know, bandaging the symptoms.
Speaker A:So, synthesizing all of this, I think the core takeaway here, regardless of your technical background, is that before we can successfully upgrade our companies with AI software, we really have to upgrade our own human software.
Speaker B:Beautifully said.
Speaker A:Right.
Speaker A:Like how we think, how we structure our questions, how we clarify our intentions, and really how we build our reality through language.
Speaker B:And that aligns perfectly with the philosophy of Heather's platform.
Speaker B:Start with AI.
Speaker A:Yeah.
Speaker B:Which champions practical intelligence for humanity first businesses.
Speaker A:Humanity first.
Speaker B:Right.
Speaker B:That concept of being humanity first encapsulates the entire paradigm shift.
Speaker B:The technology is phenomenal, obviously, and the infrastructure is necessary, sure.
Speaker B:But the human relationship to the tool dictates the outcome.
Speaker B:Always an intricate knowledge of language patterns, the ability to communicate with extreme precision, and the critical thinking stamina required to iterate through that Toho T model.
Speaker B:Yeah.
Speaker B:These are no longer soft skills relegated to like, HR seminars.
Speaker A:Right.
Speaker B:They are the ultimate hard business assets in an AI driven economy.
Speaker A:It completely flips the narrative of the last decade.
Speaker B:It really does.
Speaker A:We spent years panicking about teaching everyone to code or learning the technical architecture of neural networks.
Speaker A:Right when the real master key to the next decade of productivity is just mastering our own native language.
Speaker B:Which leads to a rather profound and perhaps unsettling implication about where society is heading next.
Speaker B:If clarity of thought and precision of language are the ultimate interfaces for commanding AI, this could create a bizarre new socioeconomic divide.
Speaker A:Wait, I can see where you're going with this.
Speaker B:Yeah, we might be looking at a future where the haves and have nots aren't divided by who has a computer science degree or who can write Python.
Speaker A:Right.
Speaker B:The new dividing line will be mastery of grammar, syntax, logic, and rhetoric.
Speaker A:Wow.
Speaker B:If the most valuable coding language of the future is simply English or Spanish or Mandarin, we may need to completely overhaul how we educate the next generation.
Speaker A:That's a huge shift.
Speaker B:We might see a future where the most highly compensated, powerful people in tech aren't software engineers.
Speaker B:They're individuals with degrees in philosophy, logic, and English literature.
Speaker B:Wow.
Speaker B:It forces us to ask, if interacting with AI exposes exactly how poorly we formulate our thoughts, is it time to stop pushing coding boot camps and urgently return to the rigorous teaching of classical language arts?
