Two Weeks, One New Hire, and a Showdown Nobody Scheduled: AI Assistant vs. The Company Wiki
Every engineering team has that one wiki page. You know the one — last edited in 2021, screenshot from a UI that no longer exists, written by a contractor who left before the framework migration. And every team also has that one engineer who just knows how the billing service works, but can only explain it in Slack threads at 4:45 PM on a Friday.
So when we got the chance to follow a new hire through her first two weeks, we designed a simple, slightly chaotic experiment: could she get productive faster leaning on an AI coding assistant than by relying on the team's official documentation? The answer tells us something uncomfortable about where knowledge actually lives inside a modern dev team.
Meet the Setup
Our new hire — call her Dana — joined a mid-sized fintech startup in Austin. Standard stack: TypeScript monorepo, a Python data pipeline, AWS everywhere, and a Confluence instance that could generously be described as 'archaeological.'
The rules were simple. For week one, Dana onboarded the traditional way: read the onboarding doc, follow the wiki, ask questions in the team channel. For week two, she switched to an AI assistant wired into her IDE, with access to the codebase via retrieval, plus her own questions. We tracked time-to-first-PR, time-to-second-PR, questions asked, and — most importantly — how confident she felt at the end of each day.
Week One: The Wiki Is a Ghost Town
Day one went fine. Environment setup went smoothly because, honestly, AI-generated setup scripts have quietly fixed the worst part of onboarding at most companies. Dana had a dev environment running in under an hour.
Then she hit the wiki.
The onboarding guide pointed her to a 'Services Overview' page that hadn't been touched since before the company split its monolith. Half the links were dead. The architecture diagram referenced a queue that had been decommissioned. She spent nearly three hours on day two just figuring out which documentation was wrong versus merely stale — a skill nobody puts on a résumé but every senior engineer quietly possesses.
By the end of week one, Dana had shipped a small bug fix to the notifications service. Total time to first meaningful PR: four and a half days. She'd asked eleven questions in Slack, mostly variations of 'where does this actually live?'
Week Two: The Assistant Knows the Code Better Than the Docs Do
Week two flipped the workflow. Now, when Dana hit something unfamiliar, she asked the AI assistant first — one that could search the actual codebase, trace call paths, and summarize what a module did based on the code itself, not the docs.
The difference was immediate. Instead of reading a 2022 wiki page about the billing flow, she got a live summary of the billing flow as it exists today, generated from the source. When she needed to add a webhook handler, the assistant surfaced three similar handlers elsewhere in the repo and matched their conventions automatically.
Time to second PR: one day, six hours. Questions in Slack: three.
That's the headline number, and it's impressive. But here's where the story gets interesting — because the AI wasn't perfect, and the failure modes were instructive.
Where the Assistant Fell Flat
The assistant knew the code. It did not know the why.
When Dana asked why the team had implemented a weird retry pattern in the payments module, the AI confidently invented a plausible-sounding rationale. It was wrong. The real reason — a legacy compliance requirement from a banking partner — lived only in a two-year-old Jira ticket and one senior engineer's head. The AI couldn't retrieve tribal knowledge that had never been written down, so it hallucinated a substitute.
Dana caught it because she asked a human to verify. But that's the trap: AI-assisted onboarding works beautifully right up until it confidently teaches a new hire something false, and new hires are the least equipped to catch it.
The Uncomfortable Conclusion
So did AI beat the documentation? Easily. The wiki lost this fight so badly it should probably be embarrassing to someone in management.
But that's the wrong takeaway. The real lesson is that Dana's AI advantage came entirely from the codebase being the single source of truth — because it had to be. Code gets updated on every merge; wikis get updated when someone remembers.
The teams that win at onboarding in the next few years won't be the ones with the fanciest AI tools. They'll be the ones whose code, commit messages, and PR descriptions are clean enough that AI retrieval actually surfaces accurate context — plus a culture where the 'why' behind decisions gets written down somewhere a machine can find it.
AI can bridge a documentation gap. It cannot bridge a knowledge-sharing gap. Dana shipped faster in week two, but she shipped correctly in both weeks because humans were still part of the loop.
Invest in both. Your next hire's first sprint depends on it.
