Why Most AI Features Fail in EdTech — and How to Fix Them

At SkillUp Exchange, we built a chapter and outline feedback mechanism into AuthPlat — our AI-assisted authoring platform. Authors would submit chapter drafts and outlines, the system would run them through Claude, and structured feedback would come back. The idea was sound. The execution was not.

The feedback was generic. Authors found it unhelpful. The team's first instinct was to swap the model, try a different provider, adjust temperature settings. Classic engineering response. But after sitting with it for a day, the problem became obvious: the prompts we had written were doing too much and saying too little. We were asking the model to evaluate technical depth, writing quality, structural logic, and audience fit — all in one instruction. The output was trying to be everything and landing as nothing.

We rewrote the prompts to do one thing at a time. Structural feedback in one pass. Depth evaluation in another. The quality improved immediately — without touching the model once.

The three things I now check before any AI feature ships

Output quality — not model quality. These are different things. A model can be technically excellent and still produce bad output if the instruction design is poor. Before you blame the model, read the prompt out loud and ask: would a smart human, given this instruction, produce what you're expecting? Usually the answer is no — because the instruction is ambiguous, overloaded, or missing the context the model needs to do its job.

Hallucination scope. Every AI feature will hallucinate sometimes. The question isn't whether it will happen — it's what happens when it does. In an authoring context, a hallucinated citation or fabricated technical claim is a trust-destroying event. We built explicit verification layers: the model was instructed to flag uncertainty rather than paper over it, and reviewers were told to treat confident-sounding claims with the most scrutiny. You have to design for failure, not just for success.

Actionability of the feedback. AI feedback that is accurate but vague is nearly useless. "This section lacks depth" is not feedback — it's a complaint. "This section introduces three concepts in two paragraphs without resolving any of them — consider addressing each in sequence before moving on" is feedback. The difference is specificity. We added output constraints that forced the model to structure every feedback point as: observation, reason, suggestion. It made the output ten times more useful without changing a single model parameter.

What most EdTech companies are getting wrong

They're integrating AI as a search engine. A student types a question. AI returns an answer. The product team calls it "AI-powered learning." But that's not learning — that's lookup. Google with better grammar.

The real opportunity in EdTech AI isn't retrieval. It's augmentation. The system that notices a student keeps getting a specific type of question wrong and adjusts the difficulty curve before the student notices they're struggling. The feedback that doesn't just tell you what you got wrong but asks you to explain your reasoning. The platform that understands where a student is in their learning journey and meets them there, rather than giving the same explanation to a beginner and an expert.

That kind of AI requires product thinking, not just engineering. Most learning experiences are still fundamentally human-designed — a teacher, a curriculum, a sequence of content. AI hasn't changed that structure. It's just been bolted on at the surface, answering questions that a better search function could have answered.

The chapter feedback feature at SkillUp wasn't a model problem. It was a thinking problem. We hadn't been specific enough about what good output looked like, what failure looked like, and what the user was actually going to do with what the model produced. Once we got clear on those three things, the model did exactly what we needed. That's the lesson. The model is not the product. The thinking around it is.