Scenario-Based Product Knowledge Assessment

The Problem

Product launch assessments at the company had always been basic recall quizzes: "What is the screen size of the Pro model?" or "How much storage does the base model include?" Reps could pass these by skimming a spec sheet five minutes before the deadline. But when they were standing in front of a customer who asked "do I need the Pro or is the standard good enough?", the quiz score didn't help them.

The sales enablement team wanted an assessment that tested whether reps could actually use their product knowledge in a customer conversation — not just whether they'd memorized the right numbers.

Design Approach

I redesigned the assessment around scenarios rather than recall. Every question begins with a realistic customer situation — a context line that sets the scene — followed by a question about what the rep should do or say in that moment.

Each question has 4 options. Only one is the best response, but I deliberately designed the distractors to reflect common mistakes I'd observed in real customer interactions: overselling features the product doesn't have, leading with discounts instead of value, using jargon the customer won't understand, or giving accurate-but-unhelpful responses.

Every question provides immediate, detailed feedback — both for correct and incorrect answers. The feedback doesn't just say "wrong"; it explains the best approach and why it works. This turns the assessment itself into a learning experience. Reps who get a question wrong learn more from the feedback than they would from a traditional quiz.

Assessment Architecture

8 questions covering the full selling cycle:

Questions 1–2 cover product positioning — how to describe the product and compare models. Questions 3–4 cover technical accuracy — compliance language, feature limitations, and what not to say. Questions 5–6 cover objection handling — price concerns, competitive comparisons, and plan compatibility. Questions 7–8 cover closing skills — moving the customer toward a decision without being pushy.

The passing score is 75% (6 of 8 correct). Reps who don't pass are directed back to the product knowledge guide before retaking. Progress is tracked through the LMS to identify topics where the broader team is struggling, which informs follow-up coaching.

Development

The production version was built in Articulate Storyline 360 and published as SCORM 1.2 for LMS tracking. I built the HTML prototype first to validate the question design and feedback approach with stakeholders before the full build.

AI tools were used in the question development process. I fed product documentation and competitive positioning guides into AI to generate initial question drafts and distractor options, which I then refined based on common mistakes observed in actual customer interactions. This cut the question writing phase by about 40%.

Outcomes

The first-attempt pass rate dropped from 91% (on the old recall quiz) to 78% — which was actually the goal. The old quiz was too easy. The new assessment surfaced real gaps. Questions 5 and 6 (objection handling) had the lowest scores across the team, which led to a targeted coaching session on competitive positioning that wouldn't have happened without the assessment data.

What I'd Do Differently

I'd add adaptive questioning — if a rep gets the first product positioning question right, skip the second one and move to objection handling. This would shorten the assessment for strong reps while focusing time where it's needed. I'd also add a manager dashboard showing team-level results by question category, so managers can see exactly where their team needs help without pulling individual reports.