MCC AI Interviewer

Each turn runs through orchestrator.ts as a deterministic pipeline with selective LLM calls:

Each turn emits structured diagnostics: action status updates, drift-control artifact deltas, gate failures, judge verdicts, and optional advance proposals.

I modeled the interview as movement through buckets (global phases) along an ideal action graph. The ideal path is the ordered actions per bucket with required / sufficient / contributing / optional states; the observed path is extracted artifacts plus completed actions plus turn-band pressure.

Canonical state vector (DriftControlArtifacts)

Control signals

• ▹Turn bands (early → light → heavy → lastTurn): correction aggressiveness increases with time in phase.
• ▹Consecutive next-bucket evidence: a guarded override requires two turns, not single-turn hallucination.
• ▹Locked artifact keys: immutable on phase ADVANCE.
• ▹Advance proposals: borderline judge verdicts become offers the candidate can accept or decline.

Instead of one monolithic prompt, interviewer behavior decomposes into composable policy dimensions mapped to prompt sections:

A two-tier prompt architecture composes these at runtime from case JSON: a universal base for behavioral realism, plus phase-specific rules (identify_problem, frame_solution, lead_analysis, provide_recommendations). bucketPromptBuilder.ts assembles case description, previous-bucket context, in-run artifacts, turn-band instructions, drift rules, calculation references, and coach-authored guidance.

The action extractor runs one LLM call per action for clearer classification with action-specific examples. Key design choices:

• ▹Dyadic acceptance: framework_proposed only completes when the interviewer turn shows acceptance language, not a refinement question.
• ▹Solution-graph matching: MECE node alignment against a case-defined semantic graph.
• ▹Artifact merge with lock respect: upsert analysis results without overwriting locked keys.

The framework compiler fixes a real bug class: solution_structure from a single acceptance turn misses multi-turn frameworks. The compiler reads the full frame_solution transcript on ADVANCE; gating still uses the gate artifact, display uses the compiled summary. Analysis extraction uses an explicit state space of dozens of granular slots (for example consumer_credit_revenue_cars versus consumer_credit_revenue) that aggregate to validation slots.

Advancement is evidence-based, not conversational:

The semantic gate requires strong shape (for example decision + objective + metrics) before identify_problem can close. The advancement judge is a single LLM call over gate failures, turn band, merged artifacts, transcript, and per-phase rubric, returning advance / propose / hold. Effective bias escalates with turn pressure so candidates are not trapped. The judge is rescue-only: it can unblock a stuck candidate, but it never overrides a gate-ready advance.

Cases are declarative programs for the engine, with no code per scenario:

• ▹Buckets with objectives, actions, gates, turn caps, and turn-band instruction overrides
• ▹Calculations with expected results for in-prompt validation
• ▹Information-to-provide with triggers (exhibit routing, whitelisted facts)
• ▹Solution-graph node IDs for structure matching, plus a runtimeConfig ontology
• ▹Phase editors in the coach dashboard; the orchestrator always runs authoritative published content

Coaches input domain knowledge once; the engine interprets it as constraints on the state space, and the four behavioral abstractions inject at compile time rather than being hand-edited in TypeScript.

Automated regression runs against the real orchestrator, not mocks:

• ▹Deterministic suite (hard gate): phase transitions, ADVANCE/ROLLBACK, artifact extraction, numeric parsing, gate readiness.
• ▹LLM scenario runners: behavioral regression, drift detection, key collection.
• ▹Turn-level telemetry and structured TurnDetail make prompt and policy iteration measurable.

The voice path shares the same orchestrator contract over HTTP, so text and voice are surfaces on one engine.

The interviewer sounds human; the orchestrator is paranoid. Mixing them creates confident-sounding premature advances.

Phase transitions require artifact readiness, action completion, and turn caps. The judge rescues; it does not govern.

Artifacts credit only when the interviewer validated them in the same turn, which prevents marking structure the coach never accepted.

Different consumers, different correctness requirements.

New cases ship as JSON and DB content, so engineering scales with engine features.

Ontology mapping before the LLM: cheaper, faster, auditable, the kind of NLU discipline production needs.

Higher-autonomy behaviors roll out safely behind feature flags (ADVANCEMENT_JUDGE_ENABLED, ADVANCEMENT_PROPOSALS_ENABLED).

The interviewer generates first; action extraction then runs on the full turn including coach acceptance, so advancement ties to interviewer behavior rather than pre-response guesswork.