User persona simulation

Reusable for any conversational agent testing

Problem

Testing a conversational agent by hand — manually typing inputs and eyeballing outputs — is slow, inconsistent, and non-reproducible. Ad-hoc test inputs do not systematically cover the range of user behaviors the agent is meant to handle. When rule changes break existing responses, there is no automated signal. Eliza needed a way to define representative user behaviors as data and drive the agent with them programmatically.

Solution

Define test personas as structured data objects, each containing a name, a description of the simulated user’s style or intent, and a list of sample utterances. Drive the agent with each persona’s utterances in sequence and assert that responses match expected patterns or do not match anti-patterns.

interface UserPersona {
  name: string;
  description: string;
  utterances: string[];
  expectedPatterns?: RegExp[];
  forbiddenPatterns?: RegExp[];
}
 
const personas: UserPersona[] = [
  {
    name: "AnxiousUser",
    description: "Expresses worry and seeks reassurance",
    utterances: [
      "I am worried about my job",
      "I feel like I am failing",
      "Nobody understands me",
    ],
    expectedPatterns: [/you (feel|are|seem)/i],
  },
  {
    name: "CuriousUser",
    description: "Asks open-ended questions",
    utterances: ["Why do I always do this?", "What does it mean to be happy?"],
  },
];
 
function runPersona(agent: Agent, persona: UserPersona): TestResult[] {
  return persona.utterances.map((utterance) => {
    const response = agent.respond(utterance);
    const passed = persona.expectedPatterns?.every((p) => p.test(response)) ?? true;
    return { utterance, response, passed };
  });
}

Personas are versioned alongside the ruleset and re-run on every change. Failures surface regressions immediately.

Usage

Apply when the agent has more than a handful of rules and manual spot-checking has become unreliable. Particularly effective when:

• The rule set is growing and edge cases are accumulating
• Multiple developers or sessions are modifying the rule file
• There is a need to demonstrate agent behavior to stakeholders without a live demo

Pair with a simple test runner or standard test framework. The pattern does not require a testing library — console.assert or a minimal harness suffices.

Trade-offs

• Pro: Persona data doubles as living documentation of the agent’s intended behavior and user model.
• Pro: Adding coverage for a new user type is additive — no test infrastructure changes required.
• Con: expectedPatterns using regex can become brittle if the agent’s response phrasing changes for legitimate reasons. Broad patterns reduce false positives but reduce specificity.
• Con: Personas capture breadth, not depth. They do not model multi-turn conversational flows; each utterance is tested independently unless the runner maintains conversation state across the sequence.

See Also

• Conversation State Machine
• Eliza Rule Matching
• Craftsmanship

Source

Synced from ops-core learning data. Original source: Eliza