Skip to main content

Conversation review best practices

Effective conversation review is crucial for continuously improving your AI assistant's performance and your product documentation. This guide explains the recommended approach for handling conversation reviews in Kapa, with a focus on team workflows and actionable insights.

Understanding the review cycle​

The ideal conversation review process follows a circular workflow:

  1. Regular review - Systematically examine relevant conversations
  2. Insight identification - Determine which conversations reveal opportunities for improvement
  3. Implementation - Address the identified issues
  4. Verification - Confirm improvements are effective

This process helps you transform user interactions into concrete product and documentation improvements while maintaining a manageable workflow.

Regular review cadence​

We recommend establishing a weekly review cadence dedicated to examining conversations identified by Review Mode. This responsibility typically works best when assigned to one or two team members who can develop expertise in recognizing patterns and opportunities.

When reviewing conversations, focus on identifying:

  • Common user questions that aren't well documented
  • Recurring pain points that might indicate product issues
  • Feature requests or use cases you hadn't considered
  • Areas where the AI assistant struggles to provide accurate information

Not every flagged conversation requires action. Many conversations marked as Uncertain may be edge cases that don't warrant documentation updates. Use the Mark as reviewed button for these to keep your review queue manageable.

Converting insights to actions​

For conversations that reveal valuable insights:

  1. Use Kapa's Status tags to categorize and identify the type of issue or opportunity:

    • Create custom status tags that reflect issue types (e.g., Doc Issue, Bug, Feature Request)
    • Apply the appropriate tag to categorize the conversation

  2. Document your analysis by adding a comment to the conversation that explains:

    • What specific issue was identified
    • Why it matters
    • Any relevant context for someone who might address it later

  3. If your team uses a task management system (like Jira, Asana, or similar):

    • Copy the conversation URL from Kapa
    • Create a task in your task management system with the relevant details
    • Include the conversation link for reference

This approach makes it easy to search for and export conversations by issue type within Kapa, while allowing your team to manage the actual work in whatever system best fits your established workflows.

Implementation and verification​

When implementing changes based on conversation insights:

  1. Reference the original conversation to understand the context fully
  2. Make the necessary updates (documentation, knowledge base, product, etc.)
  3. Update the conversation's status tag to Fixed or a similar "completed" status

This final step of updating the status in Kapa helps maintain a clear record of which insights have been addressed and closes the feedback loop, even when the actual work is managed in external systems.

Adapting to your needs​

This workflow provides a starting point that you can customize based on your team size, tools, and processes. The key principles to maintain are:

  • Regular, systematic review
  • Clear ownership of tasks
  • Documentation of insights and actions
  • Closed-loop verification

By establishing a consistent conversation review practice, you'll continuously improve your AI assistant's effectiveness and ensure your documentation evolves with your users' needs.