Off-Topic Detection

Detect and block user inputs that deviate from allowed conversation topics.

Overview

Off-Topic Detection helps maintain focused conversations by identifying when user input strays from predefined allowed topics. This is particularly useful for specialized applications like customer support bots, educational assistants, or domain-specific tools.

How It Works

1. Define allowed topics with names and descriptions
2. User input is compared against all allowed topics using semantic similarity
3. If the input doesn't match any allowed topic, it's flagged as off-topic
4. Optionally block off-topic inputs based on configuration

Configuration

Enable Off-Topic Detection

guardrails:
  check_off_topic: true
  block_off_topic: true

Define Allowed Topics

guardrails:
  check_off_topic: true
  block_off_topic: true
  allowed_topics:
    - name: "Product Information"
      description: "Questions about product features, specifications, pricing, and availability"
    
    - name: "Technical Support"
      description: "Help with installation, troubleshooting, error messages, and technical issues"
    
    - name: "Account Management"
      description: "Questions about user accounts, subscriptions, billing, and profile settings"

Detection Mode vs Blocking Mode

Detection Only

Set block_off_topic: false to detect but not block off-topic content:

guardrails:
  check_off_topic: true
  block_off_topic: false
  allowed_topics:
    - name: "Allowed Topic"
      description: "Description of allowed topic"

This mode is useful for:

• Monitoring conversation patterns
• Collecting data before enforcing restrictions
• Providing warnings without blocking

Blocking Mode

Set block_off_topic: true to actively block off-topic content:

guardrails:
  check_off_topic: true
  block_off_topic: true  # Block off-topic inputs
  allowed_topics:
    - name: "Allowed Topic"
      description: "Description of allowed topic"

Best Practices

1. Write Clear Descriptions

Provide detailed, specific descriptions for each topic:

Good:

- name: "Password Reset"
  description: "Help users reset forgotten passwords, unlock accounts, and resolve login authentication issues"

Poor:

- name: "Password Reset"
  description: "Password stuff"

2. Cover Related Subtopics

Include variations and related concepts in your descriptions:

- name: "Technical Support"
  description: "Installation help, error troubleshooting, bug reports, performance issues, compatibility questions, and system requirements"

3. Avoid Overlapping Topics

Keep topics distinct to avoid confusion:

Good:

- name: "Billing"
  description: "Payment methods, invoices, billing cycles, and payment history"

- name: "Subscriptions"
  description: "Plan upgrades, downgrades, cancellations, and subscription features"

Poor (too much overlap):

- name: "Billing"
  description: "Payment and subscription management"

- name: "Subscriptions"
  description: "Managing payments and subscriptions"

Complete Example

from elsai_guardrails.guardrails import LLMRails, RailsConfig

yaml_content = """
llm:
  engine: "openai"
  model: "gpt-4o-mini"
  api_key: "sk-..."

guardrails:
  input_checks: true
  output_checks: true
  check_off_topic: true
  block_off_topic: true
  allowed_topics:
    - name: "Product Information"
      description: "Questions about product features, specifications, pricing, availability, and comparisons"
    
    - name: "Technical Support"
      description: "Help with installation, configuration, troubleshooting, errors, and technical issues"
    
    - name: "Account Help"
      description: "Questions about user accounts, login issues, profile settings, and account security"
"""

config = RailsConfig.from_content(yaml_content=yaml_content)
rails = LLMRails(config=config)

# This will pass - on-topic
response1 = rails.generate(
    messages=[{"role": "user", "content": "How do I install the product?"}]
)

# This will be blocked - off-topic
response2 = rails.generate(
    messages=[{"role": "user", "content": "What's the weather like today?"}]
)

Result Inspection

When off-topic content is detected, the result provides details:

result = guardrail.check_input("What's the capital of France?")

if not result.passed:
    print(f"Detection: Off-topic input detected")
    print(f"Message: {result.message}")
    # Handle off-topic input appropriately

API Response Format

The off-topic detection returns a response with the following structure:

{
    "decision": "off_topic",  # Values: "allow", "off_topic", or "clarify"
    "reason": "Query is off-topic or unclear",
    "matched_topic": None  # Topic name if matched, None if off-topic
}

Decision Types:

• "allow" - Input is on-topic and matches an allowed topic
• "off_topic" - Input does not match any allowed topic
• "clarify" - Input is unclear and may need clarification

How It Works Internally

The off-topic detection system:

1. Semantic Analysis: Uses semantic similarity to compare user input against allowed topic descriptions
2. Topic Matching: Calculates similarity scores between the input and each allowed topic
3. Decision Making: Returns one of three decisions:
   • "allow" - Input matches an allowed topic (similarity above threshold)
   • "off_topic" - Input doesn't match any allowed topic
   • "clarify" - Input is unclear or ambiguous

Error Handling

If the off-topic detection service fails:

• The system returns {"error": "off_topic_service_failed", "passed": True}
• Fail-open behavior: Requests are allowed to proceed (passed: True)
• This prevents service disruptions if the detection API is unavailable

Configuration Reference

Option	Type	Default	Description
check_off_topic	bool	false	Enable off-topic detection
block_off_topic	bool	true	Block off-topic inputs
allowed_topics	list	None	List of allowed topics with name and description

Next Steps

• Configuration Guide - Complete configuration options
• Semantic Classification - Related content classification
• Examples - See more usage examples