Chat With Context

The Chat With Context component provides an enhanced conversational interface that leverages external knowledge bases in addition to the capabilities of a Large Language Model (LLM).

This component enables you to build interactive, context-aware conversations in your Pipeline Builder applications while maintaining conversation history between interactions.

The key difference from the Chat Response Without Context component is the ability to integrate relevant external information, making responses more accurate and informative for domain-specific applications.

---

Skill Level

• Understanding basic Prompt Engineering concepts is helpful.

• No Python or JavaScript knowledge is required.

Here’s the full Markdown conversion of your provided HTML content:

Overview

Chat With Context
A chat feature that uses AI plus your data to give better answers.

• Looks up helpful information from your documents.

• Remembers past messages with a session ID.

• Follows your setup instructions (system prompts).

• Keeps answers accurate by using the right context.

---

Key Terms

---

When to Use

---

Component Configuration

Chat With Context component in the Pipeline Builder interface

Required Inputs

---

Possible Chaining

The Chat With Context component can be chained with other components in the Pipeline Builder to create powerful GenAI applications.

Common component connections include:

• Create VectorDB Context → Chat With Context: Retrieves relevant documents from your vector database and feeds them as context.

• Check Condition → Chat With Context: Routes conversations based on conditions or topic detection.

• Extract Text → Create VectorDB Context → Chat With Context: Processes documents on the fly to create conversational context.

• Chat With Context → Generate Text: Uses conversation output to generate derivative content.

• Chat With Context → Call HTTP Post: Takes actions based on conversation outcomes.

---

Example Use Case: HR Policy Assistant

Scenario: Creating an HR assistant that can answer employee questions about company policies and procedures using the employee handbook.

Configuration:

• LLM Provider: Anthropic Claude

• System Prompt:
  "You are an HR assistant for our company. Answer employee questions based on the provided company handbook information. If the answer is not in the provided context, explain that you do not have that specific information and offer to connect them with an HR representative. Always be professional and maintain confidentiality."

• Topic: HR Policies

• Context Embeddings Id: company_handbook

• Word Limit for Chat Summary: 300

• Word Limit for Chat History: 800

Conversation Flow

First Iteration: User input: "What's our company policy on remote work?" Context retrieved: \[Company handbook sections about remote work policies] AI response: "According to our company handbook, our remote work policy includes: 1. Employees may work remotely up to 3 days per week with manager approval 2. Core hours of 10am-3pm must be maintained regardless of location 3. Employees must have reliable internet and a secure workspace 4. Equipment can be requested through the IT portal 5. Performance expectations remain the same as in-office work Would you like more specific details about any aspect of the remote work policy?" \[SessionId generated: session\_abc123] --- Second Iteration: User input: "Yes, what's the policy on working remotely from another country?" sessionId: session\_abc123 Context retrieved: \[Company handbook sections about international remote work] AI response: "According to our handbook, working remotely from another country requires: 1. Advance approval at least 30 days before travel 2. Maximum duration of 4 weeks per calendar year 3. The country must be on the company's approved list (available on the HR portal) 4. You must complete the International Work Request Form 5. You're responsible for ensuring proper internet connectivity and security 6. The company cannot guarantee compliance with all local employment laws For stays longer than 4 weeks or countries not on the approved list, you'll need special approval from both your department VP and the HR Director. Would you like me to explain the application process in more detail?"

---

Best Practices

• Prepare quality context data – The better your knowledge base, the more accurate the responses.

• Balance context amount – Include enough context for accurate answers but avoid exceeding token limits.

• Design your System Prompt carefully – Clearly instruct the LLM on how to use the context.

• Implement proper error handling – Handle cases where relevant context might not be available.

• Configure appropriate word limits – Balance context retention with efficiency.

• Always pass the SessionID – Maintain conversation continuity across interactions.

• Test thoroughly – Use varied queries to ensure effective context retrieval.

---

Troubleshooting

---

Limitations and Considerations