Piyush Arora.
ProjectsAboutResumeContact
Back to Projects

Project Details

LLM Chatbot with Multi-Agent System

Intelligent chatbot using Mastra Framework, AI SDK, and OpenAI with multi-agent routing and tool-based agent calls.

Mastra FrameworkAI SDKOpenAIMulti-Agent SystemMongoDBReact

About the Project

Built a production-ready LLM chatbot using Mastra Framework and AI SDK. Implemented a root agent and sub-agent architecture where the root agent (Recruiting Agent) handles simple conversations directly and routes complex queries to a specialized sub-agent (Recruiter Query Handler Agent) via tools. Features conversation continuity through MongoDB memory, streaming responses, and human-in-the-loop interactions for enhanced user experience.

Project Overview

Built an intelligent LLM chatbot using the Mastra Framework, AI SDK, and OpenAI (Azure GPT-4.1 Mini). The system implements a root agent and sub-agent architecture where the root agent (Recruiting Agent) handles simple conversations directly and routes complex queries to a specialized sub-agent (Recruiter Query Handler Agent) via tools. This ensures accurate responses while maintaining conversation context through MongoDB-based memory storage.

Mastra Framework Backend

The backend uses Mastra Framework to orchestrate agents, tools, and workflows with built-in memory management:

import { Mastra } from '@mastra/core/mastra';
import { MongoDBStore } from '@mastra/mongodb';
import { chatRoute } from '@mastra/ai-sdk';
import { recruitingAgent } from './agents/recruiting-agent';

export const mastra = new Mastra({
  agents: { recruitingAgent },
  storage: new MongoDBStore({
    url: process.env.MONGODB_URL,
    dbName: process.env.MONGODB_DATABASE,
  }),
  server: {
    apiRoutes: [
      chatRoute({
        path: '/recruiter/recruiting-chat',
        agent: 'recruitingAgent',
      }),
    ],
  },
});

Root Agent & Sub-Agent Architecture

The system uses a root agent and sub-agent pattern where the root agent handles simple queries directly and routes complex queries to a specialized sub-agent:

// Root Agent - Handles routing and simple conversations
export const recruitingAgent = new Agent({
  name: 'Recruiting Agent',
  instructions: `You're a routing agent for JobHai recruiters. 
  Your role is to handle simple conversations directly and 
  route job-related queries to the appropriate tool.
  
  Handle directly (no tool):
  - Greetings and acknowledgments
  - General conversation
  
  Route to handleRecruiterQueryTool:
  - Job posting issues
  - Account/job verification questions
  - Document issues
  - Any query requiring job-specific information`,
  model: azure('gpt-4.1-mini'),
  tools: {
    handleRecruiterQueryTool, // Tool that calls sub-agent
  },
  memory: createMemory(),
});

// Tool that invokes sub-agent
export const handleRecruiterQueryTool = createTool({
  id: 'handle-recruiter-query',
  description: 'Handles recruiter queries about jobs, accounts, verification',
  execute: async ({ context, runtimeContext }) => {
    const { query } = context;
    
    // Extract thread/resource from runtime context
    const thread = runtimeContext?.get('thread');
    const resource = runtimeContext?.get('resource');
    
    // Call sub-agent with same thread/resource for context continuity
    const handlerAgent = new Agent({
      name: 'Recruiter Query Handler Agent',
      tools: {
        getJobDetailsTool,
        checkJobStatusTool,
        getUserProfileTool,
        checkVerificationStatusTool,
      },
      memory: createMemory(),
    });
    
    const response = await handlerAgent.generate(query, {
      memory: { thread, resource },
    });
    
    return { response: response.text };
  },
});

Tool-Based Sub-Agent Invocation

The root agent uses tools to invoke sub-agents, creating a clean separation where the root agent handles routing and the sub-agent handles specialized queries with tools:

// Root Agent calls tool, tool invokes sub-agent
export const handleRecruiterQueryTool = createTool({
  id: 'handle-recruiter-query',
  description: 'Handles recruiter queries by calling sub-agent',
  execute: async ({ context, runtimeContext, writer }) => {
    const { query } = context;
    
    // Extract thread/resource from runtime context
    const thread = runtimeContext?.get('thread');
    const resource = runtimeContext?.get('resource');
    
    // Create sub-agent instance
    const recruiterQueryHandlerAgent = new Agent({
      name: 'Recruiter Query Handler Agent',
      instructions: `You handle recruiter queries about jobs, accounts, 
      and verification. Use tools to get real-time data.`,
      tools: {
        getJobDetailsTool,
        checkJobStatusTool,
        getUserProfileTool,
        checkVerificationStatusTool,
      },
      memory: createMemory(),
    });
    
    // Call sub-agent with shared thread/resource for context continuity
    const response = await recruiterQueryHandlerAgent.stream(query, {
      memory: {
        thread: thread,
        resource: resource,
      },
    });
    
    // Stream response back to root agent
    if (writer) {
      await response.fullStream.pipeTo(writer);
    }
    
    return {
      response: await response.text,
      agentName: 'Recruiter Query Handler Agent',
    };
  },
});

Memory & Context Management

Conversation context is maintained across agents using thread and resource IDs stored in MongoDB:

// Frontend sends memory context
const requestBody = {
  messages: [...],
  memory: {
    thread: 'conversation-123',
    resource: 'user-456',
  },
};

// Backend middleware extracts and stores in RuntimeContext
middleware: [{
  path: '/recruiter/recruiting-chat',
  handler: async (c, next) => {
    const body = await c.req.json();
    const memory = body.memory;
    
    const runtimeContext = c.get('runtimeContext');
    if (runtimeContext && memory) {
      runtimeContext.set('thread', memory.thread);
      runtimeContext.set('resource', memory.resource);
    }
    
    await next();
  },
}],

// Tools extract thread/resource from runtime context
function getThreadAndResource(runtimeContext) {
  const thread = runtimeContext?.get('thread') || 
                 runtimeContext?.thread;
  const resource = runtimeContext?.get('resource') || 
                   runtimeContext?.resource;
  return { thread, resource };
}

// Agents use same thread/resource for conversation continuity
const response = await agent.generate(query, {
  memory: {
    thread: threadId,    // Same thread across all agents
    resource: resourceId, // Same resource across all agents
  },
});

Frontend: AI SDK useChat Integration

The frontend uses AI SDK's useChat hook with custom transport to handle streaming responses and tool invocations:

import { useChat } from "@ai-sdk/react";
import { DefaultChatTransport } from "ai";

export function useChatbotTransport({
  threadIdRef,
  resourceIdRef,
}) {
  const { messages, sendMessage, status } = useChat({
    transport: new DefaultChatTransport({
      api: API_ENDPOINT,
      // Custom request preparation
      prepareSendMessagesRequest: ({ messages: msgs }) => {
        return {
          messages: msgs,
          memory: {
            thread: threadIdRef.current,
            resource: resourceIdRef.current,
          },
        };
      },
    }),
    
    // Handle tool calls
    onToolCall({ toolCall }) {
      // Client-side tool handling
      if (toolCall.toolName === "colorChangeTool") {
        changeBgColor(toolCall.input.color);
      }
    },
  });

  return { messages, sendMessage, status };
}

Displaying Tool Results

Tool invocations are displayed in the UI with user-friendly messages and interactive elements:

// MessageBubble Component - Renders different message parts
export function MessageBubble({ message, sendMessage }) {
  return (
    <div className="message-bubble">
      {message.parts?.map((part, index) => {
        // Text parts - formatted text
        if (isTextPart(part)) {
          return <FormattedText text={part.text} />;
        }
        
        // Tool invocation parts
        if (part.type?.startsWith("tool-")) {
          const toolName = getToolName(part);
          
          // Show tool status
          if (part.state === "call") {
            return (
              <ToolInvocation part={part} />
              // Shows: "Processing your request..." with pulsing dot
            );
          }
          
          // Human-in-the-loop: Interactive selectors
          if (isTool(part, "askJobTool") && isToolWaitingForInput(part)) {
            return (
              <JobSelector 
                jobs={part.input.jobs}
                onSelect={(jobId) => sendMessage({ text: jobId })}
              />
            );
          }
          
          if (isTool(part, "askCityTool") && isToolWaitingForInput(part)) {
            return (
              <CitySelector 
                cities={CITIES}
                onSelect={(city) => sendMessage({ text: city })}
              />
            );
          }
        }
        
        // Nested agent parts (from tool calling agent)
        if (isNestedAgentPart(part)) {
          return <NestedAgentDisplay part={part} />;
        }
      })}
    </div>
  );
}

// ToolInvocation Component - User-friendly tool status
export function ToolInvocation({ part }) {
  const toolName = getToolName(part);
  const userMessage = getToolMessage(toolName);
  
  // Show simple message during tool execution
  return (
    <div className="text-xs text-slate-500 italic flex items-center gap-1.5">
      <span className="w-1.5 h-1.5 bg-slate-400 rounded-full animate-pulse"></span>
      {userMessage} {/* e.g., "Processing your request..." */}
    </div>
  );
}

Key Features

Root Agent & Sub-Agent Pattern

Root agent (Recruiting Agent) handles simple conversations directly and routes complex queries to a specialized sub-agent (Recruiter Query Handler Agent) via tools.

Tool-Based Sub-Agent Invocation

Root agent uses tools to invoke sub-agents, creating clean separation where root handles routing and sub-agent handles specialized queries with tools.

Conversation Continuity

Thread and resource IDs ensure all agents share the same conversation context, enabling follow-up questions across agents.

Human-in-the-Loop

Interactive UI elements (job selectors, city selectors) allow users to provide input during tool execution.

Streaming Responses

Real-time streaming of agent responses using AI SDK's streaming capabilities for better UX.

MongoDB Memory

Persistent conversation storage in MongoDB enables context retrieval across sessions and agents.

Architecture Benefits

  • Modularity: Each agent has a specific role, making the system easy to maintain and extend
  • Scalability: New agents and tools can be added without modifying existing code
  • Context Preservation: Shared thread/resource IDs ensure conversation continuity across agent boundaries
  • Flexibility: Tool-based routing allows dynamic agent selection based on query content
  • User Experience: Streaming responses and interactive tool UI provide real-time feedback
  • Observability: Mastra's built-in observability features enable monitoring and debugging

Technical Stack

Mastra FrameworkAI SDKOpenAI (Azure GPT-4.1 Mini)MongoDBReactTypeScriptNext.jsStreaming