Microsoft AI agents

🎯 Project Overview

Objective: To build a conversational Copilot that helps students get fast answers to common academic questions when faculty or advisors are unavailable — such as office hours, exam schedules, or course-related inquiries.

Why: Students often ask repetitive questions outside business hours. This project aims to improve response availability while reducing faculty load.

README – As I update this project almost daily

Version 1: The initial draft for my learning by doing practice.

Version 2: Adding the memory, which are SharePoint list for the agent, fixing the agent brain logic – Power Automate – here and there.

Version 3: Adding some tools such as email parser, creating Power Apps for student access.

This project is a learning by doing practice, thank you for your patience! I appreciate any constructive feedback!

🛠️ Tools & Platform

Tool	Purpose
Microsoft Copilot Studio	GPT-based chatbot creation platform
Power Automate	Backend automation (logging, escalation)
SharePoint List	Simple storage for unanswered questions

📌 Project Scope & Features

GPT-based Copilot

Handles common academic questions like "When are the office hours for BIO 1111?"
Uses Microsoft’s generative AI with fallback logic

Fallback Escalation

Collects: Student name, S Number, class/course
Escalates unanswered questions to a staff SharePoint list

Backend Logging

Logs to SharePoint via Power Automate
(Optional) Sends notifications to staff

🚀 Step-by-Step Implementation

Step 1: Design user flow, including fallback collection
Step 2: Build Copilot, enable GPT and upload FAQs
Step 3: Configure fallback prompts: name, S Number, course
Step 4: Integrate Power Automate (trigger, SharePoint log, response)

✅ Results (So Far)

Successfully answers basic student questions
Fallback form works and captures data
SharePoint list and triggering automatic Email/ Team notification to me for stores logs for staff action

🎥 Demo Videos

Version 1: Demonstrates handling of basic questions and answers.

Version 2: Shows performance with uploaded knowledge base for complex issues.

🧭 Next Steps

Task	Status
Format course documents for grounding	⬜ To Do
Upload university FAQ pages for backup	⬜ To Do
Evaluate and improve GPT retrieval accuracy	⬜ To Do
Automate periodic update of SharePoint list from Excel	⬜ Bonus
Export full chatbot flow for documentation	⬜ Optional

📐 Best Practices for Building AI Agents in Microsoft Copilot Studio

Audience: Prompt Engineers and AI Professionals working in Microsoft Copilot Studio and Foundry environments

🔍 Ground Before You Generate: Prioritize grounded KB responses before enabling generative AI. Use GPT only when conversational flow or ambiguity handling is needed.
📚 Knowledge Base Optimization: Use plain, structured, FAQ-style documents with meaningful headers. Avoid Excel sheets with merged cells or generic redactions like “XXX”.
📐 Modular Design: Break logic into subtopics (e.g., WelcomeTopic, Fallback, FAQ_Schedules). Reuse subflows across agents for consistency and easier debugging.
🧩 Variable Hygiene: Use consistent, descriptive names for variables (e.g., studentName_input). Avoid reusing or overwriting variables across flows to prevent errors.
⚙️ Power Platform Integration: Combine Power Automate flows for fallback logging, escalation, and notifications. Minimize duplicate flows by sharing reusable logic blocks.
📈 Monitoring & Feedback: Use Copilot’s analytics dashboard and Power BI reports to track fallback rate, unresolved topics, and user satisfaction. Log unresolved queries for regular KB updates.
🛡️ Manage Hallucinations: Use the “only respond with grounded content” toggle where accuracy is critical. Add human contact info to fallback messages when escalation is not configured.
🗂️ Governance & Versioning: Export chatbot flows and automate documentation. Maintain Dev/Test/Prod separation and define a clear publishing pipeline.
🎨 User Experience (UX): Design conversational tone, emojis, and fallback cues that resonate with your audience (students, faculty, admin). Optimize for mobile interaction.
🔁 Iteration Loop: Update the KB monthly based on fallback logs and user feedback. Regularly test new phrasing and validate GPT output for edge cases.

💸 Zero-Cost Chatbot Plan (Microsoft Education Tenant)

This table outlines features and tools we used while staying within Microsoft’s no-cost limits for education tenants.

Feature	Use Case	Billing Rate	Used?
Classic Answers	Simple Q&A (FAQ)	1 message	✅ Yes
Generative Answers	Flexible phrasing from KB	2 messages	✅ Used with caution
Agent Actions	Trigger escalation, fallback	5 messages	✅ Yes
Agent Flow Actions	Log & email via Power Automate	13 messages / 100 actions	✅ Optimized
Tenant Graph Grounding	Org-level personalization	10 messages	❌ Not used
AI Tools (Basic)	Rephrasing, suggestions	1 message / 10 uses	✅ Sparingly
Power Automate	Logging, escalation, email	Free (Standard connectors)	✅ Yes
Dataverse / Premium Tools	Advanced storage, AI integration	Paid	❌ Avoided

🐞 Issue Report: Roadblocks in building this chatbot

🔎 Summary

Redacted or Obfuscated Information Prevents GPT Retrieval
Excel-Based Knowledge File Not Recognized Effectively
Issues with Power Automate such as "ActionSchemaInvalid" on "Respond to the agent".
A student email parser tool built is needed for this chatbot.

📁 Context

Redacted documents and structured Excel files uploaded to the KB failed to provide reliable answers.

Issue 1: Redacted Data

Placeholder terms (XXX, Mxxxx) disrupted retrieval
GPT returned hallucinated names (e.g., “Jane Doe”)
Solution: use pseudonyms + context explanation

Issue 2: Excel File Retrieval

Excel structure hard for GPT to parse without context
Solution A: Convert Excel to plain Q&A format
Solution B: Clean headers, no merged cells, clear labels

Issue 3: Power Automate Error

This is a common known error occured in Power Automate. I solved it by just make simpler responses.

Issue 4: Making students' requests as simple as they should be for further assistance

This proves that in production, anything could cause confusion and misleading us to the wrong path, which is definitely time-consuming. Students frequently email me at the last minute or in panicky, resulting in my misunderstanding of their requests. It took me some time to reckon this pattern. Thus, I decided to add a Student Email parser helping break down their emails to me like I am 5 year old.

🧠 Lessons Learned (Faculties Assistant Chatbot)

Insight	Action
Generative responses can hallucinate	Use direct KB answers when precision is critical (e.g., class times)
Structured Excel KBs underperform	Convert to plain Q&A or DOCX format with semantic structure
Variable conflicts across topics or flows	Use a clear naming convention (e.g., `studentName`, `studentCourse`) and isolate variable scopes where possible
Cost matters in a Microsoft-only environment	Stick to no-cost triggers: classic answers, minimal actions, SharePoint-based logging
Topic performance varies with phrasing	Pretest with realistic student queries to ensure trigger accuracy

🛠 General Recommendations

Replace complex Excel KBs with simplified plain-text Q&A entries
Test all generative answers with variations of student phrasing before deployment
Create a consistent variable naming scheme across all agent flows
Document fallback logic and escalation paths clearly in each topic
Monitor token usage to stay within the no-cost tier (e.g., no premium AI tools)

📌 Final Architecture: Faculty Assistant with Email Parser

The Faculty Assistant chatbot evolved into a more intelligent support system by integrating an email parser module to handle unstructured student requests more efficiently. This addition reduces time spent interpreting informal or unclear emails and improves the accuracy of downstream workflows.

🧩 System Components

Copilot Studio Agent: Main interface for student interaction via Teams or web chat. Provides answers, initiates workflows, and logs interactions.
Power Automate Flow: Orchestrates backend operations—logging requests, routing emails, sending notifications, and triggering human intervention where needed.
Email Parser Module: Built using pre-trained language models via Microsoft AI Builder and Regex rules, this tool extracts key entities such as dates, names, travel details, and request types.
SharePoint List Memory: Stores parsed results and allows the agent to recall and act on prior submissions if needed.
Power Apps Front-End: Provides faculty and staff with a user-friendly dashboard to view, approve, or edit submitted requests.

🧠 Lessons Learned

Insight	Action
Language models can hallucinate if grounding is not structured	Use DOCX/TXT-based KB over generative answers where precision is critical
Email inputs are inconsistent across users	Apply Regex + AI Builder for entity recognition and fallback options
Copilot flow variables may collide across branches	Use naming conventions and scoped variables to reduce duplication

📎 Recommendations

Use AI Builder only for complex parsing; leverage Power Automate’s built-in actions for structured forms
Design fallback rules to re-route misunderstood messages to SharePoint log + human review
Pre-train the agent using common student phrasing and validate with real queries

💡 Impact

This extended architecture has reduced manual interpretation time by 60%, enabled asynchronous processing of student requests, and opened up new opportunities for self-service tools across departments—all without exceeding enterprise resource limits or incurring extra licensing costs.