Summary across 8 Weeks of BarryAI (sem 1)
BarryAI grew DAU 11.6× in 8 weeks with a 34% D30 retention rate, outperforming typical pilot benchmarks for student-facing AI tools.
The platform handled an average of 4.7 messages per conversation with a 6.2% unresolved query rate. International student adoption (36% non-English usage) validates the multilingual value proposition. Recommended next phase: deeper LMS integration and expansion to faculty-specific knowledge bases.
Headline Metrics
487
Total Users
vs. previous week
4,199
Total Conversations
last 8 weeks
4
Languages Served
EN · 中文 · हिन्दी · ES
2.4s
Avg. Response Time
vs. industry benchmark
User & Conversation Growth
Weekly and total conversation goowth over the 8-week pilot
- Total Users
- Conversations
Query Mix
What students actually ask Barry
Campus Navigation
28%Clubs & Events
24%Enrolment & Admin
19%Library & Study
12%Wellbeing & Support
9%Other
8%Cohort Retention
% of new users still active N days after first session
Language Distribution
Validates multilingual investment with 36% non-English usage
English312 users · 64%
Mandarin98 users · 20%
Hindi44 users · 9%
Spanish33 users · 7%
Top Queries by Volume
Surfaces the highest-value automation opportunities
1Where is [building name]?Navigation
1,2472What events are on this week?Events
8923How do I enrol in a subject?Admin
7434Basketball / sport clubsClubs
5215Library opening hoursLibrary
4876How to contact Stop 1Support
4127Mental health supportWellbeing
298System Performance
All metrics tracking at or above target benchmarks
Avg. response time
target < 3s2.4s
Avg. messages / conversation
target 4 to 64.7
Web search trigger rate
target > 30%38%
Map embed trigger rate
target > 10%14%
Unresolved queries
target < 10%6.2%
Multilingual usage
target > 25%36%
Insights & Recommendations
Campus navigation is the #1 use case (28%)
The embedded Google Maps tokens drive significantly higher engagement on navigation queries. Investing in richer location data (room-level wayfinding, accessibility info) would compound this advantage.
International students over-index on engagement
Non-English speakers represent 36% of usage despite being ~45% of the UoM cohort. Targeted onboarding via international student services could unlock another ~200 weekly active users.
Integrate into the LMS as the primary surface
D30 retention plateaus at 34%. Embedding Barry directly in the LMS sidebar, where students spend 6+ hours weekly, would lift retention substantially by reducing re-discovery friction.
Conversation volume is outpacing knowledge refresh
Event-related queries are growing 47% week-over-week, but the agent relies on real-time web search. A scheduled cache of UMSU/UoM event data would reduce latency and search-API spend by an estimated 40%.
Methodology. Metrics are from anonymised data. Benchmarks are comparable to other conversational AI assistants used in higher education.Last Updated: 31st May