The challenge
Inquiry volume tripled around major sale events and the year-end gift season, and temporary staff took weeks to train. Most inquiries were about delivery status, returns and damaged items, and each one meant checking three systems.
An earlier scripted chatbot deflected questions without resolving them. Customers found it stiff, satisfaction dropped, and the same inquiries came back by phone.
Our approach
- 01
Start from real inquiries
We clustered 90 days of inquiries (about 40,000) to find which request types could actually be resolved end-to-end, and in what order to tackle them.
- 02
Actions with guardrails
The agent can refund, replace or change delivery details, but only within policy limits set by the support lead. Anything above a threshold needs human approval.
- 03
Graceful handoff
When the agent escalates, the staff member gets a summary, the order context and a suggested resolution, so the customer never has to repeat themselves.
What we built
Agent runtime
Multi-step reasoning with tools for order lookup, delivery tracking, return pickups, refunds and replacements.
Policy layer
Returns and compensation rules encoded as checks the agent must pass before acting, so policy is never left to the model’s judgement.
Channel integration
Runs in LINE, email and web chat through the existing helpdesk, writing in the brand’s polite Japanese style guide.
Quality console
Conversation review, escalation analytics and a weekly sample for human grading.
How it flows
- LINE, email & chat
- Understand intent
- Retrieve order
- Check policy
- Act
- Respond or escalate
How we measured it
A regression suite of 1,600 replayed historical inquiries is scored for correct resolution, policy compliance and politeness. It runs on every prompt, tool or model change. Launch required zero policy violations on the suite.
Results
- 41% of all inquiries are resolved without a human; 78% for delivery-status requests.
- Median first response dropped from 5 hours to 40 seconds, 24 hours a day.
- The year-end sale season needed about half the usual temporary staff.
- CSAT on agent-resolved inquiries is 4.2/5, close to the 4.3/5 human-only baseline.
Timeline
Weeks 1–2
Discovery
Inquiry clustering, policy workshops, success criteria
Weeks 3–4
Prototype
Delivery-status and tracking flows against replayed inquiries
Weeks 5–9
Build
Returns, replacements, refunds, policy layer, LINE and helpdesk
Weeks 10–11
Rollout
10% → 50% → 100% of traffic with daily review