AI chat helps when answers stay grounded in your docs and policies. This guide walks through creating an assistant, connecting content, setting safety rails, and publishing the widget without surprising customers — or your brand.
Create an assistant
Open AI Assistants New. Name it for a job (Support, Sales, Docs). Write a system brief that defines tone, languages, and what it must refuse (legal advice, medical diagnosis, competitor bashing, inventing pricing).
Personas should match your help center voice. A playful brand can still require the assistant to escalate billing disputes rather than invent goodwill credits.
- Create the assistant and assign an owner.
- Draft the system brief and refusal list.
- Set default language and fallback locales.
- Choose confidence thresholds for escalation.
Assistant: Support
Brief: Answer from indexed help + policies only.
Refuse: legal advice, guaranteed outcomes, inventing SKUs.
Escalate: billing disputes, security incidents, anger keywords.Connect content
Index help articles, product specs, pricing pages you want quoted, and policies. Re-ingest when content publishes. Stale chunks cause confident wrong answers — schedule reindex after major doc releases.
Exclude draft, internal, and contradicted pages. If two articles disagree, fix the source of truth before indexing both.
- Chunking: prefer coherent sections over arbitrary character splits.
- Metadata: attach product line, locale, and freshness dates.
- Test queries: build an evaluation set of twenty real questions with expected citations.
When pricing changes, update the canonical pricing doc the same day and re-ingest before campaigns mention new numbers.
Safety rails
- Answer only from retrieved sources or say you do not know
- Block PII and payment data from logs and prompts where possible
- Escalate to human chat below a confidence threshold
- Rate-limit abusive clients and empty prompt floods
- Red-team jailbreak attempts during staging
Display citations so users can verify. Provide a visible path to a human. Do not let the assistant execute privileged account actions without authenticated workflows outside the chat box.
Publish
Embed the widget on selected pages. Start with help and pricing; expand once deflection metrics look healthy. Track containment rate, escalation rate, CSAT on handed-off chats, and hallucination reports from agents.
- Evaluation set passing on staging
- Escalation path staffed during publish hours
- Widget shown only on approved templates
- Analytics events firing for open / resolve / escalate
Troubleshooting
Outdated answers: check last ingest time and whether the article is excluded by path rules.
Over-refusal: tighten refusal list; add clarifying examples to the brief.
Low adoption: place the launcher near help entry points; avoid competing live-chat popups on first paint.
Prerequisites & preparation
Before changing production settings for AI chat setup, align the people who own content, DNS, analytics, and approvals. A fifteen-minute kickoff that names owners prevents multi-day Slack archaeology later. Capture decisions in the workspace notes so the next teammate inherits context instead of guesswork.
Gather credentials and access: workspace admin or editor role, DNS control when hostnames are involved, payment or API sandbox accounts when money paths are involved, and a shared checklist link. Confirm which environment you will rehearse in — preview first, production only after a green run.
Nominate an owner, draft refusal topics, and gather canonical help articles and policies for indexing.
Detailed walkthrough
Work the happy path slowly the first time. Narrate what you expect to see after each click: a status badge, a DNS record, a webhook delivery, a Lighthouse metric. When reality diverges, stop and resolve the mismatch instead of clicking ahead — most outages begin as ignored yellow states.
- Create the assistant with a job-specific brief and escalation rules.
- Index approved content; exclude drafts and contradictory pages.
- Build a twenty-question evaluation set with expected citations.
- Enable safety rails: grounding, PII controls, rate limits, human handoff.
- Publish on help/pricing first; expand after deflection metrics look healthy.
Edge cases & failure modes
Plan for partial failure. Networks drop, registrars delay, providers rate-limit, and humans approve the wrong revision. Your runbook should say what “abort” looks like: leave preview up, roll back the release, or freeze campaigns until metrics recover.
- Stale embeddings after pricing changes.
- Over-refusal blocking simple how-to answers.
- Widget competing with live chat on first paint.
QA checklist before you announce
- Happy path verified on mobile and desktop
- Failure path messaging reviewed
- Owners named for the first hour after launch
- Rollback or freeze path documented
Operating the change
After launch, watch the metrics that prove the change worked — not vanity charts. Pair quantitative signals with one qualitative check (support ticket themes, sales feedback). Schedule a follow-up within a week to remove temporary flags, raise DNS TTLs, or archive the experiment.
Re-ingest on publish of critical docs; review hallucination reports with support weekly.
Treat AI chat setup as a repeatable playbook. The second time your team runs it should be faster because the checklist and owners already exist.
Grounding beats cleverness. “I do not know” plus a human path is better than a confident wrong refund policy.
Citations build trust and give agents a way to verify before they override the bot.
Do not let chat execute privileged account changes without authenticated workflows outside the widget.
Evaluate before launch with real questions from tickets, not only marketing FAQs.
Track containment, escalation, and CSAT on handed-off chats — not just message volume.
Red-team jailbreaks in staging; update the brief when new attack patterns appear.
Implementation checklist
Confirm job-specific system briefs and refusals on preview before you promote. Name who approves the change, which environment is authoritative, and what “done” looks like in measurable terms.
Document index freshness after pricing publishes where the team already looks — workspace notes or the engineering handbook — not only in a meeting memory.
Rehearse evaluation sets from real tickets against written acceptance checks. If you cannot name the verification event (order, DNS lookup, deploy health, or signed API call), you are not ready to announce.
Measure citation display and human escalation with anonymized but realistic data, and keep logs, headers, or screenshots for at least one release cycle so regressions have a baseline.
Operating it with your team
Automate PII redaction in logs and prompts so the next teammate can repeat the path without tribal knowledge. Prefer vaulted secrets over chat paste, and archive temporary exceptions with an end date.
Review widget placement without first-paint spam when two tools disagree. Pick a source of truth in advance so incidents do not burn the rollback window debating dashboards.
Validate containment and CSAT monitoring before paid traffic or customer emails amplify mistakes. Capture request IDs or screenshots for support if anything looks off.
Schedule a follow-up on staging red-teams for jailbreaks after launch, update the runbook when reality differs from the doc, and assign a named owner for the first hour.
When you finish this guide, you should be able to explain the happy path, the abort path, and the metrics that prove success. If any of those are fuzzy, revisit the walkthrough with your teammates before you scale traffic, spend, or automation.