Internal Docs Co-Pilot

The Problem

A logistics company had 8,000+ pages of Confluence documentation — SOPs, carrier guides, rate sheets, escalation procedures. New hires spent their first month just learning where things were. Experienced staff interrupted each other constantly with “do you know which doc has X?” questions.

The knowledge existed. Finding it took forever.

What I Built

A Slack slash command (/ask [question]) that queries the full Confluence knowledge base and returns a cited answer in under 15 seconds:

/ask what's the escalation procedure for late LTL shipments over $10k?
→ Based on the Carrier Relations SOP (updated Oct 2024): For LTL shipments 
  over $10,000 delayed by 48+ hours, escalate to the regional carrier manager 
  via the priority queue in CarrierOps. Notify the shipper proactively within 
  2 hours... [source: Carrier Relations SOP §4.2]

Architecture

Ingestion pipeline (runs nightly):

• Confluence API → fetch all pages updated in last 24h
• Clean HTML → markdown (strip navigation, boilerplate)
• Chunk with 200-token overlap (SOPs need context across paragraphs)
• Embed via text-embedding-3-small → upsert to Chroma

Query pipeline:

• Slack event → FastAPI webhook
• Hybrid retrieval (BM25 + vector) → top-8 chunks
• GPT-4o-mini generation with strict citation prompt
• Format as Slack blocks with source links
• Post as ephemeral message (only visible to requester)

Why Ephemeral Messages?

Privacy and noise. People ask sensitive questions about clients or internal processes. Posting answers to the channel would create unnecessary noise and potential information leakage. Ephemeral responses feel like a private assistant.

Results

• ~4 hours saved per person per week (self-reported in 30-person survey)
• New hire ramp time reduced from ~1 month to ~2 weeks
• 90-day retention rate of 78% of Slack active users (they kept using it)
• Zero hallucinations reported by staff on process-critical questions (citation-anchored responses are hard to hallucinate)