Webhooks & Message Ingest

Dolly receives customer messages via webhooks from Pancake (your ecom messaging platform). This section explains how to configure and validate webhooks.

Pancake Webhook Integration

When a customer sends a message on Pancake, we receive it via webhook and process it through our pipeline.

Webhook Endpoint

POST https://api.dolly.shin0x.space/webhooks/pancake/:tenantId

Request Headers

X-Pancake-Signature: sha256=<hmac_digest>
X-Pancake-Timestamp: 1678951234567
Content-Type: application/json

Request Body

{
  "message_id": "pancake_msg_12345",
  "conversation_id": "conv_12345",
  "customer": {
    "id": "cust_12345",
    "name": "Nguyen Van A",
    "phone": "+84912345678",
    "email": "customer@example.com"
  },
  "content": "Mình muốn kiểm tra đơn hàng YD-20260312-001",
  "type": "text",
  "timestamp": 1678951234567
}

Webhook Response

{
  "success": true,
  "messageId": "pancake_msg_12345",
  "status": "queued"
}

HTTP 200 must be returned within 5 seconds, or Pancake will retry.

---

Signature Validation

Pancake signs all webhooks with HMAC-SHA256. Always validate the signature:

JavaScript Example

const crypto = require('crypto');
const express = require('express');

// Middleware to validate Pancake signature
function validatePancakeSignature(req, res, next) {
  const signature = req.headers['x-pancake-signature'];
  const secret = process.env.PANCAKE_WEBHOOK_SECRET;
  
  // Get raw body (express.text() middleware required)
  const body = req.rawBody || JSON.stringify(req.body);
  
  const hash = crypto
    .createHmac('sha256', secret)
    .update(body)
    .digest('hex');
  
  const expectedSignature = `sha256=${hash}`;
  
  if (!crypto.timingSafeEqual(signature, expectedSignature)) {
    return res.status(401).json({ error: 'Invalid signature' });
  }
  
  next();
}

app.use(express.raw({ type: 'application/json' }));
app.use(validatePancakeSignature);

app.post('/webhooks/pancake/:tenantId', (req, res) => {
  // Process webhook
  res.json({ success: true });
});

curl Example

BODY='{"message_id":"123","content":"Hello"}'
SECRET='your_pancake_webhook_secret'
SIGNATURE=$(echo -n "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)

curl -X POST https://api.dolly.shin0x.space/webhooks/pancake/tenant_id \
  -H "X-Pancake-Signature: sha256=$SIGNATURE" \
  -H "Content-Type: application/json" \
  -d "$BODY"

---

Supported Message Types

Type	Content	Example
text	Plain text message	"Mình muốn đổi hàng"
image	Image with optional caption	Photo of defective item
video	Video file	Product demo video
audio	Voice message	Customer voice note
file	Document (PDF, etc.)	Invoice PDF
sticker	Emoji/reaction	😊, ❤️

---

Message Processing Pipeline

1. INGEST QUEUE
   ├── Dedup by message_id (prevent double-processing)
   ├── Media detection (image/video/audio → preprocessing)
   ├── Chunk buffering (2.5s window for rapid-fire messages)
   └── Enqueue to PROCESS
   
2. PROCESS QUEUE
   ├── Load customer memory (4 layers)
   ├── Load relevant KB (RAG)
   ├── Load persona config
   ├── Call Haiku LLM
   ├── Run 5 gates (validate response quality)
   └── Enqueue to DELIVER
   
3. DELIVER QUEUE
   ├── Typing delay simulation
   ├── Send via Pancake API
   ├── Log to database
   ├── Extract episode (async)
   └── Update customer memory (async)

Total latency: ~2-5 seconds from webhook receipt to reply delivered (varies by LLM response time).

---

Media Handling

Image Messages

{
  "type": "image",
  "content": "Áo bị rách",
  "mediaUrl": "https://files.pancake.vn/image_12345.jpg",
  "mediaType": "image/jpeg"
}

Processing:

1. Download from Pancake URL
2. Upload to Minio (temporary storage, 1h TTL)
3. Call Haiku vision API
4. Extract structured description
5. Inject into prompt as context

The AI sees: "[Image: defective polo shirt with tear in seam, customer's concern about quality]"

Video Messages

{
  "type": "video",
  "content": null,
  "mediaUrl": "https://files.pancake.vn/video_12345.mp4",
  "mediaType": "video/mp4"
}

Processing:

1. Download video
2. Extract 3-5 keyframes using FFmpeg
3. Send each frame to Haiku vision
4. Combine descriptions
5. Inject into prompt

Audio Messages

{
  "type": "audio",
  "mediaUrl": "https://files.pancake.vn/audio_12345.wav",
  "mediaType": "audio/wav"
}

Processing:

1. Download audio
2. Transcribe using Faster-Whisper (local CPU, no API)
3. Inject transcribed text into prompt

Language: Auto-detected from audio content.

---

Error Cases

Invalid Signature

HTTP 401
{
  "error": {
    "code": "INVALID_SIGNATURE",
    "message": "Webhook signature validation failed"
  }
}

Missing Tenant ID

HTTP 404
{
  "error": {
    "code": "NOT_FOUND",
    "message": "Tenant not found"
  }
}

Rate Limit

HTTP 429
{
  "error": {
    "code": "RATE_LIMIT",
    "message": "Too many messages — back off with exponential delay"
  }
}

---

Retry Policy

If your webhook doesn't receive HTTP 200 within 5 seconds:

• Retry 1: Immediate
• Retry 2: 5 seconds
• Retry 3: 30 seconds
• Retry 4: 5 minutes

After 4 retries, Pancake stops sending.

Best practice: Return 200 immediately, process asynchronously.

---

Deduplication

Dolly deduplicates by message_id within 24 hours. If the same message_id is sent twice, only the first is processed.

Use your Pancake message_id or generate a stable UUID:

const messageId = pancakeMessage.id || 
  `${tenantId}-${customerId}-${timestamp}-${content.hash}`;

---

Testing Your Webhook

Using curl

curl -X POST \
  https://api.dolly.shin0x.space/webhooks/pancake/your_tenant_id \
  -H "Content-Type: application/json" \
  -H "X-Pancake-Signature: sha256=test_signature" \
  -d '{
    "message_id": "test_123",
    "conversation_id": "conv_123",
    "customer": {
      "id": "cust_123",
      "name": "Test Customer",
      "phone": "+84912345678"
    },
    "content": "Hello",
    "type": "text",
    "timestamp": 1678951234567
  }'

Using Postman

1. Set method to POST
2. URL: https://api.dolly.shin0x.space/webhooks/pancake/:tenantId
3. Headers: X-Pancake-Signature: sha256=...
4. Body (raw JSON): paste the payload above
5. Send

---

Logs & Debugging

Check your conversation logs in the dashboard for:

• Message receipt timestamp
• Processing queue latency
• Gate results (which gates passed/failed)
• Final response sent

See Conversations API for full message audit trails.