Gmail Labeler

Governance & Responsible AI

Transparency, privacy impact assessment, and failure mode analysis for the Gmail Labeler skill.

Privacy Impact Assessment

What the Skill Reads

  • Sender address: Full email address for domain extraction and classification
  • Subject line: Used by AI-based classification (optional BYOK feature only)
  • Message snippet: First ~100 characters of body text (AI-based classification only)
  • Date/time: For sorting and filtering recent mail

What It Never Reads

  • Attachment contents
  • Full message body (only snippet for AI classification)
  • Email addresses in CC/BCC fields
  • Calendar events or other Gmail data

Where Data Goes

Rule-Based Classification (Default)

  • Processing: Entirely local - sender domain matched against predefined rules
  • External calls: None
  • Data retention: None - stateless classification

AI-Based Classification (BYOK Optional Feature)

  • Processing: Sender, subject, snippet sent to LLM provider (Anthropic or OpenAI)
  • Your API key: Never sent to our servers - calls go directly from browser to provider
  • Provider retention: Check your provider's data policy (Anthropic: zero data retention available; OpenAI: 30 days default)
  • Audit log: Stored in your browser's IndexedDB - never sent to us

Risk Mitigation

  1. Dry-run first: Preview all actions before applying - catch errors before they affect your inbox
  2. Test account recommended: Try on a non-primary Gmail account first
  3. No hosted backend: This showcase has no server - everything runs in your browser
  4. Audit trail: All AI calls logged locally for review
  5. Opt-in AI: Rule-based classification works without any AI provider

Rule Coverage Analysis

Statistical analysis of rule performance across 100 synthetic threads. All metrics reported with 95% confidence intervals.

View Interactive Coverage Report

Known Failure Modes

Edge cases where the rule system performs poorly, with proposed improvements and mitigation strategies.

1.Domain Typos and Misspellings

Issue: Typosquatting domains (e.g., arnazon.com instead of amazon.com) bypass rules.

Impact: Phishing attempts may be mislabeled as legitimate; misspelled addresses go unlabeled.

Current mitigation: None - exact domain match only.

Proposed improvement: Fuzzy domain matching with Levenshtein distance; flag domains with distance < 2 from known providers for manual review.

2.Subdomain Handling

Issue: noreply.amazon.com doesn't match amazon.com rule unless explicitly defined.

Impact: 10-15% of threads from known providers go unlabeled due to subdomain variation.

Current mitigation: Manual rules for common subdomains (noreply.*, notification.*).

Proposed improvement: Automatic subdomain inheritance - match *.example.com when example.com has a rule.

3.Role Addresses

Issue: Generic role addresses (noreply@..., notifications@...) from legitimate but unlisted domains.

Impact: Important transactional mail (password resets, order confirmations) from small providers goes unlabeled.

Current mitigation: None - role address is ignored, only domain is checked.

Proposed improvement: Heuristic: if sender is noreply@X and X is unlisted, apply generic label "Transactional/Unlisted" with keep action.

4.Multi-Sender Threads

Issue: Gmail threads combine multiple senders - classification uses only the first sender.

Impact: Replies in a thread from different providers may have wrong label.

Current mitigation: None - single-sender assumption.

Proposed improvement: Detect multi-sender threads, skip labeling (or label as "Mixed/Conversation").

5.Content Type Ambiguity

Issue: Same sender sends both receipts (keep) and marketing (archive) - action hint may be wrong.

Impact: Amazon receipts archived as marketing, or vice versa.

Current mitigation: Default action is "keep" for ambiguous providers (errs on side of safety).

Proposed improvement: Subject line analysis - "Your Amazon.com order" → keep, "Prime Day deals" → archive.

6.Internationalization

Issue: Rules are AU/US-centric - EU/APAC/LATAM providers missing.

Impact: Low coverage for non-English-speaking users (estimated 30-50% unlabeled).

Current mitigation: Users can add custom rules manually.

Proposed improvement: Multi-region rule packs - user selects region(s), loads appropriate provider list.

Open Issues Log

Tracked issues for future improvements (not blocking current functionality):

  • #1: Add subdomain inheritance logic (DR-001 "What I'd Change")
  • #2: Expand non-US/AU provider coverage (EU banks, APAC e-commerce)
  • #3: Implement fuzzy domain matching for typo detection
  • #4: Add subject line content-type classifier (receipt vs marketing)
  • #5: Multi-sender thread detection and handling
  • #6: Confidence scoring for classifications (exact vs fuzzy vs no-match)

See Methods page for full "What I'd Change" discussion.

AI Features (BYOK Optional)

When enabled with your own API key, AI features add:

  • Custom rule suggestions: "Here are 10 sender addresses from my inbox" → AI generates provider type and rules
  • Subject-based refinement: Distinguish receipts from marketing for same sender
  • Coverage predictions: Estimate how many threads would be labeled before running

All AI outputs are:

  • Visibly labeled "AI-generated"
  • Logged to audit trail (viewable at /ai-log)
  • Subject to human review before application

See Methods → AI Use Statement for full transparency policy.