Skip to content

feat(collectors): add optional auth-backed collectors without new tools #22

Description

@endgegnerbert-tech

Summary

Add optional cookie/auth-backed collectors after the no-auth collector system and consolidated emet integration are stable.

This remains separate because it has higher maintenance, privacy, credential, and platform-policy risk.

Candidate collectors

  • Twitter/X via an explicitly installed local CLI or approved API path
  • Reddit via an explicitly installed local CLI or approved API path
  • Bilibili via yt-dlp where supported
  • Weibo via safe reader/API path if available
  • Douyin only through an explicitly configured local integration

Public surface

Do not add new MCP tools for each platform. Optional collectors should plug into the same internal collector registry from #18 and the same existing emet options flow from #19.

The agent-facing surface remains:

  1. emet
  2. web_fetch

Best-practice constraints

  • Read-only only.
  • No bundled credentials.
  • No OAuth flow in core unless separately designed and reviewed.
  • Cookie/CLI collectors must default to unavailable and include setup hints.
  • User consent and local configuration must be explicit.
  • Never send tokens/cookies to telemetry or logs.
  • Respect platform terms, rate limits, and removal/block behavior.

Acceptance

  • Optional collectors are invisible or clearly unavailable until configured.
  • Doctor output explains required local setup per collector.
  • Missing tools never crash MCP/Pi startup.
  • Tests cover unavailable state and one mocked configured collector.
  • tools/list does not gain platform-specific tools.

Out of scope

  • Posting/writing.
  • Headless browser automation.
  • Credential management UI.
  • Private account indexing.

Split from #17. Depends on #18 and #19.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions