PHP Bot Detect

Author: Jarosław Szulc https://psyll.com/jarek / Psyll.com https://psyll.com

A lightweight, standalone PHP library (no external dependencies) for detecting bots, crawlers, and automated HTTP traffic. Requires no database or Composer packages - just a single file.

Detects bots based on several independent signals:

• User-Agent signatures - over 400 known bots: search engines (Google, Bing, Yandex, Baidu, DuckDuckGo...), social media bots (Facebook, Twitter/X, LinkedIn, Discord...), security tools (nmap, sqlmap, nuclei...), HTTP libraries (curl, axios, python-requests...), headless browsers (Puppeteer, Selenium, Playwright...), and a large list of scrapers/crawlers.
• Reverse-DNS verification - checks whether a bot claiming to be e.g. Googlebot is actually connecting from Google's network (protection against spoofing known bots via a fake User-Agent).
• HTTP header analysis - missing or inconsistent headers (Accept, Accept-Language, Sec-Fetch-*, Sec-CH-UA, etc.) typical of automated traffic.
• Data center IP ranges - recognizes traffic from major cloud/VPS providers (AWS, GCP, Azure, DigitalOcean, OVH, Hetzner, and others).
• Behavioral analysis - detects consecutive requests occurring faster than is physically possible for a human.
• Honeypot - support for a hidden form field that should remain empty for real users.

Requirements

• PHP 8.1 or newer
• A writable directory on disk (for logs and cache) - optional, see below

Installation

No Composer needed - download the BotDetect.php file and include it in your project:

require_once 'BotDetect.php';

Quick start

<?php
require_once 'BotDetect.php';

if (BotDetect::detect()) {
    // this request was classified as a bot
    http_response_code(403);
    exit;
}

BotDetect::detect() reads the current request ($_SERVER, HTTP headers) on its own and returns true/false.

Configuring the logs and cache directory

By default, the library stores logs of detected bots and cache (DNS verification, behavioral trace) in the system's temporary directory (sys_get_temp_dir() . '/bot-detect'). To point it to your own writable directory:

BotDetect::setBaseDirectory(__DIR__ . '/storage/bot-detect');

Call this once, at application startup, before the first use of detect().

If logging and behavioral analysis are disabled (see below), this directory is not created or used at all.

detect() parameters

BotDetect::detect(
    bool $logs = false,          // log detected bots to a CSV file (daily log)
    bool $checkHoneypot = true,  // check the honeypot field in $_POST
    bool $checkDns = true,       // verify known bots (Googlebot, etc.) via reverse-DNS
    bool $checkBehavior = true   // analyze request frequency (requires per-IP disk writes)
): bool

Example - logging enabled, DNS verification disabled (e.g. on very high-traffic sites, where the cost of DNS queries would be too high):

$isBot = BotDetect::detect(logs: true, checkDns: false);

Honeypot in forms

Add a hidden field to your form with the name defined in BotDetect::HONEYPOT_FIELD_NAME:

<input type="text"
       name="<?= BotDetect::HONEYPOT_FIELD_NAME ?>"
       value=""
       autocomplete="off"
       tabindex="-1"
       style="position:absolute; left:-9999px;">

Note: hide the field using position:absolute; left:-9999px, not display:none - some simpler bots skip fields hidden the latter way.

Checking the honeypot separately (e.g. without the full analysis):

if (BotDetect::honeypotTriggered()) {
    // the field was filled in - almost certainly a bot
}

Categories of detected bots

detect() returns only a bool. If you need details (category, confidence level, matched signature), enable logging (logs: true) and read the entry from the CSV file, or use the class as a starting point for your own method that returns a full analysis result.

Available categories (public constants):

Constant	Meaning
CATEGORY_SEARCH_ENGINE	Search engine (Googlebot, Bingbot, etc.)
CATEGORY_SOCIAL_MEDIA	Social media bot / link preview
CATEGORY_SECURITY_TOOL	Security scanning tool
CATEGORY_HTTP_LIBRARY	HTTP library (curl, axios, requests...)
CATEGORY_HEADLESS_BROWSER	Headless browser (Puppeteer, Selenium...)
CATEGORY_CLOUD_HOSTING	Cloud hosting signature in the UA
CATEGORY_SCRAPER	Other known scraper/crawler
CATEGORY_UNKNOWN_BOT	Unknown bot detected heuristically (headers/IP/behavior)
CATEGORY_SPOOFED_BOT	UA impersonates a known bot but failed DNS verification
CATEGORY_HONEYPOT_TRIGGER	Hidden form field was filled in
CATEGORY_BEHAVIORAL	Detected based on request frequency analysis

Log format

When logs: true, entries are written in CSV format, one file per day (YYYY-MM-DD.csv), with the columns:

timestamp, client_ip, flag, category, score, confidence, verified, matched_signature, user_agent, request_uri, request_method, referer, signals

How it works (order of checks)

1. Honeypot - if filled in, an immediate positive result (100% confidence).
2. User-Agent signature - matched against a database of known bots. For search engines, optionally verified via reverse-DNS (protection against spoofing).
3. Header analysis - scoring based on missing/inconsistent headers.
4. IP in a data center range - additional points if the request comes from a known hosting network.
5. Behavioral analysis - additional points for requests that are too frequent from the same IP.

The final score is the weighted sum of the points for the signals above. If it exceeds the threshold (0.7 by default), the request is classified as a bot.

Performance and scaling

• User-Agent signature and header checks are in-memory and very fast.
• Reverse-DNS verification performs real DNS queries (gethostbyaddr/gethostbynamel), which are cached on disk (DNS_VERIFY_CACHE_TTL, 24h by default). Under very high traffic, consider disabling it (checkDns: false) or keeping the cache on faster storage (tmpfs).
• Behavioral analysis writes a small JSON file per unique IP. Under very high traffic, consider disabling it (checkBehavior: false) or periodically clearing the directory.

License

MIT - see LICENSE.

Notes

The list of bot signatures and data center IP ranges is maintained manually and is not exhaustive - new bots appear regularly. Pull requests with list updates are welcome.

The library is not a replacement for full-fledged WAF/CAPTCHA solutions to protect against advanced, targeted abuse - it is a heuristic, standalone tool for traffic classification.