WildProp: Visual Estimation of Wildlife Body Proportions at Scale

[Website] [arXiv] [demo]

Figure 1: Wildlife body proportions

Population-level morphometric measurements underpin ecological and evolutionary studies but traditionally require controlled imaging or physical specimen handling, limiting scalability. We present WildProp, a training-free framework that estimates wildlife body proportion distributions directly from large-scale, unconstrained image repositories. We cast morphometric estimation as a retrieval-driven correspondence problem: given a single user-annotated canonical image, WildProp performs pose-aware retrieval using foundation model features, transfers part endpoints via dense patch-level matching, filters predictions using geometric consistency, and aggregates measurements across retrieved images to estimate population-level ratio distributions. Unlike supervised keypoint pipelines, our approach adapts to arbitrary species and user-defined parts without per-species training. Evaluations on three large morphometric datasets spanning birds and amphibians show median relative errors of 10-20%. We further highlight the broad applicability of our approach through a number of case studies measuring various proportions across diverse taxa, including birds, frogs, insects, and flowers. Ablations demonstrate that pose-aware retrieval is critical for stable estimation, while robust aggregation mitigates keypoint and pose noise. Our results indicate that carefully curated 2D correspondences over web-scale imagery can provide scalable morphometric proxies for comparative and subgroup analyses across taxa, geography, and seasonality.

Installation

All steps run in a single Python 3.10 environment. On the cluster:

conda create -n wildprop python=3.10 -y
conda activate wildprop
pip install -r requirements.txt

requirements.txt lists every package actually imported by this code, including lang-sam (which brings in its own Grounding DINO / SAM2 stack) for the segmentation step. Install a CUDA build of torch/torchvision matching your machine (see the comment in requirements.txt) before installing the rest.

DINOv3

Steps 3 (get_dino_feats.py) and 5 (keypoint_matching.py) load DINOv3 from a local checkout via torch.hub.load(..., source='local'), since the released weights require accepting Meta's license:

git clone https://github.com/facebookresearch/dinov3
pip install -r dinov3/requirements.txt

Download the ViT-B/16 pretrained weights (dinov3_vitb16_pretrain_lvd1689m*.pth) from the official DINOv3 repository (license-gated) and place them at model_weights/dinov3_vitb16_pretrain_lvd1689m.pth, or pass an explicit path via --dinov3_weights to get_dino_feats.py / keypoint_matching.py. The repo location can likewise be overridden with --dinov3_repo_dir.

Physical measurements

get_stats.py compares predicted part proportions against external morphological measurement datasets. Download them from this Google Drive folder and place them under a data/ folder in the repo root:

• data/avonet.csv (used by --dataset_name avonet)
• data/frogs.csv (used by --dataset_name frogs)
• data/shore.csv (used by --dataset_name shore)

If a ground-truth file is not available, get_stats.py will still run and report predicted statistics, but with GT values of 0 (large reported error).

Reproducing quantitative experiments

The set of commands below process a set of species through the full pipeline. Set a working directory to store all intermediate/output data:

ROOT_DIR=/path/to/evals/<name>  

Activate the environment once, before running any of the steps below:

conda activate wildprop

1. Download images (per species)

python3 download_inat.py --species "$SPECIES_NAME" --root_dir "$ROOT_DIR" --threads 16

This script sleeps --sleep_seconds (default 2) between iNaturalist API calls to respect their rate limits. If you run downloads for multiple species concurrently, increase --sleep_seconds so the combined request rate across all concurrent jobs still stays within iNaturalist's limits.

2. Segment subject with LangSAM (per species)

python3 get_sam_masks.py --species "$SPECIES_NAME" --root_dir "$ROOT_DIR" --text_prompt "$TEXT_PROMPT"

The text prompt can be simple species description like "bird" or "frog" or "butterfly".

3. Extract DINOv3 features (per species, both feature types)

python3 get_dino_feats.py --species "$SPECIES_NAME" --root_dir "$ROOT_DIR" --batch_size 128 \
    --image_dir sam_masks/masked_images --dino_type patch_concat --feature_name sam_dino_patch_concat

4. Pose retrieval based on canonical images (per species)

python3 pose_retrieval.py --species "$SPECIES_NAME" --root_dir "$ROOT_DIR" --feature_name sam_dino_patch_concat \
    --image_dir sam_masks/blur_images --dims_path sam_masks/img_dims.json --num_retrievals 100

5. Keypoint Matching (per species, per feature)

Keypoints are transferred with keypoint_matching.py, which re-estimates each keypoint's query feature over several rounds from the top-k best matches seen so far. Its --re_estimate_iter, --re_estimate_topk, --topk_selection, and --kp_aggregation defaults are already set to the final hyperparameters used for all three examples (3 re-estimation iterations, top-20 retrieval-selected matches, equal_weighted_avg aggregation), so they don't need to be passed explicitly:

EXP_NAME="equal_weighted_avg_iter_3_retrieval_top20"

python3 keypoint_matching.py --species "$SPECIES_NAME" --root_dir "$ROOT_DIR" \
    --feature_name sam_dino_patch_concat \
    --results_csv_path "./results_final_hyper/${DATASET_NAME}/sam_dino_patch_concat/${EXP_NAME}/results.csv" \
    --annotations_path annotations/annotations_final.ndjson \
    --image_dir sam_masks/blur_images \
    --dataset_name "$DATASET_NAME" \
    --ransac_repeated_parts \
    --save_dir_name "dino_keypoints_iterative_final_hyp/${EXP_NAME}"

6. Compile part-proportion error statistics (per feature)

python3 get_stats.py \
    --preds_json_path "./results_final_hyper/${DATASET_NAME}/sam_dino_patch_concat/${EXP_NAME}/results_predictions.json" \
    --stat_type median \
    --results_csv_path "./results_final_hyper/compiled/sam_dino_patch_concat/${EXP_NAME}/${DATASET_NAME}.csv" \
    --dataset_name "$DATASET_NAME"

Species lists

Example	DATASET_NAME	TEXT_PROMPT	Species
frogs	frogs	frog	Agalychnis callidryas, Osteopilus septentrionalis, Espadarana prosoblepon, Anaxyrus woodhousii, Pseudacris regilla
shore	shore	bird	Calidris alpina, Tringa flavipes, Calidris mauri, Limosa lapponica, Arenaria interpres
avonet	avonet	bird	Elanus leucurus, Ardea alba, Cyanocitta cristata, Setophaga tigrina, Larus californicus