Add vlmbench-cli-skill with SKILL.md and MODELS.md

[devin-ai-integration]

Summary

Copies the vlmbench CLI skill files from vlm-run/vlmbench/.claude/skills/vlmbench/ into a new skills/vlmbench-cli-skill/ directory in this repo.

Files added:

• skills/vlmbench-cli-skill/SKILL.md — CLI usage, backend resolution, concurrency, metrics, GPU detection
• skills/vlmbench-cli-skill/MODELS.md — Supported models table with vLLM serve-args and model profiles

Updates since last revision

• Updated MODELS.md to use uvx vlmbench instead of bare vlmbench in CLI examples (per review feedback). This means the files are no longer exact verbatim copies from the source repo — SKILL.md already used uvx vlmbench, but MODELS.md had two bare vlmbench invocations that have been corrected.

Review & Testing Checklist for Human

• Filename check: The source repo has SKILL.md (singular), not SKILLS.md as mentioned in the original request. Verify this is the intended name, or rename to SKILLS.md if preferred.
• Confirm the content of both files is up to date with the latest version in vlm-run/vlmbench (files were copied at the time of this PR and may drift).

Notes

• Requested by: @spillai
• Devin Session

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[spillai]

Use uvx vlmbench instead of vlmbench across all the README

[devin-ai-integration]

Updated — replaced vlmbench with uvx vlmbench in the CLI examples in MODELS.md (lines 6 and 25). SKILL.md already used uvx vlmbench throughout.

[spillai]

Closing as superseded by #13, which adds the same skill at the same path (skills/vlmbench-cli-skill/). Both PRs create SKILL.md there, so only one can land.

Carried over from this PR: the model → --serve-args table, which was the one thing #13 lacked. It now lives inline in SKILL.md rather than in a separate MODELS.md, keeping the skill one self-contained file.

Two things in this PR's copy were stale against vlm-run/vlmbench@main and were corrected on the way over:

	This PR says	vlmbench main
Profile storage	directories vlmbench/profiles/<name>/ with config.toml + setup.sh	flat vlmbench/profiles/<name>.yaml; no config.toml or setup.sh exists
Profile inventory	4 profiles	10 profiles ship in the package

#13 points at uvx vlmbench profiles for the live list instead of a static table that drifts. The make build/serve/benchmark block was dropped, since those targets need a clone of the vlmbench repo and do not apply to the uvx workflow the skill documents.

Also in #13 and not here: registration in .claude-plugin/marketplace.json, without which /plugin install vlmbench-cli-skill@vlm-run/skills does not resolve; a row in the README skills table; and a frontmatter name matching the directory. Its body is written for the current CLI, which this snapshot predates: results now default to ~/.vlmbench/benchmarks/ rather than ./results/, --model is optional when --base-url is set, --serve must be passed explicitly to auto-start a server, and text-only dataset runs are supported.

The uvx vlmbench review note on this PR from @spillai is satisfied in #13, where every example uses uvx.

---

Generated by Claude Code

[devin-ai-integration]

❌ Cannot revive Devin session - the session is too old. Please start a new session instead.

View session