WebsiteBench
WebsiteBench is a benchmark that tests whether AI agents can rebuild a full website by exploring it in a browser, for researchers and developers.
What it is good for
- Comparing how well AI agents can recreate a website
- Testing a coding agent on full-stack website rebuilding tasks
- Measuring how closely an agent's site matches the original
At a glance
- Who it is for
- Setup difficulty
- Technical ยท Guided command-line setup
Level 4 of 5. You copy and paste commands into a terminal by following the instructions. Comfortable territory for vibe coders.
- Cost
- Free and open source
- Type
- Other
- Tasks
- Works with
- Harbor, Docker, Hugging Face, Codex
How to get started with WebsiteBench
- Install Docker on your machine.
- Install Harbor with
uv tool install harbor==0.25.0. - Set your model provider key, for example
export OPENAI_API_KEY='<key>'. - Run an agent on all tasks with
harbor run --dataset websitebench/WebsiteBench@v1plus your agent and model options. - Read each trial's score in
verifier/reward.txtinside the job folder.
From the project's own documentation. If a step does not match what you see, the official pages have the latest instructions.
Before you try it
- Requires Docker and the Harbor framework.
- Running an agent needs an API key for a model provider such as OpenAI.
- Reference websites are third-party content, provided for non-commercial research only.
- Task files include a canary GUID, so exclude them from training data.
Momentum
- Rank this week
- #69
- Best rank
- #69 week 46, 2026
- Stars gained last week
- +299
- GitHub stars
- 447
GitHub numbers as of Nov 16, 2026. Updated weekly.
Ranking history
Official links
Questions
- Is WebsiteBench free?
- Yes. It is free and open source (Apache-2.0 license).
- Do I need to know how to code to use WebsiteBench?
- You do not need to write code, but setup means copying and pasting commands into a terminal by following its instructions.
- What does WebsiteBench work with?
- According to its own documentation, it works with Harbor, Docker, Hugging Face, and Codex.
- How do I get started with WebsiteBench?
- 1. Install Docker on your machine. 2. Install Harbor with uv tool install harbor==0.25.0. 3. Set your model provider key, for example export OPENAI_API_KEY='<key>'. 4. Run an agent on all tasks with harbor run --dataset websitebench/WebsiteBench@v1 plus your agent and model options. 5. Read each trial's score in verifier/reward.txt inside the job folder.
- What should I know before trying WebsiteBench?
- Requires Docker and the Harbor framework. Running an agent needs an API key for a model provider such as OpenAI. Reference websites are third-party content, provided for non-commercial research only. Task files include a canary GUID, so exclude them from training data.
- What is WebsiteBench good for?
- Comparing how well AI agents can recreate a website, testing a coding agent on full-stack website rebuilding tasks, and measuring how closely an agent's site matches the original.
Similar agents to WebsiteBench
- Co-Design-Bench
Tests revising existing web apps from screenshot requests instead of rebuilding whole sites from browser exploration.
- OpenSandbox
Provides isolated environments for agents to run code and browsers, not a benchmark with scored tasks.
- RSI-Anything
A research framework for helping agents learn from feedback, not a fixed set of website tasks.
- Playwright MCP
Lets AI apps control a browser for everyday tasks rather than evaluating agents.
- taste-skill
Guides a coding agent toward better landing page design instead of measuring agents on rebuilding sites.
- AgenticSeek
AgenticSeek is a self-hosted agent app that browses the web, writes code, and plans tasks for people who want to keep their data on their own machine.