PointFive Labs
Open, reproducible research on how AI infrastructure spend is actually incurred, with the benchmarks and data behind the findings.
The people doing the measuring.
PointFive Labs is the research arm of PointFive, led by Amir Hozez and Sarel Weinberger.
Amir Hozez
Co-Head of PointFive Labs
Amir is co-founder and CTO of PointFive. Before starting the company he was VP of R&D at IntSights, which was acquired by Rapid7. He is a co-author of both Labs papers.
Sarel Weinberger
Co-Head of PointFive Labs
Sarel has founded several AI companies and led AI at PwC. He lectures at Bar-Ilan University. He is lead author of both Labs papers and maintains the open benchmarks behind them.
Reducing what a coding agent sees does not necessarily reduce what it costs to complete the task.
The study compares Claude Code configurations using provider-billed cost and task outcomes. It separates delivered tool-output tokens from cache traffic and the additional work an agent may do after compression.
In the tested RTK-ML configuration, delivered tool-output tokens fell 38.4% while pooled billed cost rose 6.8%. These results describe the tested setup, not every compression tool or workload.
The same task can lead to different reasoning, verification, and costs depending on how the agent is asked to work.
The research examines prompt wording, inference effort, and the harness that manages tools and context. These factors interact: more effort can help difficult tasks, but can also add cost without improving outcomes.
Evaluation needs to track task success and end-to-end cost together, with the model, task, and harness kept explicit.
Inspect the measurements behind the findings and reproduce the analysis.
The AI Efficiency Benchmark includes task manifests, retained campaign data, analysis scripts, and the methodology used to compare coding-agent configurations.
The benchmark records provider-reported cost and evaluates task success with deterministic checks. Its retained data can be re-analyzed without running a new paid campaign.