Shopping Plugin Testing at VPT
Last week Ben Edelman, VPT’s Chief Scientist, posted a remarkable write-up about misconduct by Honey. Many people won’t be surprised that Honey is ignoring stand-down rules. But evading testers – requiring a minimum account age and minimum points to ignore stand-down – that is big news. Worst of all is Honey checking for the cookies that show a user has logged into an affiliate network dashboard. That is a clear attempt to hide from industry experts.
In light of these revelations, both merchants and networks should be asking how they missed it. VPT has been tracking Honey stand-down violations since 2021, and our archives of prior observations include 401 Honey violations as to 119 merchants on a dozen-plus networks. We tested in both low-points and high-points conditions (based on Ben’s finding that Honey behaves differently when a user has many points). Our test systems operate around-the-clock, fully automated, without a human ever needing to boot a test PC or supervise a run. We test from locations on three continents and provide full-motion videos, packet logs, and automated PDF reports from a self-service dashboard.
But Honey is far from the only problem. Other shopping plugins break the rules too, so VPT tests thousands of plugins – some widely known, others decidedly obscure. In 2025, we observed stand-down violations from 36 different shopping plugins, and we preserved a total of 1,930 observations of stand-down violations. We saw more violations from Honey than any other single shopping plugin, but our top-violators list includes a variety of well-known names.
Reviewing historic incidents makes me particularly proud of the dashboard improvements VPT deployed this year. Our new dashboard lets users sort and filter violations across numerous dimensions and group by publisher and subid to see which problems are new or unresolved. Each incident includes an “explainer”-style report that summaries the evidence and the reason for concern – helping nonspecialists understand what went wrong and what to do next.
We built VPT’s automation to detect exactly the problems in Ben’s research and MegaLag’s recent analysis of Honey. We’re proud that we can catch these problems efficiently and at scale. Fair treatment for publishers and merchants requires continuous testing and enforcement. The recent Honey debacles show that trust alone is not enough. Get in touch to request a demo.
