Introducing Provisional Reviews on Polygon
Video games don't release the way they used to.
When I proposed review updates in January of 2012 to the other founding editors of Polygon, before we were actually Polygon at all, they were an acknowledgement of the changing lifespans of games, of games as service and platforms that evolve over time. We talked about titles like Team Fortress 2, of WoW, and League of Legends, of steady DLC releases and the result of dwindling multiplayer populations, about how the experience and recommendation of a game could change over time. In hindsight it was somewhat utopian, thinking mostly about the good things that could happen to a game, or at worst, the slow twilight of multiplayer-oriented titles.
But to be honest, the first time we tested our system, it wasn't for a game that had been out for a while, it was for a game that launched and subsequently collapsed. And while we've updated a few games to reflect improved experiences and better versions on other platforms, the overwhelming majority of our score updates have hinged on game-breaking bugs and performance issues revealed after launch, and worse, games that simply do not function online as advertised, or at all.
Whatever factors were preventing publishers and developers from setting their games loose upon consumers in an unfinished state have become less pressing, apparently. I'm not actually interested in calling any particular publisher or platform holder out here, as I don't think I have enough fingers to point at them all. The point is, simply, that it's becoming harder and harder to know, even on release day, if a game will function on day one, two, three or indefinitely.
I don't think this is going away. In fact, for the time being, I am absolutely positive it won't. It will be some time before publishers get the hint that this isn't ok, where they move beyond lip service about "making it right" and actually start doing the right thing and delaying games that aren't in a state fit to be sold. I don't know what it will take for this to happen. I don't know what the final straw will be for consumers to push back.
That said, I think there's more we can do to serve our audience and offer some modicum of caution and warning about games we have reservations about.
Moving forward, Polygon editorial staff will have the ability to flag the reviews of certain games with a "Provisional" tag. If a review is marked as Provisional, then we're advising caution on the part of our audience. It means we haven't tested it enough in real-world conditions to be sure it will function as-advertised. Reviews marked this way will not be listed on Metacritic until the flag is removed. This flag should last only for a short window of time from a game's official release, until such time as we can make an informed judgment about its launch state.
We feel that this arrangement is to the benefit of the Polygon audience as well as Metacritic. Metacritic's stated policy is to only list one score for a game per outlet. This provisional review model is not only in Metacritic's best interest but also that of Polygon's audience. We want our scores aggregated when we're sure of our assessment of the "1.0" release version's actual state in the launch window.
I hope that we'll use this sparingly at first, but we'll evaluate if it's appropriate more often on a case by case basis. The first review to publish with a provisional tag will be Sunday night at 9PM PT with our review of Evolve.