This week after a two-year hiatus, Gartner published its 2019 Magic Quadrant for Data Center Backup and Recovery Solutions with a new lead analyst and supporting team. As many know, Gartner’s research can sway market perceptions–presumably based on the firm’s reputation for diligent research and, as Gartner notes, the firm’s “independence, objectivity and accuracy.”
While we continue to seek good relations with Gartner, to put it bluntly, it’s our opinion that the 2019 Backup and Recovery Solutions MQ is seriously flawed. We have engaged with the Gartner team over several months to remedy a number of significant issues and concerns to no avail, so we felt it was important that the market, including our customers, potential customers, partners and employees, have the full set of facts that are pertinent in objectively evaluating the information contained in this MQ.
Based on objective data, since the 2017 MQ, Rubrik has made significant progress in its business, out-paced all of its competitors, and has had a disproportionately large impact on the Data Center Backup and Recovery market. Yet, this progress has not manifested in any significant movement, as reflected in Rubrik’s position within the 2019 MQ.
Note: The data cited below is as of February 2017 and February 2019, when information was submitted to Gartner for the MQ.
Rubrik’s Ability to Execute Rating Ignores Our Dramatic Progress Over Two Years, Despite Rubrik’s 5x Growth, Comparable Size and Capabilities Vis-A-Vis the Other Leaders.
The 2017 Gartner Analyst Team positioned Rubrik in the lower right Visionary quadrant. This reflected a strong Completeness of Vision but lower Ability to Execute rating. The 2017 Analyst Team’s Ability to Execute rating reflected the fact that we had been shipping product for 18 months, with 250 employees. Two years later, Rubrik’s bookings and employee count have grown more than 5x, and are comparable to the data center businesses of leaders such as Commvault and Veeam. In fact, according to Gartner’s lead analyst in 2019, Rubrik showed up in 80% of his client Backup Inquiries, yet in the MQ he authored, he didn’t think we played a major role in leading and affecting the overall market. Over the past two years, Rubrik has added tech titans such as John Thompson, Enrique Salem, and John Chambers to its list of board members and advisors while Commvault, Veritas, and Dell have seen significant and well-publicized execution challenges at the leadership levels and their businesses generally over that same period of time.
Rubrik Made 17 Substantive Corrections of Fact to Gartner’s Writeup That Led to “No Material” changes in the Outcome.
Despite a comprehensive 30-page survey submission and 25 formal analyst inquiries over the preceding 12 months, Gartner failed to get many basic facts correct in the draft MQ and Critical Capabilities. In the draft summaries shared with vendors, Rubrik found 17 inaccuracies covering missing functionality, customer adoption, and deployability. In some cases, it was clear that the analysts confused us with a smaller competitor in their description of an OEM relationship and in multiple descriptions of how our technology works. Unfortunately, these corrections when submitted, had absolutely no impact on Gartner’s assessment of Rubrik or dot positioning. Gartner gave us no further assurances that all errors were ultimately corrected.
Gartner Employed an Inconsistent Standard to Rubrik in this MQ.
One argument put forward by Gartner was that Rubrik is still considered a new entrant and it is too early for Rubrik to be in the Leader Quadrant. We analyzed the stats of 10 New Leaders across adjacent markets such as enterprise storage, security and data warehousing, among others. These included companies like: Anaplan, Mulesoft, Nutanix, Palo Alto Networks, Pure Storage, Snowflake, Splunk, Skyhigh, Zoom and Zscaler, all of which made the Leader Quadrant for the first time with less than $200M in Revenue and ~1000 employees. Rubrik has exceeded these stats. In the 2014 MQ for Solid State Arrays, Pure Storage was recognized as a Leader with less than $50M in Revenue and less than 700 employees, placing higher than HP or NetApp.
Gartner Failed to Manage a Major Conflict of Interest and Bias at the Heart of the 2019 MQ Drafting.
Finally, and most critically for all Gartner’s clients, Rubrik discovered and raised to Gartner a clear analyst conflict of interest and bias against Rubrik that could significantly impact the results and positioning in the 2019 MQ. Rubrik raised this issue only when it was clear that the analyst had not disclosed his conflict of interest and that this issue was material to the MQ work product. It is well known that a Gartner analyst has significant discretion, influence, and authority over quadrant criteria, the weighting of criteria, vendor scoring, vendor analysis and narratives, and ultimately dot positioning. The background facts underlying this conflict of interest and bias follow.
In 2018, five Gartner Analysts covering Backup and Recovery separately sought to leave Gartner and approached Rubrik for employment. Of the five, Rubrik eventually made offers to only four analysts.
What happened to Analyst #5? After his sustained pursuit of a position with Rubrik, we declined to offer Analyst #5 a role, and he expressed his clear disappointment in light of his colleagues’ hirings at Rubrik. Approximately four months after communicating this displeasure, Analyst #5 would change teams and be named as a lead analyst on the MQ covering Rubrik. We expected that this conflict of interest would have been disclosed, as Gartner’s policy requires, and it was not.
During our process with the Ombudsman, Gartner did not acknowledge the conflict of interest and bias issue and continued to justify their findings with new arguments, many of which were supplied by Analyst #5 and justified by Gartner’s Peer Review Process. When the conflict of interest was clear to Gartner, a second full review was conducted over a three week period, which concluded with a product that “provides no material change” and is now the current 2019 MQ. We are disappointed that Gartner’s proposed resolution was to rely upon a short three week review performed by a new analyst relying upon Analyst #5’s work, analysis, and data. This is in sharp contrast with the typical 4-5 month research process led by dedicated analysts with deep firsthand experience with customers and vendors in our industry. We also question the adequacy of Gartner’s analyst peer review considering the attrition since 2017 of 10 experienced analysts with over 100 years of experience in Backup and Recovery, Archiving, and Storage.
Could an experienced and objective third party come to a different conclusion? Naveen Chhabra, an Analyst with 20 years experience at HPE and Forrester, recently published the Forrester Wave for Data Resiliency Solutions, placing Rubrik in the Leader Category with the highest possible score for strategy. We now know the answer to that question.