In the world according to Gomez.com, a self-proclaimed consumer services analyst, their surveys of vertical categories like cars, toys, and insurance buying are supposed to help consumers evaluate the best sites on the Internet. A review of the eyebrow-raising results of their latest "Home Buying Scorecard" leads one to question whether the surveys are more geared to expanding Gomez' subscription/consultation customer base and punishing dissenters.
Gomez.com calls itself "the e-commerce authority for both consumers and e-businesses." The company creates broad consumer categories, reviews online businesses within those categories, compares them, and then publishes a "Scorecard" every quarter which is supposed to be an unbiased tabulation of the results derived from company interviews, consumer input and mystery shopping techniques. But the methodology is so secret that even consumers don't know what criteria they should be using to judge a category. Gomez.com's founder Julio Gomez has gone on the record numerous times saying he guards the site's evaluation criteria and methodologies to avoid giving his business plan to kids to rip off. Yet others in the business-ranking sector, including Bizrate and Forrester Research, share no such inhibition. Is the secrecy to hide Gomez.com's continuously changing criteria which allows new subscribers to enter the playing field and to keep the paying customers off-balance just enough to renew their consultation subscriptions?
This quarter's survey is a mess, a grab bag of business-to-consumer and business-to-business sites, pitting publicly-traded portals against tiny start-up discount e-brokers. The site also seems confused by its own grading criteria, nit-picking over the inclusion of 800 numbers for some sites while overlooking glaring feature omissions such as listings.
Some site officials wonder how they can ever win with Gomez. Even paying them isn't necessarily the answer.
Expertise in the home buying category?
Under the guise of encouraging online businesses to improve their services to consumers, Gomez.com issues Scorecards in more than 37 categories, adding eight new categories since spring. According to two former employees who do not wish to give their names, the scorecards are produced under a one-week deadline by a senior analyst and a research analyst, a pressure-cooker of a schedule that causes the analysts to fall way short of researching the 195 criteria points customers are promised. The pressure can and does produce serious errors which are not only kept from consumers, but go uncorrected by the analysts.
Just ask HomeSeekers.com. They've had an ongoing battle with Gomez.com since the site produced its first home buying scorecard in January. The evaluation was so full of errors said Greg Costley, Homeseekers' CEO, that they were compelled to write the company a stern letter.
Gomez.com, it seems, had mispublished the number of Homeseekers listings, allowing them only about 400,000 when the company claims it had over 800,000. The correction of this omission alone presumably would have placed Homeseekers much higher in the rankings. The company apologized but refused to change the number on its site or change Homeseekers' ranking.
Deciding to take the higher road, Homeseekers agreed to pay Gomez.com $35,000 in consultation fees to learn how it could improve its evaluation. When the next (Spring) home buying scorecard was released, Homeseekers did move up one place from number 10 to number nine. The number of listings was acknowledged at 850,000, but Homeseekers wondered why there wasn't more improvement.
The company was confounded to learn that after it was told to place an 800 number on the site and did so, says John Hensley, Homeseekers' product manager, that it was chastised for not placing it on every page. Wasn't that part of the advice they were paying for? Or, was the Gomez team covering its tracks by projecting the blame back onto its paying customer?
Hensley also claimed that out of 35 criteria that Gomez.com listed, there were six errors in the Homeseekers evaluation, again causing Homeseekers' failure to move ahead in the rankings. Homeseekers asked that the corrections be made before the results of the survey were made public. Not only did the site go ahead with the survey without the corrections, but responded ungraciously when Homeseekers publicly demanded, via a widely distributed press release, that Gomez.com acknowledge and make the corrections. Gomez.com spokesperson Julie Ward was quoted April 22, 2000 by The Boston Globe: "They're ranting," she said of Homeseekers.
Gomez.com wasn't flattered by the unfavorable spotlight that Homeseekers very public complaints shone. The Internet Standard soon picked up the story on May 1, 2000. Their journalist questioned market-research firms "which position themselves as advocates for consumers, but they often make money from the very companies they evaluate, something most consumers don't know."
Was Homeseekers crushing its own sour grapes? Maybe. But now you can add Homestore to the whinery.
Knocked to second place by Homeadvisor in the summer survey, Homestore seems only just now to be awakening to the possible power Gomez.com can wield and the damage it can inflict.
Pleased with its number one rankings in the past two surveys, Homestore's chief information officer Dean Denhart had no reason to question Gomez.com's results before. But since the survey came out on Wednesday, Homestore is openly questioning Gomez's research methods.
Citing listings dominance, traffic and stickiness as a few of the reasons why Homestore should have retained its number one position, Denhart said, "We've been named to Business Week's list of top information technology companies, one of nine dot coms to make the list."
He bristled at Gomez.com's criticisms, and questioned their methods, particularly the site performance. "We use Keynote, the industry standard in tracking Web site speeds," said Denhart. "We are about a half-second faster for download times than other top Web sites." Denhart went on to explain that his site comparisons are similar high-trafficked companies such as Yahoo!, Schwab and eBay, not those with less content.
"I have to wonder what their methodologies are to come up with such completely different
results," says Denhart. "You have to be extremely careful how you measure. You can't just
track one site one day and another site another day." He explained that just last week, the
Sprint system was having problems causing site delays and slow downloads for thousands
of Web sites.
What happened to the importance of listings?
Besides their discontent with Gomez.com, Homestore and Homeseekers have something else in common - a fierce desire to protect the Realtor's role in consumer service. Why? They supply listings. Realtors represent approximately four-fifths of available homes for sale for a reason - that many home sellers believe that hiring a Realtor makes a difference in the price and speed at which their homes will sell. But Realtors don't control the market as many believe they do. Consumers do, and they want to see listings.
According to Julio Gomez, there is no single criteria that is so important that it could truly hurt a company, but Homeseekers and Homestore maintain that isn't true.
Any real estate site knows that the number one goal of online home buyers is to see listings of homes. Listings are so important in getting eyeballs to Web sites, in fact, that Homestore and its competitors engaged in an unofficial listings war, periodically releasing their gains in jubilant press releases whenever they could enlist a major broker or an MLS's database.
The company to triumph, and consequently gain a better valuation from Wall Street, was Homestore. Due to its "Gold Alliance" contracts, Homestore was able to effectively shut out or limit the number of listings available to its competitors from leading markets such as Dallas and Boston. Homestore's flagship, Realtor.com, is the official Web site of the National Association of REALTORS, enabling it to soon boast over 1.3 million listings, giving consumers hundreds of thousands more homes to look at than any other site. Homeseekers vollied back with broker alliances, getting more listings door-to-door and were able to top out at about 850,000, putting them easily in second place.
What consumers don't know and Gomez doesn't tell them is that there are really only about four real players in the national listings arena and two of them supply listings to competitors. Homeseekers supplies listings to Homeadvisor, who in turn supplies listings for Homes.com and iOwn, and vice versa. Homestore supplies the listings for number six on the top sites list, Realestate.com. With this much co-mingling, listings may one day become a moot point, but today, they are still considered highly important to consumers who want to search as many homes as possible.
Some sites learned this fact of life the hard way. Before it was reduced to asking Realtors to share listings, Owners.com was vehemently pro-FSBO and anti-Realtor, but the site's owner learned after nearly going bankrupt that while sellers may be interested in saving money on commissions, buyers weren't interested in viewing only a handful of poorly-presented homes for sale per market. Realtors' listings have value, but the reverse doesn't appear to be true.
Yet Homeseekers was punished in the spring survey for not having FSBO listings on its site (neither did any of the others, with the exception of iOWN) according to Hensley. Suddenly, that criteria also seems to have disappeared, coinciding with the addition of e-brokers to the survey. Even e-Brokers don't represent owners without a contract.
Does this change of criteria serve the consumer or serve Gomez.com? If FSBO listings were that important, there would be no home buying survey. Only two companies, Owners.com and iOwn, cater to FSBOs and they are both about to run out of cash, according to recent news stories. So did the consumer demand for FSBOs dry up or was that a criteria abandoned in favor of allowing e-brokers into the survey?
Apples-to-raisins comparisons
From the winter to the summer, the home buying survey expanded from the top 15 to the top 20, allowing the entrance of e-brokers into the arena. Start-ups with fire-in-the-belly, but hardly the sophistication and range of services to play on the same field as 10-year Internet service veterans as Homeseekers, or world-class relocation experts such as Move.com, the Internet storefront of Cendant. Cendant brands include Cendant Mobility, one of the top relocation companies in the world, and three of the most popular franchise brands representing over 25 percent of all broker-owned listings, Coldwell Banker, ERA and Century21.
A zipRealty.com spokesperson proclaimed the survey "tough, but fair," blissfully unaware of her company's good fortune to leap ahead of companies such as Homeseekers. Did the fact that the company is a $30,000 GomezPro subscriber help? Did the fact that Gomez will probably never see another dime from Homeseekers make a difference? Who knows? Meanwhile, Homeseekers spokesperson Pam Johnston wonders, "If we dropped in the rankings so much, what did the others do to improve?" Three months is a very short time to fall so significantly in the rankings, particularly after Homeseekers just added consumer-friendly features such as its new exclusive Connect2call service, which promises a return call from an agent within one minute of contact. Does Gomez.com forget that they are evaluating Internet sites? Doesn't technology matter?
Where the e-brokers gained footing, according to Gomez.com senior real estate analyst Nick Karris, was their willingness to offer discount services to consumers. However, Gomez.com does not know enough about the industry to warn consumers that reduced fees can result in reduced service and higher liabilities for consumers. If consumers are demanding reduced fees, Realtors have no choice but to respond with less service and responsibility. While some consumers may look forward to taking the reins of home searching, the question has to be asked, who will tell them what they need to do to protect themselves while moving the home transaction forward? Gomez punished Homeseekers for not providing this kind of step-by-step advice, while ignoring other sites which do the same.
Karris expressed "surprise" that the e-brokers had scored so much better than Prudential, Century-21, Coldwell Banker and ERA. Of little apparent importance was their ability to serve customers in more than a few states. zipRealty is in 11 markets in eight states. eRealty is in five jurisdictions. eHome is in only four states. Homebytes is the only discount e-broker to announce penetration into all 50 states, but they weren't included in the survey.
Predicted Karris in a press release, "The Internet will create a win-win opportunity for consumers and Realtors alike, allowing consumers to take a more active role in their home search and allowing agents to spend their time negotiating and closing more transactions.''
Flying against the formidable consumer market research of Homestore, Homeseekers, and other listings aggregators, Karris confidently said, "We're seeing from our rankings that consumers want to take a more active role. Getting listings is certainly important, as is neighborhood-related information, but consumers want a faster, easier and more efficient process."
Really? So do Realtors. Perhaps he didn't know that it was veterans such as Homeseekers, Cyberhomes and Realtor.com, working under the encouragement of the National Association of REALTORS that brought listings, neighborhood information, as well as other automated home buying services to the Internet in the first place.
