Wikia Inc., the for-profit company behind community Web-site Wikipedia, would be working to develop an open-source search engine with the aim to tap up to 5 percent of the vast search market, founder Jimmy Wales announced last Thursday.
The new search engine would be built using a model similar to Wikipedia with the results freely available. Wales explained that programmers and developers in the open source community would be given free access to tools and technology required to build the search engine.
Being open-source, developers would also find ways to prevent spamming and abuse, Wales believes. The idea of an open-source engine has been explored before and ended prematurely each time, because the search algorithm – the main code that pulls up relevant results from the Web, could not be put through large-scale tests. It calls for full-scale crawling of the World Wide Web right from the beginning and the effort behind it "actually requires a great deal of investment in hardware".
Wikia would provide the resources and back the effort to ensure the search engine is well tested. “We'll give away all the technology, all the data. Release everything under a free license” just as in Wikipedia, Waes said. He also said he wasn't worried about competition resulting from making the search engine technology freely available. Wikia's open-source search engine could still enjoy 2 to 3 percent of the vast search market – which would be “a pretty decent little business” despite multiple clones.
The clones would also help sell the original, Wales believes, because as “more people find out about the brand, more people drive traffic back”. The test version of the search engine could be ready by the fourth quarter this year, Wales said.
While admitting that he was in fact taking on the might of search giant Google, Wales also said that search is a fundamental part of the infrastructure of the Internet. Hence there is need for an open source option that could shift the power on the Internet. He rued the fact that the established search leaders Google and Yahoo were secretive about the algorithm behind the software. He described them as “black boxes” with secrets of how the search results are ranked.
Some firms that have tried and failed to rival Google, have expressed interest in the Wikia project, which “provides kind of a level playing field where lots of players can contribute” he said.
