- Develop a “web spider” – a software program to browse the Internet.
- Instruct the spider to begin its “web crawl” at popular websites, building an index of the words on the pages.
- By following the links on the sites, the spider will quickly spread out across much of the web.
- Build up an index of search words found by the spider and encode and store the data for users to access.
- Develop search engine software – a program to sift through the millions of entries in your index and rank them in order of relevance.Record Breaker
The world’s biggest and most-used search engine, Google has an index of millions of web pages and handles 250 million searches every day. The company was set up in 1998 and today has around 53,600 (Q4 2014).