How does Google find your pages on the WWW and how do they find web pages matching a specific query and from that how do they determine in what order these search results appear.

If you think of the WWW as a book with a detailed index at the front which tells you exactly where you can find everything in it, this will give you some idea.  It is similar when you type in a query, Google’s programs check their index and make a decision based on which results they think are the most relevant, and return in order, showing the highest first and they make use of three processes to do this.

Crawling

Crawling using Goolgebot’s finds new and updated pages which they want to add to the index.  Google uses an enormous set of computers to do this, crawling literally billions of pages with the sole purpose of fetching information.  The computer then uses an algorithm to determine which sites to crawl and how often they should be crawled and how many pages the spider should fetch from each site.  Crawling starts with a list of web page URL’s that have been generated from previous searches; spiders visit these pages in order to detect links and then add them to the list.  Google does not accept payment in an attempt to have a site crawled more frequently.

Indexing

Spiders process each one of the pages it crawls and then compiles a gigantic index of all the words they see and where they are situated on each page plus they process information that is included in the tags and attributed contained on a page and while Google can process many content types, they cannot process them all, especially rich media files or dynamic pages.

Bringing back results

Once a user enters a query they expect to get back the best results possible.  Google’s machines search the index for any pages matching this query and return what they believe are the most relevant and they use over 200 factors to determine this.  One is PageRank.  PageRank is measure in terms of the importance of a page based on the incoming links found on the page from other pages.  Each link that comes from another site adds to the PageRank of a site, but they are not all equal.  For this reason Google keeps trying to improve the service offered to users by weeding to spamming or other things that impact search results in a negative way.

You have to make sure that your site can be crawled correctly so that they can be ranked well in the search results page but Google do offer some tools that can help you such as Related Searches, Google Suggest and even Spelling suggestions which are aimed at helping users save time by displaying common issues related to popular queries.

If a site is ranked well, it is because Google have algorithmically determined that the content on its pages is more relevant to a users query than other sites.