A search engine is a computer program that reads as many web pages as it can find and keeps an index of the words they contain. When you make a search, it takes the words you type into the searchbox and looks for them in its index. Then it links you to the sites which contain those words.

That is why searches for common words tend to bring in several million pages. The search engine then attempts to place those pages in some sort of order. How precisely they do that is what gives them their particular character. But it often pays to look beyond the first set of results, which may not have exactly what you are looking for.

Basic search engine technique

It always makes sense to start with a search designed to bring in a small number of results, only broadening that out if you don't find what you want. If you type in a whole series of words you will (in most search engines) only find pages that have every one of those words. That makes more sense than being presented with a million results and then only looking at the top ten.

It helps to pick one search engine and learn how to use it. Today most people pick Google (and Google UK) . But it is worth knowing that for all its size, Google does not cover everything. Different search engines may have found other content that Google knows nothing about. If you want to be confident that you have exhausted all possibilities in a search, you will need to use several search engines.

Google is an excellent general purpose search tool. It is fast, uncluttered and indexes a lot of pages. Its method of ranking a page makes a lot of sense: if a page has a lot of other pages linking to it, it moves up the league table.

It also has one great advantage: unlike the other search engines, it keeps a copy of every page in its index. This saves the usual frustration when you find a page, only for it to be missing or unreachable. You find those "stored" pages by clicking the Cached link after each individual result on the results page.

But Google also has a few quirks. It only brings in pages which include ALL the words you type in. This is not a disadvantage. Type in a lot of words on your first search: then remove words if necessary to increase the number of pages you get.

Google allows you to place exact phrases in quote marks, like this: "Tony Blair", but disregards common words like "in", "on", "of", "the" and so on. If you want them in your search you must specifically include them by placing a + before each one. Most search engines allow "wild card" or "stemming" characters: for instance, hous* to bring in "house", "houses" and "housing". Google has automatic stemming: the word "house" will automatically bring in "houses" and "housing". It also allows you to use the asterisk as a wild card to replace any word in a phrase: "a * of the star" will bring in "a glimpse of the star", "a friend of the star", "a fraction of the star" and many more.

But it does have lots of extra options on its advanced page, some of which you can use just by using a few special terms in your original search. For instance, you can find things in specific sites if you use the "site:" term. For example, comedy hotel Basil site:bbc.co.uk will only bring in relevant stuff from the BBC's site.

Another advanced trick is the "inurl:" term, which means you can find stuff only from sites whose web address (or url) includes a particular word or words. I use this all the time, using inurl:gov.uk, for example, to find sites with some connection to the UK government. In the same way, you can often search within a site that has no search facility of its own by using the site: or inurl: trick.

Beyond Google

At one time, if you didn't want to use Google you had two more big search engines to try. But both AltaVista and All The Web have been swallowed up by Yahoo!, although their front pages still exist for the time being. Meanwhile, Yahoo! is now really a search engine rather than a directory of human-selected sites.

Other users swear by Ask.com (formerly Ask Jeeves) and WiseNut. Both have the clean look pioneered by Google. Give them a whirl.

Metasearch

Metasearch means searching several search engines at once. In practice, you get a shallow trawl of ten or so search engines, but don't touch resources that are more deeply hidden. Try Ixquick, Metaeureka or Ithaki. Many other metasearch engnes have gone over entirely to a paid placement model and have a limited value for non-shopping purposes.

The whole subject of search engines and search techniques is explored in more detail at Danny Sullivan's Search Engine Watch site, or you can follow search developments as they happen by keeping in touch with Tara Calishain's excellent Research Buzz site.

CONTENTS