Optimizing the Site For Search Engines

If you have cool site that bites all its competitors, but nobody knows about it - you have nothing at all. Search engines bring the most part of the traffic to site. The fact of how is high the rating that your site takes in search results determine your business success or failure. Let's see how to reach success.

How search engines operate

Any search engine consists of three parts - that are robot, or spider, the database and web interface.

Robot is a program that walks by the Net and views the sites one by one, collects certain info about it and stores it in the database. In order to allow the robot to index your page it should be registered on the search engine's site (but sometimes robots can find your site occasionally just walking by the Web).

Web interface is actually the search site where you enter the query and get results. When the user's seeking for something with the search engine, the search program computes the theme of the query and looks for the best matching records in its database, and outputs the results sorted by rating.

Robots fetch various information from the HTML code of the site and determine basing on it how is the page content relevant to the query, by math. computing of several coefficients. Here is the common list of such info (we will refer to this list as rating criterias)

• How often does the keyword appears on the page (keyword density, the higher is the best)
• How close is the keyword to the begin of the page (the closer is the best)
• Keywords in the meta-tags, and how it match to the real document theme (100% match is the best)
• Keywords in headers and outstanding text
• Comments text
• Site popularity, that is computed as a number of links from other sites to your site (the higher is the best).

Optimizing Technique

To achieve the good site rating it is necessary to analyze how well the rating criterias of the site match to the engine requirements. It is not hard to adjust the site content to meet the requirements listed above. Although it might be difficult to gain the popularity (the last item of the list above), however this can be partially solved by exchanging the links with the sites of the same theme. Generally matching of the different criterias themes is quite important for the site rating.

There are few things except the rating that are important for the search optimizing. This is the ease of finding the site pages by the robot. For example robots don't like frames; it prefer textual links to the graphic ones. Almost all robots don't follow the links with the query, e.g. links that contain question ? character, for example
http://www.site.com/cgi-bin/script.cgi?var=val&var2=val2
The reason is that such a page may contain different information depending on a query parameters, and also it is easy to robot to get lost within the query variables (for example what happen if the variables var and var2 can take values from the range 0..65535 ?) For the Apache web-server this could be partially solved using the mod_rewrite module: http://www.apache.org/docs/mod/mod_rewrite.html. Briefly, it is a technique of replacing ? char with the slash /. Therefore such a page becomes the regular one from the sight of robot. Such an URL:
http://www.domain.com/script.php?op=login&id=1&session=2
could be transformed to
http://www.domain.com/script/login/1/2

There are few tricks that search engines detect and penalise site for it:

• it is possible to put keywords in a document using the font color that is the same with background. Therefore keywords are invisible to the user but still visible to the robot;
• Non-matching keyword meta-tags and document theme, or keyword meta-tags overuse;
• duplicate pages on the site.

Using the script languages like PHP it is possible to give out totally different content of the same page to the robot and to the visitor. This can be achieved by analyzing HTTP_USER_AGENT variable and IP address of the visitor. As a result the site could gain the high rating, but the relevancy of the site theme will be broken. Currently it is unknown to me whether this can be detected by the search engines.

The rating of the page is determined mainly by its text. The lesser is the text present on the page and the more is the graphics, the less the robot will learn about the page. Robots cannot analyze graphics (so far), that's why it is just ignored. The deeper is the text goes to the nested tables the harder becomes to read it by the robot. The ideal optimized page is a plain text with minimum graphics and with frequent keywords that are close to the document beginning and formatted by <h1>, <title>, <b>,<i> tags. It is always possible to find a compromise between graphics and text percentage.

Other Ways of Gaining the Traffic

Search engines are not the only way of the site advertising. There are also:

• Banner advertising
• Link exchange technique
• Non-Internet advertising