crunchbasecom

1愛的鼓勵 16訂閱站台

Sometimes, we may want search engines never to catalog certain elements of the site, and on occasion even prohibit other SE in the site altogether. Navigating To vimeo.com/gracecamenker/ probably provides aids you can give to your family friend.

That is the place where a simple, little 2 line text file called robots.txt will come in.

Once we've a website up and running, we should ensure that all visiting search-engines can access all the pages we want them to consider.

Sometimes, we might want search-engines not to index certain areas of the site, and on occasion even ban other SE from the site all-together.

That is where a simple, little 2-line text file called robots.txt will come in.

Robots.txt exists in your internet sites main directory (on LINUX systems that is your /public_html/ directory), and looks something just like the following:

User-agent: *

Disallow:

The initial line controls the robot that will be visiting your site, the second line controls if they are allowed in, or which elements of the site they're maybe not allowed to see

Then easy repeat the above lines, If you want to deal with multiple spiders. For alternative interpretations, consider having a glance at: \u7f51\u53cb\u70b9\u8bc4-Using Wordpress As A Content Material Management Technique Or CMS-\u529e\u516c\u5546\u57ce.

Therefore an example:

User-agent: googlebot

Disallow:

User-agent: askjeeves

Disallow: /

This can enable Goggle (user-agent name GoogleBot) to see every page and directory, while in the sam-e time banning Ask Jeeves from the site completely. If you require to be taught further on go there, there are many on-line databases you should consider pursuing.

To discover a fairly updated set of robot person names this visit http://www.robotstxt.org/wc/active/html/index.html

Its still very advisable to place a robots.txt file on your own site, even if you want to let every software to index every page of your site. Identify further on \u7f51\u53cb\u7559\u8a00-Blogging On Free Hosts Not Really A Good Idea-\u80a5\u732b\u6210\u54c1\u7f51\u7ad9\u8d85\u5e02-\u4f01\u4e1a\u5efa\u7ad9\u7684\u9769\u547d by going to our wonderful web resource. It will stop your error logs filling up with entries from search-engines wanting to access your robots.txt file that doesnt exist.

To learn more on robots.txt see, the entire listing of resources about robots.txt at http://www.websitesecrets101.com/robotstxt-further-reading-resources.