السلام عليكم
حملت هالسكربت من موقع هوت سكربت
وبحثت عنها ...على انها تخدم خاصية البحث في الجوجل...والي هي ملف robots.txt
وهنا الشرح انشاء ملف robots.txt
:con2:
كود:
Entry Meaning
User-agent: *
Disallow:
The asterisk (*) in the User-agent field is shorthand for "all robots". Because nothing is disallowed, everything is allowed.
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /private/
In this example, all robots can visit every directory except the three mentioned.
User-agent: BadBot
Disallow: / In this case, the BadBot robot is not allowed to see anything. The slash is shorthand for "all directories"
The User Agent can be any unique substring, and robots are not supposed to care about capitalization.
User-agent: BadBot
Disallow: /
User-agent: *
Disallow: /private/
The blank line indicates a new "record" - a new user agent command.
BadBot should uts go away. All other robots can see everything except the "private" folder.
User-agent: WeirdBot
Disallow: /tmp/
Disallow: /private/
Disallow: /links/listing.html
User-agent: *
Disallow: /tmp/
Disallow: /private/
This keeps the WeirdBot from visiting the listing page in the links directory, the tmp directory and the private directory.
All other robots can see everything except the tmp and private directories.
If you think this is inefficient, you're right!
Bad Examples - Common Wrong Entries
use one of the robots.txt checkers to see if your file is malformed
User-agent: *
Disallow / NO! This entry is missing the colon after the disallow.
User-agent: *
Disallow: *
NO! If you want to disallow everything, use a slash (indicating the root directory).
User-agent: sidewiner
Disallow: /tmp/
NO! Robots will ignore misspelled User Agent names. Check your server logs and the listings of User Agent names.
User-agent: *
Disallow: /tmp/
User-agent: Weirdbot
Disallow: /links/listing.html
Disallow: /tmp/
NO! Robots read from top to bottom and stop when they reach something that applies to them. So Weirdbot would stop at the first record, *, instead of seeing its special entry.
Thanks to Enrico Altavilla for pointing out this problem in my own robots.txt file
وهنا مثال لل robots.txt للبيت الابيض :nice:
http://www.whitehouse.gov/robots.txt
...وابي واحد فاهم يشرح شو هي انسب طريقة لاستخدام هالسكربت