Back to Question Center
0

Wanne alama daga dogon Semalt ya kamata in yi amfani da robots.txt?

1 answers:

Ma'anar Semalt ya furta cewa za'a iya amfani da alamu da yawa, kamar yadda ya kamata ya dace.

Ina so in toshe wasu batu ta hanyar robots. txt kuma ina damuwa game da wane ɓangare na layin Semalt don amfani da, musamman ma don ƙananan bots. Alal misali:

     Mozilla / 5. 0 (jituwa; uMBot-LN / 1. 0; mailto: crawling @ ubermetrics-fasaha - gas jet vacuum ejector system. com) "
JS-Kit URL Resolver, http: // js-kit. com /
Mozilla / 5. 0 (dacewa; SEOkicks-Robot + http: // www. seokicks. de / robot. html    

Shin, zan yi amfani da alama ta biyu? Shin alamun alamu sun ƙunshi sararin samaniya, ko magoya bayan SEOkicks sun manta da bayanan SEOkicks-Robot ? Ba na ainihi nufin ɗaukar tambaya ta musamman ga 'yan bots - Ina so in san jagorancin: wane ɓangare na UA nake sanyawa a cikin' yan fashi. txt ga wadannan bots da yawa tare da UA har tsawon haiku?

     Mai amfani: uMBot-LN / 1. 0
Disallow: /    

PS: Yarda da ku amma ban buƙatar in ji cewa bots da ba'a so ba sun fi kariya tare da mod_security. Na riga na kasance da ka'idojin mod_sec kasuwanci.

February 6, 2018

Masu fasahar yanar gizo masu goyan baya. txt:

  • Google :

    Mai amfani da Google ɗin shi ne (yadda ya dace) Googlebot.

  • Yandex :

    Misalai: Mai amfani-wakili: Yandex

  • Yahoo :

    Yahoo Slurp yayi biyayya da shigarwa ta farko a cikin jigilar. txt fayil tare da mai amfani-wakili dauke da "Slurp. "

Akwai kuma bayanai na sunayen robot da za a iya amfani dashi a cikin jigilar. txt a kan tashar yanar gizo: http: // www. robotstxt. org / db. html

Abin baƙin ciki, ba daga cikin wayoyin da kake sanya a matsayin misalai suna da shafukan da zan iya samu ba, kuma ba a lissafta su ba a cikin asusun ajiya.Duk da haka, a matsayin misali, Ina fatan cewa ta yin amfani da slash a cikin User-wakili na jigilar. txt ba zai dace ba. Babu wani misali da na zo a fadin bayar da shawarar cewa. Don haka zan yi amfani da:

  Mai amfani: uMBot-LN
Disallow: /Mai amfani: SEOkicks-Robot
Disallow: /