MJ12bot 这个 bot 是干嘛的= =

This topic created in 3757 days ago, the information mentioned may be changed or developed.

151.80.41.169 - - [15/Feb/2016:13:12:14 +0800] "GET /robots.txt HTTP/1.0" 200 257 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)"; -

一天同一个页面爬好几次。

google 上只找到说是万维网搜索引擎研究项目

mj12bot

万维网

151.80.41.169

v1.4.5

1 replies • 2016-02-16 20:40:38 +08:00

zxgngl

Feb 16, 2016

http://www.majestic12.co.uk/projects/dsearch/mj12bot.php