Comments on: False Reports About Yahoo! Blocking Googlebot On del.icio.us http://news.accuracast.com/seo-7471/false-reports-about-yahoo-blocking-googlebot-on-delicious/ News from the world of Internet & mobile search and social media Fri, 18 May 2012 12:24:08 +0000 hourly 1 http://wordpress.org/?v=3.3.2 By: Kamrul Hasan http://news.accuracast.com/seo-7471/false-reports-about-yahoo-blocking-googlebot-on-delicious/comment-page-1/#comment-16543 Kamrul Hasan Sun, 01 Aug 2010 07:17:57 +0000 http://www.accuracast.com/search-daily-news/seo-7471/false-reports-about-yahoo-blocking-googlebot-on-delicious/#comment-16543 Hi, I have a question. I think you will be able to halp me. I am finding lots of bookmarking sites with user-agent disallow. For example connotea.org . When I click on |info| of any bookmark it takes me to its parmalinks. And older parmalinks (bookmarked by some one) of those sites does not have any Google cache. Yet people says those are do follow bookmarking sites. I am confused. How those sites can help our sites SEO? Hi, I have a question. I think you will be able to halp me.
I am finding lots of bookmarking sites with user-agent disallow. For example connotea.org . When I click on |info| of any bookmark it takes me to its parmalinks. And older parmalinks (bookmarked by some one) of those sites does not have any Google cache. Yet people says those are do follow bookmarking sites. I am confused.
How those sites can help our sites SEO?

]]> By: Colin Cochrane http://news.accuracast.com/seo-7471/false-reports-about-yahoo-blocking-googlebot-on-delicious/comment-page-1/#comment-3828 Colin Cochrane Fri, 22 Feb 2008 01:36:44 +0000 http://www.accuracast.com/search-daily-news/seo-7471/false-reports-about-yahoo-blocking-googlebot-on-delicious/#comment-3828 I felt it would be prudent to respond myself. 1) The directories being blocked were not the issue. The robots.txt reference was used solely as a list of user-agents to test against. 2) If del.icio.us is serving these 404s to prevent spoofing, then it is being done to prevent proxy hijacking, not content-scraping. A content scraper could just spoof a normal Mozilla user-agent if it was worried about getting caught. On a final note: I'm not sure at what point I became a "self-proclaimed pundit". I simply encountered unusual behaviour from del.icio.us, did a little investigating, and wrote about what I found. People took from that what they did. I felt it would be prudent to respond myself.

1) The directories being blocked were not the issue. The robots.txt reference was used solely as a list of user-agents to test against.

2) If del.icio.us is serving these 404s to prevent spoofing, then it is being done to prevent proxy hijacking, not content-scraping. A content scraper could just spoof a normal Mozilla user-agent if it was worried about getting caught.

On a final note: I’m not sure at what point I became a “self-proclaimed pundit”. I simply encountered unusual behaviour from del.icio.us, did a little investigating, and wrote about what I found. People took from that what they did.

]]>