Delicious was part of the Yahoo! network and used to be one of the most popular sites in its category. In December last year, there was news that Yahoo! intended to shut down the website.

Following this, Chad Hurley and Steve Chen, who had earlier developed and then sold YouTube, decided to buy it over from Yahoo! Delicious was taken over as part of their new company called Avos, which they had formed in April 2011.

Both Hurley and Chen had been thinking about the fact that there is always a flood of information on the Internet, and it is very difficult for users to go through all of it and reach the information that they are really looking for. With the revamp of Delicious, they hope to be able to simplify this process for users.

The new Delicious is supposedly a place where users can see the most recent links shared around topical events. On the homepage, users will now see browseable “stacks”, which are collections of images, videos or links to a particular topic.

Stack on Delicious about the TV programme, Merlin

Speaking about the newly launched site, Chad Hurley sa, “We want to simplify things visually, mainstream the product and make it easier for people to understand what they’re doing.” If this idea catches on, Delicious could soon become far more popular than it ever was in the past.

However, the first mover advantage that Delicious once had is no longer the case. Sites like Facebook and Google+ already have far more viewers and are extremely popular for social sharing. StumbleUpon has steadily grown in popularity as a social discovery tool.

Since the demise of Delicious a number of bookmarking sites have stepped in, but none have become quite as popular. While the organisation of information that the new service offers is great, conceptually, the new owners of Delicious might find that their absence from the market has cost them dearly and that they’ve missed the boat to become a social media powerhouse.

Due for launch in Q2 2008, Yahoo! onePlace will allow mobile users to bookmark any piece of content with a single click in one convenient place.

A built-in RSS reader and image and video management system allows users to link to news feeds, websites, videos, images, emails, search queries and all content that has already been personalized on other Yahoo! services such as Flickr and MyYahoo!

Yahoo! onePlace will be more sophisticated than their other bookmarking service, del.icio.us. The service will allow users to create collections and bundle all related information within a collection.

Every piece of information that is bookmarked will also be categorised and kept up to date automatically. The sytem will provide users a quick overview of everything that has changed in the mobile users world, via a feature called Pulse and even provide a favourites view to provide quick access to the content they view most frequently.

onePlace will be bundled with oneSearch, Yahoo! Go 3.0 and oneConnect to provide a mobile experience that goes far beyond plain-vanilla search.

Yahoo! have realised that they will not win against Google in a race to dominate search on the desktop or on mobile. Instead, they are doing what they are best at – creating a portal and a host of services that users are likely to keep coming back to.

Over the last weekend Colin found that the robots.txt file on Yahoo!’s social bookmarking property del.icio.us blocked search engine spiders including Googlebot from crawling certain directories. The extract from the robots.txt file on del.icio.us pasted below shows the “offending” code:

User-agent: Googlebot
Allow: /
Disallow: /inbox
Disallow: /subscriptions
Disallow: /network
Disallow: /search
Disallow: /post
Disallow: /login
Disallow: /rss

Colin then spoofed the Googlebot to see what was being delivered to the spider when it tried to access one of those pages, and found that he was being delivered a 404 error.

Colin’s erroneous post

Yahoo! has recently tested featuring information about del.icio.us bookmarks on its search results pages. Colin, along with a bunch of other SEO enthusiasts immediately jumped to the conclusion that Yahoo! is making use of its right to prevent competitors from benefiting from del.icio.us.

What Colin and the others are overlooking is three extremely important facts:

1. The directories being blocked by the robots.txt contain general administrative pages such as the “Add a URL” page, which do not need to be indexed by the search engines.
2. The 404 result is delivered because the user is spoofing a search engien spider, and the del.icio.us site is most likely smart enough to detect this and therefore does not allow access to its pages in order to prevent content scraping.
3. Del.icio.us, like most other websites relies on Google for traffic, and blocking the spider from accessing its content would amount to del.icio.us shooting itself in the foot!

The fastest way to check the validity of Colin’s claims is by checking if Google has been able to spider and cache any pages from del.icio.us after Colin’s original discovery…

Pages spidered by Google from del.icio.us in the past 24 hours

Clicking on the link above will immediately show that the Googlebot continues to access del.icio.us.

The Problem With SEOs

While the topic of del.icio.us blocking or allowing Googlebot is relatively minor, the buzz surrounding this post highlights a much bigger problem in the industry: the fact that a self-proclaimed pundit can cry “wolf”, and in no time a whole bunch of clueless “sheep” will run scared, turning what should have otherwise been someone’s minor error into SEO gospel.

Social bookmarking pioneer del.icio.us was acquired by Yahoo! in December 2005. The site enables users to store their bookmarks and tag them online so that they can retrieve them or add some more at any time, from any computer that they may be using at that point in time, instead of the earlier practice of being able to access ones bookmarks only from one’s own PC.

Del.icio.us incorporates a social element by allowing users to share bookmarks with friends and check bookmarks used by others. In effect the system allows people to be better organized and share information with colleagues and friends.

Google has long considered bookmarking to be an important indication of site authority, and incorporated the number of bookmarks, social as well as browser-based, as one of the factors in its algorithm to rank sites. Yahoo! has probably weighed in the number of del.icio.us bookmarks to a page when calculating the page’s rank for some time too.

However, for the first time Yahoo! is now listing the number of bookmarks alongside the search results pages, as shown in the screen grab below, found courtesy of Marketing Pilgrim. Even if Yahoo! does not use del.iciou.us bookmark popularity as a ranking factor, the presence of the number of bookmarks pointing to the site will probably influence users to click on the more popular results.

Integration of del.icio.us tags and bookmark popularity in the search results is currently being tested and has not been implemented widely. If they find that the information helps users, we will likely start seeing the del.icio.us icon more frequently alongside search results on Yahoo!

The presence of the del.icio.us popularity number could turn into a bit of a vicious circle in some ways, as more users click on the more popular results and accordingly more users will end up bookmarking those results, further increasing their exposure. While this isn’t bad news for some webmasters, for others whose sites are new or not as popular, this could mean they start the SEO game on the wrong foot and are never given the opportunity to prove their worthiness of a place in the SERPs.