Smaaz’s Blog

Microsoft explains how to verify MSNbot

Microsoft has added some information, on how to identify the different types of their MSNbot Crawlers, on the Live Search’s WebLog. The different types of the MSNbot are:

MSNBot
Main web crawler (www.live.com)

MSNBot-Media
Images & all other media (images.live.com)

MSNBot-NewsBlogs
News and blogs (search.live.com/news)

MSNBot-Products
Products & shopping (products.live.com)

MSNBot-Academic
Academic search (academic.live.com)

They also explain a method, to verify the identy of the MSNbot. Might be useful, since a lot of spam bots are cloaking themself as searchengine crawlers.

1. When you get a page view request, it specifies a user-agent and an IP address. As I described above, all requests from Live Search use a user agent starting with the word ‘MSNBot’.
2. If you see the MSNBot user-agent, it’s time to check the identity of the bot. Starting with the IP address (i.e. 207.46.98.149), you can use reverse DNS lookup to find out the registered name of the machine.
3. Once you have the host name (in this case, livebot-207-46-98-149.search.live.com), you can check that it really is coming from Live Search. The name of all live search crawlers will end with ‘search.live.com’. If the name doesn’t end with ‘search.live.com’, you know it’s not really our crawler.
4. Finally, you need to verify that the name is accurate. In order to do this, you can use Forward DNS to see the IP address associated with the host name. This should match the IP address you used in Step 2 – if it doesn’t, it means the name was fake.

Add comment November 30th, 2006

Google Answers will be shut down

Google has announced, that their Answers service will be discontinued later this week. Answers has been online since early 2002, and contains answers on more than 260,000 questions. According to Google, all the Qs ans As, that have been posted within the last 4 years, will remain online.
Seems, you now have to visit Yahoo!’s Answers service, if you still have unanswered Questions.

Add comment November 29th, 2006

YouTube has announced cooperation with Verizon

YouTube/Google and Verizon Wireless have agreed on a cooperation, which will bring videos of YouTube.com on Verizon mobile phones. Starting from early december, the service will be availabe on Verizon’s VCAST-Portal, for a fee of $15 for a month or $3 per day. Verizon will do a preselection, to make sure that only videos will be available, that fit the small displays and also have the highest quality.
YouTube also announced, that this co-operation will be exclusive only for a limited timespan.

Add comment November 28th, 2006

Cut out the Bank - Loan 2.0

Just came accross the finance co-op network of the UK company Zopa.
Zopa is an internet business that introduces borrowers to private lenders and cuts out the bank. Since the credit sum can be distributed on several credit givers, the risk is reduced. So the system of Zopa is different to the loan system of the big banks:

“At banks, people who have spare money put it into an account. The banks then use it for lots of different things. Some of it they lend to people who need to borrow. Some of it they gamble on the price of tin or the yen depreciating. Some of it they use to make their branches nice and pretty to attract more customers. And they make a lot of money doing all this, but only ever give their customers a comparatively small amount of interest.

Things work differently at Zopa. People who have spare money lend it directly to people like them who want to borrow. There are no banks in the middle, no huge overheads and no unethical investments.”

Their service is already used by more then 100,000 members and will be introduced in the US soon.

1 comment November 26th, 2006

Porn site operator jailed for life

The operator of China’s largest pornographic Web site “Pornographic Summer” was jailed for life on Wednesday, state media reported. The judges at the Taiyuan Intermediate People’s Court in Shanxi province gave the life sentence to the 28 year old Chen Hui and handed down terms of 13 months to 10 years to eight others after they were convicted of profiting from pornographic dissemination. 
Chen and his accomplices started the Web site in 2004, and opened a further three porn Web sites, attracting more than 600,000 users.
According to the police the site contained more than 9 million pornographic images and articles, when it was closed in October last year.

Add comment November 23rd, 2006

Happy Thanksgiving!

Enjoy the time with your family and friends. And here are some useful ;)Thanksgiving facts form the U.S. Census Bureau:

256 million
The preliminary estimate of the number of turkeys raised in the United States in 2005. That’s down 3% from 2004. The turkeys produced in 2004 weighed 7.3 billion pounds altogether and were valued at $3.1 billion.

44.5 million
The preliminary estimate of the number of turkeys Minnesota expects to raise in 2005. The Gopher State is tops in turkey production. It is followed by North Carolina (36.0 million), Arkansas (29.0 million), Virginia (21.0 million), Missouri (20.5 million), and California (15.1 million). These six states together will probably account for about 65% of U. S. turkeys produced in 2005.

649 million pounds
The forecast for U.S. cranberry production in 2005, up 5% from 2004. Wisconsin is expected to lead all states in the production of cranberries, with 367 million pounds, followed by Massachusetts (170 million). Oregon, New Jersey, and Washington are also expected to have substantial production, ranging from 18 million to 52 million pounds.

1.6 billion pounds
The total weight of sweet potatoes—another popular Thanksgiving side dish—produced in the United States in 2004. North Carolina (688 million pounds) produced more sweet potatoes than any other state. It was followed by California (339 million pounds). Mississippi and Louisiana also produced large amounts: at least 200 million pounds each.

998 million pounds
Total pumpkin production of major pumpkin-producing states in 2004. Illinois, with a production of 457 million pounds, led the country. Pumpkin patches in California, Ohio, Michigan, Pennsylvania and New York also produced a lot of pumpkins: each state produced at least 70 million pounds worth. The value of all the pumpkins produced by these states was about $100 million.

2.1 billion bushels
The total volume of wheat—the essential ingredient of bread, rolls and pies—produced in the United States in 2005. Kansas and North Dakota combined accounted for about 33% of the nation’s wheat production.

Add comment November 23rd, 2006

Yahoo! and Answers.com cooperate

Answers Corporation today announced the launch of question and answers content from Yahoo! Answers on Answers.com. Answers.com users can view relevant questions and answers - both specific questions and topic-related categories - from Yahoo! Answers directly on Answers.com’s own pages. In addition, users can click directly to Yahoo! Answers to ask any question.
Yahoo! Answers was released in May this yearly and has more than 60 million users globally. Answers.com was started back in 1999 and has around 3 million hits a day.

Add comment November 22nd, 2006

Google still is the most used search engine

According to the latest ranking of Nielsen/Netratings, an estimated 3.0 billion search queries were conducted at Google Search, representing 50 percent of all search queries conducted in October 2006. At the same time Yahoo! Search handled 1.456 billion requests, which is a plus 30% since October 2005. MSN/Windows Live Search has lost 8% and made it to the third place with 538 million requests.

Source: Nielsen//NetRatings MegaView Search, November 2006

 

Add comment November 21st, 2006