Amazon Web Services Gets Another Hiccup
Amazon’s Web Services experienced another hiccup today. Early this morning, its Elastic Compute Cloud (EC2) went down for about an hour for at least some customers in the U.S. This follows a major outage of its S3 storage service in February. Companies big and small use EC2 as a virtual data center to run jobs on Amazon’s computers. Customers began reporting problems on the EC2 developer forum at 1:51 AM PT. The problem seemed to be resolved about an hour and a half later.
Amazon does not guarantee 100 percent uptime for its Web Services, although it does strive to achieve that. And data centers go down all the time, no matter who is hosting your data. But more and more companies are relying on Amazon to be able to scale their computing resources on demand and do it cheaply by paying only for what they need. Many Web startups are building their entire businesses on top of Amazon’s Web Services, and even an hour of unavailability is unacceptable. At least this one happened during the middle of the night.
The outage is a reminder that, as Amazon CTO Werner Vogels said last week after a speech he gave about uncertainty, “Everything fails all the time.” And it comes on the eve of what could be Google’s entry into the on-demand computing infrastructure business with the expected announcement of its BigTable cloud database service tonight. As big tech companies such as Amazon, Google, IBM, and others start to compete around web services, reliability will be one of the main features they will compete around. (The other one will be price).
Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.
Yahoo Search Wants to Be More Like Google, Embraces Hadoop
Yahoo is following in Google’s footsteps again in search. Today, it is shifting a crucial part of its search engine to Hadoop, software that handles large-scale distributed computing tasks particularly well. Hadoop is an open-source implementation of Google’s MapReduce software and file system. It takes all the links on the Web found by a search engine’s crawlers and “reduces” them to a map of the Web so that ranking algorithms can be run against them.
Yahoo is replacing its own software with Hadoop and running it on a Linux server cluster with 10,000 core processors. The Hadoop software does the same job 34 percent faster than the old software. Yahoo is also providing some other interesting stats that gives us a view into the computing infrastructure behind its search engine:
Some Webmap size data:
* Number of links between pages in the index: roughly 1 trillion links
* Size of output: over 300 TB, compressed!
* Number of cores used to run a single Map-Reduce job: over 10,000
* Raw disk used in the production cluster: over 5 Petabytes
Compare this to some data from Google on its MapReduce computing infrastructure (it is not quite apples to apples, but Google was processing 20 petabytes a day back in September, 2007 and outputting 14,000 terabytes in compressed data per month):
Hadoop is a project of the Apache Software Foundation. It also works for large-scale computing problems beyond search. For instance, IBM is using Hadoop as a foundation for its cloud computing initiative. Competing with Google using open-source software where it can is a smart move on Yahoo’s part, especially when that software outperforms its own.
Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.
OpenID Welcomes Microsoft, Google, Verisign and IBM
As anticipated by TechCrunch UK in early January, OpenID is welcoming some big new partners to the club - Microsoft, Google, Verisign and IBM (TechCrunch UK anticipated all but Microsoft).
Google has been dabbling with OpenID for some time with its Blogger platform (and Brad Fitzpatrick, the creator of OpenID, is now a Google employee).
Yahoo also announced support for OpenID earlier this month, which more than tripled the number of OpenID accounts to 350 million. 10,000 websites now accept OpenID accounts for login.
All of the newcomers, along with Yahoo, have joined OpenID’s corporate board and, we assume, will be making their user accounts OpenID-compatible. But it’s not clear that any of them are in a hurry to become a “relying party†(allowing users with third party OpenIDs to log in to their sites). OpenID looks like it’s going to be a winner, so big companies making their user accounts OpenID compatible is a good hedge. Everyone, of course, wants to be an ID issuer, since they get to “own” the user. Less attractive is allowing users from other sites to log into your services, so don’t expect that functionality to come for some time.
Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.
IBM Acquires Storage Company XIV For $350 Million
IBM has acquired Israeli based data storage technology company XIV for what is believed to be $350 million.
XIV’s main product Nextra is a storage system based on a grid of standard hardware components. XIV will become part of the IBM System Storage business unit of the IBM Systems and Technology Group.
Andy Monshaw, general manager for IBM System Storage said that the acquisition of XIV will “further strengthen the IBM infrastructure portfolio long term and put IBM in the best position to address emerging storage opportunities like Web 2.0 applications, digital archives and digital media.â€
XIV was founded in 2002 by five graduates from the 14th class of the Israeli Army’s elite “Talpiot” program (hence the XIV); Executive Chairman and Co-Founder of XIV Moshe Yanai was previously with EMC.
Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0
Amazon Takes on Oracle and IBM With SimpleDB
Companies can now go ahead and fire their expensive database administrators—those engineers who keep the Oracle or IBM databases humming. Amazon has just added an enterprise-class database called SimpleDB to its suite of cloud-based IT infrastructure, which also includes storage (S3) and computation (EC2) available by the drink. Today, Amazon is taking sign-ups for the SimpleDB beta, which should start in a few weeks. As it points out on the new Simple DB page:
Amazon SimpleDB is a web service for running queries on structured data in real time. This service works in close conjunction with Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), collectively providing the ability to store, process and query data sets in the cloud. These services are designed to make web-scale computing easier and more cost-effective for developers.
Traditionally, this type of functionality has been accomplished with a clustered relational database that requires a sizable upfront investment, brings more complexity than is typically needed, and often requires a DBA to maintain and administer. In contrast, Amazon SimpleDB is easy to use and provides the core functionality of a database - real-time lookup and simple querying of structured data - without the operational complexity. Amazon SimpleDB requires no schema, automatically indexes your data and provides a simple API for storage and access. This eliminates the administrative burden of data modeling, index maintenance, and performance tuning. Developers gain access to this functionality within Amazon’s proven computing environment, are able to scale instantly, and pay only for what they use.
This will be especially attractive for Web startups. Amazon has just taken another major infrastructure cost off the table for them. Relational databases are expensive to buy and maintain. Whatever features or performance SimpleDB lacks, it should make up for in price. Amazon wants to democratize the database by making it available to more businesses, and even individuals, thus leveling the playing field between big companies and startups even more.
And since SimpleDB operates at Web scale, larger companies will wake up to the cost saving opportunities of such a service as well. IBM, for one, is already trying to preempt any customer defections with its copycat Blue Cloud initiative. If speed is of the essence, you might still want to keep your database on your own servers. But the Web is where most software will one day live, whether consumer or enterprise. And Amazon’s got nothing to lose by speeding that day along.
Pricing for SimpleDB is as follows:
Machine Utilization - $0.14 per Amazon SimpleDB Machine Hour consumed
Data Transfer
$0.10 per GB - all data transfer in
$0.18 per GB - first 10 TB / month data transfer out
$0.16 per GB - next 40 TB / month data transfer out
$0.13 per GB - data transfer out / month over 50 TBData transfer “in” and “out” refers to transfer into and out of Amazon SimpleDB. Data transferred between Amazon SimpleDB and other Amazon Web Services is free of charge (i.e., $0.00 per GB).
Structured Data Storage - $1.50 per GB-month
Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0
2008 Sector Outlook: Technology
The fifth in a series examining the outlook for five sectors. S&P analysts have just one overweight recommendation: information technology
Where the Affordable Suburbs Are
Demand—and prices—for homes in the best suburbs remains strong. But good deals in good communities exist if you know where to look
Supercomputing for the Masses
Advances in chip design are prompting companies to take advantage of the potential power. But the necessary new markets and software are lagging
Bad Time for Biometrics
Talk that troubled VeriFone may acquire the doubly troubled Pay By Touch, a leader in point-of-sale fingerprint payments, has turned to murmurs
Battles and Bids Over Pay by Touch
Talk that troubled VeriFone may acquire the doubly troubled Pay By Touch, a leader in point-of-sale fingerprint payments, has turned to murmurs