The Pervasive Datacenter

A bunch of us were debating over Twitter yesterday whether it's desirable to have separate personal and professional identities on the service. The consensus seemed to be: "it depends." It depends on your professional situation. It depends on how personal and workplace-safe you want your posts. And so forth.

I find this whole question of what I call "identity 2.0" fascinating. Increasingly, there's a blurring line between personal and professional identities--and even between multiple compartments within those buckets.

As Wendell comments in a post: "It's kinda like living in a small town again." There are a lot of analogs. Just as locality and small size break down barriers between public and private in a small town or village, so, too, do the Internet and the search engine.

This is a trend that we're all going to be wrestling with for years to come. Although things I've written back in my college days are readily available online, if you know where to look, it was mostly stuff written for newspapers or Usenet posts.

There are doubtless matters on which I've changed my thinking, but there is probably nothing that I'd find especially embarrassing. What I don't have online--because it didn't exist back then--is "off the record" commentary written purely for a circle of friends. (In Here Comes Everybody: The Power of Organizing Without Organizations, Clay Shirky describes how many blogs are clearly written for a close circle of friends, even though ... Read more

I spent the past couple of days attending Technology Review's EmTech08 conference at MIT. Lots of interesting speakers and ideas, some in areas of tech that I follow day-to-day (such as cloud computing) and others that I follow more in the vein of an interested observer (alternative fuels, open voting systems). In many respects, it's a refreshing change of pace from the events I commonly attend that tend to be more focused on today's immediate IT concerns.

EmTech08 gave me lots to mull--and I'll roll that mulling into more in-depth pieces down the road. For today, though, I'm just going to expand a bit on a few statements and thoughts I ran across in the course of the two days that particularly caught my attention.

The state of the market for tools in parallel computing is abysmal. (Marc Snir, University of Illinois)

There seems to be a default assumption around the IT industry today that, as processors evolve to more cores and more heterogeneous processing (and as computing architectures get more distributed), the software will evolve apace. Changes will be required, of course, but nothing to really worry about. I'm not so sure. Even if one discounts the most dramatic doomsayers, lots of researchers and IT executives see serious gaps in both tools and training to deal with highly threaded processing. Consider that several of the panelists in the Parallel Programming session spoke warmly of the potential for functional languages. Yet it's very ... Read more

In October of 2000, I hopped a Las Vegas-bound flight to attend a developers' event being thrown by the InfiniBand Trade Association.

By way of background, InfiniBand was one of the hot technology properties of the pre-bubble-bursting days. It was touted as a better (faster, more efficient) way to connect servers than the ubiquitous Ethernet. Its more vocal backers, of which there were many, went so far as to position it as a "System Area Network"--a connective fabric for data centers. A whole mini-industry of silicon, software, host bus adapter, and switch vendors supported InfiniBand. One sizable cluster resided in Austin, Texas, but there were many of them scattered around the U.S. and elsewhere--to say nothing of significant InfiniBand initiatives at companies such as IBM and Intel.

I don't remember all the details of that past InfiniBand event but it filled a decent-sized hall at the Mandalay Bay and was followed by a party that took over the hotel's "beach" on a balmy Vegas evening.

Last week, I attended another InfiniBand event, Techforum '08. It was also in Las Vegas. More modest digs at Harrah's reflected that InfiniBand hasn't exactly lived up to those past hopes. However, the fact that there even was a TechForum '08 also reflects that InfiniBand is still with us--primarily as a server connect for high performance computing (HPC) applications where low latency and high bandwidth are especially important.

Given that I've been following InfiniBand since its early days, ... Read more

Guy Kawasaki recently interviewed Chunka Mui to discuss Mui and Paul Carroll's new book, Billion-Dollar Lessons: What You Can Learn from the Most Inexcusable Business Failures of the Last 25 Years. All the problems that Mui lists make lots of sense and I could offer plenty of my own examples from high tech and elsewhere.

• Companies overestimate the power that comes with additional size.

• Companies underestimate the complexity that comes with additional size.

• Companies overestimate their hold on customers.

• Companies don't consider all their options.

• People routinely overpay for acquisitions. But you already knew that.

However, I think it's worth highlighting one in particular:

Companies play semantic games to convince themselves that they have something that matters in a new market. Avon decided in the 1980s that its "culture of caring" equipped it to operate retirement homes. Not even close. We blame this phenomenon partly on all the talk that the railroads fell by the wayside way back when because they thought of themselves as being in the railroad business, that the railroads could have captured the nascent automobile industry if they'd thought of themselves more broadly as being in the "transportation" business. We think the railroads would have lost their shirts if they'd gotten into cars. Railroads and cars had nothing at all in common.

I think it worth highlighting because there's considerable Marketing 101 logic to defining yourself by markets rather than technology. There's a reason for that. This admonition often

One of the sure signs that a new technology is having some real impact on the industry, as a whole, is when it starts changing other technologies, products, and processes that touch it. In part, this is a simple reflection that a vendor or a small group of vendors aren't the only ones who care about their new shiny-ness. Press releases, consortia, and partnerships are all well and good. But the real proof of acceptance is when other companies and customers start spending real money and changing their own plans and products.

We're seeing this happening with server virtualization. IT shops have started to rethink the processes that they use to allocate new computing resources to users. Some at the forefront have even made virtual servers, rather than physical ones, their default unit of computing.

We're also seeing changes in the way that servers are designed and built. In the x86 processor world, Intel VT and AMD-V attacked some of the most fundamental difficulties of virtualizing x86 hardware. Both companies are continuing to introduce hardware virtualization enablers to address things like I/O performance, virtualization's handling of memory, and compatibility of virtual machines across multiple generations of hardware.

We're also seeing changes in the way that servers are designed and built. Fundamentally, the issue is this. Server virtualization's first big win was in providing a path to consolidate x86 servers that were otherwise very lightly utilized--5 percent or below in many cases. Virtualization can ... Read more

In the 1980s, New York City undertook a huge project to begin metering residential water service and charging for individual apartment and condo water use rather than simply assessing a flat fee as had been the norm. It was as huge and contentious a project as one would imagine. Why go to all this trouble and expense?

Well, you hardly have to be a radical free market economic thinker to accept that there's generally a connection between how much something costs and how much people consume. The higher the price, the less you use. In this case, the incentives have more to do with installing water-efficient fixtures, fixing leaky faucets, and the like. But the idea is the same. In short, New York wanted residents to start conserving water and realized that financial incentives were the only practical way to make this happen.

On the enterprise IT scene, I'm seeing the same sorts of issues replayed around datacenter power consumption. Most everyone professes great concern for how "green" (to use the trendy lingo of the moment) their operations are. In practice, if the power bill isn't in their budget (and is therefore effectively a flat rate of $0) nothing much usually happens until someone higher up who is responsible for all the relevant money flows makes it so.

But incentives for efficient datacenters is a topic for another day. Let's talk broadband instead. There's considerable ruckus brewing over whether it's right and proper for ... Read more

Last week's big virtualization news was Red Hat's purchase of Qumranet for $107 million.

By way of brief background, Qumranet has two overlapping, but somewhat independent, technology sets. The first--for which it is probably best known--is KVM, an open-source hypervisor that is in the process of being added to the Linux kernel. The other is its SolidICE virtual desktop solution that uses a back-end Linux server (virtualized with KVM) connecting to clients with the company's own Simple Protocol for Independent Computing Environments (SPICE) protocol. The virtual desktops themselves can be Windows, as well as Linux.

Some aspects of this buy are pretty straightforward and obvious. Others less so. As a result, I held off writing until I had the chance to discuss some of the specifics with Red Hat and my colleagues.

My conclusions? What seemed straightforward is straightforward. What didn't seem straightforward? Well, that's going to need some time to play out. Nonetheless, I got some good color and food for thought, which I share here.

My first observation is that virtualization remains a hot acquisition property. Now, $107 million may not seem like a huge sum. After all, Citrix bought XenSource for something closer to $500 million about a year ago. But XenSource was the well-known entity behind the Xen Open Source hypervisor project, and its commercial XenEnterprise product was gaining at least some market traction. By contrast, Qumranet is largely unknown by all but the most serious virtualization watchers--$107 million for ... Read more

Time was when most enterprise software came in the front door as part of a formal, signed-off-at-the-highest levels procurement process. Or it got written in-house as part of an equally formal, multi-year development plan. Or some combination of the two. You didn't expect that expensive packaged software you bought to just work out of the box did you?

Lots of software still gets purchased and developed that way of course. However, the truly striking story of the past decade is how so many of the tools and other software that we take for granted today are essentially bottoms-up phenomena. They largely came in the back door and made their way into what's often called the "Shadow IT" of organizations. Official IT didn't make this software ubiquitous and mainstream. For the most part, it was already ubiquitous and mainstream by the time IT departments got around to blessing it.

Linux (and, more broadly, open source in general) is perhaps the canonical example of this trend. In some respects, Linux adoption just mimicked past adoption patterns for distributed computing in general--from Windows NT servers to PCs and even Unix in the early days. However, open source licenses make backdoor sourcing one big quantum step easier. Indeed, the basic idea that open source licensing helps to build a developer and user base that can then be monetized when the software goes into production underpins a lot of the thinking around business models associated with open source.

However, the ... Read more

Over five years ago, I wrote a research note titled "Latency Matters!" The impetus was the following observation:

What's the best way to estimate travel time? Would you rely on an estimate based solely on the number of lanes in the road and the sound of the engine? Nope. You need to know, at minimum, how far you have to travel, the condition of the road, and how fast you'll likely be able to go. Obvious, right?

You'd think so. But system and networking specs rate computer performance according to bandwidth and clock speed, the IT equivalents of just measuring the width of the road and the engine's revolutions per minute...

Latency is the time that elapses between a request for data and its delivery. It is the sum of the delays each component adds in processing a request. Since it applies to every byte or packet that travels through a system, latency is at least as important as bandwidth, a much-quoted spec whose importance is overrated. High bandwidth just means having a wide, smooth road instead of a bumpy country lane. Latency is the difference between driving it in an old pickup or a Formula One racer.

Some of the particulars discussed in that research note are less central to everyday IT concerns than they were at the time. For example, although as much engineering attention goes into designing high-end systems as ever, the details of memory architectures, internal processor interconnects, and the ... Read more

Kathy Sierra tweets:

Twitter extreme eg.--many of us non-users couldn't perceive benefits. Low barrier made it OK to say, "just try it..." Not true w/all things

This is a sometimes overlooked advantage of software as a service (SaaS) in its various forms. Even installing free or trial software can be challenging enough that all manner of virtual appliances and application virtualization have been suggested as possible solutions to this "pain point."

Of course, no barrier is truly zero height. Even signing up with a Web site, getting the hang of the basics, and (perhaps most of all) figuring out how or if it fits into the flow of your lifestyle and work don't just happen. This is especially true when the service in question is new and different. When it makes you approach an activity in a genuinely different way or otherwise shift an established mindset.

New is hard for developers and designers. It's also hard for users.

That said, the freedom to tell prospective users/customers to just press their browser at a URL and "play" is an incredibly powerful concept. Especially when the product in question lends itself better to experience than explication.

Kathy is right that Twitter is one such example. Before I gave it a serious run, I thought it sounded sort of silly. It was actually using it that convinced me otherwise.

Compare and contrast this to the case of TiVo and the digital ... Read more