Friday, November 18, 2011

Cloud Computing and Programmable Infrastructure (Part I)

This post will be a bit different, but if you bear with me, I plan to relate it back to investing. This will be part I of a series (as I found out earlier today).

A few weeks ago, I attended the Structure 09 Conference put on by GigaOm Media in San Francisco, CA. Not a bad little conference. While I'm glad I attended, and I had a great time in the later sessions and the "meet and greet" held by Canaan Partners after, it wasn't quite as technical as I was hoping. Either that, or I missed the best parts. (I arrived about 3 hours late since I was driving through rush hour traffic on the 101 from "Man" Jose.) I did get more technical meat from O'Reilly Media's Velocity 2009 conference the previous 3 days, so it was all good. Besides, I met Paul Kedrosky of Infectious Greed fame and had a nice, though brief, conversation with Jonathan Heiliger, who was my personal hero for a great many years at the start of my career.

During the conference, I shot off this tweet about cloud computing. (Much of Structure seemed to revolve around this topic.) It seemed such a simple thing to say, and so disgustingly obvious that it wasn't even worth saying, but apparently it struck a chord with some folks. So I want to expand on this idea.

For ages, the supremely competent engineers and sysadmins out there have been developing custom tools to allow them to manage their systems and networks. Most of these tools have probably revolved around some combination of expect/TCL, Perl, and shell. The reason is simple - most device operating systems were not accessible programmatically, e.g. via an API. Most still are not. I attribute this to vendor lock-in. If the only way to interface with the device is by the vendor's prescribed methods (web based, software application, or command line), then you are forced to learn the vendor's products. As you become accustomed to how a given vendor's products work, you're going to be reluctant to invest the time and energy to learn a competing vendor's products. Voila! This is the biggest reason that so many networking products have command line interface syntaxes similar to Cisco's IOS and CatOS. This has been a huge contributor to Cisco's financial success over the past 20 years. Cisco's command line syntax has become de facto standard in the networking world.

Vendors would probably say there are other reasons for not exposing their devices to be inspected or controlled programmatically. One would be security. Most network devices are pretty insecure, in terms of default passwords and configuration settings (although this has been changing SLOOOOWLY over time). I imagine a great many customers didn't particularly care to create tools and programs to manage network devices, servers, and other systems. They just wanted to configure the device and go on with their lives. Only in the last decade has it become apparent to people that Sun was right and "the network IS the computer". The network infrastructure has taken on a level of criticality that was reserved only for servers for a long time.

Finally, the rise of cloud computing and the idea of automated infrastructure has changed people's thinking about the flexibility of their systems. On-demand computing can now be realistically accomplished with commodity hardware and open systems, provided the infrastructure supports it. Many companies have been born to make this flexibility and elasticity a reality for the masses who would rather not deal with the intricacies of building their own. Amazon Web Services, Joyent, RightScale, Mosso a.k.a. The Rackspace Cloud, Enomaly, and countless others come to my mind immediately. They've taken on the challenge of providing this infrastructure for regular people. Not everyone is Google, Amazon, Yahoo! or Microsoft, with both the resources to devote to building these tools from scratch, never mind the inclination or need.

There's also a certain level of difficulty in implementing APIs correctly and efficiently that vendors were
probably unwilling or unable to address. Market dynamics being what they are, the demand probably wasn't there for APIs as much as for new features, better performance, and lower costs. Even security was a higher priority! Finally (and this is largely related to the concept of lock-in), vendors probably didn't want to expose their devices to hacking and other tampering, or reverse engineering (via black box testing, etc.) by their competitors. If you're forced to deal with a command line, software application GUI (Java or Windows-based, many times these days) or web GUI, you were inherently limited in how much information about the internal workings of the device you could derive/describe.

However, with the exception of Juniper Networks, most systems and network devices still do not offer the programmatic option for people who are willing to build their own. Thus we have the menagerie of scripts and other homegrown tools which have to make do with clunky workarounds, scraping output from text commands or collecting SNMP data. Yes, it works, but it can hardly be considered optimal. These tools become somewhat second class citizens from a performance perspective, only able to issue so many commands in a given time quantum, and limited by the range of functionality exposed. Most of these scripts don't run with the highest levels of privilege, again for security reasons. (That is, a configuration management script for a Cisco Catalyst switch may run as a regular user as opposed to having "enable" privileges, and cannot change configuration settings on the switch, but I'm sure this varies too.)

However, I see this changing. I don't know if a company like Vyatta will lead the charge, but there's no reason for them not to. The question will be who follows (or leads, if not Vyatta)? Again, Juniper has already allowed a certain amount of access, which is a good start. However, I don't think the dominoes have really fallen as yet. In time, I believe more vendors will offer programmatic access to their devices through custom APIs. The ones who start this trend will do it as a differentiator, the way most innovation shows up commercially. Eventually, more vendors will offer this ability, at least for network devices (routers, switches, firewalls, IDS/IPS devices, etc.). Servers may not need programmatic access as much, although it would not surprise me to see a vendor, possibly an open source vendor, come up with a cross platform management layer that exposed a single, consistent API for multiple operating systems. Again, maybe this already exists but I can't think of who offers it.

What does all of this mean? It means, simply, that the ability to program your network will be within the reach of the average "man" and "woman" (or rather, system administrator and network engineer). This won't remove the business case for the cloud providers named previously, such as AWS or Joyent, but it will enable automated infrastructure for the masses. This will be a good thing, I believe.

Whew! That ended up being a lot longer than I thought it would, so I'm going to pause right here. In part 2 of this series, I'll go a bit deeper the investing implications of automated infrastructure and cloud computing, as I see them. Hopefully I haven't bored anyone to tears, but if I have, I apologize. I think a bit of context is required for this discussion, and there were some things that I wanted to make sure were said.

Until next time, gentle readers...

No comments: