Evolving Private Clouds: Welcome Fishermen, Hunters, Cloud Admins & Other Liars

Liars Welcome

I’ve been meditating on Tom Bittman’s recent blog post, Why Are 95% of Private Clouds Failing? and what it means for the evolution hybrid cloud applications.

My colleagues at newScale, then Cisco and now at my current employer have been watching the same phenomenon.  We thought there was a high rate of struggle / flailing and probably a high rate of failure, but the number in my head was more like 40%.

To me, 95% is stunning. Rarely in tech do we 95% of anything. But 95% failure tells there’s something deeply wrong about what people think a private cloud is.

Tom’s recent poll shows 31% fail to change the operational model, 13% fail to change the financial model, and 11% spend too much time defending the status quo / doing too much and 10% on the wrong benefits.

From my point of view, these really all go into the bag of “failure to change the operational model.”

What does “failure to change the operational model mean?  On the ground, it looks feel like this:

IT doesn’t know what a service offering should be, the capacity management process is barely sufficient for static infrastructure. It  involves long planning processes,  ok for static capacity, but not adequate for real time infrastructure.

  • IT policies are executed through manual processes and forms that involve many reviews, not automated and controlled by software in real time. This introduces friction and delays to provisioning.
  • There maybe a little automation, but the developers don’t have access to this automation nor the API’s.
  • The financials are difficult. Either there was no chargeback, or IT doesn’t know the unit costs of their offerings–hard to know if IT doesn’t know the offerings.
  • It’s impossible to provide elasticity on a fixed amount of finite infrastructure, so more policies and friction are added to prevent over-consumption
  • App teams want all the goodies of cloud (elasticity, on-demand, etc) but are not able or aware that they probably need to make changes to their development process and architecture to really take advantage of a cloud architecture.

In other words, IT Operations “private cloud” is  something that is an evolution of virtualization, but it’s really not a cloud (standard, metered, on-demand, automated, elastic, etc).  As I wrote a few year ago, we go from From cruising at self-service speed to “we gotta have a meeting”

Meanwhile, over the last few years, developers have adopted cloud services en masse.. These services are elastic, on-demand, metered, elastic, etc.  No work needed to achieve that level of maturity required –there are other challenges, but those have become increasingly easier to deal, while the transformation challenge remains stubbornly immovable.

The result has been a kind of silo’d hybrid cloud. Things work one way in public, the tools are different, the people are different.  Something that Gartner has started calling bimodal IT. A kind of truce under the clouds.

Unfortunately, apps have gone hybrid and this truce has become war by other means rather than peace.  The private cloud that isn’t real can’t deliver what the developers need.

Hybrid cloud gets real

The biggest opportunities for enterprises today lie in how they will engage and interact with their customers. And these customers want to interact on their smart phones and tablets, while hanging out on Facebook, Pinterest, Twitter. The kids on Snapchat and WhatsUp, the parents deciding where to live on Zillow. These are new ways of living, shopping, entertaining, and socializing.

Meanwhile, the marketing and sales groups want campaigns to engage their customers where they are, in the way the customer wants to engage.  And the business wants measures and metrics to understand what’s effective and working in their marketing spend.  And which boss wouldn’t like to have a (positive) viral video?

This implies that to measure things that couldn’t be measured before, the enterprise may need big data to process those Google search streams, it needs a scalable video architecture to deliver those viral videos, it  need it’s login integrated with Facebook and Twitter to make it easy for the customer to engage — plus it gets better data from Facebook on what they like than from its traditional transaction systems.

All of these new capabilities also need to integrate with the enterprise traditional systems of record: ERP, sales, etc.

This is how we arrive at our hybrid app era: on the back of cat videos and a billion likes. These are today’s equivalent of the old appointment TV or water cooler shows of the sixties and seventies; which were underpinned by the need for consumer marketing and brand building by enterprises.

Marketing is not the only use case, but today this encompasses any enterprise that uses the internet to engage and transact business: real estate, banking, credit, insurance, travel, etc, etc — all are now internet businesses with a clickstream and a need to engage with their customers.

This year hybrid gets real because applications have gone hybrid

Hybrid cloud management now needs to be put in place. Not because hybrid clouds require it, but because there’s now enough applications across multiple places that this brings a new management problem: how do you managed both?

We noticed that the market is asking for a single control, management and service plane that encompasses both public clouds and private clouds.

And this is where the old private cloud – the one that failed to deliver – has to be matured to the next level or it won’t be able to be managed along with public cloud resources.

We are at the end of the era of fake private clouds and at the beginning of the hybrid cloud era.

Watching Clouds. Enabling Crop Rotation

Furrow and Ridge.

Recently I’ve been thinking about innovation and cloud. Basically my observation is that cloud allows C-level executives to run more experiments faster and safely.

The fact that most ROI/TCO calculations don’t effectively account for this phenomenon is a problem of bad math / lazy thinking.  It is simply a matter of time before we have a new accounting model that measures innovation as capital. Any model that can’t account for opportunity size, market growth or opportunity costs is insufficient in 2015.

Which brings me to this article from the NY Times about risk.  Reading it helped me think through what I’m seeing about the value of cloud computing: which is not the only the ability to take more risks, but also the diminution of the negative consequences of failure.

In pre-modern times, when starvation was common and there was little social insurance outside your clan, every individual bore the risk of any new idea. As a result, risks simply weren’t worth taking. If a clever idea for a crop rotation failed or an enhanced plow was ineffective, a farmer’s family might not get enough to eat. Children might die. Even if the innovation worked, any peasant who found himself with an abundance of crops would most likely soon find a representative of the local lord coming along to claim it. A similar process, one in which success was stolen and failure could be lethal, also ensured that carpenters, cobblers, bakers and the other skilled artisans would only innovate slowly, if at all. So most people adjusted accordingly by living near arable land, having as many children as possible (a good insurance policy) and playing it safe.

Our relationship with innovation finally began to change, however, during the Industrial Revolution. While individual inventors like James Watt and Eli Whitney tend to receive most of the credit, perhaps the most significant changes were not technological but rather legal and financial. The rise of stocks and bonds, patents and agricultural futures allowed a large number of people to broadly share the risks of possible failure and the rewards of potential success. If it weren’t for these tools, a tinkerer like Perkin would never have been messing around with an attempt at artificial quinine in the first place. And he wouldn’t have had any way to capitalize on his idea. Anyway, he probably would have been too consumed by tilling land and raising children.

via Welcome to the Failure Age! – NYTimes.com.

Power in large organizations is clannish. Who you grew up with, who knows you, who you know, the network of favors you built provide both protection and support to get projects done.

But there are other clans who have different agendas.  Hard to experiment in such an environment.

For C-level execs, innovation today is a bit like betting on crop rotation: They can’t be wrong!  The C-level executive has to assure they are  right, then report they are right, and worse-comes-to-worse bend the metrics to show they were right.

No room for  I think this might work, but I’m not sure.

The enterprise corporate culture requires certainty: Commitment to metrics before they are reasonable, models before these are proven and projections that cannot be grounded. This is why failed projects last three years; it’s the minimum depreciation time for assets. Everyone knows it’s a zombie project, but no one can tell the C-level executive that they failed.

Who would want to sign up for that mission?  Most enterprise managers don’t.

It’s the equivalent of experimenting with crop rotation.

It might work. Or your family might starve. Even if it works, the Warlord will come take your results anyway.

The ability to run more experiments safely is right up there with the legal and financial innovation that increased our culture’s ability to innovate.  Making it easier, safer to try new things changes the economics of running experiments.

The fact that most don’t know how to model this on a spreadsheet is not stopping innovators from going forward. In facts, most CxO’s know this intuitively.

Not everyone is comfortable with change and ambiguity. There are many people like me: permanent immigrants. We need to try new things because the past is not our friend.

I believe we are all immigrants into a cloudy future and the risk of not taking a risk is now higher than the risk itself.

The Three Freedoms of Cloud – A CxO view

"Freedom 2001" by  Zenos Frudakis

I have been CEO, CTO or board member of both a small start up and a large enterprise these last 10 years. Every budget year has been about achieving economic growth and meeting objectives. I have never seen “cost” reduction as a primary objective.

Cost reduction might show up as a secondary objective: either as a way to free up resources to fund innovation, or as a logical step to improve margin after the main objective has been achieved.

I’ve never been to a board meeting or operations review about cost reduction. It’s always about growth objectives. Sure, we review the budget, but as a CxO – that is just the basics–We are expected to stay on budget. It’s like wearing clean clothes to work; of course you should, but that’s not the job.

Business success is all about innovation. Growth does not occur without innovation. Innovation feeds on investment—So where do those resources come from?

The good news: You likely already have the capital. You just don’t see it, because because it’s stranded into a pile of amorphous activities and legacy maintenance.

You are probably using those resources to address the daily routines that keep the lights on.  Like patching long-time legacy apps with spit and baling wire. Like managing your aging application implementations from a rattling, smoking collection of dashboards and consoles.

It’s a fact that 70 to 90 percent of IT budgets are earmarked for “run operations,” otherwise known as Keeping The Lights On. This is where your innovation capital is stranded.

These resources can be freed up by cloud products and services to make your organization faster, more agile—more innovative.

The Three Freedoms that cloud can underwrite:

1. Freedom to experiment. The cloud allows you to run more experiments due to lower start up capex costs per idea tried. For every executive, using cloud to demonstrate early success and scale, all with a provident solution, is the equivalent of having the Sword of Excalibur when next year’s budget battle gets going.

2. Freedom to Fail (Quietly). It’s true that we celebrate innovation, but there’s also a very real tendency to punish failure. The desire to experiment is quashed and promising ideas don’t get considered if they are seen as too risky. Meanwhile, other projects get over-funded to mitigate risk, or they wheeze forward on life support because no one wants to admit failure. In both cases, working capital that could be used for interesting initiatives is unavailable. Too bad, because the ability to run small projects without capital expenditures represents freedom to fail, allowing experiments to succeed step by step or to be shut down quickly, thus avoiding the three-year run to equipment depreciation.

3. Freedom to be agile. I’ve been a startup guy, founder of VC-funded newScale, which was acquired by Cisco in 2011. My background is all about competing against incumbent behemoths with established reputations and customer relations. So how do startups compete? By moving fast, running faster than the competitor’s innovation cycle time, and marshaling superior domain expertise at the point of sale. Large companies have a hard time competing against such agility unless they are agile themselves. Cloud represents the opportunity for large organizations to streamline—getting rid of all kinds of friction in the innovation process and enabling senior management to focus on core problems and differentiated valued-added efforts. This can happen through offerings such as on-demand infrastructure or platform as a service, which can save months of configuration and planning; software as a service, which delivers a solution ready to go; and business process as a service, which gets us away from reliance on hardware, software and operations. The ability to strategically have these as-a-service tools in your innovation toolkit is a game changer.

The Great Surrender of Oh-14: Come Out of Shadows with Your Hands Up to The Cloud

fog surrounds

I just came back from two weeks in ASEAN countries (Singapore, Thailand, Indonesia, Malaysia – and that was the first four days) as well as a week in Australia.

I met with many of our teams working in cloud as well as with customers. A theme emerged that actually matched some of the trends I’m seeing on the waning days of the year Oh-14.

I call it the Great Surrender.

We were not comfortable with public cloud. Policies were enacted to prevent usage of non-sanctioned infrastructure.  Eventually we built a private cloud, but it still takes six weeks to provision.

Our developers didn’t take it up.  The business had needs and they responded by developing on public clouds.

Now we have a bunch of apps that are important to the business and growing.

We need to bring our public cloud consumption under management. We have no idea if they are backed up, patched, using the proper firewall policies or if it’s legitimate use.

But we are going forward with public cloud usage. We now want to figure out how, not if.

The great surrender of Oh-14 is now at a cloud near you.

No, cloud CANNOT be sold like a barrel of oil

Oil Drums

It seems every year the word commodity appears when discussing cloud computing.  Yet we are seeing the dominance of Amazon, the emergence of Microsoft Azure as a strong contender. Google’s deep pockets & architecture keep them a credible competitor. None of these players is playing a commodity game despite regularly trumpeting price cuts.

Price cuts are the inverse side of Moore’s Law: instead of trumpeting number of transistors, they trumpet the reduced cost of the compute unit.

So reading this article from GigaOm, “Yes, IT can be sold like a barrel of oil (but it’s going to take some work)“, reminds me that commodity dreams die hard.

Choice quote:

Big companies use commodity contracts to ensure predictable prices for oil, wheat, electricity, metal and other crucial supplies that keep their businesses going. These days, a crucial supply for many companies is cloud computing power — raising the question of whether that too can be bought and traded in the same way as oil or oranges.

A recent partnership suggests the answer is yes, and that we’re heading to a world where companies won’t just turn to Amazon Web Services or Microsoft Azure for cloud services, but to a commodities market that offers the best price, on the spot or in the future, for a range of interchangeable IT infrastructure.

I strongly disagree this is likely in the foreseeable future (3-6 years). Yes there will be price competition but that’s normal.

I wrote about why I don’t believe in this a little over a year ago. Not much has changed.

Here’s the original post where I dismissed brokers: “Brilliant! I Dismiss Service Brokers on Monday. Someone Announces One on Tuesday. Enron Bandwith Exchange Anyone?”

That’s my opinion. Not for the next 3-6 years. And maybe not after that either. But I’m product guy, not a futurist. Just a working class product guy.

Working class CTO

Progressive Working Class Caterer

It’s been a while, but I’m back to blogging.

I’m  naming this new blog Working class CTO because for the last couple of decades that’s been my job and I have a few observations. It’s a job that I have enjoyed and cherished but, one that has always been hard to explain, especially to my neighbors in Oakland.

Exactly, what does a CTO do? Like the cloud, no one really knows. Is it like Netscape’s Jim Barksdale used to say, “An engineering VP without a delivery date” (ouch) or is it fundamentally a kind of marketing job wrapped in the pretension of a vision? Or something else?

Over the last couple of years, I’ve been recruited by some of the biggest names in the industry for cloud CTO jobs.  Each one required a very involved conversation to understand what they were really looking for.

The answers ranged from silly, “You need to know the CAP Theorem” to honest, “We don’t know for sure” to incomplete, “Communicate our vision to the larger market”.  I passed on these opportunities.

They were missing the working class part, making great products. When I asked about team size, product influence, budget the answers were, ahem, interesting. As in the ancient Chinese curse, “May you live in interesting times.”

One  useful learning from my time at Cisco came from interacting and collaborating with several of our CTO’s.  Why several? Very different disparate businesses in different areas require different expertise – there’s no way for a single person to know all industries at the right level.

Some were very technical, some were more business oriented.  All were great communicators of the vision and trends driving their industry segment.

But while that’s cool, I’m a bit different. Back then, I started joking I was a working class CTO to distinguish what I did from those higher up in the hierarchy.

I meant to say, I spent time with my product teams getting into the detailed work of translating strategy and vision into products. The retail-detail, mundane toil of of bending the lathe of vision into facts on the ground.

Followed by working with our marketing team to turn that into compelling value propositions. Getting into the field to help the our sales force get the initial customers for the new innovation.

Doing thirty-four meetings in two weeks in Australia, dodging punches at a Newark train station at 10 pm waiting for my train while german shepherds jump,  sleeping in my suit at a freezing holiday inn 80 miles north of Detroit. That working class CTO job.

This blog will not cover the topics I have covered over the last few years, mainly service catalogs. The old blog remains up; 600+ articles and seems to be useful to a lot of people.

The (new) adventures of a working class CTO begin here.