I think this quote really hits the nail on the head and confirms what a lot of people may have intuitively known about the value of cloud providers:
But once certain startups turn into big companies with hundreds of millions of users, with computing needs that they’ve come to intimately understand, it can be far more efficient to set up computing infrastructure designed exactly with those needs in mind.
I think the main advantage of cloud providers is to offset the risk of purchasing equipment that eventually is no longer needed, which is ideal for younger companies that are still trying to reach their market capacity or unsure about whether they'll still be around in a year. Of all the things on a startup's todo list, I can't imagine setting up their own infrastructure is the best way to improve profits or revenue.
But once the constraints of a userbase are more established, it should be easier to migrate off these platforms, since their pricing is optimized for users of all business sizes and use cases, whereas your specific hardware can be optimized for your specific users.
My biggest question is whether cloud providers could achieve a scale where they are able to offer the most optimal infrastructure costs for specific businesses. Maybe this is the case for smaller or mid-size companies, but I'd be interested to see where the inflection point lies.
You’ve described the historical value of cloud computing perfectly. That said, I think the days where all but the largest or most stubborn companies run their own datacenters are coming to a close.
The problem will be finding skilled labor. Short-haul networking, power configurations, thermal load, hardware maintenance; these and many more are specific skills that can’t be learned overnight. Data center work used to be a viable middle-class career, but the pay scale for it has gone down and down. Companies that do run their own DCs like Google and Facebook have a few centralized experts, a thin professional staff on-prem, and an army of minimum wage disk swappers who are told what to do by a ticket system, just like an Amazon warehouse worker. The knowledge of how to build and run these things is all at the top now.
I’m not saying the jobs or talent pool are gone. Just that they’re shrinking, and will continue to shrink. Like the manufacturing industry, the fewer people there are who are comfortable working with real hardware, the harder it will be to start anew.
I'm not in the web or cloud business, but I've filled a rack with my stuff before. My impression is that hardware has become a lot more capable even relative to its tasks. With high iops memory, many cores and obscene amounts of RAM, I would expect companies of a much larger scale (in $, FTEs, or most other metrics) can be served by one 4HE machine, or by one rack, or by one room. Thus I would expect the knowledge of how to handle 5000 hard drives to become more obscure, naturally, but the skill to run a decently sized web application to remain almost constant.
Does this math work out, or have the tasks become more demanding at the same speed that hardware has improved?
A bank running 50 different services, on different platforms, with serious audit requirements, physical and logical access control, strict change and configuration management, etc., has two orders of magnitude more complexity. And that shit is very expensive in manpower.
There are now businesses that explicitly depend on the elasticity of the cloud and can never really be moved on premise without massive up-front investment in hardware that may only be used a few times a year for their biggest customers. Trying to hybridize these workloads hasn't been very successful as of yet. It is possible that K8S could relive this problem but I haven't seen it in practice, at scale.
Instant Elasticity in Cloud is a myth. If you think you are going to get 1k hosts just like that from AWS you will have an unpleasant experience.
I work at one of the decent size tech company and we are split between cloud and on prem. From our experience you have to inform AWS/GCP in advance (sometime way early) if you are looking to meaningfully increase capacity in zone/region.
Sure, auto scaling few hundreds of hosts may be possible but people who run a service which needs few hundreds of hosts run it directly on AWS, they will run it some kind of scheduler+resource manager which will have some kind of operational buffer anyway (as in you would already have those hosts so cloud elasticity is not a factor here).
How early is "way early"? Because as long as it's shorter than the two-three weeks it'd take to order boxes, rack them, provision them (which would be automated but might still take a afternoon), deal with any QA hiccups... I'd much rather call my AWS rep and say "can we add 30% by Thursday" and have them figure it out (and at such a large scale you might be able to spread it out across a couple regions anyway unless you only serve a specific part of the world).
From what I have seen it is actually of the same order or sometimes more. In one of the region/zone we add few hundreds hosts every week but that is after telling them we plan to upscale this in this region upto some big X number.
This is the same with disaster recovery too. The idea that "oh, our main DC went down, we'll just spin it up in another region" is great until you realize that means you need reserved instances in another zone, that just like another physical DC, you won't be using.
Right now there is no specific distinction between what we want to run in Cloud vs On Prem. Important thing to note here is we use Cloud as an IaaS only. We have our own stack which sort of prepares the hosts before it is ingested into clusters as usable capacity.
We actually recommend not using custom cloud providers Databases or any other value added services.
Why not completely either way (on prem vs cloud) is something that happened way before I joined the group but I think the main reason is to have a tactical edge in the long run such that we avoid lock in. I guess in some ways it helps us negotiate pricing better.
Imagine moving a certain workload from GCP region to an AWS region as part of a failover drill.
I was talking about at scale, not a rack. If you can get by with a rack, you will pay more for the people to support it than the incremental cost of the cloud.
> If you can get by with a rack, you will pay more for the people to support it than the incremental cost of the cloud.
Probably a whole lot less.
At larger scale - I would guess it's the same thing. If an organization needs more than a rack during peak use, it can probably benefit from setting up its own infrastructure. Only in the uncommon case of short extreme peak use and almost no use most of the time does such elasticity make a could solution attractive. IMHO.
> My impression is that hardware has become a lot more capable even relative to its tasks.
Indeed. The margins are bonkers high. As an example, the amount of ram that you can stuff into a physical machine has at least doubled in the last five years, but the price of the average virtual machine has not.
You still want ha, failover, and disaster recovery. Then you need to set up stuff like bgp, dns, security rules, etc, etc etc. Complexity mounts pretty quickly.
Indeed. It seems that most of the people saying that cloud hosting is expensive have never run into the issues of making their own SAN, managing the provisioning of 20 different teams, etc.
The organizational complexity and specialist knowledge is mind-boggling and there is zero chance that your in-house knowledge is better than what Amazon can provide.
Installing rack servers and setting up services to run a site used to be a sort of rite-of-passage 15-20 years ago, but that time period of the web was different. Still, I would consider basic familiarity with the infrastructure necessary also today.
Increasing hardware performance relative to task load created the rational for virtualization. Virtualization also turned out to be rational with respect to consistency, convenience, maintenance, and so on. At that point, outsourcing to a cloud can be rational.
But fewer people get hands-on experience with the infrastructure, and it sounds like many consider it almost mythical. For example, realizing the amount of work that can be done in 4U today. What does amazon charge for 96 cores and 256GB?
It's not just managing the complexities of managing the bare metal, although that's certainly a huge component of it.
There's some other huge arguments against running your own datacenters.
One is being able to properly provision resources. Being able to write just a function and have it consume just that tiny amount of resources rather than a whole VM is huge. Being able to spin instances up and down as you need them is huge.
I think that's been obvious for a long time, but what I've seen less obvious to the business analysts is the impact of more advanced cloud services. The direction of cloud computing is managed services where they run your databases, container platforms, etc for you. Trying to run a huge Cassandra cluster or Kubernetes cluster takes up a ton of expensive labor's time and there's a good chance the cloud providers are a lot better at it than you.
Sure, cloud services tend to be really expensive, and besides cost, there's also concern about things like vendor lock-in, IP protection/data privacy, and ability to tweak the small details of your platform. But cloud platforms in 2020 have a lot more features than in 2010 and in 2030 will have even more. The direction is pretty obvious. Running your own datacenters will be about as common as running your own power plant.
> One is being able to properly provision resources. Being able to write just a function and have it consume just that tiny amount of resources rather than a whole VM is huge. Being able to spin instances up and down as you need them is huge.
A 75 million dollar price tag is also huge.
Bothering about the operational impact of a VM or a request sent to a function-as-a-service might be a significant operational issue if your whole team can be moved around with a small sports utility vehicle.
Once you've grow over the point where your monthly cloud pricetag eclipses your company's paycheck budget, operating your own hardware is a no-brainer.
> But cloud platforms in 2020 have a lot more features (...)
That really doesn't matter at all, does it?
I mean, cloud providers are already repackaging FLOSS services as their Serverless offering.
And besides pursuing the latest fad, how many of those features are killed off and vanish from the face of the earth?
It's always great if we can get others to do the work for us, but if we consider the absurd premium charged by cloud providers for their services... Well, those "others" doing the work can be employed by your company and you still save money.
Is it? Over the two years they saved, this is about the salary for 100 engineers. Can you replace and maintain all the cloud aspects that AWS provides you with (I mean the ones you actually use) with 100 engineers? Maybe, if they are good engineers (which is kinda implied by the 300k salary tag in the calculation). If it's worth it, remains to be seen. Definitely nothing for any medium sized company.
Dropbox is huge and has a relatively simple, highly optimized use-case, for which cloud perhaps doesn't offer too much. This is NOT the norm. For most companies, no matter the size, building their own cloud is a no go.
> those features are killed off and vanish from the face of the earth?
Don't use Google Cloud then ;).
> but if we consider the absurd premium charged by cloud providers for their services...
Do you have any data on backing this up? This "absurd premium" includes the salaries of engineers to develop it, maintain it, do DevOps, keep the hardware/data centers, do marketing, etc. etc. There is of course a margin, these companies aren't doing it as a social service... That margin is highly variable from service to service and also between cloud providers. Some may not have a margin at all, others may run at a loss. There is not easy "uh everything is overpriced". Most companies will have a VERY hard time providing the offering at the price of large cloud providers. And the simple "back of the envelope" calculations often miss all the work & cost that needs to be done, but you don't know about...
> Well, those "others" doing the work can be employed by your company and you still save money.
Yeah, if your company is really big, then yes. If your revenue is below 100 million, there isn't even room for any discussion on this: Don't run your own cloud, it's not gonna work. Most of the "cons" I see are about misunderstandings of the offerings and failure to navigate the pricing models and picking the cheapest offerings that do the job. If you fail to do even that, how on earth are you going to run your own cloud?
I work at a medium sized company. Depends on who you count, but let's say around ~30 devs.
Recently we basically did just this, and it's been a great success. We haven't fully migrated and still use AWS for prod, but have seen substantial savings already.
We spent $2k on servers, Dell r720s. We bought a UPS and mount, and racked them in our office. I installed OpenShift 4 on it, which is Red Hat's Kubernetes offering with a nice web GUI, and setup a few terabytes of NFS to automatically provision storage.
To be fair, installing OpenShift for the first time took a while, around 3 weeks. Since then it's been smooth. We still use AWS, but our usage has gone down dramatically. We are still only migrating dev and test environments, leaving prod in AWS (we don't want to be responsible for uptime SLAs, and clients pay prod hosting costs). Some of these projects are CPU heavy, machine learning and computer vision projects too. They're not just simple web-apps. I'm not privy to our entire AWS budget, but I know that one project which we migrated saved over $500/mo.
After installation, maintenance has taken barely any time. Around 10-20% of my time is dedicated to OpenShift cluster maintenance. The rest I do normal project work. I often go weeks without having to touch anything, and the most common task I do is onboard new users. We've had 2 outages in over 6mo, one was an expiring cert and one was an airflow issue on the rack. I've learnt a lot and am certainly not an expert. These were the firs rack servers I'd ever worked with personally, although I had been researching used models for home use for a while (shoutout to /r/Homelab).
In fact, I had such success doing this that I personally bought a Dell r720 and have used it to selfhost a bunch of stuff at home. A co-worker of mine hosts his self hosted lab on AWS. Things like Plex, private photo storage, a few other toys, etc. He says he pays $300/mo, which seems insane to me, but I guess people streaming 4K plex adds up. The used r720 server I bought was $1,500CAD and has way more horsepower than he's paying for. (There are also electricity costs I haven't factored in here, as I'm trying to control for other changes in my power bill. Might be $100/mo at most.)
You're not doing anything even remotely close to the features offered by cloud providers or even managed hosting providers.
Disaster recovery? Geographically separate redundant servers with failovers? Automated (and proven to work) backups? One-stop access control for infra maintenance? Audit controls for your database and storage objects? Tape backups?
Even today to support all those things you need a small army of specialists. Granted, a heck of a lot of things can get away with not having any of this. But the use cases are out there and hosting and maintaining all of that in-prem is another different level.
I understand your use case, but your is very, very far from the sheer and absolute complexity and features that enterprise data centers have.
> You're not doing anything even remotely close to the features offered by cloud providers or even managed hosting providers.
So what?
Who in their right mind believes in, say, you need to operate and maintain half a dozen types of RDBMS in three flavors along with two or four or eight different message brokers and your own convoluted infrastructure-as-code multiplied by three along with a repackaged FLOSS offering... And a ground station?
Let's not be mad, here. There are proper, full-blown, popular, global-scale cloud service providers. That. Only. Offer. VMs.
Are we so drunk with corporate kool-aid to believe that we are missing out because we are missing... What do you believe you're missing, actually?
I repeat: there are popular professional cloud service providers whose business consists of providing either VMs or access to bare metal. That's where real-world companies run their real-world businesses. Why are we supposed to believe that you need more to operate your own stuff?
You are assuming that that vast majority of shops have the capacity to impose a very limited number of technologies, and secure them through common best practices.
This is about as far from the truth as I have experienced in life.
Fortune 500 companies have an innumerable number of platforms for software, use hundreds of products from dozens of vendors, many dead long ago. Same thing with governments, at every level of scale. Telecoms? Utility providers? Medium-sized businesses who are not in tech? Specialist software that runs in a basement rack and that eventually gets moved to a datacenter and compliance requirements begin demanding all the bells and whistles I just mentioned.
Without a doubt there's a lot of gross compute power that lives on the VMs you just mentioned. But all their financial processing is probably about a fraction of what some AS/400 or mainframe doing a nightly batch job, with software running from decades ago and licensing costs going into 7 figures a year.
What you're asking for just doesn't exist. You can do what you're mentioning across, maybe, a single product line and a half-dozen teams. But even that company needs to use CRMs, ERPs, and custom stuff for which you cannot possibly define platform requirements on your own, limited, terms.
A customer that I used to admin their Unix servers on had software on IBM mainframes, IBM AS/400s, Solaris, AIX, two SCO Unix machines running some proprietary hardware control plane, a few thousand Windows machines, etc. You want a "real" ERP product? It's gonna run on Oracle or DB2, forget about Postgres. That app you made 15 years ago running on MySQL with the ISAM storage engine? Forget about ever upgrading that. Need to interact with banks? Holy smokes have I got bad news for you. You need software to interact with medical records that requires special legal compliance across multiple jurisdictions? Well, no one cares what that runs on as long as it keeps the millions rolling in.
>Disaster recovery? Geographically separate redundant servers with failovers? Automated (and proven to work) backups? One-stop access control for infra maintenance? Audit controls for your database and storage objects? Tape backups?
These are our dev+test setups, and we're looking far more carefully at prod for the reasons you touch on. Those aren't necessary for every project too, eg hosting computer vision demos.
For our government projects, the government hosts it on their own OpenShift cluster that they maintain (including their own data centre), due to requirements for all data to be hosted within our boarders. The OpenShift cluster I setup is no-where near as well maintained as the governments, they have multiple FTE and it runs most of the open source gov't code. They have tape backups, rolling on-call staff, public developer chat for support, the whole deal.
What I setup is far more simple. We have daily/weekly/monthly rolling backups of postgres pods. We store some backups of those on digital ocean, but that's just a cheapo litttle linux server.
But now a team of 30 developers can easily spin up their own projects using a web-based GUI from basically just providing a Dockerfile or a link to a git repo. One of the oft-touted organizational benefits of "cloud" is that you don't have to wait a week for Ops to provision a VM. We get all that.
>I understand your use case, but your is very, very far from the sheer and absolute complexity and features that enterprise data centers have.
My point is that many things people host in AWS do not need enterprise quality. If you're a startup, then almost by definition you do not need enterprise quality (though, as always, it depends). We made a tonne of savings. I'm sure many others would by self-hosting and learning a moderate amount of Linux / Kubernetes.
I think that overstates the problem. It does not require a whole pile of skill to purchase a few rack mount servers from Dell or Supermicro with onsite 24 hour warranty, and plonk them in a co-lo. In the rare event the hardware does break ring Dell and ask them to fix it for you. When the onsite warranty end's in 7 years it's time to replace the servers.
The expertise required it literally minimal - not much beyond the ability to use a screw driver to install it into the rack and know how to connect the Ethernet cables. Then you have to plug in a USB and install whatever OS you want, of course, but you don't have to be onsite to do that. They all come with iDRAC's or the equivalent.
They will cost about $1000/yr with maybe 10TB raid disk, co-lo costs of around another $1000/yr for unlimited bandwidth. To rent the same dedicated metal is about $500/mo from OHV where I live, so over twice the cost.
Obviously, this is all impossible if you aren't big enough to have dedicated IT staff. And obviously, if you are likely to go through rapid change (well, something more extreme than adding a new server every now and then), it isn't the best plan. But for a stable mature business that employs several hundred people, all you are really doing is cutting out the middle man.
But it's insane that running a for-higher data center is considered a high-margin business. (And testament that the customers are VC-gorged price-unconcious baby gremlims.) In a sane economy, data centers for higher would be a fully-commoditized barely-profitable common carrier with little natural monopoly.
Reliability concerns make datacenters resilient to commoditization. A datacenter that’s available 90% of the time is worth vastly, vastly less than 90% as much as one that 99.95% of the time. Commodity businesses are largely built on presumptions of linearity. Produce 90% as much corn/iron/wood/widgets as you expected and you’ll probably make something like 90% of the money you expected. Produce a 90% available datacenter and you’ll have a hard time finding anyone willing to pay you anything. And that’s just availability, not to mention data durability, which is even more critically nonlinear.
> A datacenter that’s available 90% of the time is worth vastly, vastly less than 90% as much as one that 99.95% of the time.
That depends on your workload and the nature of the loss of availability. Is the completed work still there but just unreachable for an hour every few months? That might be okay for some folks.
Is your data center always available mon-fri but constantly has scheduled downtime on weekends? Might still be okay.
You can sue your provider for breaches in SLA. The rebate you automatically get likely exceeds what you'd otherwise recover. (Which is why they grant one). SLA payouts are usually very generous - I've gotten credit for the entire month for a one-hour outage of a service.
I don't know where this idea that cloud providers and other DCs don't pay out for SLA agreements is, but they absolutely do.
>But it's insane that running a for-higher[sic] data center is considered a high-margin business. (And testament that the customers are VC-gorged price-unconcious baby gremlims.)
Your analysis is incomplete which is why it looks like insane high-profit margins.
Amazon AWS (and MS Azure, Google Cloud) also sell high-value services on top of raw datacenters. It's not just commodity rack servers. Amazon keeps iterating on new value-added services (e.g. see new announcements at annual AWS re:invent conference[1]). E.g. AWS DynamoDB service was announced in 2012 and Netflix is one of the customers that use it.
In contrast, other datacenter companies that don't have the same higher value-added portfolio like Rackspace and DigitalOcean are losing money[2] or not even profitable yet[3]. Yes, the lower-tier datacenters are also adding value-added services but the breadth of their product portfolio is not in the same league as AWS/Azure/GCP.
Rackspace was losing so much business to AWS that they're trying to sell the idea of customers paying their RS employees to manage AWS.[4]
>(And testament that the customers are VC-gorged price-unconcious baby gremlims.)
Most of the revenue comes from non-VC businesses. A lot of old Fortune 1000 companies where IT is a cost center shrank their self-run datacenters and moved the workload to the cloud vendors. Another example is AWS winning the big $600 million contract from the CIA.
Recently I just began using cloud services. I started with Firebase, thought it was super cool, then moved to GCP, which was super inflexible (and super shit support). So I decided to try Azure, since we used that at a previous startup, but it was too complicated to even get started. So I decided to give Jeff my money (or rather avail of his free tier), and started using AWS. While it's super complicated to use compared to Firebase, it was relatively easy to learn from scratch due to huge amount of support online and what they have on offer. But what honestly stumped me was the HUUUUUUUGE amount of services they have, and the pricing on that. DynamoDB was half the price of Cloud Firestore, the free tier on most services was half that of GCP, and the offerings were insane. Need to host a website? Cloud Front. Launch an app? Lightsail. Satellite connection? Covered. Streaming Data? Kinesis. Queried Database? Elasticsearch. And that's just scratching the surface, I know.
Honestly, in a few years, I think we'll see Google forced to exit this space, Azure and AWS competing on price for big name corporate contracts, while others such as Digital Ocean being crumpled, simply because the AWS offering is so vast and widely supported online. And AWS' lock-in is pretty damn good.
The only solution to break out of such a duopoly would be for tech oriented companies to stop being lazy and start building out their own cloud infra.
> The only solution to break out of such a duopoly would be for tech oriented companies to stop being lazy and start building out their own cloud infra.
What competitive advantage do you get by making your own in-house, inferior version of an infrastructure service that won't benefit from AWSs economies of scale?
Not being overly dependent on an outside provider? AFAIK, most traditional corporates using clouds often use multiple cloud services of the same type from different providers.
Some of it is becoming more democratized too. Facebook as an example open sources it's datacenter design, it's servers, switches etc. I know they are the big bad for privacy these days, but the OCP project has contributed state of the art datacenter tech that legacy providers should be jumping at.
There is a lot of spectrum in the middle between AWS micro and your own datacenter(s). One don't have to jump from the cloud to datacenter - just rent/buy a server or two, or rack of server or two.
Sure, but the management overhead from 1 server to 20 racks scales, at best, linearly. I've seen a lot of places just past the point where a single sys-ad person with a thumb drive is viable, yet they choose to not invest here and instead spend 2-5x on going to the cloud.
all but the largest or most stubborn companies run their own datacenters are coming to a close
You're right about the stubborn part, but not necessarily largest. Many IT leaders at companies of all sizes have their political capital tied up in the data center. What you're talking about doing (with cloud adoption) is outsourcing 90% of what they control. For them it's existential, uptime and agility be damned.
Serverless is even worse for them, as far as IT fiefdoms are concerned.
You say "stubborn" but when the savings can be in the millions of dollars, it becomes logical to own a datacenter. I can't believe datacenter people suddenly all disappeared or are retired. If the pay scale is down, they will surely appreciate working for a big company with datacenter needs.
Obviously, this only applies to huge companies like Dropbox. Everyone else is better served by AWS.
They aren’t shrinking at all, we have the same talent pool and are trying to thinly spread it over all of the problems we can solve with networked computers. The amount of problems you can solve with networked computers has scaled way faster than our industries capability to find talent and train employees.
Who is “we”? Always curious what remains of my field—I’ve worked in the storage and infrastructure industry for over 20 years. If you mean the industry as a whole, I simply disagree. The talent pipeline is not there anymore. You used to be able to get started without an Engineering degree by “getting in at the bottom.” Now somebody on this path will never get past power supply swaps because they’re just doing drudge work fed to them on a tablet (“replace disk 17 on row 23, rack 9, slot 4”—“oh, yup, there’s the yellow light”). If you do have the degree, there are a billion more profitable paths, and most take them.
We is engineers who work in datacenters. Colocation datacenters have existed for several decades and remote hands working there would probably not see much difference between getting a call/fax/email/app telling them to do a break fix order. Having worked in dozens of datacenters over the past two decades is that there will always be newer semi-technical workers there doing break fix. A lot of those people would consider those jobs dead ends too and so this cloud datacenter thing is nothing new, its just on a much larger scale. The datacenter operators don't have to keep working there to level up their careers but working in a datacenter gives familiarity and the confidence to try more technical roles later.
Dropbox is in a relatively specific situation wherein their cloud costs would be extremely high (storage, bandwidth) and their in-house technical skills are probably quite good.
Second point is spot on. Not many companies have the engineering capabilities. Dropbox has so much potential, but their business vision wasn't great to say the least.
Dropbox is itself a cloud provider. A consumer/SMB cloud provider. It makes total sense they build up their own infrastructure, not only from a purely profit/scale point of view.
It's about their core values and identity, at least because in this way they are seen in the market as a big player and not just as another AWS reseller with some added benefits.
There's a lot of companies that have the scale to benefit from running their own tech, fewer who have the skills and number of people, but it's not a small niche even so.
Netflix is a counterpoint? A good fraction (though not majority probably) of their cost is probably bandwidth and computation yet they continue to offload that to a cloud provider, who's a direct competitor no less.
netflix does not serve their bits over amazon. they have their own major infra, and operate a large and distributed cdn, including edge caches colocated within eyeball networks.
it would be insane for them to deliver their video content over aws.
Netflix does run their own CDN for video content, presumably to avoid that exact problem. I would imagine that's by far the most expensive part of their business.
It's not to offset risks, if you have very little load (a site with 10k visits a day, for example), you can share the costs on the cloud and save nearly the entire bill.
Companies with a high revenue/load ratio tend to stay at the cloud even after they get big. That is because even though the cloud is very expensive for their needs, it adds speed on their internal processes by saving the time to decide and buy equipment. But when that ratio is small, they just can't afford it.
There are not a lot of those companies which are that big.
You also might not want to manage that many it experts for your infrastructure or you are not able to get them.
Also if your companies product is very technical, i would argue that those companies are much better equiped doing it by themselfs then others.
Nonetheless, it also doesn't need to be all or nothing. You can easily combine a MultiCloud approach.
Build only the stuff which is easy to build and costs a lot on cloud yourself. I would say Buildsystems or compute instances are good candidates.
Like i could imagine putting netflix authentication system on a cloud provider while doing the compute stuff in my own data center and building the CDN myself.
> Nonetheless, it also doesn't need to be all or nothing. You can easily combine a MultiCloud approach.
There may be reasons to go multicloud but ease isn’t one of them. You double your infra support overhead (or more likely, half its quality) and have a “least common denominator” experience.
The natural tendencies of large organizations is a diffusion of investment but the cheapest costs frequently come from a concentration of investment.
My employer does have a luxury of focus in its product offering, though we do have a moderately heterogeneous approach in development, certainly compared to many of the peers that operate at similar scale.
Heterogeneity in compute location has a multiplicative effect on accounting, security, capacity management, network management and is dilutive in terms of expertise -- instead of being able to justify the worlds leading experts in one system, you now need more staffing to cover a wider surface area (and they all need to have collaboration overhead to ensure they arent working at cross-purposes in strategy or tactic.)
I think this belief in marginal benefit from "right tool for the job" is a local-optimization where the costs of coordination and overhead are not borne locally and so are generally undervalued/discounted.
My employer runs on a single cloud provider, but -- do to its scale and closeness to core competency of our business -- we do operate our own CDN infrastructure, and this is a decision I happen to agree with. As a result of this division, I am acutely aware of the impact it can have on an engineering organization and only in certain specialized use-cases would advise considering DIY or multi-cloud.
I hear this sentiment repeated frequently, but I’ve never heard multicloud as leverage actually getting a better deal than an exclusivity deal. If you have a different experience I’d love to connect and learn more - email in my profile.
There are lots of entities that are big enough. Given that the cloud stacks change from time to time, you don't reduce your need for engineers and other SMEs -- in some cases you need more.
I would say as someone who supports lots and lots of apps that cloud services are usually financial winners in a SaaS perspective and in a rapid growth scenario. Nobody can deliver Exchange cheaper than Microsoft. My team stood up apps for covid related activity for 20-40% of the cost and more importantly type than services under our organizations control.
That said, for what I would call "base load" scenarios, in many scenarios it's exactly the opposite.
> I think the main advantage of cloud providers is to offset the risk of purchasing equipment that eventually is no longer needed,
Every corporate use case I have seen is labor based. They dont want the overhead of salary and healthcare for the IT department. Even if long term they end up paying more, they always view it has pay for it now or pay for it later. And they always choose later because they dont know better.
(none of these have been the scale of dropbox, that is different)
> And they always choose later because they dont know better.
This is a dangerous assumption to make. Delaying payments and going with the crowd are both safe decisions. Safe decisions are smart decisions under normal circumstances.
True there is not one answer for everyone. I just know Ive been a part of companies where we go IAAS, get rid of the people that have the knowledge to manage infrastructure and when it comes time to need the knowledge again we either need to get consultants or hire at a much more expensive rate as for infrastructure management talent pool is getting smaller everyday. To be fair I deal with the headaches more than the everything is fine and dandy so my views are skewed as a result.
Is it because Starbucks has the most efficient cost outlay for long term investment in a user's needs?
Or is it because people just want some damn coffee and there's one on every corner?
Or is it because, in a world full of places to buy coffee, one place gives you everything you could ever dream of in a coffee place?
That is what the big cloud providers are. They are Starbucks. They are not the cheapest. They aren't even the best. But they are everything you want.
If your company gets Starbucks-huge, you don't need to buy your coffee from Starbucks. You have your own deals with roasters and your own supply chains and baristas and coffee logistics experts.
That's why some companies build their own. Not because it's a better idea (it's not), or because Starbucks costs too much (compared to the investment in re-creating Starbucks?). It's because they are a business that effectively makes their own coffee already, so it makes no sense to pay Starbucks for it. Of course, they won't have a Starbucks once they make their own coffee, but they will have served their needs well enough.
This is not a great analogy. Sure people go to Starbucks for one off coffees and sure some do that exclusively. But almost every startup to enterprise company has an expresso machine in the kitchen.
So in most cases, if I want a coffee I don't go to Starbucks. They I go to the kitchen, it's faster, cheaper, and more convenient than going to Starbucks.
However, if I'm out and about it doesn't make sense for me to invest in temporary coffee infrastructure. In those cases it's easier to go to a cafe like Starbucks.
This matches the idea of the article. If you have consistent demand it makes sense to buy infrastructure to meet that demand. But if the project is temporary or extremely bursty it may make sense to have someone else do it for you.
Their product is extremely close to basically just reselling storage space. Of course it makes sense for them to build their own infrastructure.
For a company whose product is a saas application (business logic in code) with users spending hundreds or thousands of dollars per month, those cost savings may never materialize relative to the amount of infrastructure each customer is using per dollar.
Dropbox is essentially buying a barrel of gasoline and selling it in gallons. Their product can never be profusely more valuable than the underlying infrastructure.
As I recall there are giants like Netflix that still run on AWS...which brings up another point! If you’re large enough to consider your own data center, you’re large enough to negotiate contracts with cloud providers at below-retail rates.
> My biggest question is whether cloud providers could achieve a scale where they are able to offer the most optimal infrastructure costs for specific businesses. Maybe this is the case for smaller or mid-size companies, but I'd be interested to see where the inflection point lies.
Disclosure: I work at Microsoft on Azure, but I’m on the product/dev tool side not on infra.
I think this is already happening to a certain extent and will happen more in more verticals as time goes on. There are massive government use cases for the cloud and it isn’t as if governments and agencies haven’t been maintaining their own datacenters and servers before. Clouds optimized for healthcare are also a thing and are only becoming bigger — again, industries that have long maintained their own infra.
You also have the private cloud model, which OpenStack pioneered but Azure Stack and AWS Outpost have put their own spin on, which essentially lets you host specific cloud services and tools on your own infrastructure.
There are always going to be some businesses that reach a size and scale where it doesn’t make sense to offload to the cloud, where paying for people to do maintenance and support, build out monitoring, handle everything soup to nuts makes sense. I think Dropbox, which is a storage provider, is a key example of that.
I talked with the then CTO of Dropbox right after it finished moving from AWS to it’s own datacenters and the process was extraordinary and really impressive. For what Dropbox is doing, it makes sense that it owns and operates its own infrastructure and storage and tooling.
Of course, you can also have the inverse. Zynga famously moved off AWS as its demand peaked and it saw the cost savings, and then had to move back to it, after demand died down and the numbers of owning and maintaining its own infrastructure no longer made sense.
Netflix has moved much of its stuff in-house, but still relies on AWS and likely will for quite some time.
But on the whole, yes, I absolutely see cloud providers moving to offer specific business and business vertical centric solutions with pricing that is lower than what those businesses could achieve on their own, even if you take some of the “services” stuff out of it snd are just looking at raw infrastructure costs.
> it should be easier to migrate off these platforms, since their pricing is optimized for users of all business sizes and use cases
Isn't the financial implications of capex vs opex a huge consideration? I've heard that opex is is a lot simpler to account for. Technically once you get big enough cloud becomes more expensive. But hiring people to manage both your own datacenters and cloud services does complicate things. I find it understandable that companies are willing to pay more for cloud providers if their core business doesn't require expertise in cloud computing.
colo'd datacenter or 'off brand' cloud provider can easily be much cheaper than the big 3, still get you out of 'dealing with hardware' and either way you are still 'setting up infrastructure' in terms of developing software management tools for your system
> My biggest question is whether cloud providers could achieve a scale where they are able to offer the most optimal infrastructure costs for specific businesses.
IMO, this has happened. AWS GovCloud. Since the USGov has near unlimited spending power, it's better for integrators to just pass the costs along. Compared to certifying your own infrastructure, this will probably be much cheaper for most everybody.
But once the constraints of a userbase are more established, it should be easier to migrate off these platforms, since their pricing is optimized for users of all business sizes and use cases, whereas your specific hardware can be optimized for your specific users.
My biggest question is whether cloud providers could achieve a scale where they are able to offer the most optimal infrastructure costs for specific businesses. Maybe this is the case for smaller or mid-size companies, but I'd be interested to see where the inflection point lies.