On Thursday, Amazon’s chief technology officer, Werner Vogels, announced the launch of a tool for running clusters of the Apache Kafka open-source software for processing data.
Kafka was originally developed inside LinkedIn, and its creators formed a start-up called Confluent that spun out of LinkedIn in 2014. Today, Confluent fields a cloud-managed version of Kafka that can be run on Amazon’s cloud, or in other clouds and even in on-premises data centers.
Confluent’s product is designed to be easier for people to use than the open-source project that developers would have to set up and manage computing and data storage infrastructure on their own. For years, AWS has suggested that developers interested in Kafka use its proprietary product called Kinesis, but AWS now has a product specifically with Kafka in mind. It goes by the name Amazon Managed Streaming for Kafka.
“People basically want the real thing, so in that sense, it’s good for the open-source companies,” Jay Kreps, a creator of Kafka and co-founder and CEO of Confluent, told CNBC on Thursday. “This is obviously hard, because it’s kind of a dual thing. Rather than collaborate with the company and kind of build the ecosystem of those offerings in their cloud, they obviously want to do it themselves.”
What’s more, Kreps said, Amazon has not contributed a single line of code to the Apache Kafka open-source software and is not reselling Confluent’s cloud tool.
“They’re profiting from our use of resources,” Kreps said. “We’re charging our customers for the service, paying for compute and network resources that we use.”
Also this week, AWS introduced a marketplace for artificial intelligence algorithms that can be used in SageMaker, an AWS tool for training and running artificial intelligence models. A start-up called Algorithmia offers a marketplace of more than 5,000 AI algorithms.
But that’s not everything that Algorithmia does. “Our real power comes from our ability to deploy and manage models at scale,” Algorithmia co-founder and CEO Diego Oppenheimer wrote in an email.
That can happen atop Amazon’s cloud, or in other clouds, like Microsoft Azure — or, alternatively, in corporate data centers.
“I think AWS’s marketplace will cement the notion that discoverability and accessibility of AI models is key to success and adoption in the industry, which we have been preaching for years,” Oppenheimer said.
Amazon is also reportedly working on a service that would compete with MongoDB, a company that sells subscriptions and services around the MongoDB open-source database software — an AWS-based version of the database is available — although AWS didn’t announce anything along those lines this week.
In past years, Amazon has announced other products that competed with select customers, including Dropbox, which has since become more reliant on its own data center infrastructure, although it does still use Amazon, according to its latest earnings report. Amazon announced its file-sharing service Zocalo in 2014 and has since renamed it WorkDocs.
Another example is Elastic, which went public last month. Elastic’s team commercializes the Elasticsearch open-source search software, among other components. In 2015, months after Elastic announced a cloud implementation of Elasticsearch that can be run on AWS, AWS itself launched a hosted version of Elasticsearch. But Elastic doesn’t support Amazon’s version.
Elastic addressed the competitive dynamic between itself and Amazon in regulatory filings this year.
“Amazon competes with us for potential customers, and while Amazon cannot provide our proprietary software, the pricing of Amazon’s offerings may limit our ability to adjust the price of our products,” Elastic said in a filing in September.
In a statement, Amazon said, “If you look at the history of AWS, we deliver what our customers ask us to build. There are times when there is some degree of overlap with what our customers offer, but most of these market segments are quite large and support several successful entries. And, for customers who’ve built meaningful offerings with significant functionality, we’ve yet to see these companies struggle to keep growing simply because AWS offers something in that area, too.”