CASE STUDY Segment Uses Containers and Optimizes Operations to Drive Efficiency

Segment acts as a central infrastructure that collects, stores, and routes first-party user data to hundreds of tools in seconds so that businesses can understand their customer-facing interactions. The company eliminates the headache of messy analytics installations to provide customers with the agility to try new tools and test campaigns quickly. With over 5.8 billion end-users worldwide, Segment’s dynamic cloud environment requires increased governance and visibility to efficiently manage their containerized workloads.
 

Segment's Challenge

The majority of Segment’s cloud environment is containerized and runs on Amazon Web Services (AWS), where the team relies heavily on Amazon Elastic Container Service (ECS) to manage their stateless workloads, with a smaller percentage of resources running on Google Cloud Platform (GCP). Because of their growing cloud footprint, Segment’s IT team no longer has the resource bandwidth to manually analyze their cloud bill and determine where costs could be optimized.

Segment also relies heavily on auto-scaling, which in turn makes their cloud spend highly dynamic. “To onboard a customer, Segment starts ingesting all of their first-party data — web activity, clickstream, you name it,” said Tido Carriero, Chief Product Development Officer at Segment. “A single customer coming online could require 50,000 requests per second. And we total about 300,000 requests per second, on average. There are spikes when a customer has a high traffic event, so it’s quite a dynamic, high scale load.”

Customers were pushing on the limits of Segment’s infrastructure, and company leadership wanted to ensure this new demand wouldn’t have a negative impact on cost of goods sold (COGS) and overall gross margin.

To solve for these challenges, the Segment team determined they needed a cloud management platform that would provide visibility into their dynamic cloud environment, automate manual/time-intensive processes, and better govern their overall cloud infrastructure.
 

Finding a Solution

To ensure that the business would continue to scale efficiently while maintaining high levels of customer satisfaction and support, Segment chose CloudHealth as its cloud management platform to support visibility into container workloads, optimize cost savings, and assist with tagging.

Segment has seen considerable success in a short period. “Implementing CloudHealth was really the beginning of the turnaround for us. As soon as we had that visibility from CloudHealth, we were able to chip away at the problem and make sure our COGS didn’t spike if certain customers pushed the limits of our infrastructure,” shared Carriero.

“Being able to share CloudHealth dashboards helped our engineers understand the cost problem more,” said Albert Strasheim, Director of Engineering at Segment. To more closely manage their cloud spend, CloudHealth set up auto-alerts to notify the Segment team if there were significant unplanned spikes in cost. Segment teams treat the auto-alerts as high priority, which enables them to easily address cost drivers and mitigate risks of unexpected cost increase.

CloudHealth’s proven ability to support Amazon ECS was one of the main reasons Segment chose the platform. Segment has many multitenant clusters, which present unique challenges for understanding cost. With multiple products and services on the same ECS cluster, Segment struggled with finding cost visibility into those complex environments, so the team built a CloudHealth report that provided valuable insight into business metrics. “This report went out to all key stakeholders, allowing us to take action quickly when we saw a change in cost,” shared Travis Cole, Director of Site Reliability Engineering at Segment.

With CloudHealth, Segment can easily analyze and identify problem areas to drive greater efficiency. Some changes are as simple as rightsizing while others require bigger engineering effort. The ability to share analysis and breakdowns across many groups and teams has proved invaluable.

From the CloudHealth Platform, Segment has scheduled a daily cost report to ensure rapid response and constant improvement. “The ability to drill down to the individual line items in the bill with CloudHealth drove some very interesting changes in how we think about our system. It has helped us deeply understand our costs,” Strasheim explained.

CloudHealth acts as a singular view into cloud and business metrics that makes it easy for Segment to compare and contrast details with real-time data. CloudHealth allows Segment to drive costs as a metric/KPI for better control.

Results

CloudHealth has provided Segment with a better overall understanding of how their cloud environment functions and has enforced greater accountability on team members. As a result, Segment has greatly scaled operations, increased revenue, and expanded cost drivers. Additionally, Segment’s finance team is able to feel confident in their cost drivers, allowing them to craft more competitive and agile pricing for enterprise customers. “CloudHealth’s cost optimization dashboards have empowered Segment to easily analyze cost and gain visibility into our infrastructure to ultimately increase our gross margin by about 20 percentage points. That blew our board of directors away,” Carriero shared.

CloudHealth’s cost visibility tools have enabled engineers at Segment to analyze dashboards and drive greater cost savings initiatives. The team is able to understand costs on a per container basis and a multi-tenant cluster basis, as well as costs associated with other AWS services. Specifically, application load balancers have allowed the engineering team to drill down into specific line items and create new ways to architect the system to reduce cost.

Segment’s rapid maturity within the cloud is quite impressive. The team now uses a combination of CloudHealth and custom Tableau reporting to combine business data with cloud usage and cost data to gain deeper visibility into their cloud environment. “It is incredibly important to understand the efficiency of each part of our product, so we can price it as accurately as possible. Without combining metrics, it’s impossible to get a crystal clear metric,” explained Cole.

What's Next for Segment?

CloudHealth has given Segment valuable insights into their cloud and containerized environments. Segment partners with CloudHealth to analyze where the team can maximize efficiency and this data is used alongside business metrics to impact long term business planning. Segment has developed a strong foundation to act proactively on any cost issues and will continue leveraging CloudHealth for additional governance and optimization features.
 

DOWNLOAD PDF    View All Case Studies