中文Digitally Imported Standardizes Architecture on NGINX to Reduce Complexity, Increase Reliability, and Provide Scalability
NGINX Plus HA Configuration Easily Supports More Than 400,000 User Requests per Minute
Overview
Digital Imported (DI.FM) is a multi‑channel Internet radio service that exclusively streams electronic music. Each channel is curated by a team of avid electronic music fans who sift through petabytes of new releases and back catalogs to find only the best from each genre or style, including house, trance, techno, drum and bass, and dubstep. One of the first online radio networks to go live (in 1999), DI.FM has grown into a destination and lifestyle for over three million unique listeners who tune in every month through web browsers, dedicated mobile applications, and third‑party vendor products. In 2010, DI.FM was awarded Best Global Radio Station in the International Dance Music Awards at Miami’s annual Winter Music Conference.
Digitally Imported originally operated its infrastructure in dedicated co‑location facilities which featured numerous vendor appliances and software. But as their subscriber base grew, DI.FM recognized the need to reduce the complexity of their architecture and improve operational efficiency. Not only was the collection of different technologies cumbersome and difficult to manage, but DI.FM’s operations engineers had poor visibility into performance‑related issues, which significantly hampered their ability to ensure a scalable, reliable, and consistent service.
“The architecture we had was cobbled together over time to meet the needs as we grew. But it had come to a point where it was increasingly difficult to manage individual components and even more so to ensure the level of service our users expected.”
Challenge
Digitally Imported had created an architecture over time to support a growing, global user base that was increasingly streaming more music. They employed a mix of different special‑purpose tools to provide such network functionality as load balancing, SSL termination, application delivery, static file storage, and more. These tools included hardware appliances, Varnish, Apache, and even NGINX Open Source. But with a small operations team, maintaining such a diversity of technologies proved problematic, especially under peak loads which could reach up to 400,000 requests per minute. When Digitally Imported engineers began to seek a way to consolidate network functionality, they specifically looked for a high availability (HA)/failover load balancer to ensure traffic remained flowing to the primary API application powering much of their music service if a hardware load balancer failed. And they wanted a solution which could handle that failover without manual intervention by a Digital Imported operations engineer and yet include vendor support in the event it was needed. Given that Digitally Imported was already familiar with NGINX Open Source, it was an easy choice to standardize their architecture on NGINX Plus. By doing so they not only removed a significant amount of complexity, but also gained additional control, insight, and functionality to ensure optimal performance and consistent uptime.
Solution
Digitally Imported maintains a family of client applications (mobile applications and websites) which all communicate back to a central API to obtain metadata about streaming content, get schedules for radio shows, fetch URLs to assets on their CDN, and report what people are listening to so Digitally Imported can calculate artist royalty payments. The API is a monolithic Ruby on Rails application running on a fleet of servers behind the load balancers. The application server is NGINX Open Source employing Phusion Passenger to host the Ruby application and provide upstreams.
When Digitally Imported operations engineers explored solutions for highly available HTTP load balancers for the API, they landed on NGINX Plus. Already familiar with NGINX Open Source, they decided against expanding their existing hardware load balancer footprint as they were no longer delivering as much traffic from their co‑located locations and were relying more heavily on CDNs. They no longer required the raw power of a hardware‑accelerated appliance. In the new architecture, NGIN Plus replaces Varnish for load balancing and HTTP caching, Apache for WebDAV services, and proprietary hardware for application firewalls and various TCP proxying functionality.
“Although our primary reason for purchasing NGINX Plus was the straightforward active‑passive HA feature provided by its keepalived and VRRP support, our familiarity with NGINX community edition, coupled with the industry‑leading support and enhanced features of NGINX Plus, such as the integrated WAF functionality, made it a no‑brainer to standardize our new architecture on NGINX Plus,” says Seth Hochberg, Director of Engineering.
In the revised architecture, NGINX Plus serves as a highly available Layer 7 load balancer, the primary HTTP cache, SSL terminator, TCP proxy, web application firewall, and health monitor for upstream servers. The NGINX Plus load balancer proxies a JSON‑based API as well as static file assets like website images. During the busiest times of day, it handles around 400,000 requests per minute, serving roughly 70% of responses directly from its cache. This improves round trip time by eliminating the need for requests to continue upstream to the API server or a storage device. The NGINX Plus load balancers deliver sustained outbound traffic of roughly 2 Gbps from a combination of caches and upstreams in response to incoming requests. NGINX Open Source still acts as a static file server, web application host (with Phusion Passenger Enterprise), WebDAV server, and even performs real‑time precache image manipulation at the CDN origin. 100% of application traffic transits NGINX in some form.
“We run NGINX in some form on essentially every server in our network, with the exception of a few isolated roles like databases. Even our CDN vendors are running NGINX at their edge! It’s a vital tool for our core business of streaming music. The product is great and the integration, support, precompiled modules, and more that come with the NGINX Plus support packages are well worth the cost.”
Results
Increased Server Uptime
Prior to standardizing on NGINX Plus, Digitally Imported operations engineers had little insight when a hardware load balancer failed. With NGINX Plus, they can now better monitor load balancer performance and, by employing an HA/failover configuration, rest assured that even if a load balancer goes down users will experience uninterrupted service while engineers bring it back online. This not only ensures consistency and reliability for end users, but also significantly reduces the operational complexity of providing a high‑performance streaming service. Ultimately, server uptime is improved with better monitoring which enables engineers to proactively fix issues before they cause or contribute to server failure.
Improved Visibility into Delivery Performance
As delivery has moved from co‑located servers to CDN caches, visibility into the delivery workflow becomes increasingly important. But the combination of different technologies and hardware appliances previously used to service user requests significantly hampered such insight. NGINX Plus includes advanced monitoring functionality that provides enhanced visibility into the upstream and downstream performance of stream deliver. This enables Digitally Imported operations engineers to keep close tabs on how API requests are being handled and spread across the NGINX Plus architecture.
Better Operational Efficiency and Lower TCO
Managing multiple vendors, hardware appliances, and software deployments can eat up a lot of resources. From upgrades to patches and fixes to continual monitoring, any architecture employing a wide variety of technologies and products can eventually collapse underneath its own weight. For Digitally Imported, the difficulty managing such an architecture was compounded by a small operations team. NGINX Plus enabled Digitally Imported to radically consolidate network functionality from multiple vendors into a single, software‑based approach that can be deployed on commodity hardware. This provided a level of redundancy the team didn’t have before and a way to easily deploy, test, and monitor within a single configuration language and deployment strategy. DI.FM didn’t have to find or hire specialized resources to manage several vendor products, ensuring they could stay lean and agile while also gaining a high level of confidence in managing their network. The consolidation of multiple vendor tools into a single tool significantly reduces the overall time needed to maintain the streaming infrastructure (lowering operational cost) while simultaneously improving overall operational efficiency through a natively supported HA/failover configuration. This ensures, even in the event of downtime, that user requests are still handled in a high‑capacity, scalable manner.
About Digitally Imported
Founded in 1999, Digital Imported (DI.FM) is a multi‑channel Internet radio service that focuses exclusively on electronic music.
