LOCAWEB PIONEERS BRAZILIAN WEB SERVICES WITH NGINX PLUS

Overview

Locaweb is a pioneer and leader in professional hosting services in Brazil. Their services range from common user to the large corporations, through three verticals of business: Internet, data center, and communication.

Challenge

Locaweb is the largest provider of web hosting and infrastructure services in South America, and is the clear leader in Brazil. The company serves more than 265,000 customers, hosts more than 300,000 sites, and operates 20,000 total servers, both on premises and in the cloud. Although hosting is a large part of Locaweb’s business, the company also offers a full suite of Internet services, such as ecommerce, search marketing, software as a service (SaaS), cloud infrastructure as a service (IaaS), platform as a service (PaaS), and email. As a large email provider, Locaweb supports more than 10 million inbound email addresses, and processes roughly 200 million email messages a day.

Not surprisingly, Locaweb has faced a growing need for load‑balancing solutions over the past few years. The need became acute in 2014 when Locaweb enhanced email security. Locaweb used to make SSL/TLS optional for email, but switched to secure connections for all mail traffic to better meet its own security goals and the needs of its customers. This pleased customers greatly, and resulted in better security overall, but the hardware load balancers couldn’t handle the throughput over SSL/TLS. Locaweb began investigating alternative load balancers.

“We tried other software and hardware load balancers, but in our internal tests we didn’t get the high throughput and performance that we needed, especially with SSL. In addition, hardware licenses are very expensive,” says Danilo Acquaviva, Systems Administrator at Locaweb. “We were already familiar with NGINX as a cache system, proxy server, and web server. We were very interested in trying NGINX Plus for its advanced load‑balancing features.”

Solution

NGINX is the world’s most popular open source web server for high‑traffic websites. With NGINX Plus, customers get additional enterprise‑ready features such as HTTP and TCP load balancing, health checks, session persistence, advanced monitoring, and more.

In Locaweb’s internal tests, NGINX Plus clearly outperformed the other load balancers. Now, all email services at Locaweb are running behind NGINX Plus.

The company replaced a cluster of hardware appliances with NGINX Plus load balancers. For its internal email platform, 3 NGINX Plus machines load balance traffic across about 80 backend servers hosting 3.4 million mailboxes. For its large enterprise customers, Locaweb has deployed 2 additional clusters – one with 3 NGINX Plus machines handling 37 servers and 2.2 million mailboxes, and the other with 3 NGINX Plus machines load balancing 60 servers and 800,000 mailboxes. In the second cluster, NGINX Plus is load balancing ActiveSync protocol traffic too.

Further, because all Locaweb websites use the same environment for email, it was an easy decision to switch to NGINX Plus for load balancing all of the websites as well. Locaweb deployed another three NGINX Plus machines for this purpose. All NGINX Plus instances are working as load balancers and some have caching enabled. They are deployed on physical servers, the biggest with 196 GB of RAM, 16 processors, 2 SAS disks in RAID1 (1T), and a 10‑Gbps network interface.

Migrating a large email platform to a new load‑balancing solution is a significant project, and one that Locaweb wanted to implement carefully in order to avoid negatively impacting customers as it deployed the enhancements to its offerings. Senior sysadmins migrated a few domains per week in small steps. The overall migration from hardware load balancers to NGINX Plus took about four months.

Results

Amazing Performance

NGINX Plus is a complete application delivery platform, optimized for handling high traffic volumes. Its sophisticated event‑driven architecture enables NGINX Plus to scale and handle a very large number of simultaneous connections using a very small memory footprint.

Better Visibility Allows for Rapid Response

Using the NGINX Plus live activity monitoring dashboard, Locaweb’s teams have deep visibility into how their backends and load balancers are working. They can easily see how much traffic each server is handling, what kind of traffic is coming in, where there are failures, and more. This is extremely helpful for system maintenance and for addressing possible incidents. NGINX Plus provides this data in a JSON format, which Locaweb feeds into its own monitoring system.

“The metrics from NGINX Plus help us achieve the uptime and performance we need,” says Claudio Borges da Silva Filho, Technical Leader of the Linux Team. “We developed a system for monitoring using the JSON output from NGINX Plus. If a server stays out of its pool for a specific period of time, we automatically trigger an internal incident report and our staff check the issue. The visibility we get with NGINX Plus is very useful to us.”

Fast and Easy to Deploy New Configurations

NGINX Plus easily integrates into the DevOps culture and workflows at Locaweb. The technical teams are using a DevOps style for deployment to production. They have defined a configuration manager role, which allows specific people to deploy new configurations. With the vast size of their environment, the role of configuration manager is a big deal.

“One small mistake in deployment can result in a big problem over here. We were delighted that NGINX Plus easily integrated into our DevOps environment. NGINX Plus works well with CFEngine, our configuration management system. We can replace variables in the templates and can have the same configurations deployed across the entire environment in no time at all,” says Filho.

More Flexibility Helps Ensure a High‑Quality User Experience

Locaweb uses the dynamic reconfiguration feature in NGINX Plus to easily add and remove servers from upstream groups. Locaweb’s sysadmins are even building a control panel on top of NGINX Plus to further optimize it for their environment. Locaweb needs to respond quickly if a server crashes, or a hard drive or network interface needs to be replaced. Building on NGINX Plus’s dynamic reconfiguration capability, Locaweb’s control panel will give its sysadmins control over the specific servers they maintain within the larger environment. This will enable more sysadmins to remove or add servers rapidly without physical access to them, and to respond quickly to potential issues to ensure a high‑quality user experience.

Exceeds Expectations at a Fraction of the Cost

Locaweb found that costs are much lower with NGINX Plus, on several levels. As a software load balancer, NGINX Plus replaces hardware appliances and add‑on licenses that cost much more – and still weren’t doing the job. To get the same performance without load balancing rquires a very large number of regular servers, a very expensive proposition. And Locaweb was already very familiar with NGINX Open Source, which meant they could quickly get up and running with the advanced capabilities of NGINX Plus.

Locaweb plans to migrate even more hardware load balancers to NGINX Plus and use it as its main load‑balancing solution.

About Locaweb

Locaweb is the leading provider of hosting infrastructure services in Brazil, according to IDC. With 17 years of experience, about 1,000 employees, and 265,000 customers, Locaweb offers software, platform, infrastructure, and marketing solutions. Its Enterprise Solutions unit is responsible for project delivery, from simple projects to fully customized scopes. With capacity for more than 25,000 servers, according to the Hostmapper report, the company is the leader in the market for .com.br domains. Between 2012 and 2013, Locaweb acquired four new companies: Eventials, Tray, SuperPay, and All In Mail. Locaweb believes that innovation, quality of service, and its highly qualified team are the key factors to achieving success in the market. For more information, visit www.locaweb.com.br.