NGINX Plus 和 Elasticsearch：更好地合作

Rick Nelson of F5

方案工程区域副总裁

2015-03-05

Elasticsearch is an advanced, high‑performance, and scalable open source search engine that provides full‑text search and real‑time analytics for structured and unstructured data. Its features are available using a RESTful API over HTTP, making it easy to fit into modern web architectures. Because NGINX Plus load balances HTTP traffic with outstanding performance at very high concurrency rates, it’s ideal for use as a reverse proxy load balancer as you scale up to more than one Elasticsearch server.

Editor – The first Elasticsearch User Conference – Elastic{ON} – took place at Pier 27 in San Francisco on March 9–11, 2015, shortly after the original publication of this blog. It was the single largest gathering of Elasticsearch, Logstash, and Kibana expertise anywhere in the world at the time. NGINX, Inc. was a conference sponsor.

For more about Elasticsearch with NGINX Plus, see Monitoring NGINX Plus Statistics with ELK on our blog.

The Advantages of Deploying NGINX Plus with Elasticsearch

Elasticsearch has a few features designed for scalability, but you can free up resources on your Elasticsearch servers by offloading the load balancing of requests to NGINX Open Source or NGINX Plus, which has even more enterprise‑grade features.

Putting NGINX or NGINX Plus in front of a single Elasticsearch server provides benefits, such as request logging, but the real value comes when you scale to multiple Elasticsearch servers.

In addition to logging each API request, NGINX and NGINX Plus:

• Support a very large number of client connections (with or without keepalives enabled) with a much smaller of number of persistent connections (using keepalives) to the Elasticsearch servers.
• Load balance requests across the Elasticsearch servers.
• Cache data locally to reduce repeated requests to the Elasticsearch servers for the same content. NGINX Plus further provides an HTTP API for purging objects from the cache.
• (NGINX Plus only) Provides application health checks to continually check whether the Elasticsearch servers are up and functioning. Down servers are automatically removed from the load balancing rotation and returned to it when they come back up.
• (NGINX Plus only) Reports a wealth of real‑time metrics to improve monitoring and management. A demonstration of the metrics is available at demo.nginx.com.
• (NGINX Plus only) Supports dynamic reconfiguration of upstream server groups via the NGINX Plus API. You can add and remove servers from the upstream group, mark them up or down, and change their weights.

The following table compares the functionality available with Elasticsearch by itself, combined with NGINX Open Source, and combined with NGINX Plus:

When you use NGINX Plus in a high‑availability active‑passive configuration to load balance a cluster of Elasticsearch servers, another benefit is that you can configure the Elasticsearch clients to direct requests to NGINX Plus, rather than to the Elasticsearch servers. This means you can scale the Elasticsearch server cluster as needed without having to update the clients.

Deploying NGINX Plus with Elasticsearch

Deploying NGINX Plus with Elasticsearch is very straightforward. The directives in the following sample NGINX Plus configuration define settings for load balancing two Elasticsearch servers:

• upstream – Load balance requests across two Elasticsearch servers with IP addresses 192.168.187.132 and 192.168.187.133 that listen on port 9200.
• proxy_cache_valid – Cache valid responses for 10 minutes and 404 responses for 1 minute.

• health_check – Actively check the health of the servers, taking them out of the load‑balancing rotation when they are down and adding them back into the rotation when are healthy again. The health check references the match block, which not only makes sure that the server returns a status code of 200, but checks that the response is formatted in JSON and that it returns the string expected from Elasticsearch:

  "status" : 200

• Final server block – Enable the NGINX Plus API to collect statistics and listen on port 8080 for requests for the dashboard.html page which displays the statistics.

# in the 'http' context
proxy_cache_path /var/cache/nginx/cache keys_zone=elasticsearch:10m inactive=60m;

upstream elasticsearch_servers {
    zone elasticsearch_servers 64K;
    server 192.168.187.132:9200;
    server 192.168.187.133:9200;
}

match statusok {
    status 200;
    header Content-Type ~ "application/json";
    body ~ '"status" : 200';
}

server {
    listen 9200;
    status_zone elasticsearch;
    location / {
        proxy_pass http://elasticsearch_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_cache elasticsearch;
        proxy_cache_valid 200 302 10m;
        proxy_cache_valid 404 1m;
        proxy_connect_timeout 5s;
        proxy_read_timeout 10s;
        health_check interval=5s fails=1 passes=1 uri=/ match=statusok;
    }

    # redirect server error pages to the static page /50x.html
    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
    }
    access_log logs/es_access.log combined;
}

server {
    listen 8080;
    root /usr/share/nginx/html;

    location = /api {
        api write=on;
        # directives to restrict access to authorized networks or users
    }

    location / {
        index dashboard.html;
    }
}

Summary

Elasticsearch, a powerful and flexible search engine, and NGINX Plus, an enterprise‑grade application delivery platform, fit naturally together as components of a modern scalable web architecture. As software products, they provide the same features and functions wherever you install them: on bare‑metal servers, in virtual machines, in the cloud, or in containers.

To try out NGINX Plus with Elasticsearch for yourself, start your free 30‑day trial of NGINX Plus today or contact us to discuss your use cases.

Further Reading

Playing HTTP Tricks with Nginx on the Elasticsearch blog

Securing Elasticsearch with Nginx by James McFadden

Monitoring NGINX Plus Statistics with ELK on our blog