Difference between revisions of "Nginx/Caching"

From WhyAskWhy.org Wiki
Jump to: navigation, search
m (Added examples of what does/doesn't work.)
m (Fixed invalid closing tag which hid recent changes from view.)
 
(8 intermediate revisions by the same user not shown)
Line 22: Line 22:
 
Parameters of caching can also be set directly in the response header. This has a higher precedence than setting of caching time using the directive. The <code>X-Accel-Expires</code> header field sets caching time of a response in seconds. '''The value 0 disables the caching of a response'''. If a value starts with the prefix <code>@</code>, it sets an absolute time in seconds since Epoch, up to which the response may be cached. If header does not include the <code>X-Accel-Expires</code> field, parameters of caching may be set in the header fields <code>Expires</code> or <code>Cache-Control</code>. If a header includes the <code>Set-Cookie</code> field, such a response will not be cached. Processing of one or more of these response header fields can be disabled using the <code>proxy_ignore_headers</code> directive.
 
Parameters of caching can also be set directly in the response header. This has a higher precedence than setting of caching time using the directive. The <code>X-Accel-Expires</code> header field sets caching time of a response in seconds. '''The value 0 disables the caching of a response'''. If a value starts with the prefix <code>@</code>, it sets an absolute time in seconds since Epoch, up to which the response may be cached. If header does not include the <code>X-Accel-Expires</code> field, parameters of caching may be set in the header fields <code>Expires</code> or <code>Cache-Control</code>. If a header includes the <code>Set-Cookie</code> field, such a response will not be cached. Processing of one or more of these response header fields can be disabled using the <code>proxy_ignore_headers</code> directive.
 
</blockquote>
 
</blockquote>
 +
 +
=== http section ===
 +
 +
==== Logging cache hits/misses ====
 +
 +
The following log format <ref name="log-format" /> is used to help troubleshoot what content is and isn't being cached.
 +
 +
Here is what I'm using for now:
 +
 +
<syntaxhighlight lang="nginx">
 +
log_format cache '***$time_local '
 +
    '$upstream_cache_status '
 +
    'Cache-Control: $upstream_http_cache_control '
 +
    'Expires: $upstream_http_expires '
 +
    '$host '
 +
    '"$request" ($status) '
 +
    '"$http_user_agent" '
 +
    'Args: $args '
 +
    'HTTP Cookie: $http_cookie ';
 +
</syntaxhighlight>
 +
 +
 +
==== Cache path ====
 +
 +
Setup cache <ref name="cache-path-explanation" /> for later use in server block(s):
 +
 +
<syntaxhighlight lang="nginx">
 +
    fastcgi_cache_path /var/cache/nginx/
 +
        levels=1:2 keys_zone=fastcgi_cache:8m max_size=250m inactive=3d;
 +
    fastcgi_temp_path /var/cache/nginx/tmp;
 +
</syntaxhighlight>
 +
 +
 +
=== server section ===
 +
 +
==== Using the cache log ====
 +
 +
We defined it earlier, and now will use it in a server block (I still think of them as vhost blocks from my dealings with Apache HTTPd) where we're testing the various cache settings:
 +
 +
<syntaxhighlight lang="nginx">
 +
server {
 +
    ...
 +
    access_log /var/log/nginx/testing.cache.log cache;
 +
    ...
 +
}
 +
</syntaxhighlight>
  
  
 
=== What works ===
 
=== What works ===
  
So far I've determined that a nested setup works:
+
Nesting a FastCGI-enabled <code>location</code> block inside another with appropriate cache settings (r411) works. The benefit here seems to be that any location block that ''is not'' nested under <code>location /</code> does not have caching enabled unless explicitly turned on.
  
 
<syntaxhighlight lang="nginx">
 
<syntaxhighlight lang="nginx">
 
+
server {
 +
    ...
 
     # Nested location block (PHP/FastCGI content)
 
     # Nested location block (PHP/FastCGI content)
  
 
     location /
 
     location /
 
     {
 
     {
        proxy_buffering on;
 
        proxy_cache proxy_cache;
 
        proxy_cache_valid 200 302 3d;
 
        proxy_cache_valid 404 1m;
 
        proxy_cache_use_stale  error timeout invalid_header http_500;
 
        proxy_cache_key    "$scheme$request_method$host$uri$is_args$args";
 
 
 
 
         fastcgi_cache fastcgi_cache;
 
         fastcgi_cache fastcgi_cache;
 
         fastcgi_cache_valid 200 302 3d;
 
         fastcgi_cache_valid 200 302 3d;
Line 52: Line 91:
 
         location ~ \.php$  
 
         location ~ \.php$  
 
         {
 
         {
 
 
           # Placing these in a separate file for now so they can be referenced
 
           # Placing these in a separate file for now so they can be referenced
 
           # directly when needed (until I learn of a better way)
 
           # directly when needed (until I learn of a better way)
Line 62: Line 100:
 
          
 
          
 
     }
 
     }
 +
}
 +
</syntaxhighlight>
 +
 +
 +
The flip side of that (r414) is placing the <code>fastcgi_cache</code> directives at a <code>server</code> level. This turns on caching for ''all'' <code>location</code> blocks within the <code>server</code> block. You then have to disable caching within a <code>location</code> block where you don't want it to be used.
 +
 +
<syntaxhighlight lang="nginx">
 +
server {
 +
    ...
 +
 +
    # Placing the fastcgi_cache settings at the server level (outside of location blocks)
 +
 +
    fastcgi_cache fastcgi_cache;
 +
    fastcgi_cache_valid 200 302 3d;
 +
    fastcgi_cache_valid 404 1m;
 +
    fastcgi_cache_use_stale  error timeout invalid_header http_500;
 +
    fastcgi_cache_key  "$scheme$request_method$host$uri$is_args$args";
 +
 +
    # Content matching this location block WILL be cached due to server-level cache settings
 +
    location ~ \.php$
 +
    {
 +
 +
      # Placing these in a separate file for now so they can be referenced
 +
      # directly when needed (until I learn of a better way)
 +
      include php_parameters.conf;
 +
 +
      # Misc general FastCGI and Proxy settings.
 +
      include proxy.inc.conf;
 +
    }
 +
}
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
=== What does not work ===
 
=== What does not work ===
  
Not all that surprising, but this does not result in cached FastCGI (PHP) content:
+
This is because location blocks don't inherit from <code>location /</code> nor each other unless one is nested within another. In some cases even that doesn't seem to provide reliable results <ref name="passenger-enabled-directive" />.
 +
 
  
 
<syntaxhighlight lang="nginx">
 
<syntaxhighlight lang="nginx">
 +
server {
 +
    ...
 
     # Location blocks on the same level
 
     # Location blocks on the same level
  
 
     location /
 
     location /
 
     {
 
     {
        proxy_buffering on;
 
        proxy_cache proxy_cache;
 
        proxy_cache_valid 200 302 3d;
 
        proxy_cache_valid 404 1m;
 
        proxy_cache_use_stale  error timeout invalid_header http_500;
 
        proxy_cache_key    "$scheme$request_method$host$uri$is_args$args";
 
 
 
 
         fastcgi_cache fastcgi_cache;
 
         fastcgi_cache fastcgi_cache;
 
         fastcgi_cache_valid 200 302 3d;
 
         fastcgi_cache_valid 200 302 3d;
Line 88: Line 151:
 
     }
 
     }
  
     # Content matching this location block will not be cached
+
     # Content matching this location block WILL NOT cached
 
     location ~ \.php$  
 
     location ~ \.php$  
 
     {
 
     {
Line 99: Line 162:
 
       include proxy.inc.conf;
 
       include proxy.inc.conf;
 
     }
 
     }
 +
}
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
  
 
== References ==
 
== References ==
  
<references />
+
<references>
 +
<ref name="log-format">[http://blog.infoentropy.com/Custom_Nginx_logging_format infoentropy.com - Custom nginx logging format (cache log)]</ref>
 +
<ref name="cache-path-explanation">[http://www.djm.org.uk/wordpress-nginx-reverse-proxy-caching-setup/ www.djm.org.uk - Using Nginx as a reverse caching proxy for Wordpress: the setup]</ref>
 +
<ref name="passenger-enabled-directive">The Phusion Passenger module directive <code>passenger_enabled on</code> [https://groups.google.com/forum/?fromgroups#!topic/phusion-passenger/UvsMKK1hzEU needs to be specified in '''every single''' <code>location block</code> that you have].</ref>
 +
</references>
  
 
* [http://wiki.nginx.org/HttpFastcgiModule nginx Wiki - HttpFastcgiModule]
 
* [http://wiki.nginx.org/HttpFastcgiModule nginx Wiki - HttpFastcgiModule]
Line 109: Line 178:
 
* [http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html nginx Documentation - ngx_http_fastcgi_module]
 
* [http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html nginx Documentation - ngx_http_fastcgi_module]
 
* [http://nginx.org/en/docs/http/ngx_http_proxy_module.html nginx Documentation - ngx_http_proxy_module]
 
* [http://nginx.org/en/docs/http/ngx_http_proxy_module.html nginx Documentation - ngx_http_proxy_module]
 +
 +
 +
== More information ==
 +
 +
* [http://www.evanhoffman.com/evan/2012/03/07/using-nginx-as-a-caching-proxy-in-front-of-wordpress/ Using Nginx as a caching proxy in front of WordPress]
 +
* [http://blog.martinfjordvald.com/2012/08/understanding-the-nginx-configuration-inheritance-model/ Understanding the Nginx Configuration Inheritance Model]

Latest revision as of 13:01, 25 August 2012



Pay close attention to what version of nginx a book or web resource is based on as behavior may change between releases [1]


For the last few weeks I've been trying to figure out how to properly configure caching for nginx sites and have had hit/miss (pun intended) results. The notes here are my attempt to keep up with what has worked so I can go from there.

Caching FastCGI content

From the FastCGI module wiki page:

The cache honors backend's Cache-Control, Expires, and etc. since version 0.7.48, Cache-Control: private and no-store only since 0.7.66, though. Vary handling is not implemented.

From the http_proxy_module page:

Parameters of caching can also be set directly in the response header. This has a higher precedence than setting of caching time using the directive. The X-Accel-Expires header field sets caching time of a response in seconds. The value 0 disables the caching of a response. If a value starts with the prefix @, it sets an absolute time in seconds since Epoch, up to which the response may be cached. If header does not include the X-Accel-Expires field, parameters of caching may be set in the header fields Expires or Cache-Control. If a header includes the Set-Cookie field, such a response will not be cached. Processing of one or more of these response header fields can be disabled using the proxy_ignore_headers directive.

http section

Logging cache hits/misses

The following log format [2] is used to help troubleshoot what content is and isn't being cached.

Here is what I'm using for now:

log_format cache '***$time_local '
    '$upstream_cache_status '
    'Cache-Control: $upstream_http_cache_control '
    'Expires: $upstream_http_expires '
    '$host '
    '"$request" ($status) '
    '"$http_user_agent" '
    'Args: $args '
    'HTTP Cookie: $http_cookie ';


Cache path

Setup cache [3] for later use in server block(s):

    fastcgi_cache_path /var/cache/nginx/
        levels=1:2 keys_zone=fastcgi_cache:8m max_size=250m inactive=3d;
    fastcgi_temp_path /var/cache/nginx/tmp;


server section

Using the cache log

We defined it earlier, and now will use it in a server block (I still think of them as vhost blocks from my dealings with Apache HTTPd) where we're testing the various cache settings:

server {
    ...
    access_log /var/log/nginx/testing.cache.log cache;
    ...
}


What works

Nesting a FastCGI-enabled location block inside another with appropriate cache settings (r411) works. The benefit here seems to be that any location block that is not nested under location / does not have caching enabled unless explicitly turned on.

server {
    ...
    # Nested location block (PHP/FastCGI content)

    location /
    {
        fastcgi_cache fastcgi_cache;
        fastcgi_cache_valid 200 302 3d;
        fastcgi_cache_valid 404 1m;
        fastcgi_cache_use_stale   error timeout invalid_header http_500;
        fastcgi_cache_key   "$scheme$request_method$host$uri$is_args$args";
        
        # Will content be cached if embedded within another location block?
        # In this configuration, the answer is yes.
        location ~ \.php$ 
        {
           # Placing these in a separate file for now so they can be referenced
           # directly when needed (until I learn of a better way)
           include php_parameters.conf;

           # Misc general FastCGI and Proxy settings.
           include proxy.inc.conf;
        }
        
    }
}


The flip side of that (r414) is placing the fastcgi_cache directives at a server level. This turns on caching for all location blocks within the server block. You then have to disable caching within a location block where you don't want it to be used.

server {
    ...

    # Placing the fastcgi_cache settings at the server level (outside of location blocks)

    fastcgi_cache fastcgi_cache;
    fastcgi_cache_valid 200 302 3d;
    fastcgi_cache_valid 404 1m;
    fastcgi_cache_use_stale   error timeout invalid_header http_500;
    fastcgi_cache_key   "$scheme$request_method$host$uri$is_args$args";

    # Content matching this location block WILL be cached due to server-level cache settings
    location ~ \.php$ 
    {

       # Placing these in a separate file for now so they can be referenced
       # directly when needed (until I learn of a better way)
       include php_parameters.conf;

       # Misc general FastCGI and Proxy settings.
       include proxy.inc.conf;
    }
}

What does not work

This is because location blocks don't inherit from location / nor each other unless one is nested within another. In some cases even that doesn't seem to provide reliable results [4].


server {
    ...
    # Location blocks on the same level

    location /
    {
        fastcgi_cache fastcgi_cache;
        fastcgi_cache_valid 200 302 3d;
        fastcgi_cache_valid 404 1m;
        fastcgi_cache_use_stale   error timeout invalid_header http_500;
        fastcgi_cache_key   "$scheme$request_method$host$uri$is_args$args";
    }

    # Content matching this location block WILL NOT cached
    location ~ \.php$ 
    {

       # Placing these in a separate file for now so they can be referenced
       # directly when needed (until I learn of a better way)
       include php_parameters.conf;

       # Misc general FastCGI and Proxy settings.
       include proxy.inc.conf;
    }
}


References


More information