/ Without Label / Amazon Cloudfront Dynamic Content Delivery With A WordPress Blog

Amazon Cloudfront Dynamic Content Delivery With A WordPress Blog

December 31, 2013
Atypical WordPress blog contains a mix of static stuff such as images, javascript, style sheets and dynamic content such as posts, pages and comments posted by users. You can speed up your blog by serving static content via content delivery network such as Akamai, Edgecast and so on. The big boys of CDN business also offered the solution to accelerate dynamic content to improve the performance and reliability of the blog. However, solutions offered by big and traditional CDNs are expensive. Amazon cloudfront recently started to serving dynamic content at lowered price. In this blog post, I will explain:
  1. How to serve your entire blog using cloudfront.
  2. DNS settings.
  3. WordPress settings.
  4. Documenting limitations of cloudfront.
  5. Documenting performance improvements.

How does dynamic content delivery network works?

A dynamic content delivery network or content distribution network (CDN) is a system of computers containing copies of data, placed at various points in a network so as to maximize bandwidth for access to the data from clients throughout the network. A client accesses a copy of the data near to the client, as opposed to all clients accessing the same central server, so as to avoid bottleneck near that server. A cdn will increase speed and efficiency for your blog. A typical setup looks as follows:
Fig.01: A typical setup for wordpress based blog for dynamic CDN
Fig.01: A typical setup for wordpress based blog for dynamic CDN

Amazon cloudfront limitations

From the aws cloudfront faq:
What types of HTTP requests are supported by Amazon CloudFront?

HTTP GET and HEAD requests are currently supported by Amazon CloudFront. Over time, we will add support for POST requests.
Translation: CloudFront will not accept any POST requests i.e. you will not able to post comments, post, or upload images using WordPress. Currently, cloudfront will not act as a proxy, passing all post requests via the custom origin. However, you can configure WordPress to accept POST requests on a subdomain. This will allow you to post comments, upload files and much more.

Our sample setup

You can convert your existing blog or setup a new blog from scratch to use cloudfront. For demonstration purpose, I'm going to setup a new blog called www.contentdeliverynetworklog.com and serve the whole blog via cloudfront as follows:
  1. Blog domain name: www.contentdeliverynetworklog.com (CNAME to cloudfront)
  2. Blog origin domain name: cp.contentdeliverynetworklog.com. To fetch web content from this origin web server.
  3. Static asset domain name: s0.contentdeliverynetworklog.net (CNAME to cloudfront).
  4. Static asset origin domain name: origin.contentdeliverynetworklog.net.
  5. Custom origin IP address for both static/dynamic assets: 75.126.153.203
  6. Caching ruleset for your dynamic and static assets: I am going to use a combination of Batcache+Memcached. However, you can use other plugins such as "W3 Total Cache" (not tested).
  7. DNS server: Route 53 (or use your existing BIND9 based dns servers).

Step #1: DNS setup for cp.contentdeliverynetworklog.com

First, point your custom origin domain cp.contentdeliverynetworklog.com to 75.126.153.203. A typical BIND 9 entry will look as follows in your zone file:
cp                     600      IN A     75.126.153.203

Step #2: Create a bucket

You may want CloudFront to log all viewer requests for files in your distribution. This is useful for stats program such as webalizer and awstats. You need to create a bucket by selecting a bucket name and region. This is required to store web server logs. Login to aws console > open the Amazon S3 console at https://console.aws.amazon.com/s3 > Click Create Bucket:
Fig.02: Create a bucket for access log
Fig.02: Create a bucket for access log
Please note that access logging is an optional feature of CloudFront. There is no extra charge for enabling access logging. However, you accrue the usual Amazon S3 charges for storing and accessing the files on Amazon S3. Please see this guide for more information on log file formats.

Step #3: Configure AWS Cloudfront Dynamic CDN for www.contentdeliverynetworklog.com

Now, let us see how to configure and use CloudFront to distribute dynamic and static content. I am going to create a CloudFront distribution and configure it to fetch web content from my origin web server called cp.contentdeliverynetworklog.com. First, open the Amazon Cloudfron console at https://console.aws.amazon.com/cloudfront > Click Create Distribution > Set a delivery method to Download > Continue.

Origin settings

  1. Set Origin Domain Name to cp.contentdeliverynetworklog.com
  2. Set Origin ID to CustomWWW-cp.contentdeliverynetworklog.com
  3. Set Origin Protocol Policy to HTTP Only (CloudFront will connect to my origin using only HTTP).
Fig.03: Origin settings
Fig.03: Origin settings

Default cache behavior settings

  1. Set Viewer Protocol Policy to HTTP and HTTPS.
  2. Set Object Caching to Use Origin Cache Headers. My origin server is adding a Cache-Control header to control how long your objects stay in the CloudFront cache. However, you can specify a minimum time that objects stay in the CloudFront cache regardless of Cache-Control headers by selecting Customize option and setting Minimum TTL in seconds (default is 24 hours).
  3. Set Forward Query Strings to Yes.
Fig.04: Configure default cache behavior settings
Fig.04: Configure default cache behavior settings

Distribution settings

The distribution settings affect both cdn performance and pricing. You need to select the price class associated with the maximum price that you want to pay for CloudFront service.
  1. Set Price Class to Use All Edge Locations.
  2. Set Alternate Domain Names(CNAMEs) to www.contentdeliverynetworklog.com. I want to use my own domain name instead of the CloudFront domain name for the blog URLs. You need need to create a CNAME record with DNS server to route queries for www.contentdeliverynetworklog.com to *.cloudfront.net.
  3. Set Logging to On.
  4. Set Bucket for Logs to webserverlog-contentdeliverynetworklog.s3.amazonaws.com (see step #2 create a bucket for more info).
  5. Set Log Prefix to stats-logs/.
  6. Finally, set Distribution State to Enabled.
Fig.05: Distribution edge settings
Fig.05: Distribution edge settings

Finally, click on the Create Distribution button. You will see the status as follows:
Fig.06: Cloudfront distribution setup in progress
Fig.06: Cloudfront distribution setup in progress

Please note down the domain name d3qrb8why8gyke.cloudfront.net.

Step #4: DNS CNAME setup for www.contentdeliverynetworklog.com

You need to create a CNAME record with DNS server to route queries for www.contentdeliverynetworklog.com to d3qrb8why8gyke.cloudfront.net. A typical BIND 9 CNAME entry will look as follows in your zone file:
www                    600      IN CNAME d3qrb8why8gyke.cloudfront.net.

Verify DNS settings

Use the following host command dns lookup utility to verify dns settings:
$ host cp.contentdeliverynetworklog.com
Sample outputs:
cp.contentdeliverynetworklog.com has address 75.126.153.203
Make sure your cloudfront CDN domain is resolving:
$ host d3qrb8why8gyke.cloudfront.net.
Sample outputs:
d3qrb8why8gyke.cloudfront.net has address 54.240.168.62
d3qrb8why8gyke.cloudfront.net has address 54.240.168.82
d3qrb8why8gyke.cloudfront.net has address 54.240.168.98
d3qrb8why8gyke.cloudfront.net has address 54.240.168.190
d3qrb8why8gyke.cloudfront.net has address 54.240.168.214
d3qrb8why8gyke.cloudfront.net has address 54.240.168.30
d3qrb8why8gyke.cloudfront.net has address 54.240.168.38
d3qrb8why8gyke.cloudfront.net has address 54.240.168.54
Also, make sure d3qrb8why8gyke.cloudfront.net set as CNAME to www.contentdeliverynetworklog.com:
$ host www.contentdeliverynetworklog.com
Sample outputs:
www.contentdeliverynetworklog.com is an alias for d3qrb8why8gyke.cloudfront.net.
d3qrb8why8gyke.cloudfront.net has address 54.240.168.146
d3qrb8why8gyke.cloudfront.net has address 54.240.168.166
d3qrb8why8gyke.cloudfront.net has address 54.240.168.170
d3qrb8why8gyke.cloudfront.net has address 54.240.168.198
d3qrb8why8gyke.cloudfront.net has address 54.240.168.206
d3qrb8why8gyke.cloudfront.net has address 54.240.168.26
d3qrb8why8gyke.cloudfront.net has address 54.240.168.126
d3qrb8why8gyke.cloudfront.net has address 54.240.168.134

Step #5: WordPress configuration

You need to install WordPress at cp.contentdeliverynetworklog.com sub-domain. This domain will be used for the following purpose:
  • Blog installation and/or upgradation
  • Post / edit comments.
  • Post and edit pages or blog entries.
  • Manage your blog by visiting the url http://cp.contentdeliverynetworklog.com/wp-admin/ or https://cp.contentdeliverynetworklog.com/wp-admin/

Create the Mysql database and a user

Type the following command:
$ mysql -u root -p
Type the following sql commands to create the database and a user account:
mysql> CREATE DATABASE cdnblog;
mysql> GRANT ALL ON cdnblog.* TO cdnblogadm@localhost IDENTIFIED BY 'aZ7pHqG$b3#';Where,
  1. DB Name: cdnblog
  2. DB User: cdnblogadm
  3. DB Password: aZ7pHqG$b3#

WordPress installation

Download WordPress using the wget command, enter:
$ cd /tmp/
$ wget http://wordpress.org/latest.tar.gz
$ tar xvf latest.tar.gz
cd to your Apache/Lighttpd/Nginx DocumentRoot:
$ cd /home/httpdocs/contentdeliverynetworklog.com/home/http
## create a sud-dir to keep all wordpress files ##
$ mkdir dyn
$ cd dyn
$ cp -avr /tmp/wordpress/* .
## temporary set permission ##
$ chmod -R 0777 ../dyn
To install WordPress visit the following url:
http://cp.contentdeliverynetworklog.com/dyn/wp-admin/install.php
If you placed the WordPress files in the root directory, you should visit the following url:
http://cp.contentdeliverynetworklog.com/wp-admin/install.php
Just follow on screen instructions or see installing WordPress page for more information. Make sure you set read-only permission on all files:
# chown -R www-data:www-data /home/httpdocs/contentdeliverynetworklog.com/home/http
# chmod -R 0444 /home/httpdocs/contentdeliverynetworklog.com/home/http
## make sure web-server and process can read and list the dirs ##
# find /home/httpdocs/contentdeliverynetworklog.com/home/http -type d -print0 | xargs -0 -I {} chmod 0445 {}

index.php

Next, create a file called index.php
$ cat /home/httpdocs/contentdeliverynetworklog.com/home/http/index.php
Sample outputs:
 
<?php
/* Short and sweet file for www.contentdeliverynetworklog.com */
define('WP_USE_THEMES', true);
define('WP_IN_ROOTDIR', true);
require('./dyn/wp-blog-header.php');
?>
 

wp-config.php configuration

Edit the file /home/httpdocs/contentdeliverynetworklog.com/home/http/dyn/wp-config.php:
Add the following at the top of the file
 
/**
/* WP_SITEURL: the WordPress address (URL) where your WordPress core files reside.
 */
define('WP_SITEURL', 'http://cp.contentdeliverynetworklog.com/dyn');
/**
/* WP_HOME: the WordPress Cloudfront CDN (URL). This is the address you want people
 * to type in their browser to reach your WordPress blog.
 */
define('WP_HOME', 'http://www.contentdeliverynetworklog.com/');
 
Save and close the file.

Install and configure WordPress memcached object caching engine

You need install memcached object cache plugin to speed up dynamic database-driven wordpress blog by caching data and objects in RAM to reduce the number of times an external data source must be read. The WordPress memcached object cache plugin provides a persistent backend for the WordPress object cache. See how to install memcached server andWordPress memcached object cache plugin for more information.

Install and configure Batcache

The Batcache plugin uses Memcached to store and serve rendered pages. Grab Batcache using wget command, enter:
$ cd /tmp
$ wget http://downloads.wordpress.org/plugin/batcache.1.2.zip
$ unzip batcache.1.2.zip
cd into plugins directory:
$ cd /home/httpdocs/contentdeliverynetworklog.com/home/http/dyn/wp-content/plugins/
Copy batcache.php using cp command, run:
$ cp /tmp/batcache/batcache.php .
Copy advanced-cache.php using cp command, run:
$ cp /tmp/batcache/advanced-cache.php ..
$ cd ..
## configure advanced cache max-header ##
$ vi advanced-cache.php
Set values as per your requirements:
 
 var $max_age =  300; // Expire batcache items aged this many seconds (zero to disable batcache)
 var $group   = 'cp_wp_cdn_log_r1'; // Name of memcached group. You can simulate a cache flush by changing this.
 
Save and close the file. Edit the file/home/httpdocs/contentdeliverynetworklog.com/home/http/dyn/wp-config.php. Add the following line the top of wp-config.php to activate Batcache:
 
define('WP_CACHE', true);
 
Save and close the file.

Testing headers

Type the following curl command to test headers
$ curl -I -H 'Accept-Encoding: gzip,deflate' http://cp.contentdeliverynetworklog.com
Sample outputs:
 HTTP/1.1 200 OK
    Server: nginx
    Date: Wed, 27 Feb 2013 10:40:29 GMT
    Content-Type: text/html; charset=UTF-8
    Content-Length: 3488
    Connection: keep-alive
    Keep-Alive: timeout=60
    X-Whom: l1-com-conte
    Last-Modified: Wed, 27 Feb 2013 10:39:41 GMT
    Cache-Control: max-age=252, must-revalidate
    Vary: Cookie
    Vary: Accept-Encoding
    X-Pingback: http://cp.contentdeliverynetworklog.com/dyn/xmlrpc.php
    Content-Encoding: gzip
    X-Origin-Type: Dynamic

Step #6: Configure WordPress to use a CDN for static assets

You need to configure s0.contentdeliverynetworklog.net to serve static assets such as css, js, and images using cloudfront. First, open the Amazon Cloudfron console at https://console.aws.amazon.com/cloudfront > Click Create Distribution > Set a delivery method to Download > Continue.
  1. Set Origin Domain Name to origin.contentdeliverynetworklog.net
  2. Set Origin ID to CustomWWW-origin.contentdeliverynetworklog.net
  3. Set Origin Protocol Policy to HTTP Only (CloudFront will connect to my origin using only HTTP).
You need to set default cache behavior settings as follows:
  1. Set Viewer Protocol Policy to HTTP and HTTPS.
  2. Set Object Caching to Use Origin Cache Headers.
  3. Set Forward Query Strings to No.
Next, set the distribution settings:
  1. Set Price Class to Use All Edge Locations.
  2. Set Alternate Domain Names(CNAMEs) to s0.contentdeliverynetworklog.net.
  3. Set Logging to Off.
  4. Finally, set Distribution State to Enabled.
You need to create a CNAME record with DNS server to route queries for s0.contentdeliverynetworklog.net to de10i7qbzuz6c.cloudfront.net. A typical BIND 9 CNAME entry will look as follows in
your zone file:
; zone: c/contentdeliverynetworklog.net
s0                     600      IN CNAME de10i7qbzuz6c.cloudfront.net.
origin                 600      IN A     75.126.153.203
Verify dns settings with the dig or host command:
$ host s0.contentdeliverynetworklog.net
s0.contentdeliverynetworklog.net is an alias for de10i7qbzuz6c.cloudfront.net.
de10i7qbzuz6c.cloudfront.net has address 54.240.168.210
de10i7qbzuz6c.cloudfront.net has address 54.240.168.234
de10i7qbzuz6c.cloudfront.net has address 54.240.168.250
de10i7qbzuz6c.cloudfront.net has address 54.240.168.10
de10i7qbzuz6c.cloudfront.net has address 54.240.168.58
de10i7qbzuz6c.cloudfront.net has address 54.240.168.70
de10i7qbzuz6c.cloudfront.net has address 54.240.168.150
$ host origin.contentdeliverynetworklog.net
origin.contentdeliverynetworklog.net has address 75.126.153.203

Configure WordPress to upload files to CDN server

Edit the file /home/httpdocs/contentdeliverynetworklog.com/home/http/dyn/wp-config.php and add the following line:
 
define( 'UPLOADS', '/home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http/uploads' );
 
Save and close the file. Create a directory:
# mkdir -p /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http
Finally, creeate a symbolic link using the ln command:
# cd /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http
# ln -s ../http/wp-content/uploads .
Copy required images and css files in assets directory:
# cd /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http
# mkdir assets
# mkdir assets/lib/
# cp /path/to/your/ie.css assets/
# cp /path/to/your/style.css assets/
# cp /path/to/your/layout.css assets/
### copy all css images ###
# cp -avr /path/to/your/lib/images assets/lib/
Finally, set read only permissions:
# chown -R www-data:www-data /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http
# chmod -R 0444 /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http
## make sure web-server and process can read and list the dirs ##
# find /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http -type d -print0 | xargs -0 -I {} chmod 0445 {}
Finally, serve all WordPress media files via CDN url http://s0.contentdeliverynetworklog.net/uploads. Edit your themes functions.php and append the following code:
/**
 * Origin server: origin.contentdeliverynetworklog.net
 * CDN url: s0.contentdeliverynetworklog.net/uploads
 * WordPress Media Upload path: /home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http/uploads
 */
function wp_cdn_log_upload_url()
{
    return 'http://s0.contentdeliverynetworklog.net/uploads';
}
add_filter( 'pre_option_upload_url_path', 'wp_cdn_log_upload_url' );
 

Lighttpd max-age settings for static assets

You need to use a combination of mod_expire and mod_setenv to setup caching control headers:
 
### Origin host config for CDN s0.contentdeliverynetworklog.net ###
$HTTP["host"]  == "origin.contentdeliverynetworklog.net"{
  server.document-root = "/home/httpdocs/contentdeliverynetworklog.com/home/origin-cdn-http"
  accesslog.filename   = "/var/log/lighttpd/origin.contentdeliverynetworklog.net/access.log"
        $HTTP["url"] =~ "^/" {
            expire.url = ( "" => "access 365 days" )
        }
  setenv.add-response-header += (
    "Cache-Control" => "public"
  )
}
#### end CDN ##
 

Configure preview post button

You need use the hook or filter called preview_post_link under wordpress to change the default "preview" button when posting. Edit your themes functions.php and append the following code:
 
function wp_cdn_log_preview_link($link) {
        $link = preg_replace('/www/', 'cp', $link);
        $link = preg_replace('/\?p=/', 'dyn/\?p=', $link);
        return $link;
}
add_filter( 'preview_post_link', 'wp_cdn_log_preview_link' );
 
Save and close the file. Finally, edit your themes file (header.php) to use cdn stylesheet urls. For example:
 
<link rel="stylesheet" href="http://s0.contentdeliverynetworklog.net/assets/layout.css" type="text/css" media="screen, projection" />
 

Putting it all together

You can test working setup at the following urls:
I ran the following three basic tests:

wget download test

wgeting CDN url http://www.contentdeliverynetworklog.com/uncategorized/hello-world-13.html vs origin url http://cp.contentdeliverynetworklog.com/uncategorized/hello-world-13.html from my home:
  1. CDN URL download time: 0.07s
  2. Origin url download time: 0.3s

ICMP ping test

Ping test for cdn domain www.contentdeliverynetworklog.com vs origin domain cp.contentdeliverynetworklog.com (ave. rrt):
LocationAWS Cloudfront [1]Origin server at Dallas, TX [2]
Amsterdam2, Netherlands0.8114.5
Florida, U.S.A16.332.6
Sydney, Australia123.0192.3
Vancouver, Canada4.645.5
London, United Kingdom2.1108.3
Singapore3.0218.3
Hong Kong, China3.1230.2
Mumbai, India75.3278.9
Dublin, Ireland12.5128.6
Cape Town, South Africa351.7261.8
Tampere, Finland11.0156.4

Web page performance test

I used webpagetest.org from Singapore (IE8/DSL):

First run: CDN [3] vs origin server [4]

The first time content is requested, it's pulled from the origin server to the CDN network and stays there for other users to access it as per TTL (max-age header):

Second run: CDN [5] vs origin server [6]

The second time content is requested, the CDN network will serve the content directly from geographically closer to my end-users. If max-age header is expired the CDN server will pull the fresh copy from the origin server again.

Conclusion

With cloudfront, I can get my blog post and other static content to users faster and more efficiently. I set the max-age ttl to 300 seconds for all blog posts. It should be increased to 1800 seconds or more for better performance. However, higher ttl means comments will not get rendered immediately. A better solution is to use third party commenting systems. This is my first attempt running a WordPress based site using Amazon CloudFront to deliver a dynamic and static content. Amazon CloudFront is really easy to use. The cost of running such a site from AWS cloudfront for 4 weeks:
  • United States - $68.91 (343 GB with 36,969,170 HTTP requests)
  • Europe - $81.79 (372 GB with 41,252,596 HTTP requests)
  • Total - $150.70
REFERENCES:
  1. Ping CDN test result.
  2. Ping origin server result.
  3. CDN test # 1 (without edge cache) cdn server result.
  4. CDN test # 2 (with edge cache)cdn server result.
  5. Origin test # 1 origin server result.
  6. Origin test # 2 origin server result.
  7. AWS cloudfront guide.

Disqus comments