Repository files navigation

This lab is provided as part ofAWS Summit Online, clickhere to explore the full list of hands-on labs.

ℹ️ You will run this lab in your own AWS account. Please follow directions at the end of the lab to remove resources to avoid future costs.

In this lab we will learn how we can use lambda@edge functions to serve different variants of the same static resources from a CloudFront distribution.

This ability can be used to enable A/B testing of staticly deployed websites without resorting to complex and resource intensive conditional server-side rendering of content.

Headings with the ⓘ symbol indicate Information only sections of this README

Those with the ☛ symbol indicate action to be taken to complete the lab

To run this lab, you will require an AWS account. You will be using Cloud9, which is a web-based development environment that provides a terminal program running on a virtual machine that has the AWS CLI pre-installed and configured

1. Login to yourAWS Account Console
2. From theServices menu, selectCloud9

If you are prompted for a region, select the one closest to you.

3. Click theCreate Environment button
4. for Name, enter:lambda-edge-lab
5. ClickNext step twice, to accept the default options, then clickCreate Environment

Cloud9 will take a few minutes to launch the environment. Once it is ready, continue to the next step.

6. In the bash terminal at the bottom of the screen (showing/environment $), run the following command to download a copy of this lab to your Cloud9 environment:

git clone https://github.com/justasitsounds/lambda-edge-lab

Hint: You can expand the size of the terminal pane.

Our imaginary scenario is that our colleagues in the marketing team have some ideas about improving customer engagement on our company's website by tweaking some of the design features of our site

The functionality we need:

1. Randomly split unassigned traffic into two groups or pools
2. Serve two different variants of content from either of two origin S3 buckets (theA orB bucket)
3. Ensure that subsequent requests from the same clients will be served the same content that they first received (session stickyness)
4. Record and compare the impact the changes make to the customer engagement

To get started we will deploy:

• the CloudFront distribution
• an S3 bucket to retain the CloudFront access logs
• a CloudFront Origin Access Identity - a special CloudFront user that is associated with our CloudFront distribution (https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html)
• the two origin S3 buckets with attached bucket policies that grants the Origin Access Identitys3:getObject permission on all the resources within the bucket

There is a SAM template in this solutiontemplate.yml that we will use to deploy this stack. SAM templates are an extension of AWS CloudFormation templates that are focussed on providing a shorthand way of deploying AWS Lambda functions.

AWSTemplateFormatVersion:'2010-09-09'Transform:AWS::Serverless-2016-10-31Description:A/B testing using cloudfront and lambda@edgeResources:OriginAccessIdentity:Type:AWS::CloudFront::CloudFrontOriginAccessIdentityProperties:CloudFrontOriginAccessIdentityConfig:Comment:Origin Access Identity for lambda@edge dev labLogBucket:Type:"AWS::S3::Bucket"CFDistribution:Type:AWS::CloudFront::DistributionProperties:DistributionConfig:Logging:Bucket:!GetAtt LogBucket.DomainNameIncludeCookies:trueEnabled:trueDefaultRootObject:index.htmlComment:'AB testing Cloudfront distribution'Origins:          -Id:s3DomainName:!GetAtt OriginABucket.DomainNameS3OriginConfig:OriginAccessIdentity:!Join [ "/", [ origin-access-identity, cloudfront, !Ref OriginAccessIdentity ]]DefaultCacheBehavior:TargetOriginId:s3ForwardedValues:QueryString:falseCookies:Forward:whitelistWhitelistedNames:                -poolViewerProtocolPolicy:redirect-to-httpsDefaultTTL:30MinTTL:0AllowedMethods:            -HEAD            -GETCachedMethods:            -HEAD            -GETSmoothStreaming:falseCompress:trueOriginABucket:Type:AWS::S3::BucketProperties:BucketName:!Join      -"-"      -- "ab-testing-origin-a"        -!Select          -0          -!Split            -"-"            -!Select              -2              -!Split                -"/"                -!Ref"AWS::StackId"AccessControl:PrivateTags:      -Key:purposeValue:lab      -Key:projectValue:lambda-edge-abOriginBBucket:Type:AWS::S3::BucketProperties:BucketName:!Join      -"-"      -- "ab-testing-origin-b"        -!Select          -0          -!Split            -"-"            -!Select              -2              -!Split                -"/"                -!Ref"AWS::StackId"AccessControl:PrivateTags:      -Key:purposeValue:lab      -Key:projectValue:lambda-edge-abOriginAAccessBucketPolicy:Type:AWS::S3::BucketPolicyProperties:Bucket:Ref:OriginABucketPolicyDocument:Version:'2008-10-17'Id:PolicyForCloudFrontPrivateContentAStatement:        -Sid:'1'Effect:AllowPrincipal:CanonicalUser:!GetAtt OriginAccessIdentity.S3CanonicalUserIdAction:s3:GetObjectResource:             -!Join[ "", [ "arn:aws:s3:::", !Ref OriginABucket, "/*"]]OriginBAccessBucketPolicy:Type:AWS::S3::BucketPolicyProperties:Bucket:Ref:OriginBBucketPolicyDocument:Version:'2008-10-17'Id:PolicyForCloudFrontPrivateContentBStatement:        -Sid:'1'Effect:AllowPrincipal:CanonicalUser:!GetAtt OriginAccessIdentity.S3CanonicalUserIdAction:s3:GetObjectResource:             -!Join[ "", [ "arn:aws:s3:::", !Ref OriginBBucket, "/*"]]Outputs:CFDistribution:Description:Cloudfront Distribution Domain NameValue:!GetAtt CFDistribution.DomainNameBucketADomain:Description:Regional Domian name of the A bucketValue:!GetAtt OriginABucket.RegionalDomainNameBucketBDomain:Description:Regional Domian name of the A bucketValue:!GetAtt OriginBBucket.RegionalDomainNameLogBucketDomain:Description:Regional Domian name of the log bucketValue:!GetAtt LogBucket.RegionalDomainName

In the bash terminal at the bottom of the screen, and enter these commands to deploy this CloudFormation stack

cd~/environment/lambda-edge-labsam deploy --stack-name lambda-edge-dist --region us-east-1 -g

TheSAM CLI will ask a series of questions as you deploy for the first time.

Stack Name [lambda-edge-dist]:AWS Region [us-east-1]:#Shows you resources changes to be deployed and require a 'Y' to initiate deployConfirm changes before deploy [y/N]: N#SAM needs permission to be able to create roles to connect to the resources in your templateAllow SAM CLI IAM role creation [Y/n]: YSave arguments to samconfig.toml [Y/n]: Y

Just hitenter for each question to accept the default option.

It will take a little while (up to 10 minutes) for the stack to complete deployment. When it does you should see output similar to the following in your terminal - listing the outputs of the cloudformation stack you have deployed:

So, now we have a cloudfront distribution that is set to serve content from theA S3 bucket. The subdomain name of your new CloudFront distribution is randomly generated. To find out what the full domain name is, either refer to the OutputValue that corresponds to the OutputKeyCFDistribution in the return from thesam deploy command described above, or find the deployed CloudFormation stack in your AWS console calledlambda-edge-dist (In the AWS Console, selectCloudFormation fro theServices menu) make sure you are looking at the N. Virginia region) and find it listed under the output tab there.

Point your browser to your new CloudFront distribution domain name and you'll see:

This is because there is no content in the bucket to serve. Or rather the cloudfront distribution is trying to serve the default root object (index.html) from the S3 bucket, but it's not there.

1. Download and unzip the source files from the github repository to your local machine from thegithub repository
   The unzipped files should be arranged like this, with two subfolders:origin-a andorigin-b, both containing two different versions of a single web-page (index.html)

── content│   ├── origin-a│   │   ├── favicon.ico│   │   └── index.html│   └── origin-b│       ├── favicon.ico│       └── index.html

2. Navigate to the S3 service dashboard in the AWS console (https://s3.console.aws.amazon.com/s3/home)
3. Find theA bucket - it's name should start withab-testing-origin-a
4. open the origin A bucket by clicking on it's name
5. Upload theindex.html andfavicon.ico files from the localcontent/origin-a folder.Once you have selected the files to upload, simply click the 'Upload' button on the left hand side of the dialog to accept the default permissions and properties for those files

Once this is done, check that you can see theA content being served from your CloudFront domain by pointing your browser to the Cloudfront distribution dmain name again.

Repeat the steps listed above to upload the filescontent/origin-b/favicon.ico andcontent/origin-a/index.html to theB content bucket (the name starts withab-testing-origin-b)

If you point your browser at the CloudFront distribution domain again, you will only see theA content being served.

In order to change the origin for this CloudFront distribution as we want, we'll need to add some lambda@edge functions to the distribution.

lambda@edge functions can be triggered by 4 different CloudFront events that correspond to the following stages of a Cloudfront content request:

• After CloudFront receives a request from a viewer (viewer request)
• Before CloudFront forwards the request to the origin (origin request)
• After CloudFront receives the response from the origin (origin response)
• Before CloudFront forwards the response to the viewer (viewer response)

The CloudFront distribution has been configured to forwardpool cookies to the Origin - meaning that thepool cookie is part of the cache key. This allows us to utilise the caching abilities of CloudFront so that subsequent requests that include apool cookie (pool=a orpool=b), will fetch the same content from the edge cache without making a request to the Origin.

This function intercepts the viewer request before it is routed to the Cloudfront cache. The code simply adds apool cookie, value:pool=a orpool=b to the request header if one is not already present. Users without an existing pool cookie are randomly assigned eitherpool=a orpool=b with an equal probability (50/50) of being assigned either.

This function changes the origin bucket location in the request to point to either originA or originB depending on the value of thepool cookie.

Adds aSet-Cookie header to set thepool cookie to match the origin from where the content was served - ensuring that clients that made a requests without apool cookie are instructed to store and send the pool cookie value that matches the origin for subsequent requests.

Open theedge-functions/viewerrequest.js file in this solution in your Cloud9 editor. Copy and paste the following code and save.

'use strict';// the `pool` cookie designates the user pool that the request belongs toconstcookieName='pool';// returns cookies as an associative array, given a CloudFront request headers arrayconstparseCookies=require('./common.js').parseCookies;// returns either 'a' or 'b', with a default probability of 1:1constchoosePool=(chance=2)=>Math.floor(Math.random()*chance)===0 ?'b' :'a';//if the request does not have a pool cookie - assign oneexports.handler=(event,context,callback)=>{constrequest=event.Records[0].cf.request;constheaders=request.headers;constparsedCookies=parseCookies(headers);if(!parsedCookies||!parsedCookies[cookieName]){lettargetPool=choosePool();//pass a Number as argument to change the chance that user is assigned to Pool 'a' or 'b'headers['cookie']=[{key:'cookie',value:`${cookieName}=${targetPool}`}]}callback(null,request);};

Both theorigin_request andorigin_response lambda@edge functions depend on a common configuration file that holds the bucket names for theA andB S3 buckets.

You'll need to update this file with the names of theA andB origin buckets that were created when you first deployed the solution stack. You can find this name by either looking at the output of thesam deploy function if you still have that open in your terminal window: IE:

or by looking at the outputs in thelambda-edge-dist stack in the Cloudformation console.

In this deployment example, the name of theA bucket is:ab-testing-origin-a-01207890 and the name of theB bucket is:ab-testing-origin-b-01207890 (note that the name is the first segment of the bucket domain name)

Using these two values, update the code inedge-functions/origins_config.js to match. So for the deployment shown above the file would change from:

module.exports={a:'<REPLACE WITH THE NAME OF YOUR ORIGIN A BUCKET>',b:'<REPLACE WITH THE NAME OF YOUR ORIGIN B BUCKET>'};

to

module.exports={a:'ab-testing-origin-a-01207890',b:'ab-testing-origin-b-01207890'};

Copy and paste the following code into theedge-functions/originrequest.js file in this solution and save.

'use strict';// the S3 origins that correspond to content for Pool A and Pool Bconstorigins=require('./origins_config.js');constparseCookies=require('./common.js').parseCookies;// the `pool` cookie determines which origin to route toconstcookieName='pool';// changes request origin depending on value of the `pool` cookieexports.handler=(event,context,callback)=>{constrequest=event.Records[0].cf.request;constheaders=request.headers;constrequestOrigin=request.origin.s3;constparsedCookies=parseCookies(headers);lettargetPool=parsedCookies[cookieName];lets3Origin=`${origins[targetPool]}.s3.us-east-1.amazonaws.com`;requestOrigin.region='us-east-1';requestOrigin.domainName=s3Origin;headers['host']=[{key:'host',value:s3Origin}];callback(null,request);};

Copy and paste the following code into theedge-functions/originresponse.js file in this solution and save.

'use strict';// the S3 origins that correspond to content for Pool A and Pool Bconstorigins=require('./origins_config.js');//returns a set-cookie header based on where the content was served fromexports.handler=(event,context,callback)=>{constresponse=event.Records[0].cf.response;//response from the originconstreqHeaders=event.Records[0].cf.request;//request from cloudfrontletpoolorigin='a';//default origin poolif(reqHeaders.origin.s3.domainName.indexOf(origins.b)===0){poolorigin='b';}response.headers['Set-Cookie']=[{key:'Set-Cookie',value:`pool=${poolorigin}`}];callback(null,response);};

Below is the updated SAM template that includes the lambda@edge functions and integrates them with the CloudFront distribution. New sections have comments to show the additions. Open the filetemplate.yml in the base folder of the solution in Cloud9. Copy and paste the following to update and replace the content of the file.

AWSTemplateFormatVersion:'2010-09-09'Transform:AWS::Serverless-2016-10-31Description:A/B testing using cloudfront and lambda@edge### The Globals section is a SAM specific element that defines the common attributes of the Lambda@Edge functions we are deployingGlobals:Function:Runtime:nodejs10.xTimeout:5AutoPublishAlias:live###Resources:OriginAccessIdentity:Type:AWS::CloudFront::CloudFrontOriginAccessIdentityProperties:CloudFrontOriginAccessIdentityConfig:Comment:Origin Access Identity for lambda@edge dev lab### the EdgeFunctionRole is an IAM role that is granted to the Lambda@Edge functionsEdgeFunctionRole:Type:AWS::IAM::RoleProperties:RoleName:!Sub ${AWS::StackName}-edgeFunctionAssumeRolePolicyDocument:Version:2012-10-17Statement:Effect:AllowPrincipal:Service:              -lambda.amazonaws.com              -edgelambda.amazonaws.comAction:sts:AssumeRoleManagedPolicyArns:        -arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole        -arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess###### The Lambda@Edge functions - these require the EdgeFunctionRoleViewerRequestLambda:Type:AWS::Serverless::FunctionProperties:Description:Assigns pool cookie if not present on requestRole:!GetAtt EdgeFunctionRole.ArnCodeUri:./edge-functions/src/Handler:viewerrequest.handlerOriginRequestLambda:Type:AWS::Serverless::FunctionProperties:Description:Changes request Origin depending on value of pool cookieRole:!GetAtt EdgeFunctionRole.ArnCodeUri:./edge-functions/src/Handler:originrequest.handlerOriginResponseLambda:Type:AWS::Serverless::FunctionProperties:Description:Appends Set-cookie header to match response originRole:!GetAtt EdgeFunctionRole.ArnCodeUri:./edge-functions/src/Handler:originresponse.handler###LogBucket:Type:"AWS::S3::Bucket"CFDistribution:Type:AWS::CloudFront::DistributionProperties:DistributionConfig:Logging:Bucket:!GetAtt LogBucket.DomainNameIncludeCookies:trueEnabled:trueDefaultRootObject:index.htmlComment:'AB testing Cloudfront distribution'Origins:          -Id:s3DomainName:!GetAtt OriginABucket.DomainNameS3OriginConfig:OriginAccessIdentity:!Join [ "/", [ origin-access-identity, cloudfront, !Ref OriginAccessIdentity ]]DefaultCacheBehavior:TargetOriginId:s3ForwardedValues:QueryString:falseCookies:Forward:whitelistWhitelistedNames:                -poolViewerProtocolPolicy:redirect-to-httpsDefaultTTL:30MinTTL:0AllowedMethods:            -HEAD            -GETCachedMethods:            -HEAD            -GETSmoothStreaming:falseCompress:true### Lambda Function Associations - associating the functions with the CloudFront request/response eventsLambdaFunctionAssociations:            -EventType:viewer-requestLambdaFunctionARN:!Ref ViewerRequestLambda.Version            -EventType:origin-requestLambdaFunctionARN:!Ref OriginRequestLambda.Version            -EventType:origin-responseLambdaFunctionARN:!Ref OriginResponseLambda.Version###OriginABucket:Type:AWS::S3::BucketProperties:BucketName:!Join      -"-"      -- "ab-testing-origin-a"        -!Select          -0          -!Split            -"-"            -!Select              -2              -!Split                -"/"                -!Ref"AWS::StackId"AccessControl:PrivateTags:      -Key:purposeValue:lab      -Key:projectValue:lambda-edge-abOriginBBucket:Type:AWS::S3::BucketProperties:BucketName:!Join      -"-"      -- "ab-testing-origin-b"        -!Select          -0          -!Split            -"-"            -!Select              -2              -!Split                -"/"                -!Ref"AWS::StackId"AccessControl:PrivateTags:      -Key:purposeValue:lab      -Key:projectValue:lambda-edge-abOriginAAccessBucketPolicy:Type:AWS::S3::BucketPolicyProperties:Bucket:Ref:OriginABucketPolicyDocument:Version:'2008-10-17'Id:PolicyForCloudFrontPrivateContentAStatement:        -Sid:'1'Effect:AllowPrincipal:CanonicalUser:!GetAtt OriginAccessIdentity.S3CanonicalUserIdAction:s3:GetObjectResource:             -!Join[ "", [ "arn:aws:s3:::", !Ref OriginABucket, "/*"]]OriginBAccessBucketPolicy:Type:AWS::S3::BucketPolicyProperties:Bucket:Ref:OriginBBucketPolicyDocument:Version:'2008-10-17'Id:PolicyForCloudFrontPrivateContentAStatement:        -Sid:'1'Effect:AllowPrincipal:CanonicalUser:!GetAtt OriginAccessIdentity.S3CanonicalUserIdAction:s3:GetObjectResource:             -!Join[ "", [ "arn:aws:s3:::", !Ref OriginBBucket, "/*"]]Outputs:CFDistribution:Description:Cloudfront Distribution Domain NameValue:!GetAtt CFDistribution.DomainNameDistributionID:Description:Cloudfront Distribution IDValue:!Ref CFDistributionBucketADomain:Description:Regional Domian name of the A bucketValue:!GetAtt OriginABucket.RegionalDomainNameBucketBDomain:Description:Regional Domian name of the A bucketValue:!GetAtt OriginBBucket.RegionalDomainNameLogBucketDomain:Description:Regional Domian name of the log bucketValue:!GetAtt LogBucket.RegionalDomainName

To deploy the updated stack, we just need to invoke thesam deploy command again - with specific CAPABILITY_NAMED_IAM capabilites because now our template defines an IAM role:edgeFunctionRole that will be assumed by our lambda@edge functions.

sam deploy --capabilities CAPABILITY_NAMED_IAM

This will take between 5-10 minutes to complete. Once it is finished, you can proceed to the final stage: Testing!

As a quick recap, our requirements are:

1. Randomly split unassigned traffic into two groups or pools
2. Serve two different variants of content from either of two origin S3 buckets (theA orB bucket)
3. Ensure that subsequent requests from the same clients will be served the same content that they first received (session stickyness)
4. Record and compare the impact the changes make to the customer engagement

To test the first three requirements:

1. Point your browser at the CloudFront distribution

2. You will see either of these two pages:

   

   

3. If you open the dev tools (in Chrome: it's under View > Developer Tools > Developer Tools) and examine the Cookies (Chrome Developer Tools > Application tab, Storage : Cookies) you will see that your browser will now have apool session cookie for the current domain, the value of which will match the Origin that the content was served from:

   

4. Open the Network tab in the developer tools and ensure that thePreserve log checkbox is checked, then refresh the page any number of times. You will see the same content with each page refresh - thepool session cookie ensures that your request will be served content from the same origin. If you examine the response headers while you do this (Chrome: Developer Tools : Network tab) you will see a couple of things:

   • The initial response will have a200 (OK) status code and a customX-Cache header with a value:Miss from cloudfront - this is a hint from cloudfront that tells us that this content was fetched directly from the origin, and not served from the cloudfront cache. The matching request will have noCookie: pool=[[X]] header, but the server will add aSet-Cookie: pool=[[X]] header to the response - this instructs your browser to create the pool cookie and append it to subsequent requests to the same domain.
   • Subsequent requests will have theCookie: pool=[[X]] header and may have a304 (Not Modified) status code along with anX-Cache: Hit from cloudfront header and anAge header - this type of response means a couple of things:
     • 304 - not modified. The server (cloudfront) has recognised that the content being requested has not changed since the client last received a valid copy - this is mediated by theIf-Modified-Since andIf-None-Match request headers sent by the client. When your browser receieves the304 response it actually serves the page content from it's internal cache rather than the full content from the server - this is the purpose of 304 responses, they do not contain a message-body for this reason
     • X-Cache: Hit from cloudfront - cloudfront has checked the request against it's edge cache - and not the origin.
     • TheAge response header tells us how old the cached copy of the content is in seconds. By default this distribution is configured to only keep a copy of the content in it's cache for 30 seconds - the default cache lifetime can be overridden by adding amax-age cache-control header to the content in the S3 origin.

5. Change the value of thepool cookie to the other origin (if it is currentlya, change it tob) and then refresh the page. You will now see the alternate content for the same resource.

6. Delete thepool cookie and refresh the browser, you have a 50/50 chance of seeing the alternate content.

To test the last requirement (record and compare the impact the changes make to the customer engagement) you can:

7. Query the Cloudfront access logs using Athena. You can segment your user traffic based on thepool cookie value in the logs so that you can compare the customer journeys for those customers who received theA content andB content

1. Delete all content from the log and origin S3 buckets

   a) Open theAWS Console and selectS3 from theServices menu to open the S3 dashboard

   b) click on the bucket name for each of the buckets

   c) select all content in the bucket using the radiobuttons and click the 'Delete' button

2. Delete the CloudFormation stack:

   a) Open theAWS Console and selectCloudFormation from theServices menu to open the CloudFormation dashboard

   b) Select the radiobutton for thelambda-edge-dist stack and click the 'Delete' button

   c) If there is any content in the the log or origin buckets you will get a DELETE_FAILED status on the cloudformation stack. Delete the content (see above) and try the delete cloudformation stack operation again