Problem Statement
During a recent trip to London to speak at our Incapsula Deep Dive Day (D3), I discovered that a small app that I’m developing was suffering from poor loading times. It wasn’t an entirely unexpected problem, as the servers were hosted in AWS under the US-West region so each HTTP request and response had to travel the globe, but it got me thinking about how Incapsula can be used to decrease this latency.
Issue
- After debugging the issue a bit further, I noticed that the latency was exacerbated by the need to support Cross-Origin Resource Sharing (CORS) Preflighted requests as the API for the app is hosted under one domain (api.example.com) and the clients communicate to the API from an entirely different domain (app1.example.com, app2.example.com, etc.).
- In situations where there may be implications to user data, web browsers use preflighted requests to first send an HTTP request with the OPTIONS method to the resource on the other domain, in order to determine whether the actual request is safe to send.
- The CORS specification (https://www.w3.org/TR/cors/) stipulates that if the client must pre-flight the request if any of the following conditions are true:
- The request uses the PUT, DELETE, or PATCH methods (among others)
- The request sends custom headers not defined in the W3C Fetch spec
- The Content-Type header has a value other than application/x-www-form-urlencoded, multipart/form-data, or text-plain
- Or if a few other conditions are met
- Long story short, every call to the API results in four trips across the globe and back. If each web request takes approximately 250ms, this means the user is waiting 500ms for each action. If their local internet connection or mobile network connection is slow this can quickly grow to becoming a frustrating experience for users that demand fast loading applications.
Potential Solutions
- Option 1 — Caching
At first I thought the solution to this problem would be to simply cache the OPTIONS responses and be done with it. However, I quickly realized that when CORS requests contain the “Access-Control-Allow-Credentials” header, the “Access-Control-Allow-Origin” header cannot be a wildcard “*” and instead must be a single domain name. This is particularly problematic because the API needs to be able to support cross-origin requests from app1.example.com as well as app2.example.com. - Option 2 — Additional Servers
The next solution that crossed my mind was to simply build out additional servers to service users in the EU. But what happens when users in Singapore or Brazil are looking to access the site and experience the same issue? Unfortunately, the application in question does not have a large enough user base to justify the additional cost and complexity of building out a multi-region architecture, so I quickly scrapped this idea. - Option 3 (Winner) — Incapsula Application Delivery Rules + AWS Lambda@Edge
Then I remembered that AWS recently rolled out Lambda@Edge, a solution that is designed for exactly this purpose. By using Incapsula application delivery rules (ADR) I could carve off HTTP OPTIONS requests and forward only these individual requests to AWS Lambda functions hosted around the globe. At a cost of only $0.0000002 per request this certainly met the budget I had in mind.
Implementation
AWS Configuration
To get started there are three main things that we need to configure in AWS:
- An S3 bucket (this won’t be used for anything other than being required for the CloudFront distribution)
- A new CloudFront distribution
- The Lambda@Edge function
Step 1: Configuring S3
As a first step, we need to create a new S3 bucket. In reality, the bucket will never actually be used since the Lambda Function will be returning responses, but we are required to map our CloudFront distribution to a back-end resource, so we’ll use S3 for the sake of simplicity. Simply give your bucket a name that indicates that it will be used for your CORS requests, eg. cors-lambda-edge.example.com and use the remaining default properties.
Step 2: Create a CloudFront Distribution
Next, we’ll create a CloudFront web distribution that will be the trigger for our Lambda@Edge function.
In terms of settings, we need to point the CloudFront distribution to our newly created S3 bucket and also enable the HTTP OPTIONS method.
We also need to ensure that we have the name of the domain added in as a CNAME under the alternate domain names section. This will match the name of the site you are protecting in Incapsula and will ensure that the CloudFront distribution will respond to requests that have the website name in the Host header.
Step 3: Configuring Lambda@Edge
Now for the fun part, writing your Lambda function to respond to CORS requests. The Lambda function is fairly simple, we just want to look to see if the incoming request is using the HTTP OPTIONS method, and if it is, we want to validate that the Origin header is in our list of white-listed domain names. If it is in our white-listed domain names, we reflect that value back to in the response headers. If the domain is not in our white list, then we’ll respond back with a default value. Additionally, for debugging purposes if the request is a HTTP GET request we’ll just reflect back the request headers in the message body.
Once we’ve saved and published our Lambda function we next need to add a trigger to ensure it’s invoked. To do this, simply select the “Triggers” section of the Lambda function and add a new trigger for “Viewer Request”, which will ensure this Lambda function runs on every single request to the CloudFront distribution.
Optional: Validating your Lambda Function
As an optional step before integrating this into your site, you can validate the lambda function works properly. You can use a number of tools to test this, but for the sake of simplicity we can leverage a single cURL command that passes in the origin header and validate the response contains the reflected back origin value. It’s a good idea to test all white-listed origins to validate they work, and to also test an origin header that is not on the white list to ensure that we do not inadvertently expose our application to requests that we should not.
Incapsula Application Delivery Rules
My favorite part of this process was just how easy it was to test and implement this feature using Incapsula application delivery rules.
Step 1: Create a new Data Center for the Lambda@Edge distribution
The first thing we need to do in Incapsula is to create a new Data Center that can be used with the application delivery rule. Simply take the CNAME of the CloudFront distribution and make that the Active Server and be sure to select the “Support only forward rules” checkbox which will enable the server to be used with ADR.
Step 2: Create a new Application Delivery Rule
Our application delivery rule is actually quite simple, we just want to direct all HTTP OPTIONS requests to the Lambda@Edge distribution. As you are implementing this rule you can even validate that everything is working correctly by selecting the “Test on my IP” check box.
And that’s it! Once we enable the rule all incoming HTTP OPTIONS requests will be automatically sent to our CloudFront Distribution/Lambda@Edge function instead of our normal origin server.
Results
So, did the changes that I made make a significant difference to the overall performance of the application? Well, it’s hard to say since I returned to Seattle before I had the opportunity to do a before and after performance test. But what I can say is that after monitoring the Lambda job execution over a period of a week, the Lambda jobs were all completing in under 20ms (although there is a lag for the first request since Lambda needs to re-load the function back into memory).
Needless to say, I’m actively lobbying my managers for another trip back to London so I can validate the results in person!
References
- https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
- https://aws.amazon.com/lambda/edge/
- https://www.w3.org/TR/cors/
Try Imperva for Free
Protect your business for 30 days on Imperva.
