As you might know, API Gateway is an integral part of AWS serverless ecosystem that covers several other services that integrate tightly with each others as well as third party systems and provides enormous benefits to customers. However, I won't be talking about API Gateway benefits, use-cases or even how to use it, but rather a very specific use-case that's currently not available out of the box from AWS API Gateway offering. This write-up can be considered as a workaround until AWS decides to release a change. The use-case that I'll be covering is when you have a private AWS API Gateway and you want to use a custom domain name. AWS documentations clearly states that:
"A Regional custom domain can be associated with REST APIs and HTTP APIs. You can use API Gateway Version 2 APIs to create and manage Regional custom domain names for REST APIs. Custom domain names are not supported for private APIs.
A bit of a context on what I'm presenting here. When you create an API Gateway in AWS, the URL that will be generated which acts as the endpoint URL that will be used by consumers and will have this form: https://api-id.execute-api.region.amazonaws.com/stage
where api-id is generated by API Gateway, region (AWS Region) is specified by you when creating the API, and stage is specified when deploying the API. Since the api-id hostname is a random ID, it can be difficult sometimes to remember besides the fact that some organizations have a standard naming convention for the services and resources and it's mandatory to follow that standard.
To elaborate more, let's take an example and assume that you created an AWS private API gateway and the endpoint URL that was generated by AWS is:
However, your organization domain name is example.com and you want the API Gateway endpoint URL to be: api.example.com
Having said that, we would need to introduce an internal NLB and a private endpoint fronting the private API gateway. In other words, clients will be contacting Route53 to find the corresponding CNAME record of api.example.com and that would point to the internal NLB in Route53 hosted zone. The NLB will have the private endpoint as a target and it will send the traffic to the API Gateway once it receives it from clients. Keep in mind that, you can use ALB (Application Load Balancer) instead of NLB if needed and it will perfectly work fine but I just chose NLB here for demonstration purposes.
Now you might be wondering what if I have several private API Gateways in the region, how would the private endpoint know which API Gateway to send the traffic to and the answer is that, the private endpoint has an internal mapping with API Gateway service in the region and the service knows which Gateway to send the traffic to based on the Server Name Indication aka SNI field in the Client Hello packet during the TLS handshake which contains the domain name that is associated with the API Gateway.
For clarity, the architecture will look like the diagram below:
Before delving into the details, it's a good idea to summarize the steps that will follow to build this setup:
1- If you don't have an existing R53 public hosted zone, you probably need to create one and ensure your name servers are correctly set.
2- Create a TLS certificate for the URL that you will use. Example, api.example.com
3- Create a private endpoint for com.amazonaws.region.execute-api
4- Create a TLS target group of type IP and add the private endpoint IPs as targets.
5- Create an internal NLB with a TLS listener and assign the TLS certificate in step 2 and the target group in step 4 to the NLB listener.
6- Create a private API Gateway and assign the same TLS certificate in step 2 to the API custom domain name.
7- Optionally for the sake of completion, create a Lambda function and set the integration between API Gateway and Lambda, preferably proxy-mode integration.
To automate the entire the process, I have created a Terraform module that can be used to build the entire setup.
Here is a step by step guide on how you can achieve this using AWS Console:
1) Request a TLS Certificate:
a) Navigate to AWS Certificate Manager (ACM) ---> Request a Certificate ---> Choose Request a public certificate.
b) Provide the FQDN of the certificate that you want to issue. This would be the URL that you want your clients to visit. For instance, if your domain name is "example.com", you can choose an FQDN such as "api.example.com".
c) On the validation method, this will dictate how you want AWS to verify the ownership of the domain that you want to create the TLS certificate for. You could either use DNS verification or Email. If you have your domain hosted in Route53, then choose DNS validation as this would instruct AWS to create a DNS entry in your Route53 hosted zone and finish the verification quickly without any hassle.
Once verification is complete, navigate to List Certificate to see the issued TLS certificate status. It should look like the screenshot below.
2) Create a endpoint for the private API Gateway:
Navigate to VPC --> Endpoint ---> Create Endpoint:
You will name to provide the following information:
a) Provide a name tag for the endpoint.
b) Select AWS Services in the Service category.
c) Select "com.amazonaws.region.execute-api" in the Services section.
d) Select the VPC that you want to place the endpoint at.
e) Select a private subnet in each availability zone. Although you can proceed with a single subnet, it's a good practice to choose a subnet in each availability for redundancy purposes in case of any AZ failure in the region.
f) Select a security group. The SG should allow inbound TCP port 443.
g) Leave the full policy as is. You can customize it if you want but we'll revisit this subject again when we deploy the API Gateway.
3) Create the NLB Target Group:
In this step, we'll create a target group of type IP with TLS protocol and point it to the private endpoint that created in the previous step. In other words, the endpoint that we created in step 2 will be the targets of the NLB. However to do that we need to get the IP addresses of the endpoint and to get them, navigate to the VPC Console ---> Endpoint ---> Select the endpoint ---> choose subnets tab ---> note down the IP addresses as shown the screenshot below.
Once you have the IP addresses, navigate to EC2 Console ---> Target Groups ---> Create Target Group.
You will need to ensure the following configurations in place:
a) Target Type: IP addresses
b) Provide a name for Target group name
c) Protocol: TLS, Port: 443, IP address type: IPv4
d) Choose the same VPC that you chose in step 2 when creating the API Gateway endpoint.
e) Keep the health check as is which is TCP using the default settings. You can customize the intervals but the default settings will work fine.
f) In the Register Targets part, provide the IP addresses of the API Gateway endpoint and then click on "Include as pending below".
Now the target group is created but it's still not configured to receive traffic from the NLB, hence the health status of the targets should be reported as "unused" and the healthy count is zero. You can also see the Console is reporting that the target group isn't associated with the NLB. This is fine at this stage as we will associate the target group with the NLB in the following section.
4) Create the NLB and associate the Target Group:
Navigate to EC2 Console and choose Load Balancers ---> Create Load Balancer ---> Choose Network Load Balancer.
In the configurations section, you need to provide the following information:
a) Provide a name of the NLB. This would be a name suffix that AWS will concatenate to a randomly generated name used as a URL to reach the NLB. Don't confuse this name with the actual URL (api.example.com) that we want our clients to access as we'll create an alias record in Route53 that will point to the NLB randomly generated name.
b) Choose "Internal" in the scheme.
c) Choose IPv4 in the IP Address type.
d) Select the same VPC that you chose in step 2.
e) Choose a private subnet in each availability zone like what we did with the endpoint in step 2.
f) In the Listeners and routing section, choose TLS as the protocol, and TCP port 443. Set the default action to route to the target group that we created in step 3.
g) In the Secure listener settings section, you can keep the default policy as is or you can choose a different one if you have certain security policy in your corporate for compliance purposes. Choose the TLS certificate that we created in step 1.
e) In the ALPN section, leave it as NONE.
It will take approximately five minutes to create the NLB and bring up the targets to a healthy state. if everything was configured correctly, your targets in the target group should now report a healthy status as depicted in the screenshot below:
5) Create an alias for the NLB in Route53:
Navigate to Route53 Console and select your public hosted zone and click on "Create record". You will need to provide the following information:
a) Record name: api ---> This would be the hostname part of the URL. If the desired URL is api.example.com, then the record name should be "api"
b) Toggle Alias
c) Alias to Network Load Balancer
d) Select the region of your load balancer
e) Select the NLB you created in step 4.
f) Select simple routing in the routing policy.
g) Keep Evaluate target health set to Yes.
To verify if the record was propagated globally, you can use either dig or nslookup utilities to see if you can resolve the DNS name.
6) Create a private API Gateway and assign the custom domain name and TLS certificate.
Navigate to API Gateway Console and click on Create API and choose "REST API Private"
You will need to provide the following information:
a) In the protocol: REST
b) Select New API unless you want to import the configurations from an existing or previously created Restful API using Swagger or Open API 3 configurations file.
c) Provide a name for the API. This could be any meaningful name that you can assign as it doesn't affect how the clients access the API.
d) Endpoint Type: Private
e) VPC Endpoint IDs: Grab the VPC endpoint ID that we created in step 1.
Now we need to set the Resource Policy to allow requests from the VPC endpoint that we created in step 1. To do that, click on Resource Policy and add this policy:
You'll need to substitute the value of "aws:SourceVpce" with the correct value of yours.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "*"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "*",
"Condition": {
"StringNotEquals": {
"aws:SourceVpce": "vpce-088e85394d70eeb2b"
}
}
}
]
}
Now we need to create a stage and deploy the integration between the API Gateway and Lambda. To do that, Select Resources ---> Actions ---> Create Method ---> Select the relevant HTTP method - in our example we're going to choose GET.
Integration Type: Lambda Function
Select Lambda Proxy Integration
Provide the Lambda Function name.
Once you click on Save, you'll be prompted to agree on granting the API Gateway the permission to invoke the Lambda function.
Next, we need to create a stage by deploying the API Gateway. To do that, click on Resources ---> Actions ---> Deploy API
Finally, we need to create the custom domain and associate the API Gateway with it.
Navigate to the API Gateway Console ---> Custom domain names ---> Create
You will need to provide the following information:
- Domain name: api.example.com
- TLS version: TLS 1.2
- Endpoint Type: Regional
- ACM Certificate: Select the same TLS certificate that you assigned to the NLB in step 1.
Next, we need to associate the custom domain name with API Gateway. Select the custom domain name in the API Gateway Console ---> API Mappings ---> Configure API Mappings ---> Add new mapping
Select the API and the stage:
Now fire up your cli from an EC2 instance in your VPC to test the connectivity and curl into your URL:
[ec2-user@Linux_SSH ~]$ curl -i https://api.example.com
HTTP/1.1 200 OK
Server: Server
Date: Fri, 11 Mar 2022 13:53:17 GMT
Content-Type: application/json
Content-Length: 17
Connection: keep-alive
x-amzn-RequestId: 37c956cb-47aa-4133-854c-56bae597ae33
x-amz-apigw-id: O0osGEZMSwd2F3fw=
X-Amzn-Trace-Id: Root=1-622b544d-3ceda224707ffe4f13518904;Sampled=0
"Hello from Lambda!"And it worked, voila!
As mentioned earlier, you can simply run the Terraform module located in Github to automate the provisioning of the entire setup:
Comments