SAM CLI and Certificate Authority Bundles

I’ve been working with AWS CodePipeline and AWS CodeBuild. We are currently integrating our on premise GitHub Enterprise installation with these AWS Tools and are looking to get the same (or near same) functionality we currently have with on premise TeamCity. Part of that functionality is reporting the build status back to GitHub Enterprise so that our pull request process is fully informed. Our team norms dictate that a pull request is not merged until it has passed all of the build steps. Those steps include the build, unit tests, integration test, etc. Once those tests pass, and the status is reported back to GitHub, only then is the pull request ready to be reviewed by another team member.

Unfortunately CodePipeline doesn’t automatically report the status back to GitHub. You have to utilize SNS notifications and a Lambda function. Jens Eickmeyer has a good post on how to wire up the two along with the code for the Lambda function.

Jens’ post was a great start for me, but I soon ran into some challenges with our corporate environment. Many companies run a web proxy with custom certificates for internal clients. This helps to secure the enterprise by decrypting, most importantly inspecting, and then re-encrypting traffic in and out of the network. This helps to insure corporate data that shouldn’t leave the network doesn’t. Since these custom certificates and certificate authorities are not trusted outside of the enterprise, this poses a challenge with tools like the AWS SDKs. When running locally and interacting with the AWS APIs, we often see certificate errors. With the AWS Serverless Application Model (SAM) CLI it is even more difficult because your code is now running in a docker container.

Many people would be tempted to disable these certificate checks. I’ve seen many forum comments or blog posts advocating for the ability to disable them. THIS IS NOT A GOOD IDEA! What if that disabled configuration made it to your production environment and your organization was compromised in some way? Some would call that a “resume generating event” and at the very least a very long day for everyone. You should run your non-production environments with as close to production configurations as possible.

We were able to work through the challenge of including our corporate Certificate Authority bundle in the SAM container by introducing it into the node.js certificate chain. You can accomplish this via the NODE_EXTRA_CA_CERTS environment variable by making a couple of changes in your package.json file.

First, make sure you have the main property defined otherwise your index.js or app.js file will not be included in the sam build bundle.

"main": "index.js",

Next, add the CA bundle file to your project directory using a prepack command. We store our CA bundle centrally in Artifactory, so we download it and put it in the root of the build folder.

"scripts": {
    "prepack": "powershell -command \"Invoke-WebRequest https://my_artifact_repository.tld/certificates/CA-Bundle.crt -OutFile CA-Bundle.crt\""
}

Finally (still in the package.json file), we want to make sure the bundle is included when packaged up by the sam build command from the SAM CLI.

"files": [
    "CA-Bundle.crt"
]

When developing with SAM on my local machine, I usually have a environment variables file to override the settings I’m going to use in a production environment. In the env.json file, I’m going to add a parameters block to include the NODE_EXTRA_CA_CERTS environment variable.

{
  "Parameters": {
    "NODE_EXTRA_CA_CERTS": "CA-Bundle.crt"
  }
}

Lastly, I want to make sure I have the same environment variable declared in my SAM template so that when I deploy using the SAM CLI (sam deploy), the environment in Lambda is properly configured.

Environment:
  Variables:
    NODE_EXTRA_CA_CERTS: "/var/task/CA-Bundle.crt"

That’s it! Now when we make calls from the Lambda function running in the docker container, we can make secure requests out to the AWS API.

If you have questions, connect with me on twitter