Easy AWS Lambda function deployment with Apex
Introduction
Recently I have been spending a lot of time playing with AWS Lambda functions for various tasks like (small) ad-hoc data imports, billing alerts and building Amazon Alexa Skills.
For simple tinkering I was reluctant to do more than use the web-(and cloud9)-based IDE to avoid the need to set up a build and deploy pipeline, as a result the code only lived on AWS and not even in version control.
Enter: apex.
As the site and GitHub pages state:
Apex lets you build, deploy, and manage AWS Lambda functions with ease.
And this is certainly true as I demonstrate below with a (newly supported) ruby function.
Installing Apex
On (Ubuntu) Linux, installation is as simple as:
curl https://raw.githubusercontent.com/apex/apex/master/install.sh | sudo sh
Full instructions can be found here.
A sample project
Project Structure
Starting with an empty folder run:
paul@box [16:05:29] [~/Documents/Code/demo-lambda-function]
-> % apex init
_ ____ _______ __
/ \ | _ \| ____\ \/ /
/ _ \ | |_) | _| \ /
/ ___ \| __/| |___ / \
/_/ \_\_| |_____/_/\_\
Enter the name of your project. It should be machine-friendly, as this
is used to prefix your functions in Lambda.
Project name: demo
Enter an optional description of your project.
Project description: A demo of apex
[+] creating IAM demo_lambda_function role
[+] creating IAM demo_lambda_logs policy
[+] attaching policy to lambda_function role.
[+] creating ./project.json
[+] creating ./functions
Setup complete, deploy those functions!
$ apex deploy
This will create the basic structure for a simple nodejs function:
paul@box [16:05:56] [~/Documents/Code/demo-lambda-function]
-> % tree .
.
├── functions
│ └── hello
│ └── index.js
└── project.json
There do not appear to be any flags that will create a template for another language.
Functions and Ruby
AWS Lambda and Apex support a number of languages, for this example we will assume we have one function that uses Ruby, but in practise a single Apex project can have a number of functions and use any number of languages so long as the language is distinct per function.
Each sub-directory under the functions directory is (as the name suggets) a function in it's own right.
Each function name is derrived from the project name (prefix) and the function name (suffix), so here we will rename hello to ruby_example, as well as deleting index.js and creating a lambda.rb file as follows:
puts "start simple function"
def handler(event:, context:)
puts "processing event: #{event}"
return {
event: event
}
end
Now our file tree looks like this:
paul@box [16:10:00] [~/Documents/Code/demo-lambda-function]
-> % tree .
.
├── functions
│ └── ruby_example
│ └── lambda.rb
└── project.json
If we run apex deploy now the following error presents:
paul@box [16:11:33] [~/Documents/Code/demo-lambda-function]
-> % apex deploy
⨯ Error: loading ruby_example: validating: Runtime: zero value
It seems the sample project (and documentation on GitHub) is a little unclear on the required settings, we need to change project.json:
{
"name": "demo",
"description": "A demo of apex",
"memory": 128,
"timeout": 5,
"role": "arn:aws:iam::181984840591:role/simple-lambda-function_lambda_function",
"environment": {},
"runtime": "ruby2.5",
"handler": "lambda.handler"
}
runtime and handler needed to be added by hand as example did not generate values for them and Apex complained of their absence.
Deploying
Deploying is then very simple:
paul@box [16:13:17] [~/Documents/Code/demo-lambda-function]
-> % apex deploy
• creating function env= function=ruby_example
• created alias current env= function=ruby_example version=1
• function created env= function=ruby_example name=simple-lambda-function_ruby_example version=1
Every time the code is updated we can simply deploy again, if no change has been made Apex will detect this and the version will remain the same, and if there has been a change a new version is deployed:
paul@box [16:14:47] [~/Documents/Code/demo-lambda-function]
-> % apex deploy
• config unchanged env= function=ruby_example
• updating function env= function=ruby_example
• updated alias current env= function=ruby_example version=2
• function updated env= function=ruby_example name=simple-lambda-function_ruby_example version=2
Executing
The function can be easily executed using apex invoke, for example if we have a file with a sample requesta, events.json:
{
"hello": "world"
}
Then it can be executed as follows:
paul@box [07:48:25] [~/Documents/Code/demo-lambda-function]
-> % apex invoke ruby_example < event.json
{"event":{"hello":"world"}}
Logs
The logs for Lambda functions are available in CloudWatch Logs, but I often find this is a little clunky to use.
Apex lets you view and tail logs:
paul@box [07:49:56] [~/Documents/Code/demo-lambda-function]
-> % apex logs -f
/aws/lambda/simple-lambda-function_ruby_example start simple function v2
/aws/lambda/simple-lambda-function_ruby_example START RequestId: 983bd400-01b6-4675-9727-b2841d96605d Version: 2
/aws/lambda/simple-lambda-function_ruby_example processing event: {}
/aws/lambda/simple-lambda-function_ruby_example END RequestId: 983bd400-01b6-4675-9727-b2841d96605d
/aws/lambda/simple-lambda-function_ruby_example REPORT RequestId: 983bd400-01b6-4675-9727-b2841d96605d Duration: 14.65 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 33 MB
Or alternatively you can use something like saw.
Dependencies
Dependencies for most lambda functions must be packaged with (relative to) the code. With Ruby, bundler can do a manual install for deployment which will put the dependencies in vendor/bundle.
Apex supports a concept of "hooks" which can run pre-build, pre-deploy or on clean-up.
Some properties, including hooks, can be specified at the project level (project.json) or at the function level (function.json inside the function's subfolder).
Here I have added a Gemfile, with a few mysql gems, as well as including them in the lambda itself:
paul@box [09:30:39] [~/Documents/Code/demo-lambda-function]
-> % tree .
.
├── event.json
├── functions
│ └── ruby_example
│ ├── function.json
│ ├── Gemfile
│ └── lambda.rb
└── project.json
function.json:
{
"description": "External Gem dependencies",
"hooks":{
"build": "bundle install && bundle install --deployment"
}
}
Gemfile:
source 'https://rubygems.org'
gem 'sequel'
gem 'mysql2'
lambda.rb:
require 'rubygems'
require 'sequel'
puts "start simple function v2"
def handler(event:, context:)
puts "processing event: #{event}"
return {
event: event
}
end
After deploying this the ruby_example folder contains the vendorised gems (these can be added to .gitignore). It is wise not to clean them up each time as that will mean there is no cache which in turn will slow down the deploy.
Version Control
All of the configuration required for Apex is safe to keep in version control as it is only really the IAM role, the sensitive data like AWS Key and Secret are still kept elsewhere (ideally in aws-vault if you are doing things properly). As these are also required to enable any (potentially harmful) changes like function creation, deployment and so on it does not matter if others in the team have access to the Apex configuration files.
Apex also supports using different aws profiles through command line arguments (-p or --profile) as well environment-based configuration (-e or --env) files meaning you can have a different configuration and/or profile to deploy to dev and prod respectively, still using IAM profiles (keys and secrets) to control who can deploy and to where.
Sadly missing values in an env-specific file are not inherited from the base file so some duplication seems to be required.
Continuous Deployment
It is likely tempting to set up deployment as part of a CD pipeline, deploying on every commit (perhaps to master), and for a production environment this might make sense, but for development, staging, testing and other non-prod environments it may prove to be overkill as every tweak, attempt at a bug fix, bit of extra diagnostic data would require a commit (and push) which could quickly make the revision history quite noisy.
Apex is still an excellent tool to handle the production deploys in a CD pipeline, along with providing a convenient way to deploy during development.