Serverless Anything: Using AWS Lambda Layers to build custom runtimes
Ben Ellerby4 min read
Late in 2018 AWS released Lambda Layers and custom runtime support. This means that to run unsupported runtimes you no longer need to ‘hack’ around with VMs, docker or using node.js to `exec` your binary.
Recently I needed to setup a 100% serverless PHP infrastructure for a client. PHP is one option, but similar steps can allow you to run any language.
Lambda layers provide shared code between different lambda functions. For instance, if you wanted to share your vendor code between lambdas (e.g. node_modules for node.js).
We will create a Lambda layer to provide the PHP binary for our custom runtime API. You could also create a second to provide the vendor folder for composer dependencies.
# Step 1: Compiling thePHPbinary
We will need a `/bin` directory containing the PHP binary. Because we are compiling a binary this step needs to happen on the same OS and architecture that our Lambda will use.
- This page lists the AWS Execution environment, but last time I checked their version was out of date.
- To find the correct AMI I used the latest version for the region I was deploying in, found here by a quick regex for AMIs containing `amzn-ami-hvm-.*-gp2`.
Once you have the correct AMI, spin up a large EC2 instance and `ssh` in.
Run the following commands as listed in AWS’s docs.
sudo yum update -y
sudo yum install autoconf bison gcc gcc-c++ libcurl-devel libxml2-devel -y
curl -sL http://www.openssl.org/source/openssl-1.0.1k.tar.gz | tar -xvz
cd openssl-1.0.1k
./config && make && sudo make install
cd ~
mkdir ~/php-7-bin
curl -sL https://github.com/php/php-src/archive/php-7.3.0.tar.gz | tar -xvz
cd php-src-php-7.3.0
./buildconf --force
./configure --prefix=/home/ec2-user/php-7-bin/ --with-openssl=/usr/local/ssl --with-curl --with-zlib
make install
Checkpoint: The following should give you the version number of PHP you wanted
/home/ec2-user/php-7-bin/bin/php -v
Move this into a /bin directory.
`mkdir './bin && mv php ./bin
Now we can zip up the code for our Lambda layer that will provide our custom runtime API.
zip -r runtime.zip bin bootstrap
# Vendor files
On the same EC2, we need to use composer to get our vendor code.
curl -sS https://getcomposer.org/installer | ./bin/php`
Add some vendor code:
./bin/php composer.phar require guzzlehttp/guzzle
zip -r vendor.zip vendor/
# Bring down to local
Now use scp to copy the PHP binary down to your local machine:
- From local:
scp {YOUR EC2}:/home/ec2-user/php-7-bin/bin/runtime.zip .
Now use scp to copy the vendor zip down to your local machine:
- From local:
scp {YOUR EC2}:/home/ec2-user/php-7-bin/bin/vendor.zip .
Don’t forget to terminate your large EC2 instance
# Creating a custom runtime API
To use a custom runtime AWS requires you specify a bootstrap file which will provide the interface for lambda events. As we will be writing PHP support we can write it in PHP (very meta).
Create a bootstrap executable:
touch ./bootstrap && chmod +x ./bootstrap
Example adapted from AWS docs
#!/opt/bin/php
<?php
// This invokes Composer's autoloader so that we'll be able to use Guzzle and any other 3rd party libraries we need.
require __DIR__ . '/vendor/autoload.php';
// amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2
function getNextRequest()
{
$client = new \GuzzleHttp\Client();
$response = $client->get('http://' . $_ENV['AWS_LAMBDA_RUNTIME_API'] . '/2018-06-01/runtime/invocation/next');
return [
'invocationId' => $response->getHeader('Lambda-Runtime-Aws-Request-Id')[0],
'payload' => json_decode((string) $response->;getBody(), true)
];
}
function sendResponse($invocationId, $response)
{
$client = new \GuzzleHttp\Client();
$client->post(
'http://' . $_ENV['AWS_LAMBDA_RUNTIME_API'] . '/2018-06-01/runtime/invocation/' . $invocationId . '/response',
['body' => $response]
);
}
// This is the request processing loop. Barring unrecoverable failure, this loop runs until the environment shuts down.
do {
// Ask the runtime API for a request to handle.
$request = getNextRequest();
// Obtain the function name from the _HANDLER environment variable and ensure the function's code is available.
$handlerFunction = array_slice(explode('.', $_ENV['_HANDLER']), -1)[0];
require_once $_ENV['LAMBDA_TASK_ROOT'] . '/src/' . $handlerFunction . '.php';
// Execute the desired function and obtain the response.
$response = $handlerFunction($request['payload']);
// Submit the response back to the runtime API.
sendResponse($request['invocationId'], $response);
} while (true);
?>
- Note: #!/opt/bin/php links to our /bin/php created earlier.
Manual Deployment
- Go to AWS lambda page.
- Create a function selecting to use a custom runtime.
- Create a layer called
phpand uploadruntime.zip. - Create a layer called
vendorand uploadvendor.zip. - Apply the layers to the function you created in the merge order: 1) runtime, 2) vendor
Writing a handler function
mkdir src
touch src/hello.php
Add some basic function called hello:
<?php
function hello($data)
{
return "Hello, {$data['name']}!";
}
?>
Then zip this up to be uploaded:.
zip hello.zip src/hello.php
Upload the function handler zip to the function and change the handler name to the name of the php file without the extension. e.g. hello.php => hello
# Automation
Many of the steps here can be automated once you have compiled your binary. Either by using the AWS API, cloudformation or the serverless library which supports layers.
# Other languages
These steps should allow any language that can compile on the AWS AMI used by Lambda to be used as your runtime, e.g. Rust.
Resources:
