AWS Lambda with Ruby: A Complete Getting Started Guide

Mon Nov 11 2019

It's five o'clock on a Friday afternoon. There are no new bug reports and everything is looking smooth. Your plan of a relaxing weekend is in sight when you get a call—the website you look after isn't responding. Yikes.

AWS Lambda aims to minimize the chances of those truly terrifying events happening by taking care of server maintenance while you focus on coding robust applications.

AWS Lambda is a function-as-a-service platform that stores and executes your code when triggered by an Amazon Web Service (AWS) event. These events range from making an API call to saving a file to updating a database. You only pay for the execution of the function—there's no other associated running costs or server maintenance. And you don't need to worry about server load because AWS will handle scaling. That gives you a consistent experience, no matter how much your function is used.

This post introduces AWS Lambda, discusses when you should and shouldn't use it, and takes an in-depth look at how to create your first Ruby function using the AWS console.

The Origins of AWS Lambda

At the 2014 AWS re:Invent conference, Amazon introduced Lambda to the world. Dr. Tim Wagner, general manager of AWS Lambda at the time, explained the benefits of using Lambda over EC2, the popular compute engine. Renting virtual machines in the form of EC2 instances from AWS brings flexibility and choice, but the tradeoff is that there's an unnecessary cost when idle. Plus, there's the additional complexity maintaining the infrastructure even when it's not in use.

The Benefits of Lambda

The benefits of AWS Lambda, as explained by Tim Wagner at re:Invent 2014, are

Reduced complexity. There's a fixed operating system, fixed software language, and a fixed version.
No infrastructure to manage. AWS owns and manages the infrastructure. You only pay for the time your code runs.
Implicit scaling. AWS will handle scaling, no matter how much you use your functions.

An Explanation of AWS Events

An AWS event is a JSON message containing the origin and associated event information, depending on the service. For example, if you add a file to the Amazon Simple Storage Service (S3), this will create an event containing the origin as the Simple Storage Service, the location of the file, and information that an item was added.

The services that can raise events that can trigger a function are

Events Trigger a Lambda Function Execution

Events trigger a Lambda function to execute. It's executed either directly or is added to a queue to be executed in the future. If you'd like the full list of all how services create events, you can check out the AWS documentation. But in short, a Lambda function subscribes to the notification-specific event from a specific source and will ignore all other events from any other source.

Programming Language Support

AWS periodically introduces support for new programming languages and currently supports seven different languages. Lambda supports writing code in Java, Go, PowerShell, Node.js, C#, Ruby and Python. As for the latest supported language, that's Ruby support for version 2.5.

Use Cases Where Lambda Excels

Lambda excels when the task has one or more of the following characteristics:

It's isolated. Tasks that perform only one function—for example, updating a database or storing a file—are great.
It's asynchronous. We're talking about low priority tasks that don't expect an immediate response. This could be sending an email or SMS.
It broadcasts. That means it's updating lots of different systems from one action, like creating a repository on GitHub.
There are big peaks in demand. Lambda works well when, during different times of the year or day, the demand is different. This could occur during peak holiday season.
There's low throughput. This occurs when a task happens rarely, such as a sensor that only reports on change (like a door).
It's a prototype or mock. Lambda shines when you're working with a proof of concept before committing more time to a project.
It's stateless. Nothing needs to be stored in memory.

Use Cases Where Lambda Is Not so Useful

Lambda doesn't excel where the task has the following characteristics:

It's long-running. Functions are limited to 15 minutes, so any tasks that exceed that will fail.
It has a consistent load. It may be cheaper to use a dedicated instance if the load is predictable.
It's intensive. Functions can only allocate 3GB of memory, so any intensive tasks will fail.
It requires a specific operating system or language. If tasks are tied to a specific operating system and have limited language support, Lambda might not help.
It has a stateful system. Here, we're talking about tasks that rely on a state to be stored between functions—for example, storing and retrieving files locally on the server.

A Use Case for Writing a Lambda Function

Imagine you're building a solution for users to store files. You choose AWS S3 as the location of your files because it's easy to manage and scale. The users can name the images anything they want, but you want to shorten the names to fit with the user interface design of a new app.

Renaming files is an ideal task for a Lambda function because it's a small isolated function, and it doesn't have to be run immediately. Plus, it's stateless.

The Starting Point for Your Ruby Lambda Function

The starting point for creating this function is a Lambda concept known as a handler. A handler is a method written in a Ruby file. In Ruby, file names are normally all lower case and separated by an underscore. For this example, we'll use the default file name when creating a new Lambda, lambda_function.rb.

The lambda_function.rb file contains a method, and its default name is lambda_handler. This method contains the logic, and it will be run on an event trigger.

The first named argument for the lambda_handler method is the event. The JSON AWS event converts into a Ruby object and is assigned to this argument.

The second method argument is the context. The context is assigned to a hash, which contains methods that provide information about the function, such as the name and any limits. A full list of all the information context holds can be found in the AWS docs.

The below code shows my lambda_function.rb file and the method, lambda_handler.

  function">#lambda_function.rb
  def lambda_handler(event:, context:)
  end

The above code is the base of a Lambda function and the minimum required for the Lambda to execute.

Reading Information From an Event

The main method that handles the processing of our event, named lambda_handler above, accepts event as an argument. This parameter is a Ruby hash converted from a JSON string containing the origin of the event and any contextual information. For our example, this will contain the name of the bucket and key of the file stored in S3.

The bucket name and key in S3 uniquely reference the image we're storing. To retrieve these values from the event hash, update the lambda_handler method to reference the bucket name and key values from the event. See the code below for how this looks in the lambda_handler function:

   def lambda_handler(event:, context:)
     first_record = event["Records"].first
     bucket_name = first_record["s3"]["bucket"]["name"]
     file_name= first_record["s3"]["object"]["key"] 
  end

The above code shows retrieving the first record of the event. There will only be one record in total when adding a file to an S3 bucket. The bucket_name and file_name are then extracted from the record.

Shorten the Name of the File

To shorten the name of the file, we'll use standard Ruby libraries to extract the first 21 characters of the original file name.

One of the great aspects of using Ruby for writing Lambda functions is the simplicity of string manipulation. The below code will assign the first 21 characters of the variable file_name to the variable short_name:

  short_name = file_name[0..20]

This code uses the fact that a string can be accessed the same way as arrays and selects the range from zero to 20.

Using the AWS SDK

Lambda functions are already configured to use the AWS SDK for Ruby, so no gems need to be installed before we can use the library. To reference the SDK, add a require statement to the top of your lambda_function.rb file. The below code shows the require statement at the top of the lambda_function.rb file:

  require "aws-sdk-s3"

The code above loads the gem containing helper functions to communicate with the S3 service.

Moving Your File From One Bucket to Another

To move your files from one bucket to another, reference the original image and call the move_to function on that object. The code below shows how to use the built-in SDK functions to move the file:

# Get reference to the file in S3
client = Aws::S3::Client.new(region: 'eu-west-1')
s3 = Aws::S3::Resource.new(client: client)
object = s3.bucket(bucket_name).object(file_name)

# Move the file to a new bucket
object.move_to(bucket: 'upload-image-short-name', key: short_name)

This code will create a new client in the eu-west-1 region, create a new s3 resource, and use the resource to reference a specific object: in this case, a file. The next action is to move the object to a new location. The move_to method calls the S3 service to copy the original file to the new location. Then, it deletes the original.

Putting the Code Together

When put together, the above code will achieve our goal of moving a file from one bucket to another, shortening the name to 21 characters.

require "aws-sdk-s3"

def lambda_handler(event:, context:)
  # Get the record
  first_record = event["Records"].first
  bucket_name = first_record["s3"]["bucket"]["name"]
  file_name = first_record["s3"]["object"]["key"]

  # shorten the name
  short_name = file_name[0..20]

  # Get reference to the file in S3
  client = Aws::S3::Client.new(region: 'eu-west-1')
  s3 = Aws::S3::Resource.new(client: client)
  object = s3.bucket(bucket_name).object(file_name)

  # Move the file to a new bucket
  object.move_to(bucket: 'upload-image-short-name', key: short_name)

end

The above is a working Lambda function containing the code from the previous sections of this post.

Now that we have our function, we can configure AWS to run it in a Lambda when we upload a file. To achieve this we need

An AWS account.
Two buckets—one for the initial upload and the other for storing the files once the name is changed.
A Lambda function triggered by upload of a file.

Setting Up an AWS Account

If you don't already have an AWS account, you'll need to set one up with Amazon. They have a generous free tier, which will cover creating this function and testing it. Amazon has a great guide on how to create and activate an AWS account, so follow this guide if you don't already have one.

Create Two S3 Buckets

Create two buckets: one that will allow initial upload of the files and another to store the files once they have been renamed. Amazon has a great guide on how to create a bucket, if you wind up needing help.

Creating Your Lambda Function Using the AWS Console

You can create a function by writing the code directly into the AWS console or by writing the code on your computer and then uploading to AWS. The rest of this post covers creating your function via the AWS console.

It's important to note that functions are located in a specific region. A region relates to a specific datacenter owned and ran by AWS. Services can't communicate between regions easily, so make sure the Lambda is in the same region as the S3 buckets.

If you're choosing a region for the first time, choose the region closest to you. You should also consider that some regions don't have all the features that others do. Concurrency Labs has a great blog post called "Save yourself a lot of pain (and money) by choosing your AWS Region wisely," and it'll help you choose.

Create Your Function Using the Console

To create a Lambda function through the AWS console, navigate to https://console.aws.amazon.com/lambda in your browser. By default, this will open the console in your default region. My default region is eu-west-2, located in the nearest location to me, London.

Select Create function to create a new Ruby function in the eu-west-2 region.

The above image shows the values you should select to create a new Ruby function successfully. Once you have selected these values, click Create function. This will display the designer view shown below:

This is a visual representation of your Lambda workflow. On the left is the trigger and on the right is the flow from the trigger to the output of the function.

Trigger Your Lambda Function From an Event

To set up a trigger for your Lambda function, select S3 from the designer view. This will open the configuration menu for the S3 trigger, as shown below:

The above trigger view shows the configuration options for S3. Select the bucket that contains the uploaded images in the Bucket dropdown and press Add. Your Lambda function will now run whenever an image is uploaded to that bucket.

Configure Your Lambda Function

Select the Lambda function box from the designer view. This will open the Function code view, shown below the designer, already populated with an example function. Copy the code we created earlier in this blog post into the lambda_handler method in the lambda_function.rb file:

Give Lambda Permissions to Write to S3

To allow your function to copy the file from one bucket to another the function requires some extra privileges. Select the Lambda function and scroll down to the configuration section shown below:

The above image shows the Lambda configuration section, focused on Execution role. Click the link named View the RenameFile-role-<unique value>. The link will redirect you to the AWS IAM service. Click Attach policy.

The above shows the attach policy view of the IAM service.

Now, search for s3. This will show all the policies relevant to S3. Select AmazonS3FullAccess and press the Attach policy button. You should now see an extra box on the designer view, and Lambda will now have access to all buckets in S3.

Test Your Function

Upload a file with a name longer than 21 characters to your image bucket. The file will shortly be deleted and transferred to the small names bucket and be 21 characters long. And there you go—test passed!

Writing Lambda Functions in Ruby

Ruby is a great language, and I'm really happy AWS is now supporting Ruby functions in Lambda. Once you're comfortable with the basics and want more advanced functions, Serverless Framework is a great next step. Serverless Framework allows you to package up gems to use in your function outside of those provided by Lambda. It also helps with versioning and organizing your code, along with storing the configuration for your functions.

Another tool that could speed up your development is Ruby on Jets, an easy-to-use Rails-like framework to create API endpoints and recurring jobs. These tools provide an abstraction on top of Lambda, and the generated functions can always be verified via the AWS console.

CloudWatch is the default monitoring tool created by AWS to keep track of errors in your Lambda functions. It's a good tool, but it's very labor intensive because looking through the CloudWatch logs is a manual process. Using a tool like Retrace could help monitor how and when each of your functions are being executed and track any errors together with context in a convenient dashboard.

Lambda is not the solution to every problem, and overuse might lead to unnecessary complexity. Take each use case one at a time and decide if Lambda is the correct solution for you.