Mail to Lambda, Part 1

For a while now, I’ve been interested in experimenting with the new Serverless fad and see what it is all about. The idea is that you don’t need to care any more about installing a real or virtual server, or even about creating a container image. Instead, you just write some code and deploy it somewhere in the cloud as a function.

The hype is all about being able to scale your code without pain and being able to upgrade it easily, but of course this technology also makes it very easy to experiment without having to worry about the deployment method du jour – do I need to create a VM? A container? A pod? Nope, just a function.

Alternatives

In the past I’ve used services such as Heroku and Google App Engine (which is now part of the Google Cloud Platform) that allow doing something similar, though the abstraction layer there is different. The classic App Engine (now called Standard Environment) has a special limited runtime, which requires adapting your code to use its specific APIs. Google has a newer version (called Flexible Environment) which is based on containers and allows writing more standard code. Unfortunately, it currently lacks the ability to receive email, which the older environment did support and which was needed for this experiment.

Going Serverless

The leading Serverless offerings include AWS Lambda and Google Cloud Functions. Initially I preferred to play with the newer Cloud Functions, but it is still in alpha and requires asking for access, which I did but have not yet received at the time of this writing. Another option is Azure Functions, but I haven’t yet had a chance to work with Azure and didn’t want to get sidetracked too much.

So, AWS Lambda it is.

The Project

I’ve been using the very nice HoursTracker app for quite a while now to manage my work hours. It allows me to easily track the amount of hours worked for several clients. When the time comes to report hours, I use the app’s export function to email myself a CSV file and then I do some manipulations on the data using Google Docs before sharing them with the client.

For a while now, I’ve wanted to automate these manipulations since they tend to become a bit tedious. Being one of those things you do once a month and take only ten minutes or so, it never became just important enough to me to actually go and automate. It was therefore a perfect candidate for playing with some new technology and possibly getting a usable result as a bonus.

Inventory of Parts

I needed the following parts to assemble the full solution:

1. An email address to which I could send the exported data from the HoursTracker app.
2. A service that would execute some custom code when a new mail is received at the above address.
3. Some code that would process the email, extract the CSV data, manipulate it as needed, convert it to the final format, and send it to its destination.

The new stuff for me were the first two items, so I started with those.

Receiving Email

To receive emails in a way that would work with AWS Lambda, I used Amazon Simple Email Service (SES). Amazon provide clear instructions for setting it up, and I encountered no issues when following them.

If you want to do the same thing, note that it requires having a custom domain for which you can modify DNS records. I chose to delegate a specific subdomain of my domain (e.g. api.example.com) to Amazon’s Route53 DNS service, and added an MX record for that subdomain so that mails sent to it would be handled by SES. All this is explained in the documentation mentioned above.

As part of completing the instructions, I then created a mail Receipt Rule for the address hourstracker@api.example.com with an action that adds any received mail to a specific S3 bucket.

At this point, the first part of the inventory is complete: We can receive emails sent to hourstracker@api.example.com, and these are then stored in the hourstracker S3 bucket, ready for further processing.

Next Time

Receiving and storing the mail brings us to the point where we can call our custom code to process it. This is where AWS Lambda comes in. In the next part of this series, we will write a simple function to get some information from the email, and will tell SES to call it when a new mail arrives.