I've recently been considering adopting a dog and during my search, I realized how useful it would be if I could receive notifications whenever new animals arrived at my local shelter. I looked around and this seems to be a feature not many shelters, if any, have. Seeing what looked like a fun project to tackle, I went about mapping out the solution within AWS.
My goal with this project was to give myself some further experience with JS, API, and chaining Lambda functions together. I'm aware my solution involving several Lambda is likely not efficient, but I was more interested in gaining some experience in this area than creating a perfect solution. Though, I do have ideas for improvements that I will discuss later in this post.
Github link: https://github.com/bdaley94/Shelter_Notification_Service
Solution Overview
The goal of this solution is to provide email updates to users who would like to know when new animals arrive at the shelter. I am using Lambda functions written in Python for all of the processing. There is a DynamoDB database for both cats and dogs which I used to keep track of what animals were in the shelter last time the code ran. Users sign up for the notifications at https://brandon-daley.com/HPA-Form and choose whether they want emails regarding cats, dogs, or both. Users can unsubscribe by clicking a link at the bottom of any email they receive. The Lambda chain ends by querying the email address list DB and sending an email for each new cat/dog arrival depending on what alerts a user is signed up for.
Eventbridge + Lambda
My solution starts with an Eventbridge rule that triggers the start of the Lambda chain every day at 7pm CT. The first Lambda function (Dog1/Cat1) is responsible for three things:
Gathering the current info on available animals from a 3rd party API (Shelterluv) used by the shelter. The data comes in JSON format.
Scanning the appropriate DB to gather a list of the animals that were available the last time the solution ran.
Compare the list of animals in step 1 to the list in step 2. The animals which were in the DB, but are no longer in the newly gathered list from Shelterluv, have been adopted. The animals which are in the new list and not in the DB are new arrivals to the shelter.
Two dictionaries, one for new arrivals and one for adopted animals, are passed to Dog2/Cat2. This function then removes adopted animals and adds new arrivals to the appropriate DB.
Upon completion, the Dog2/Cat2 functions pass a dictionary containing the new arrival animals to Dog3/Cat3. This function handles several things:
Gathers a list of email addresses currently signed up for dog or cat emails depending on which function is running.
Parses all the relevant data that will be needed in the email to users such as an embedded photo, breed, age, weight, etc.
Iterates through each new arrival animal and sends an email containing the above info as well as inserting an unsubscribe link pre-filled with the recipient's email.
DynamoDB
My solution includes three DynamoDB tables. There are two separate tables for dogs and cats. These tables are used to store the animal info from the previous day. This allows me to compare the present-day animals against the previous and determine what animals are new. Something to note is the birthday data provided by Shelterluv is in Unix time, so my Dog3/Cat3 function includes code to convert this into human-readable years, months, and weeks to display the age of the animal. Below is a sample of the data I store.
The third table stores all email addresses signed up for notifications, and what animal they want emails about. This information is used in Dog3/Cat3 to determine what users should receive dog or cat email
SES
I used Simple Email Service to send the email notifications to users. The code is within the Dog3/Cat3 function. I iterated through each new arrival animal and sent an email for each animal to every user requesting emails for that type of animal before moving on to the next animal to repeat these steps. The email body is HTML which allowed me to embed an image of the animal as well as place hyperlinks for the animal's adoption page and for the user to unsubscribe from future emails.
Subscribe Functionality
Web form
A user wishing to subscribe to these notifications must visit https://brandon-daley.com/HPA-Form. The form is a simple webpage that allows users to submit their email address and what notifications they want. Upon submitting the form, my Javascript code runs and sends the form data to my API route /form. The Javascript also includes checks for valid and complete form data such as ensuring the user fills out all fields and submits a valid email address. I did start building out the functionality for SMS notifications as can be seen in this form, but ultimately cut this feature due to the new regulations surrounding registering toll-free numbers in the USA
API_Sub - Lambda Function
The API /form route invocation triggers this Lambda code to run. This function retrieves the data sent by the form and uploads it to the Email Address DB. It also returns status codes to determine if the upload was successful which the aforementioned JS code will then use to inform the user if their info was submitted successfully or not. It will also tell the user if they had previous notification settings for the submitted email address which were updated.
Unsubscribe Functionality
Unsubscribe link
Every email notification a user receives will contain an unsubscribe link which is pre-filled with the user's email for ease of use. All the user has to do is click the link and they will be unsubscribed.
API_Unsub - Lambda Function
When a user clicks the unsubscribe link it will trigger this Lambda code to run. The code will retrieve the user's email address from the API invocation query string. It will then check the Email Address DB to confirm the user is currently subscribed. If this is confirmed to be true they will be deleted from the DB and the API will return a confirmation message to them. If not, they will receive a notice that they are not currently subscribed.
Security
Each lambda function follows the principle of least privilege regarding their permissions to other services.
The API form is hosted on my website with the appropriate HTTPS security encrypting the data sent via the form.
Data at rest in DynamoDB is fully encrypted with AWS managed/owned keys via KMS.
AWS SES uses opportunistic TLS by default which means it "... always attempts to make a secure connection to the receiving mail server. If Amazon SES can't establish a secure connection, it sends the message unencrypted."
Improvements
Here I'm going to list some things that I could do to improve or expand the project upon returning to it at a later date.
SMS notification functionality - This would've been a major improvement and something I would've liked to include. However, as I mentioned briefly elsewhere in this blog, after attempting to register a phone number via AWS Pinpoint I found it would've taken longer than I scheduled to work on this project. Regardless, I did build out 99% of the requirements to add this functionality so it could be easily added, all I'm missing is the phone number.
Alternative service usage - During my research I found one way to go about sending bulk emails is to use SQS. Essentially, you add all recipient addresses to a queue and your Lambda is triggered for each message in the queue and sends an email to that address.
The issue with this is the code sending the emails needs to have access to the new arrivals animals dictionary from Dog2/Cat2 which is currently passed to Dog3/Cat3 which sends the emails using that data. Since the email sending Lambda would be triggered by each message in the SQS queue, I wouldn't be able to pass this data. I believe I could store the dictionary as JSON in each message, but that is a lot of redundant data being sent. I believe the best option here would be to have another DynamoDB table used as ephemeral storage to contain the new arrivals and have the email-sending Lambda pull from that upon each SQS message invocation. I couldn't find any good way to determine when the queue is empty without polling it constantly which would be bad practice for reducing costs. The best option here may be to clear it at a specified time, say 30 minutes after 7:00 pm CT, using an Eventbridge rule triggering a Lambda function.
Flesh out unsubscribe UI - Currently the unsubscribe UI is very barebones and doesn't offer features a user may expect, such as the ability to resubscribe if you accidentally unsubscribe. Additionally, it currently returns a blank response page directly from the API with just the text of whether their unsubscribe was successful or not. This gets the job done but would need to be spruced up a bit in production. To improve this I'd create a simple webpage where the user would be sent instead of the direct API URL, and for the API to return the response which would then be displayed to the user.
Infrastructure-as-code - As always, integrating IAC into this project would allow me to tear it down and redeploy it as well as make configuration changes more easily.
Combine animals in emails- If many animals come to the shelter on one day, the subscribers will receive numerous emails due to each animal getting an individual email. I could implement some code to check if there is more than one cat or dog within the Dog3/Cat3 Lambda function, and if so combine the emails into one before iterating through the list of email addresses and sending them off.💡This improvement has been implemented and is live as of 7/12/2023!Email to confirm subscription upon signup - Instead of (or in addition) to my JS code which notifies a user their subscription was successful, I could also send an email to notify them they are now signed up.