Introduction
Online retail marketplaces present various business and technical challenges — and none more so than when you become really successful! Although having been around more than twenty years, Bokbörsen is a market for books, comics, records and movies that has enjoyed almost exponential growth over the last five years and while they initially approached PCG to help them reduce costs, it quickly became apparent that we could help them in a number of other ways.
About Bokbörsen
Bokbörsen is a Swedish online marketplace for books, music and films, founded in 2000, with 50,000 registered sellers and over 3.6 million books listed.
The Challenge
It’s beyond doubt that getting in early to any new market is a massive advantage for a business in all sorts of ways, from establishing a clear brand identity to developing insights and expertise that help you keep ahead of the game. Bokbörsen
’s arrival in 2000 was pivotal in their evolution into being the major player in the online retail market for books in Sweden, a position that enabled their rapid expansion over the last five years. From a point in 2017 where they sold 366,000 books, 2021 saw an increase to 609,000 sales from a catalogue of 500,000 unique Swedish language titles which is around five times larger than comparable retailers.
However, rapid growth can often be a mixed blessing and is a challenge in itself to ensure that your infrastructure and logistics can keep up with rocketing demand. Furthermore, while the cloud forms a significant part of any meaningful solution to this issue by providing almost limitless technical scalability, there is always plenty of room for improvement in design and configuration. In order to have increased operational security, safer management of the IT environment, and problem-free continuous operation, without risk of interruption, in 2022 Bokbörsen chose to ask us to set up the environment with AWS.
An interesting side point to their situation was that, despite being a fully online company with a website at the heart of their operations, all development work was performed by external service providers, and they had no developers of their own. This meant that they felt exposed to certain technical matters, such as issues they had with managing email queues in their AWS services, and that they also lacked an internal procedure to resolve such issues.
Playing by the rules
A common theme with the management of an AWS cloud is the need to make adjustments in order to abide by the rules of the service and to make sure that there are processes in place to make sure that this is the case. One such challenge that arose for Bokbörsen was that their outgoing email had exceeded certain agreed limits, which can lead to service disruptions if left unfixed. Despite being a relatively simple issue, obviously, this was a matter of priority for any successful solution.
The Solution
AWS costs can be optimized in a number of ways, following the principles outline in the AWS Well-Architected framework, including right-sizing the resources you use, making use of the available cost management tools and pricing options, and identifying and eliminating idle or underutilized resources.
Technical issues as a top priority
However, before implementing cost optimization strategies, it’s important to address any technical issues that may be affecting the system's performance. Therefore, as a first step, we first focused on resolving the email system issues at Bokbörsen.
One way to manage email queues with AWS is to use Amazon Simple Queue Service (SQS). SQS is a fully managed, scalable message queuing service that enables you to decouple and coordinate the components of your application. It allows you to send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
Here is an example of how you can use SQS to manage email queues:
- Create a new SQS queue through the SQS console or using the SQS API.
- Send emails to the SQS queue instead of sending them directly to recipients.
- Create an AWS Lambda function that polls the SQS queue for new messages and sends them as emails using the AWS SES service.
- Set up a CloudWatch event to trigger the AWS Lambda function on a schedule (e.g., every minute) or when a new message is added to the SQS queue
- Set up a dead-letter queue to catch any messages that cannot be sent. If a message fails to process multiple times, it will get moved to a dead-letter queue for further troubleshooting.
By using SQS to manage your email queues, you can decouple the process of sending emails from the rest of your application and scale the number of emails sent easily by adding more servers to process the queue. Additionally, SQS's dead-letter queue and retry logic provide a high level of reliability for ensuring emails are delivered.
Note that this solution is a very simplified one and you should consider more options, such as customizing the retry logic or adding a notifications mechanism. Our solution at Bokbörsen is slightly more complicated and is built on a number of different Amazon Web Services (AWS) to manage and process emails.
The Bokbörsen Email Solution
The Bokbörsen system uses Amazon Simple Email Service (SES) as the primary tool for managing emails. An SES configuration set is created to send all bounce events to an Amazon Simple Notification Service (SNS) topic. From there, the Amazon SNS event is passed on to an Amazon Simple Queue Service (SQS) queue. An AWS Lambda function pulls messages from the SQS queue every 15 minutes through a CRON event which is set up using AWS EventBridge .
The process of handling the bounced emails then begins in Amazon Relational Database Service (RDS). In Amazon RDS, a counter is incremented, and relevant data fields are returned for the notifications. There’s always a chance that a user’s server was unavailable or there was some other kind of temporary technical error on their side, so we don’t block a user if only one bounce is found.
As a result, if the increment count is 1, the email address is removed from the suppression list in Amazon SES. However, if the count is 2 or more, the system begins the notification process. This process includes creating a notification in the database, blocking the account (forbidding sales and purchases), and sending an email to our support team using Amazon SES. A notification is also generated for the user, which is visible when they log in. This informs them that their account has been blocked because of too many bounces and that they either need to fix their email hosting issues or use a different address.
Architecture diagram
Bokbörsen Email System
Results and Benefits
Cost reduction strategies
We began by implementing cost-reduction strategies and monitoring their usage to identify areas for improvement. As a result, we were able to lower their overall costs by eliminating idle or underutilized resources, and by making the best use of cost management tools and pricing options.
Cost-optimization also helps make the most of the scalability and flexibility offered by AWS, which can be particularly useful for businesses with fluctuating workloads or those that are growing rapidly such as Bokbörsen. Likewise, cost-optimization also enabled them to identify and manage resources more effectively, helping prevent overspend and improve overall efficiency.
After seeing these positive results, Bokbörsen decided to engage us for Managed Services and this allowed us to provide additional recommendations on their architecture and make other adjustments that will further improve efficiency in the future.
A better email system and onwards to serverless
One of the pressing technical issues we addressed was the problem with their email system, which we were able to fix and prevent them from being blocked again. Consequently, they now have increased internal control and procedures in place for resolving technical issues.
As we continue to work together, we are also exploring the use of serverless architecture to further reduce costs and improve scalability while Bokbörsen sustain their existing business growth.
Conclusion
Bokbörsen's rapid growth in the online retail market presented various business and technical challenges. By working with us, Bokbörsen was able to optimize their AWS costs and resolve issues such as managing email queues. The AWS Well-Architected Framework played a key role in providing guidelines and best practices for designing and operating a cost-effective and reliable system.
Bokbörsen's experience also highlights the importance of addressing technical issues before implementing cost optimization strategies and the benefits of using managed services. If you're facing similar challenges in your business and looking to optimize your AWS costs, we encourage you to contact us to learn more about how we can help.
About PCG
Public Cloud Group (PCG) supports companies in their digital transformation through the use of public cloud solutions.
With a product portfolio designed to accompany organisations of all sizes in their cloud journey and competence that is a synonym for highly qualified staff that clients and partners like to work with, PCG is positioned as a reliable and trustworthy partner for the hyperscalers, relevant and with repeatedly validated competence and credibility.
We have the highest partnership status with the three relevant hyperscalers: Amazon Web Services (AWS), Google, and Microsoft. As experienced providers, we advise our customers independently with cloud implementation, application development, and managed services.