MARCH 22, 2024
MARCH 22, 2024
Performance, reliability and accuracy are among the few nouns that aptly describe engineering teams. The great architects and engineers of human civilization have toiled to perfect systems and processes to build everlasting monuments. From SpaceX’s Starship to the engineering of the fountain pen, each engineering feat is built across many iterations, in a never-ending quest to find perfection. This blog is an ode to the engineering journey where we sought to perfect the magic of sharing happiness with branded payments at YOUGotaGift.
The founding engineering team at YOUGotaGift was a closely-knit group of tech professionals with diverse skill sets. In the early years of the company, diversity was beneficial; when budgets and resources were lean, teams relied on each other to design the most efficient gift card system with a foundation that was meant to last. It was simple, but there is great beauty in simplicity – the initial YOUGotaGift setup included one web server, one database server, a few cache servers, and a CDN to speed things up. YOUGotaGift preferred to develop applications in Python, which was stable, versatile, easy to integrate, had strong library support, and became a must-have tool to solve tasks from web development to machine learning, and of course, today it is the go-to language for tasks related to Generative AI. As the business from B2C and B2B grew at YOUGotaGift, the challenge was to ensure YOUGotaGift’s technology infrastructure could continue to meet business demands. The ability of the team to learn new technologies and adopt agile methodologies made it easy for engineering teams to evolve. In the early days, technologies such as Django, REST APIs, and Rackspace Cloud helped handle the growth and keep pace.
One of the earliest and most difficult Product/Engineering challenges we encountered was transforming and delivering the physical gifting experience into a digital one. Given the lack of tangibility in digital gifting and rewarding, we aimed to combine our product with the most personalized and memorable touch possible. Our ultimate goal was to bring the same emotions and sentiments that accompany the physical gifting experience into the digital form.
Our monolithic architecture was stable for years as we managed a few services, customers, and partners. However, as time went on and the business grew along with the traffic, we onboarded more clients, who required further API integrations, and we developed more product features (primarily for banks and telecoms). All this made releasing new products with the latest technology stack very challenging. Adding core features became difficult because it demanded comprehensive system-wide testing, and new team members struggled to learn the extensive codebase. Sometimes, a minor bug in the code could bring the entire system down. Moreover, the situation worsened when it came to maintenance. Updating our software stack could take months or even years, with some of them still being a chasing dream. Similarly, the operations department started to grow in a way that resembled our monolithic structure and encountered challenges in restricting access to certain products, a task absolutely crucial for our business.
In keeping pace with our business expansion and to ensure better maintenance of our product, and integrate the latest technology into our stack, and provide better user experience, we transitioned from a Monolithic Architecture to a Microservices Architecture. This reorganization into smaller, independent services enhanced our scalability, flexibility, and maintenance efficiency. Beyond the additional effort required for ensuring data consistency and effective service communication, the benefits of microservices architecture (including the ability to adopt suitable stacks for specific use cases) were invaluable. Nevertheless, one operational challenge often underestimated in distributed systems architecture is the effective allocation of tasks among various teams. Utilizing Jira Software and agile methodology has facilitated our ability to establish a structured process and direct tasks to the appropriate team.
Breaking down the big monolith was a tedious task, as it demanded the operational availability of the service. Initially, We created clones of the monolith application for each independent product; in parallel, our team began developing applications from scratch with the latest technology. This approach allowed us to maintain service availability during the transition.
Cloud Infrastructure has seen an incredible transformation, moving from basic single server setups on platforms like Rackspace to more complex, distributed systems managed by AWS Elastic Beanstalk, and eventually, to Kubernetes clusters that span multiple regions. These clusters are now able to manage thousands of pods at once. This jump in technology has been greatly enhanced by the introduction of serverless database systems and services like AWS Global Accelerator, which reduce data transfer times, improving both performance and user experience worldwide.
A major step forward in our cloud infrastructure was combining Kubernetes with Karpenter, which changed the way we launch and manage instances based on workload demands. This setup has allowed us to cost-effectively use spot instances, a topic we plan to explore more in a future article.
In terms of cloud infrastructure management, adopting AWS Organizations and the sub-account framework has made a huge difference. By using Infrastructure as Code (IaC) tools like Terraform, we’ve made it easier to set up multiple, separate accounts for different teams and products. This approach not only makes audits easier but also strengthens our security by applying the principle of least privilege across various environments and teams.
A major win in our cloud evolution was centralizing the management of network and Virtual Private Cloud (VPC) setups through a specific AWS account. This method ensured a uniform network setup while allowing teams to manage their own resources independently, upholding strict security and compliance standards.
Our progress in improving cloud infrastructure is marked by a series of strategic “small wins” that, over time, have greatly enhanced our ability to deploy, scale, and manage applications more effectively and securely. Key among these wins is the adoption of Kubernetes and Terraform, and a wider use of containerization. These technologies have not only made our operations smoother but also built a solid base for agile and resilient development of cloud-native applications.
A standout example of our platform’s flexibility is the move of our rewards platform to the Oracle Cloud Infrastructure (OCI) in the Saudi Arabia region. This was driven by strict data sovereignty laws requiring data to be stored and processed within the country. This move highlighted several strengths of our cloud strategy, like flexibility, adherence to compliance and data sovereignty.
Today, our diverse tech stack includes Python, GraphQL, React NextJs, Swift, Kotlin, Go, ELK, Sentry, K8S, terraform, k8S Git Action runners, Prometheus and Grafana, all of which provide seamless integration with a microservices approach.
Over the years, we’ve introduced a dozen products to cater to various use cases within the digital gift card industry. A few of these are eCommerce experiences, white label solutions, YOUGift API, YOUProcess, YOURewards, and YOUCampaigner, YOULoyalty among others.
In May 2021, we were thrilled to become the first digital gift card business in our region to meet PCI-DSS compliance, highlighting our dedication to the security and trust of our users. Our security framework is designed around NIST standards, further strengthening our security practices. Plus, we make use of AWS’s advanced security technologies, like identity and access management, encryption, and threat detection, to safeguard our infrastructure. A special thanks goes to SonarQube, GitHub Security, Hashicorp Vault and WireGuard for helping us elevate our security standards.
