ET Network
Migration Of ET Network to AWS
Building the Foundations for a New Vision of Coffee Supply
Over a series of design and development meetings, teams from Era of We, Opsio, and AWS envisioned an overall architecture with three main components: front end, ecommerce, and backend.
Different teams were tasked with building each component. Opsio took responsibility for overall project management and for creating the common management and systems that brought everything together within a reasonable timeframe and with the ability to grow. “Era of We has a unique vision and intense business focus, and we offered the experience and resources to realize those with confidence,” says Premjeet Sahu, senior technical architect at Opsio.
Opsio partitioned the system’s three main components into a core for business logic, validation, APIs, and backend service provision. A front end of multiple websites exposes appropriate services to each class of user. And there’s ecommerce support for handling purchases, payments, and logistics—it’s a full-featured shopping platform for food and drink.
External teams were tasked with ecommerce and front-end development, while Opsio created the management systems and service provisioning that brought everything together securely, scalably, and swiftly.
Only the combination of cloud-based services and modern DevSecOps principles could bring a system with global scope and complexity within the grasp of a startup. This made it possible to deliver viable development within budget and on time. It also allowed the teams to create something that’s manageable day-to-day and supports future innovation.
The project called for collaborative development among different teams and team members: enterprise and software architects, QA experts, analysis and security specialists, suppliers, and partners. “The challenges in collaborative development between global teams are significant, but so is reinventing a global supply chain as a community,” said Erik Hedlund of Era of We. “The scope of one matched the scope of the other, so we were comfortable that Opsio had the breadth to manage both.
The full implementation relies on many other AWS services. All of these services work together to automate provisioning and security system components, providing continuous data for service analytics and business model planning, assurance of business continuity at scale, and the adherence to best practices for effective management. “The decision to build the company around such a complex and multifaceted project was bold, but in Opsio and AWS we found partners that not only shared our vision but committed fully to working with us to make it happen,” says Erik Hedlund. “We are very happy with our experience so far, and excited about inventing more of the future together.”
External teams were tasked with ecommerce and front-end development, while Opsio created the management systems and service provisioning that brought everything together securely, scalably, and swiftly.
The Solution
Opsio were the obvious choice for ETNetwork to use as their cloud architecture partner because of our rich culture of DevOps and reputation for working with high-profile clients who depend on Opsio-built solutions for their line of business applications every day. Being one of the more established cloud solutions organizations, Opsio understands what is important to businesses, and has the ability to shape technology solutions around business problems – not the other way around. The Opsio team are able to work with client organizations at all levels to help project manage, resource, deploy and test solutions.
ETNetwork needed a partner to work with who could deliver a full solution. The Kwick infrastructure was hosted from one datacentre, with inadequate redundancy and resilience when weighted against their growing customer base and with that the increasing demands on performance, availability and security. The nature of their business is such that they need to store millions of small files (digital receipts), and this was causing issues for their storage tier.
The business also demanded the highest standards for security, great performance, high availability, scalability and speed to market, and these goals were becoming difficult to achieve with in-house resources. It was also clear that a partner was needed to help with this digital transformation and to keep that continuous transformation. ETNetwork reached out to Opsio because of their status as an AWS Advanced Consulting Partner and their unique managed services offering. The goal had to be to secure and grow the ETNetwork business, and to use the AWS global network and their nordic presence with datacenters in Sweden to achieve this.
The Path to Transformation
Opsio have the benefit of experience working with lots of businesses of varying sizes, and are able to share our experience with each new client we work with. The result of such experience is the Opsio Transition process and the Migration Framework model: ? Requirements Gathering ? Cloud Strategy ? Proof of Concept ? Design and Build ? Data Migration ? App Migration ? Cloud transition ? Run and Optimize
Infrastructure as Code
As per Opsio’s strong culture of DevOps, everything we created for ETNetwork was deployed using CloudFormation. We achieved a flexible and complete infrastructure by developing separate CloudFormation templates for the core infrastructure, another for the continuous integration/continuous deployment pipeline for Code Commit and Code Build (covered next), and lastly a template for the tasks running in AWS’s managed Container service, ECS. As with all Infrastructure as Code deployments, the infrastructure can be versioned, and controlled with all the tools available to application developers. This results in a consistent and reliable deployment every time. Containerized Workloads:
Opsio designed a solution whereby the Kwick application would run within Docker containers, and for those containers to be managed and maintained by AWS managed Docker service, Elastic Container Service, or ECS.
With this in place, developers can trigger new code to be built, tested, deployed and migrated through the various testing and staging environments, just by committing into a code repository as then normally would.
SSL Mutual Authentication
Another key deliverable for the Kwick platform was the use of SSL Mutual Authentication in specific cases, which is not supported by AWS Application Load Balancers (ALB) by default - a technical solution to this had to be found. This requirement was satisfied by deploying a separate CloudFormation stack to ECS Fargate; the AWS service which hosts docker containers without the need to provision and manage the underlying virtual servers hosting the containers. A Network Load Balancer (NLB) was placed in front of the Fargate resources, and SSL connections were routed in this way, all the way to a container which would handle the SSL traffic itself. Migration
Opsio have the benefit of experience working with lots of businesses of varying sizes, and are able to share our experience with each new client we work with. The result of such experience is the Opsio Transition process and the Migration Framework model: ? Requirements Gathering ? Cloud Strategy ? Proof of Concept ? Design and Build ? Data Migration ? App Migration ? Cloud transition ? Run and Optimize
Static
Assets
The key data assets to migrate were the Postgres database and the many millions of files which were currently hosted on a Network File System (NFS) solution. Opsio developed scripts which utilised the AWS Command Line Interface (CLI), and specifically the sync functionality there-of. This allowed for a large-scale copy of all files up to AWS’s Simple Storage Service (S3), with subsequent lightweight copy operations only referencing additional files which should be copied. Such low-impact file synchronisation operations were key in a smooth final migration from the old platform to the new.
Postgres
Database
After discussions between ETNetwork and Opsio, it was decided that in order to maintain the highest levels of security, the new Aurora database instance inside the new AWS cloud platform should be hosted on a private subnet, and therefore not accessible from outside of the Virtual Private Cloud (VPC). This meant that a continuous SQL logfile synchronisation from the old database to the new Relational Database Service (RDS) hosted database would not be possible. This resulted in a more traditional export and import task, with additional scripts to ensure every last bit of data was captured and copied to the new environment right at the time of migration.
Other
Improvements
During the course of work to deploy the new cloud infrastructure and migration tasks, it was discovered that so-called secrets, meaning things like passwords and other data which attackers might use to compromise a part of the platform. Opsio worked with ETNetwork and their developers to migrate the secrets into the AWS parameter store, a part of the Systems Manager suite, and have these accessed as part of the CodeBuild step in the CI/CD pipeline. Outcomes
Managed Support
Now that ETNetwork’s Kwick solution has been migrated into the AWS cloud, they are able to take advantage of Opsio’s talented and experienced resources as part of the managed service. ETNetwork have now access to AWS certified technicians to monitor and manage their platform 24/7, with fast response times whenever intervention is required, and as per the agreed Service Level Agreement (SLA). ETNetwork also have access to CloudOps and DevOps engineers who ensure continued use of automation and continuous optimization of the platform. And all of this is managed by a dedicated Service Delivery Manager, which works closely with the client to ensure their business flourishes as a result of a well delivered managed cloud service.
Scalability and Reliability
As mentioned, a key metric for success was a performant and reliable platform which would handle the growth that ETNetwork and their Kwick product continues to experience. Through a blend of technical solutions and a well managed cloud deployment, these benefits and business objectives have been achieved, resulting in every piece of the Kwick platform being elastic, infinitely scalable and secured according to best practices. There are no more limits, and that results in a happy client, and a great partnership.
The Path to Transformation
Managed Support
Cost Effectiveness
Only the combination of cloud-based services and modern DevSecOps principles could bring a system with global scope and complexity within the grasp of a startup. This made it possible to deliver viable development within budget and on time. It also allowed the teams to create something that’s manageable day-to-day and supports future innovation.
The project called for collaborative development among different teams and team members: enterprise and software architects, QA experts, analysis and security specialists, suppliers, and partners. “The challenges in collaborative development between global teams are significant, but so is reinventing a global supply chain as a community,” said Erik Hedlund of Era of We. “The scope of one matched the scope of the other, so we were comfortable that Opsio had the breadth to manage both.
The full implementation relies on many other AWS services. All of these services work together to automate provisioning and security system components, providing continuous data for service analytics and business model planning, assurance of business continuity at scale, and the adherence to best practices for effective management. “The decision to build the company around such a complex and multifaceted project was bold, but in Opsio and AWS we found partners that not only shared our vision but committed fully to working with us to make it happen,” says Erik Hedlund. “We are very happy with our experience so far, and excited about inventing more of the future together.”
External teams were tasked with ecommerce and front-end development, while Opsio created the management systems and service provisioning that brought everything together securely, scalably, and swiftly.
Why work with Opsio
ETNetwork infrastructure was hosted from one datacenter, with inadequate redundancy and resilience when weighted against their growing customer base and with that the increasing demands on performance, availability and security. The nature of their business is such that they need to store millions of small files, and this was causing issues for their storage tier
