Creating a scalable platform with GitOps, Kubernetes-based Terraform, and ArgoCD for flexible service consumption. Challenges in modularization and referencing outputs pose issues. Seeking guidance on writing Terraform modules within a platform for easy consumption by developers and product teams through a UI.
I'll address this without delving into specific technologies. Stating a preference for conducting Platform Engineering using technology X is a step towards focusing on the solution rather than the underlying problem.
Platform Engineering fundamentally embodies the principles of DevOps at scale. While the "you build it, you run it" approach has been in place for a considerable time and can be effective, it comes with its drawbacks. In an enterprise with numerous efficient DevOps teams, inefficiencies may arise. There's a tendency for multiple teams to independently build similar products with slight variations. This leads to the use of different CI/CD platforms, the development of separate scripts accomplishing the same tasks, and so on.
While each team operates as a self-sufficient and highly capable unit, the enterprise as a whole experiences redundancy. Despite the autonomy of individual teams, there is a collective investment of hours in duplicative efforts.
n comes Platform Engineering, a mechanism of sharing practices and/or tooling where at scale teams re-use knowledge or tools to accomplish tasks, but there is a caveat here.
You cannot build a platform that does everthing
If your team isn't significantly large, managing a platform for multiple teams becomes challenging without standardizing their practices. The methods for achieving standardization warrant a separate and detailed discussion.
How can platform engineering be implemented?
- Identify a common problem that multiple teams face but are unwilling to solve or maintain.
- Gain a comprehensive understanding of each team's specific implementations and needs.
- Propose and advocate for a standardized solution that alleviates their workload and cognitive burden.
- Design a system that implements the standardized solution to cater to the needs of multiple teams.
- Select a technology that aligns with the designed solution.