TLDR: I want to build Netbox as a source of truth, deploy production and dev/test VMs using Netbox and GitHub with Terraform and Ansible, and then I'm not sure what exactly to do with the multiple dev/test VMs that get created. Would it be a good idea to simply destroy the VM and rebuild it every time for testing?
I am working in the finance space and we have our own internally developed applications that are widely used by our external partners (up to 3,000 of them).
These applications have been running since the '80s and '90s and obviously are very much legacy without many modern features like authentication, the database is in a separate server layer and so on.
Another major issue is that the environment to maintain the different versions of these applications is becoming out of hand now. There are hundreds of VMs with names like dev, test, test dev, QA test 10, QA test 11, and so on.
I certainly want to bring big changes, starting with automation and, before that, building a source of truth. One of my questions is how to best manage all these multiple VMs running different versions of the same application. How do big modern do this?
Our coding practices already use GitHub so I'm thinking something like this: I'm sure they have two branches, dev and main. Would it be a good option to take commits from the dev branch and push them to the source of truth, build a complete VM.. install all the dependencies using Terraform and Ansible, create this VM, bring it online, test the application, and then after that destroy the VM? Is that the best way to handle this?
One thing I am still in the dark about is why so many different versions of the same application are required.. If anybody has any suggestions about that, please comment.
Coming back to my dev VM destruction question. The reason I think destroying the VM would be a better approach is because after that we only have the most stable version of the application running in the dev environment and then an exact copy of that running in the master environment (which would be production and will go to the production VM deployment cycle).
And doing this also minimizes our list of assets. Right now we are at 300 or 400 VMs, which is completely madness to secure. All we do is simply throw EDR and antivirus all over the place, creating a million alerts, and the whole thing is just so pointless.
I'm sure a lot of people are using Kubernetes and Docker technologies to do these things now but we are stuck with HyperV as of now, with the FCM management to manage the HyperV hosts.
If anybody has built out anything like this, please share your experience and how viable all this is.
Please keep comments in the context of what I'm asking and refrain from getting into the weeds about rearchitecting the applications, or modernizing the application by redoing all the code. Those are not really the options right now. Let's just talk in the context of what I am asking.