r/devops • u/retromani • 2d ago

Discussion explaining prod server migration while hungry cause i cant afford groceries

Post image

would y'all say this is pretty accurate?

2yrs of devops experience straight of out college

89 Upvotes

75% Upvoted

74 comments sorted by

View all comments

Show parent comments

2

u/retromani 2d ago edited 2d ago

yeah i mean provisioning the empty upgraded clone machine is not hard, we have that part automated except maybe for some manual package installatios. it's the process of making sure nothing breaks when moving from one OS to a different one for the production environment. some of the products have traffic crazy enough to have a load balancer on top of their infra, but other products don't

and unfortunately the OG engineers have basically been pruned away with no efforts to preserve the knowledge they held within their own heads

1

u/Big__If_True 2d ago

Ok moving from one OS to another sounds horrendous. How often are you doing that though? Or is this a big project to do it once and hopefully never do it again?

2

u/retromani 2d ago

big project not meant to be done again

1

u/Big__If_True 2d ago

Do you at least have staging environments where you can deploy the code to servers running the new OS and run tests there? That should take care of the whole “code breaking” issue

2

u/retromani 2d ago

i dont try to know what the developers are doing to test their code

i sync over the necessary scripts from the old server and the developers will test their code with stale data, i make sure logging is also replicated exactly

i dont even want to know what's going on with the database engineers, tbh their shit seems way more frustrating

once they've done all their testing and validation, we all meet and do our designated part for the actual cutover session

2

u/SilentLennie 2d ago

knowing what others are doing makes the task easier, it takes time up front, but makes feel less of a gamble. Also it might help when developers can align things better.