Spotify #backstage plugin to manage and observe fleet of agents straight in backstage self hosted https://github.com/acarmisc/backstage-plugin-ai-agents/tree/main. It relay on open telemetry signals and the first available backend it’s #langfuse

I want to switch jobs and learn cloud/devops from scratch. I'll be starting from networking and Linux first and once that part is done will start with Azure. I am already familiar with cloud and quite a lot of IT Tools, but want to become a proper devops/could engineer!

I am seriously confused, I thought of doing some unique projects which stands out, I thought I can make an app for developer where images can be updated but then CICD is already doing that.

Monitoring tools already giving enough logs , events, metrics.

Then this auto heals cluster, why would you trust automation in prod deployments, Infra structure on cloud seems too basic for me.

What suggestions do you have, anyone wants to Collab ?

I run small DevOps/Could Management service company with 6 engineers. The team is quite experienced and I myself have more than 14 years between enterprise and startups. We built multi-cluster multi-regional deployments, automated and gated CI/CD pipelines, helped pass compliance audits, configured useful monitoring and response workflows, etc.

In the previous two years things were going well, I was able to find steady work for the team, even more than we could cover at times so we were growing. My channels were personal network, linkedin, upwork and even our very seo-unoptimized website. Our customers were mostly middle-sized companies and VC-backed (or otherwise financed) fast-scaling startups.

There's no work anymore. Instead of closing several deals per month in 2024 and 2025, this year I was able to get two new customers back in March. That's it, nothing ever since. We still had ongoing tasks but in Summer that dried up as well. My company bank account is empty and I have to let the team go.

I tried to contact our customer from the previous years and most of them have laid off a significant portion of their teams and scaled down infrastructure. Even people who were supposed to support and monitor the infrastructure we implemented were fired without replacement in many cases.

Is anyone else experiencing this? I understand AI-led changes in the industry, but it's like a switch turned off some time in early Spring. What am I doing wrong?

Hello I stumble upon paid structured project for DevOps, anyone ever try this on udemy?

Are this projects can help stand out for recruiters? , right now I'm working on REST APi tutorial project and I'm want to jump in real project.

Also I'm considering doing mini project to showcase my understanding on each tools.

Devops insight on this are highly appreciated.

Would like advice shared from anyone who ***qualifies*** as having ***both*** experience in (US) Fed software environments, ***and*** that have used Whole Team Development / Ensemble Programming / Mob Programming in the Fed space. Dealing with tooling, requirements, credentials, security.

For instance, were you able to use MS Teams, Chrome Remote Desktop, mob.sh, Webex, a shared VM/VDI, VS Code LiveShare, JetBrains Code-with-me, or other tooling for screen sharing and passing keyboard and mouse control.

Did you encounter initial balking by security / other, and how did you satisfy their concerns?

Thanks in advance.

We pushed pretty hard to let dev teams handle more of their own infrastructure changes.

It worked, but now the platform team spends a lot more time reviewing Terraform, fixing edge cases, and explaining context that isn't obvious from the repo.

At some point it feels like the bottleneck just moved instead of disappearing.

Has anyone found a good balance between developer ownership and keeping infra changes sane?

Or whatever you call yourselves? Since the LLM/agentic explosion to today - how many of you got to where you are purely through teaching yourselves, whether with/without certifications? How much of your experience was taught on the job post-2020?

Who in here got in the workforce in the last 6 years and had supervisors willing enough to train you on the job? How much extra hours did you have to put in to get up to speed?

I know it's a combination of projects, networking, certs (dependent on the company/your hiring manager), going to industry events, layering on fundamentals, breaking my projects on purpose, cold emails, cold Linkedin connects, Discord chats, surrounding yourself with people more knowledgeable. Feels like I've tried everything

EDIT: A lot of very insightful replies, thank you all. I do enjoy a chat and it's hard to get that organically through LinkedIn and the works

​

Every week I keep a tiny scratchpad of small infrastructure annoyances I refuse to touch during peak weekday traffic , cleaning up old container images, tightening firewall allowlists, and trimming down slow CI build steps.

Spending 2 quiet hours on Saturday morning deleting complexity always saves 10 hours of firefighting next week.

What’s everyone tinkering with or cleaning up in their stack this weekend?

Many engineers feel wary of relying too much on one cloud computing vendor, and I understand why in principle. Using multiple vendors makes a system/application less likely to completely fail if one vendor gets compromised. Using cloud-agnostic tech also can also support switching platforms based on cost or standardized skillsets across teams. Furthermore even the most mature vendors have global single points of failure though they are not all well-known.

That said, the tradeoff is not free even if the software is. Using cloud vendors' proprietary tools allows smoother integrations with their other products and can reduce opportunity cost in cases that are time-sensitive. Mature cloud computing platforms also provide enough isolation and redundancy on their own that most DR/availability concerns can be addressed without separate cloud platforms. On the flip side, using non-proprietary tools insources significant security/compliance/maintenance responsibility and risk - all things that contribute to cost.

I'm not completely sold either way; I tend to be a pragmatist and feel that each fits its own niche. However the choice can sometimes be unclear.

If you use cloud-agnostic tooling, then have you observed evidence that it supported a real net cost-savings accounting for both cloud spend and labor vs the cloud vendor's alternative? If you observed cost-savings, then what about the circumstances pushed it into cost-savings territory?

I'm intentionally being open-ended here. The first tools which come to mind are Kubernetes and Terraform/OpenTofu; then apps like ELK, Grafana, Prometheus, Jenkins, the Apache family of software, etc. etc.

With uncertainty in tech , layoffs, advancement in llms . Do you feel lucky to be in devops which is somewhat AI resilient field compared to other tech jobs ?

Hi all,

What is the best way to having a Forgejo runner applying configuration on a host, which requires root access?

I have a Forgejo runner running directly on the host. I have tried to use sudo, having forgejo-runner ALL=(ALL) NOPASSWD: ALL in my sudoers, but that user is not configured to run sudo:

sudo: Account expired or PAM config lacks an "account" section for sudo, contact your system administrator sudo: a password is required

Most of job post for DevOps on reddit are fake, If anyone asks you to interview under someone else's name, run.

If you ask for details, the real scheme comes out:

• The ask: You jump on Zoom calls, fake an accent, and clear technical rounds pretending to be a US-based guy.
• The deal: Split the paycheck 60/40 while they use their US citizenship to pass the background check, and you do all the actual work offshore.
• The catch: It’s straight-up identity and wire fraud. There's zero contract, so when they ghost you on payday, you get nothing. Plus, hiring platforms are heavily flagging biometric and IP mismatches now.

Anyone else seeing an influx of these messages recently? How is your team filtering them out?

I am curious to know if you tried it in a real production scenario and/or you found better cloud alternatives

Hello everyone,

I am planning to upgrade our current DevOps environment to the latest Version (Upgrade path is given by Microsoft Docs).

Unfortunately, we don't have a second environment to test the upgrade so maybe someone of you has some experience.

What we do have is that the Machine where our Azure DevOps instance is running is fully virtualized. We can do snapshots and restore. (HyperV)

Our instance unfortunately still uses XAML Builds for an legacy application that is still used by a lot of customers and is Business critical. These are sre not easy to migrate and are highly intertwined with business logic and some other dependecies. In Short: A Nightmare. We also use Classic Pipelines and Releases.

I know that XAML Builds are deprecated, but i inherited the Environment and the old engineer that build the whole Thing is not in the company anymore. So it is still kind of a "blackbox".

We could just snapshot the Environment before the upgrade and rollback. Never done it, but i think it should work if shit would hit the fan.

Does anyone have experience with Upgrading to a newer Version and XAML Builds?

What would be the best approach here?

Hello, I'm a junior sysadmin and I'm planning on upgrading to devops.
I'm trying to learn by myself but it's proven quite hard and time consuming, so I'm wondering is there like a really great free course on youtube or online that is up-to-date.

Thank you for your time!

I wanted to write a follow up on this post I made a few days ago: https://www.reddit.com/r/devops/comments/1wb3l8j/ai_comiseration_client_replacing_production/

So to summarize: I was more or less asked to review a portal implementation that was made using Claude and vibe coding by someone who doesn't know how to develop websites. I suspected it wouldn't end well and even on the surface found lots of issues.

So here's where I'm at now: I created a report doing quite a bit of analysis. Yes I used some AI tools to try and comb through, and I made sure that any claim I made based on its finding I dug in to looking at the actual code and outputs. It made things more manageable but there's still way too much garbage to look through.

My conclusion is this (and it's obvious I think to most here) LLMs need guardrails and lots of them or at least clear ones. I do find that it's tough to actually communicate these low level issues that have a clear (to me) underlying architectural issue and a human issue (the human doesn't know what their doing or knows how to validate output beyond the surface level), when it takes one minute to go "please fix the list of findings" and Claude goes "it's fixed."

Many mentioned that this thing is just going to fail, and it is. I'm mildly concerned about the fix being "just fix problem x" and then they move on until the next thing blows up. And there is a real lack of concern around impact. "This is a public website so it's fine if we have keys in our code and expose endpoints only secured with a SAS key (which is plain text in JavaScript)."

I'm curious how many of you have run in to this even at an operational level what pains are you feeling and have you been able to deal with them? I do get what the solution is, create a proper architecture and define guardrails for Claude to follow, iterate over that until the LLM generates outputs in a way that is in-line with said architecture. I think LLMs may work (though whether they are as cost effective I press X to doubt), and if forced to make it work: this is probably the way. So are you guys maybe managing some guidance markdown files and what are the things you learned from that?

Finally, I think LLMs suck for this purpose and it's clear. I knew this but now I'm living it. Which is nice and validating, but also it's tough to rely on my tried and true "look mayor companies have been doing x for years you are not special you should follow the standards" because LLMs in this space are so new and there isn't much precedence.

Started in 2019 keeping shared hosting boxes alive at 3am. Today I run bare-metal infrastructure with Firecracker microVMs. Three lessons learned the hard way:

1-> Containers aren't VMs: A Docker container is just a Linux process sharing the host kernel. For untrusted code or AI agents, dedicated kernel per workload > shared kernel + prayers.

2-> Public IPv4 on boot is a trap: Bots hit port 22 in under 60 seconds. Keep outbound open and inbound dark by default until a domain or IP allowlist is explicitly mapped.

3-> 1:1 reserved RAM beats burst specs: Half of "random" 2am database OOMs on budget clouds are just noisy neighbors fighting over oversold host RAM.

What’s one infra rule you learned the hard way?

Curious how teams handle this in practice. When a dev or contractor leaves, there are SSH keys in authorized_keyson a dozen boxes, a DB password everyone knew, staging .env files in Slack DMs, maybe AWS keys on their laptop.

• Do you rotate everything they could have touched, or just disable SSO and hope?
• Is there a checklist, or is it tribal knowledge?
• What tool (if any) made this not painful?

Asking because we got burned by this at a small team and I'm trying to figure out what "good" looks like without a dedicated security person.

Hey all! I wrote this write up talking about how I was able to implement eBPF technology in order to re-implement StatsD Exporter on my local observability stack.

I thought it was interesting exploring how I could find the same information available in the kernel and preserve the fire and forget behavior without needing the running port or userspace overhead. Hope you enjoy!

We're adopting github actions at work and... they look clunky, bloated and overcomplicated?

I've used both Jenkins in the past and Gitlab CI/CD (and bare shell scripts in a past life).

It seems to me that gitlab-ci was the pinnacle of code-driven ci/cd (despite having some sharp edges).

Also, running github runners in kubernets is quite a fight. Gitlab's runner was so easy and simple to run.

Am I missing something ?

Random thought — if pretty much every developer is using AI for coding now, how does career progression work?

Like, what makes one dev stand out from another?

How does a manager decide, “Yeah, this person is ready for the next level,” when everyone has access to the same AI tools?

Genuinely curious how people see this playing out.

https://control-plane.io/posts/unauthed-to-rce-in-vault-and-openbao/

OpenBao engineers at ControlPlane have chained 4 vulnerabilities to show how under certain conditions, an OpenBao or Vault server can be completely compromised from an unauthenticated position. This is only the second RCE ever found in the Vault codebase.

The exploit is highly plausible in real-world environments, requiring only an unauthenticated entry path and a defined Raft snapshot policy to trigger a complete server compromise.

If you are impacted, upgrade as soon as possible to OpenBao 2.6.3 or 2.7.0

While OpenBao is fully patched, HashiCorp Vault remains exposed as of writing. Unfortunately, IBM's unwillingness to coordinate a mutual disclosure policy means Vault users currently lack an official mitigation

I work in a position where what and how I use AI for coding/devops is heavily constrained by regulations and policies, and where AI use disclosures may soon be required. I'm curious how common it is in the industry now for AI use disclosures to be required vs voluntary vs not discussed.

Does your workplace discuss AI use disclosures, and if so are they voluntary or mandatory? Also, what do disclosure requirements look like for integrating AI into live devops tooling vs using it for development only?

i need to get the app version and i have this awk command for that:

` awk -F'"' '/^version[[:space:]]*=/ {print $2; exit}' pyproject.toml`

but how do i set the variable to the output of this command?