Experimenting with ChatGPT for DevOps

I had been working in the DevOps and Site Reliability space for roughly 5 years by the time that ChatGPT made it’s ground shattering debut to the world, and it truly revolutionized how we all think about work and technology. At Endowus, we strive to see how we can leverage modern technology to its fullest, and in this case, how ChatGPT can help DevOps engineers optimize parts of their workflow.

We had all heard the rumours of ChatGPT being able to write a complete application from scratch. With a healthy level of skepticism, I thought “it can’t really be that good”. So I decided to test it out and see for myself.

Initially, I started asking it to produce small scripts in Python and Bash, just to dip my toes in the OpenAI/ChatGPT waters.

The results were impressive to say the least! 🤯 Not only did it produce code with perfect syntax, but it also explained the code to me and listed assumptions!

At the time, I had been working on deploying Airflow to an EKS Cluster using Helm via Terraform. This setup took roughly 3–5 days to get all squared away and tested. Curious how ChatGPT would perform, I asked it to write a configuration for this. While it did produce the configuration, it wasn’t exactly what I was looking for or hoping for, in fact, it wasn’t really even that close.

Next, I turned to its OpenAI sibling, the Platform Playground, and asked it the same question:

The OpenAI Platform Playground tool was such a game changer! It produced much better code as it appears to have internet access where ChatGPT does not, and therefore can provide more well-rounded configuration.

The configuration seen here that was output by the Playground tool was similar to the one I had configured a few days prior manually, and it took this AI just a few seconds. This can help reduce manual overhead of typing out the various fields, effectively, what it produced was a template which I could then modify and use to deploy at a quicker rate.

I tried asking ChatGPT about specific infrastructure design and configuration options for setting up a theoretical metrics system, a notoriously painful area to set up and maintain. ChatGPT was able to give an answer in a matter of seconds that perfectly answered my questions and advised about best practice where hours of reading Thanos docs and different resources online couldn’t give a definite way forward.

A few weeks later, I was troubleshooting and looking for a solution to an error that kept coming up in some logs for the task that I was working on. It occurred to me that maybe I could optimize this troubleshooting process by seeing if ChatGPT had a solid resolution:

While ChatGPT didn’t provide much new information about how to troubleshoot or resolve the error from what I had already tried or found online, it can always help remind you to check something that you may have missed or forgotten. I’m sure with different errors it may give more detailed responses and may even provide the specific solution to a given error.

This opens the door for so many, especially more junior engineers, to hone their troubleshooting and triaging skills through ChatGPT. It may also change the entire flow for how we investigate various issues not only in CI/CD and Infrastructure, but even for debugging code.

These two tools combined, ChatGPT and OpenAI Playground, have truly allowed me to improve and optimize my workflow. They have unblocked me at various points over the last two months, either when I simply couldn’t find what I needed on Google or when I just needed a quick question answered.

Getting the most out of ChatGPT for your workflow

• Don’t tell ChatGPT your secrets: it’s important to be careful when you ask ChatGPT questions. Clean and remove any proprietary information, personal information (PII) or credentials/application secrets. For example, if your prompt contains any of this information, you can replace this sensitive information with other terms. Then if you use what ChatGPT generated, replace it with the correct details locally.
• Specify Versions: if you want ChatGPT to generate configuration based on a Helm Chart (for example) that may have multiple versions, I’ve found that it doesn’t always take the latest by default. It can be helpful to include the version of the configuration you want in the request to get more accurate results.
• Trust but verify: as the quote goes “trust but verify”. You can trust that the output that ChatGPT is giving you is good and is error free, but as with any code obtained online, you should always read through it yourself and ensure you understand it before adding it to your project. Never assume that code generated by ChatGPT, no matter how amazing or excellent, is production ready out of the box.
• Try the Playground: If ChatGPT isn’t giving the responses you want, you can also try using OpenAI’s Playground. It’s important to add that there is a free trial for this product line and once the trial ends, it becomes paid.
• Give Examples: it can be helpful to provide more information when asking ChatGPT something, this can also be done in the form of giving it an example. Instead of just asking regex to find an ip address in a string, you can instead ask regex to find an ip address in a string, for example, in a string 1.1.1.1:0000 the regex should match 1.1.1.1 and it will perhaps generate a more exact response for your query.

ChatGPT offers plenty of ways and opportunities for engineers to increase their productivity and help unblock themselves in ways that never existed before and can help optimize their workflow in certain aspects.

However, at the current version of ChatGPT, it is unlikely that DevOps Engineers or SRE’s are at risk for being replaced. While it’s an impressive tool, it’s important to know how best to use and leverage it for the outcome you wish to achieve.