ChatGPT in Software Development – The good, the bad and the tricky
Everyone’s talking about ChatGPT, but is it really all it’s cracked up to be? We look at the benefits of ChatGPT in software development, how developers can make use of this tool, and what our own team thinks of OpenAI’s new chatbot making waves around the world.
Let’s get into it…
What is ChatGPT?
ChatGPT is a state-of-the-art language model developed by OpenAI, designed to understand and generate human-like text. This technology is built using advanced deep-learning algorithms and has been trained on a massive dataset of diverse texts. ChatGPT can be utilised by software development companies, including developers, industry professionals, junior and senior developers, as well as business analysts, to assist with various tasks such as text generation, text classification, question answering, compiling email marketing copy, emails, client responses, and more. More recently, GPT-4 was released, a large multimodal model that accepts image and text inputs, achieving even more intelligent, human-like performance on various professional and academic benchmarks.
Here’s a screenshot of what ChatGPT thinks of itself…
How can ChatGPT help developers?
With the help of this extremely powerful and intuitive tool, software developers can benefit in multiple ways:
- It can provide code snippets and examples for programming languages and frameworks.
- It offers explanations and solutions for common coding problems and errors.
- It can answer questions about programming concepts, best practices, and industry trends.
- It provides a source of inspiration for new ideas and approaches to coding, which can improve the accuracy of the software.
- It can create unit tests based on natural language input (test component X with the inputs a, b, and c and expect the output to be Y).
- It can be used to find edge cases in written code, resulting in better-quality software.
- It can explain new concepts with examples (just think of it as your own personal virtual mentor). Developers can learn more about programming languages, software development techniques, and other related topics. By asking ChatGPT questions and engaging in conversations, developers can access a vast repository of knowledge and improve their skills and expertise.
By incorporating ChatGPT into their day-to-day work, developers can work more efficiently, learn new skills and ways of working, get assistance with language-related tasks, and provide more value to their team and organisation.
The good, the bad and the tricky
What’s ChatGPT good for?
- Speed: ChatGPT can quickly provide answers to coding-related questions, helping developers save time and increase their productivity.
- Accuracy: With its advanced language processing capabilities, ChatGPT can accurately understand and respond to complex coding queries.
- Availability: ChatGPT is available 24/7, allowing developers to access information and support at any time, regardless of their location.
- Scalability: ChatGPT can handle multiple queries simultaneously, making it a scalable solution for developers who need to resolve many coding issues quickly.
Here’s the tricky bit…
Through various testing and trials, we’ve found that while ChatGPT by OpenAI is an extremely valuable and powerful resource, there are some things to remember:
- Contextual understanding: ChatGPT is based on the Transformer architecture and is trained on large amounts of text data, which makes it capable of understanding context. However, it is important to remember that it is not perfect and may not always provide the correct answer in the context of a conversation.
- Limitations of training data: The quality and nature of the training data can affect the output generated by ChatGPT. If the model has not seen a certain type of question or information during its training, it may not provide accurate answers for that.
- Open-endedness: ChatGPT is designed to generate text and can answer a wide range of questions. However, it may generate irrelevant or nonsensical answers if it is not constrained in any way.
- Bias in training data: AI models like ChatGPT can reflect the biases present in the training data. It’s important to be aware of this and to mitigate it by using diverse and inclusive training data.
- Overgeneralisation: As with any machine learning model, ChatGPT can overgeneralise and make mistakes when faced with examples it has not seen during training.
- Safety and ethics: ChatGPT can generate potentially harmful content, such as hate speech or misinformation, so it’s important to use it responsibly and consider its ethical implications. It can also produce correct-sounding but incorrect answers. That’s because the model itself has no true inherent understanding of code, but rather knows what correct code ‘looks like.’ Keep in mind that this is a model trained on data, not an actual software engineer.
- Being precise with your prompts: Rephrasing can result in quite different results. It is important to be extremely precise in the input prompt for the highest probability of the correct answer.
- The data it’s been trained on is limited to 2021 data: This means the model will be lagging behind the most up-to-date industry standards and can produce depreciated code.
- It should not be used to generate code that requires a lot of context: It’s impossible to provide the model with the context of an entire codebase or product. If you don’t already have a strong idea of how to approach the task, then ChatGPT will not be especially useful.
What does the Haefele team think?
We asked some of our team members trialing ChatGPT what they thought of the AI tool, how they’ve been using it in their day-to-day operations, and what future opportunities they see.
Bodhi Middleton – Business Analyst
“While I haven’t integrated ChatGPT into my day-to-day operations yet, I have found it to be particularly useful to create “boilerplate” templates and documents. These foundations would have taken me at least an hour in the past, which is fantastic!
For example, I’ve used AI to generate on/offboarding, project-plan, and process foundation templates, which I then flesh out and tailor to the specific scenario. I’ve played around with quite a few AI tools now, which are all interesting to varying degrees, and there are loads of ways they can be integrated into our jobs.
If you start with the problem you’re facing, there’s likely a tool out there to help resolve it to some degree.”
Jonathan Balmer – Delivery Director
“I’ve been using ChatGPT for a couple of weeks now, focussing on the client side and from a communication perspective. I’ve used it for email generation for sales, and I can actually see a change in tone around my emails, which is pretty cool. Also, about 90% of the message sent out was generated by ChatGPT. It has certainly provided a great foundation and I’m keen to see what the next phase will look like.”
Wayne Wilson – Functional QA
“I can definitely see the value in ChatGPT. Since the launch of its new GPT-4 version, the tool is more powerful and predictable than before, which is incredibly useful. It’s quite impressive seeing some of the things it can do, for example, writing performance tests. It seems super powerful as is, so I’m looking forward to seeing what else can be done in the future.”
At Haefele Software, we’re regularly testing and trialing innovative concepts and resources to improve the way we do things. Visit haefelesoftware.com or contact us to find out how we can help you.