How Can You Use ChatGPT For Software Testing?

All eyes have been on OpenAI and its brainchild ChatGPT in recent months. ChatGPT’s ability to understand and respond to complex instructions and deliver detailed responses to user prompts has led to an explosive rise in its popularity with the public. If you were to search online for “ChatGPT tips,” you can find content to help you use the tool to tailor resumes to job postings, create menu plans on a budget, and even create and play a text-based adventure game. For business professionals, there are videos with hundreds of thousands of views that show you how to use ChatGPT for marketing purposes, writing and optimizing website content, and more.

When used effectively, ChatGPT can help people to overcome creative roadblocks, learn new ways of performing tasks, better understand information, and become more productive. This extends into the realm of software development and testing. Software development teams often run into test automation debt, where a lack of test case automation can cause the team’s momentum to lag. When equipped with ChatGPT’s powerful tools, QA (Quality Assurance) engineers and software testers are not only able to overcome test automation debt, but they are also dramatically more productive and able to deliver consistently high-quality products to market faster.

“ChatGPT fits very nicely into a test-driven development regime,” says Paul Gerrard, founder and host of the Technology Leadership Forum and renowned thought leader in the software testing space. “You can put a complete test process together. In fact, you can put a complete software development process together.”

In this article, we will explore some ways that software testers can use ChatGPT-4 and share tips to maximize your use of ChatGPT for testing applications.

Test case creation

ChatGPT is surprisingly valuable for software testing and development. It can generate test cases in different programming languages, create detailed test plans, and supply descriptions for application functionality. It can also devise scenarios and corresponding test cases, expanding test coverage in ways the testers may not have considered. “It will generate more test cases than you can probably think of for yourself,” Gerrard says.

As a result of these capabilities, Jonathan Wright, Chief Technology Evangelist at Keysight Technologies, sees a shift occurring with the increasing availability of AI (Artificial Intelligence) tools like ChatGPT that is moving from traditional manual testing and following scripts to more exploratory testing that sees testers following missions. It can “help the average testers to have superpowers to be able to write automation when they’re not fluent in automation,” he explains.

ChatGPT also stores conversations. This means that testers have a log of every test case it creates, which proves invaluable for regression testing. “You can craft the specification of testing by saying, ‘Create tests that cover the code changes we’ve just made’,” Gerrard explains. This takes the sting out of test-driven development for developers, who often see the process as time-intensive and tedious.

Test data generation

There are several ways software developers can leverage ChatGPT for generating test data. It can help with scenario generation, automating test scripts, and offering suggestions and advice for ensuring test data adheres to standards. ChatGPT supplies invaluable tools and guidance for QA engineers and developers.

ChatGPT can generate synthetic datasets that mirror real-world data to ensure that testers are able to test against a wide range of scenarios without exposing sensitive or proprietary information. This is invaluable for testing applications that will handle sensitive information where privacy concerns are paramount, such as electronic medical records for healthcare institutions. It can generate randomized data that adheres to the testers specified formats and constraints, which is particularly useful for stress-testing and performance analysis.

When testers encounter bugs or challenges, ChatGPT can supply troubleshooting tips and help them to find and resolve issues. Organizations will have unique requirements and specifications, documentation, and many historical bugs alongside their test cases. Expanding out from there, organizations have massive databases of information. ChatGPT can help them to condense their metadata down to more consumable formats which you can then train against specific requirements to identify gaps, flaws, or ambiguity.

“This is where I see the natural evolution going,” says Jonathan Wright. “I can see people deploying these kinds of models within their organization, and I can see it learning their organization. In this way, the uses of ChatGPT from HR to infrastructure…it will understand and be capable of a lot more.”

Code generation

If you are looking to write small, straightforward software programs, developers can directly set prompts in ChatGPT to get a desired result. Even more impressive, software developers can upload scans of rudimentary sketches of applications from white boards or even the back of a napkin to generate code. On the social media app Reddit, one user described ChatGPT’s capabilities in this arena as “magic, simply put. No other words to describe it.”

Jonathan Wright used ChatGPT 3.5 to create a Gherkin script to find stock prices, then turned the script into a model. From that model, he asked the program to generate the code to write the application which would find and visualize stock prices over the last year. “Not only did it build the actual app itself,” he says, “but then it also created all the Selenium scripts for each one of the pages, ran those scripts, and validated them.”

Code is often highly domain specific, so testing it historically required you to be not only an expert in the domain, but also the testing discipline to ensure coverage.

ChatGPT can help in creating examples of automated test scripts necessary for deployment pipelines and aid in maintaining a high standard of code quality for any number of programming languages and frameworks. It is also invaluable in helping to explain programming concepts, design patterns, and architectures which can help developers consider how to test and even identify edge cases for domain-specific applications they are not widely familiar with. Further, it can supply feedback on code structure and style and name areas for improvement.

ChatGPT tips for software testers

If you’ve read this far and are thinking, “well, there goes my job,” think again.

“It’s actual human augmentation,” says Wright. “If anything, it’s augmented testing through the power of being able to provide testers with superpowers.” The thought process behind designing test strategies is extremely complex and requires substantial contextualization to drive the testing, data, and code it generates. But how do you leverage ChatGPT effectively to augment the capabilities of your development team?

With the right approach and outlook, “I can’t think of anything superficially that it couldn’t do that would support a human tester,” says Paul Gerrard.

Here are some best practices and tips to help you make the best use of ChatGPT for your next development project.

Define your problem clearly and concisely: Before you even log into OpenAI, ensure you fully understand the problem you’re trying to solve. Be sure you can articulate it as concisely as you can by breaking the problem down into more palatable components and defining the outcomes or functionality you are looking for.

Be specific: ChatGPT will furnish general information for general prompts. “What we’re doing when we’re using tools like this is specifying what we need the tool to do for us,” Gerrard says. Define preconditions, rules, and desired features or outcomes like you would when you are writing code. This will increase the likelihood of extracting precise and actionable recommendations from ChatGPT.

Contextualize prompts: By giving the program contextual information, you can define the coverage you are looking for and ask ChatGPT to generate comprehensive testing coverage for that branch of the application. Paul Gerrard sums this up nicely: “That is almost certainly better than the level of testing that any developer today would try and achieve.”

Expect to iterate and refine: ChatGPT is “like a box of tools that we haven’t got the manual for yet,” Gerrard says. You aren’t going to be able to just log in, write a prompt, and have ChatGPT spit out the ideal solution the first time around. Experiment with different approaches to prompts, refine your questions, and iterate on ChatGPT’s answers each time to receive further guidance.

Have a conversation with ChatGPT: ChatGPT is more of a programming partner than just a simple and useful tool at your disposal. “We need to understand how to have a conversation” with the tool, Gerrard says. By viewing ChatGPT as a collaborator, you think more critically and unlock more creative problem solving as well as encourage the AI to ‘think’ more deeply about the problem you are solving together.

Stay curious: Read posts on forums, attend webinars, and seek out information from others who are experimenting with ChatGPT’s capabilities. In turn, share your explorations with others. Combined and shared knowledge can help to expand your abilities and hone a broader and more well-rounded skillset.

Navigate the AI frontier

ChatGPT is democratizing access to powerful productivity-boosting tools across sectors. It represents transformative potential for how developers approach testing and how organizations streamline operations. You can learn more about ChatGPT from Paul Gerrard and Jonathan Wright by watching the Chat GPT-4 Testers webinar on-demand.