Technology leaders striving to expedite software development can anticipate significant time savings with Generative AI. However, harnessing the full potential of this disruptive technology requires more than just having the right tools at hand.

The most recent empirical research reveals that tools based on Generative AI can achieve remarkable improvements in speed across various common developer tasks. When it comes to documenting code functionality for maintainability, which considers how easily code can be enhanced, this task can be accomplished in half the usual time. Writing new code can be completed in nearly half the time, and optimizing existing code through code refactoring can be accomplished in nearly two-thirds less time (Figure 1). By combining the appropriate upskilling and enterprise support, these speed enhancements can be translated into increased productivity that surpasses previous advances in engineering efficiency, driven by both new tools and processes.

However, while the potential for a significant boost in productivity exists, the research reveals that the extent of time savings can vary greatly depending on task complexity and developer experience. On tasks that developers considered highly complex, such as those involving unfamiliar programming frameworks, time savings dwindled to less than 10 percent. A similar trend was observed among developers with less than a year of experience, where in some instances, junior developers took 7 to 10 percent longer to complete tasks with the aid of these tools compared to without them.

The use of these tools did not compromise code quality in favour of speed when developers collaborated effectively with the AI tools. In fact, code quality, with regards to aspects like bug reduction, maintainability, and readability (essential for reusability), was slightly improved when AI was involved. However, participant feedback indicated that developers actively fine-tuned the tools to achieve this level of quality, highlighting that the technology is most valuable as an augmentation to developers rather than a replacement. Ultimately, to maintain high-quality code, developers must possess a deep understanding of the factors that contribute to code quality and guide the tool to produce the desired outputs.

Collectively, these findings suggest that optimizing productivity gains and mitigating risks when implementing Generative AI-based tools will necessitate engineering leaders to adopt a structured approach that encompasses training and coaching in Generative AI, thoughtful use case selection, workforce upskilling, and risk management controls. In this article, we share insights into where Generative AI excelled in the research, which tasks required developer expertise, and what steps engineering leaders can take to ensure the most effective utilization of this emerging technology.

Where does Generative AI demonstrate its capabilities?

Through the research, Generative AI-based tools offer substantial productivity enhancements in 4 key areas:

• Streamlining manual and repetitive tasks: Generative AI can effectively handle routine assignments such as auto-populating standard code functions, automatically completing code statements as developers' type, and documenting code functionality in predefined formats based on developer prompts. By doing so, these tools can liberate developers to focus on tackling more intricate business challenges and expediting the development of new software capabilities.
• Facilitating the initial draft of new code: When developers are confronted with a blank canvas, the integration of Generative AI-based tools allows them to request suggestions by entering prompts in a separate window or within their integrated development environment (IDE). Developers who utilized this approach reported that the Generative AI tools provided valuable code suggestions, helping them overcome writer's block and kickstart their work more efficiently. As one participant noted, these tools enable developers to enter the "flow" state sooner.
• Speeding up modifications to existing code: Participants also observed that, with effective prompting, they could make updates to existing code more rapidly when utilizing these tools. For example, to reduce the time spent adapting code from an online coding library or enhancing pre-existing code, developers would copy and paste it into a prompt and submit iterative queries instructing the tool to adjust based on their specified criteria.
• Enhancing developers' capacity to address new challenges: The technology can assist developers in quickly familiarizing themselves with unfamiliar code bases, languages, or frameworks required for their projects. Additionally, when faced with new challenges, developers can turn to these tools for assistance akin to what they might seek from an experienced colleague, including explanations of new concepts, synthesizing information (e.g., comparing and contrasting code from various repositories), and providing step-by-step guides on using a framework. Consequently, developers using Generative AI-based tools for complex tasks were 25 to 30 percent more likely to complete them within the specified time frame compared to those without such tools (Figure 2).

The research demonstrates that empowering developers to maximize their productivity also leads to a notable improvement in the overall developer experience, ultimately aiding companies in retaining and energizing their top talent. Developers who utilized Generative AI-based tools were over twice as likely to report a heightened sense of happiness, fulfilment, and achieving a state of "flow" (Figure 3). They credited this improvement to the tools' capacity to automate mundane tasks that had previously diverted them from more rewarding endeavours and to their ability to access information swiftly, without the need to scour various online platforms for solutions.

Identifying tasks requiring developer expertise

While Generative AI technology possesses significant capabilities, the research underscores the critical role of human engineering skills in optimizing these tools. Participant feedback highlighted three key areas where human oversight and engagement proved indispensable:

• Quality Assurance for Code: Research participants noted instances where Generative AI-based tools provided incorrect coding suggestions and even introduced errors into the code. In some cases, developers had to input numerous prompts to correct the tool's mistaken assumptions before obtaining a correct answer to a question. In another scenario, a developer described the need to carefully guide the tool through the debugging process, almost like "spoon-feeding" it to ensure accurate code validation.
• Providing Organizational Context: Off-the-shelf Generative AI-based tools possess a wealth of knowledge about coding, but they lack awareness of the specific requirements of individual projects and organizations. This contextual understanding is crucial in coding to ensure that the final software seamlessly integrates with other applications, meets the company's performance and security standards, and ultimately addresses end-user needs. As emphasized by research participants in their feedback, the responsibility falls on software developers to furnish these tools with the necessary context through prompts. This context includes details about how the code will be used, the intended users, the interfaces and systems the software will interact with, the relevant data, and more.
• Navigating Complex Coding Challenges: Participant feedback also indicates that Generative AI-based tools excel in addressing straightforward prompts, such as optimizing a code snippet, but may struggle with more intricate tasks, such as merging multiple frameworks with disparate code logic. To obtain a viable solution for complex requirements, developers sometimes had to manually combine components or break down the code into smaller segments. As one participant elucidated, Generative AI is less helpful when grappling with multifaceted challenges that necessitate a holistic perspective, stating, "[Generative AI] is least helpful when the problem becomes more complicated and requires consideration of the bigger picture."

What Implications Do These Discoveries Hold for Technology Leaders?

Considering these findings, what actionable steps can technology leaders take to transform these time-saving benefits and quality enhancements into tangible productivity gains while mitigating potential risks associated with the adoption of Generative AI in software development? Drawing insights from the experiences of the research participants, it is recommended to prioritize 4 key areas: skill enhancement, exploring advanced use cases, strategizing for skill transitions, and implementing robust risk management measures.

Provide developers with Generative AI training and coaching

To enable developers to effectively harness this technology to enhance their daily workflows, a combination of training and coaching is likely necessary. Initial training should encompass the dissemination of best practices and hands-on exercises focusing on the creation of natural-language prompts, often referred to as "prompt engineering." Additionally, workshops should furnish developers with an understanding of Generative AI-related risks, including potential data privacy or intellectual property concerns specific to their industry. Developers should be educated on best practices for reviewing AI-assisted code, which encompasses considerations like design, functionality, complexity, adherence to coding standards, and overall code quality. This training should also encompass the ability to discern between valuable recommendations and suboptimal ones provided by the AI tools.

For developers with less than a year of experience, the research suggests a need for supplementary coursework in fundamental programming principles. This includes topics such as coding syntax, data structures, algorithms, design patterns, and debugging skills. Such additional education is essential to achieve the same level of productivity gains observed among developers with more experience.

Once developers commence integrating these tools into their daily routines, their skill development should be an ongoing process, facilitated by continuous coaching provided by senior team members. Additionally, building a supportive community environment, such as dedicated online channels and team meetings for sharing practical examples, is crucial. These efforts foster continuous learning, enable the dissemination of best practices across the organization, and aid in the early identification of any potential issues. The research participants highlighted that, as they generated more prompts and shared their learnings, the quality of their prompts improved over time.

Explore Advanced Applications Beyond Code Generation

While there is significant industry excitement surrounding the Generative AI's capacity to create new code, the research demonstrates that this technology can have a meaningful impact on a wide range of typical developer tasks, including the critical task of refactoring existing code. This ability presents an opportunity for leaders to address resource-intensive modernization efforts that have often been deprioritized due to time constraints. For instance, if Generative AI-based tools can help teams swiftly refactor a legacy application, those teams can allocate their time toward resolving a backlog of enhancements that have long been on their company's agenda or enhancing the architectural performance of the entire software platform.

Introducing new applications necessitates a thoughtful evaluation of available tools, given the influx of Generative AI tools entering the market, each excelling in specific domains. Leveraging multiple tools can yield more benefits than relying on one. During the study, participants had access to two tools: one utilizing a foundation model trained to respond to user prompts, and another employing a fine-tuned foundation model specifically trained for code-related tasks. Participants noted that the former, with its conversational capabilities, was particularly adept at answering questions during code refactoring. On the other hand, they found that the latter tool excelled in generating new code, thanks to its ability to seamlessly integrate into their integrated development environments and provide code suggestions based on descriptive comments within their documents. Remarkably, when developers utilized both Generative AI tools concurrently for a given task, rather than using only one, they realized an additional time savings of 1.5 to 2.5 times.

Prepare for Skill Evolution

As developers experience heightened productivity, leaders must be primed to redirect their workforce toward higher-value tasks. Establishing a productivity baseline and continuously monitoring progress can help identify emerging capacity within the organization. Leaders should contemplate how to leverage this additional capacity and assess any skill gaps that may arise, necessitating targeted upskilling efforts. For instance, they might channel their talent towards facilitating new business expansion or more frequent product updates. Such endeavours would require developers to acquire new proficiencies in software design and architecture.

Implement Robust Risk Management

The adoption of Generative AI-based tools introduces new data, intellectual property, and regulatory risks. Considering the rapid pace at which developers can generate or modify code using these tools, it is crucial to anticipate potential issues, such as:

• Data Privacy and Third-party Security: Leaders should be vigilant about the possibility of developers inadvertently exposing confidential information when interacting with the AI tools.
• Legal and Regulatory Changes: Ongoing legal, regulatory alterations and other restrictions on technology usage should be carefully considered as they impact the adoption of this technology.
• AI Behavioural Vulnerabilities: Potential concerns may arise if malicious entities attempt to introduce harmful code into the public domain, affecting the training of large language models or infiltrating organizations.
• Ethical and Reputational Concerns: There is a possibility of ethical and reputational issues arising from using code snippets copyrighted by other entities or amidst debates regarding the ownership of code generated by these tools.
• Security Vulnerabilities: Vigilance is required to address security vulnerabilities that may manifest in AI-generated code, which could jeopardize systems and the overall organization.

By proactively addressing these risks and implementing robust risk management strategies, leaders can ensure the safe and responsible integration of Generative AI-based tools into their software development processes.

Final thoughts

Generative AI is set to revolutionize software development, boosting task completion speed by up to two times. Yet, tooling alone won't unlock its full potential. To harness this technology, organizations must follow a structured approach, including Generative AI training, smart use case selection, workforce upskilling, and robust risk controls. This paves the way for remarkable productivity and groundbreaking software innovation.

VNG Cloud plans to launch GPU Cloud, a dynamic and cutting-edge platform with a focus on delivering exceptional GPU performance in 2024. This cloud solution is tailored for a wide range of applications, from AI, Machine Learning, Deep Learning, Large Language Model (LLM) to high-performance computing (HPC) workloads. Our dedicated GPU servers are ready to meet the high demands of today's data-intensive tasks.