The integration of Artificial Intelligence (AI) into various technological aspects has become a prevailing trend ever since the release of ChatGPT in 2022. Many IT ventures have contemplated the possibilities of embedding AI in their products and services. Recently, OpenAI introduced the GPT-o1 model, optimized for scientific and programming tasks, sparking even more interest in AI adoption. Although there may be skepticism about whether AI is the next big hype, we can't deny its potential to revolutionize certain processes before the excitement fades.
AI excels in processing natural language and handling straightforward, repetitive tasks that allow for some errors. This makes AI an excellent assistant - one that efficiently executes repetitive tasks but requires guidance to achieve results. So, how does this apply to CI/CD systems? At OneRingCI, we've explored several practical AI applications in our managed CI/CD services, and we're excited to share some fruitful experiences.
Generating Release Notes
Automated release notes generation is a straightforward application of AI in CI/CD. At OneRingCI, every commit is linked to a ticket and an impact label by default. During release, we gather all commits, extract relevant information from your project management tools, and employ the GPT-4o model to craft structured, readable release notes. These notes are then automatically distributed to stakeholders, ensuring that even non-techies can comprehend them. And the best part? It's all from the source of truth - your source code and work items.
Code Reviews: Enforcing Standards and Conventions
While AI-driven code quality reviews may not be foolproof, we found AI extremely helpful in enforcing coding standards and internal conventions. Using the preview version of the GPT-o1 model, we've achieved consistent outcomes in enforcing agreed standards and conventions, such as variable casing, file formatting, spacing, preferred usage, nesting depth, method size and more. It's like linting on steroids, defined in plain text. This initial check ensures compliance with all rules, allowing human reviewers to focus on the important things like design, architecture, security, and performance considerations.
Build Log Analyzer
A single build can often generate thousands of lines of logs, making it difficult to extract meaningful information. More often than not useful warnings and messages are hidden there. To tackle this, we've implemented a build health check system. Once a week, the GPT-3.5-turbo model analyzes logs from a random main/master build, summing up potential issues needing attention. It flags long-running tasks, outdated packages with vulnerabilities, and other suspicious elements. Although the process occasionally results in false positives, it provides valuable insights and time savings when properly monitored by a human - a core component of our services.
Lessons Learned: AI Applications with Less Impact
In our lab, we've experimented with several AI applications that yielded inconsistent results, including:
Validation of platform architecture constraints
General best-practices reviews for code
Build time and process optimization
Security and vulnerability auditing based on context
Test automation results analysis
Why do these use cases fall short? AI models are limited by the average knowledge base used during their training. Tasks demanding context, experience, and deep, sometimes undocumented, technology knowledge cannot be fully derived from universal models like GPT. This is where human expertise plays a critical role - offering a level of efficiency and insight that surpasses that found in documentation or StackOverflow.
Another reason why more complicated tasks should be avoided for now is the constant evolution of all models. Even if some consistency is achieved with a certain prompt, we've noticed that this can change in time as the model is optimized and altered and unlike well versioned APIs - we don't get an indication that something is wrong. This can hide long term risks and mission critical applications should be carefully considered.
Elevate Your Software Development with OneRingCI
Have you explored other potential solutions to optimize your software development? With OneRingCI, you don't have to undertake this journey alone. If gaining a competitive advantage without straining your budget appeals to you, we are here to help. If you want to learn more about our solutions and get that much needed competitive edge just reach out to us. Even if you decide to do it internally, or use another company we are always happy to share experience and get to know a new business.