Lightrun is providing early access to an ability to verify whether a pull request (PR) will actually run in a production environment as part of its artificial intelligence (AI) platform for automating site reliability engineering (SRE) workflows.
Company CTO Leonid Blouvshtein said the Runtime Aware PR Verifier enables DevOps teams to assess the impact a PR will have on a live production environment before it is deployed.
At the core of the Lightrun platform is a Runtime Context engine that enables DevOps teams to understand how code truly behaves. Armed with those insights, it becomes possible to both identify issues and bottlenecks before and after an application is deployed in a production environment.
Lightrun is now extending that capability to assign risk scores to PRs based on how the change behaves against live execution paths, dependency interactions, and real traffic. Those scores are then natively shared within the context of an existing continuous integration/continuous delivery (CI/CD) platform from GitHub, GitLab or Atlassian.
As more PRs are generated using AI coding tools, DevOps teams are increasingly being overwhelmed as both the number and size of PRs continue to increase. Unfortunately, code that has not been thoroughly tested and validated as it should be is finding its way into production environments. Much of that code not only contains vulnerabilities but also tends to be more verbose than code that a human application developer might write. As such, it is more challenging than ever to manually review code.
In fact, a recent Lightrun survey finds 43% of AI-generated code requires manual debugging in production, even after passing QA or staging tests. Furthermore, an average of three manual redeploy cycles are required to verify a single AI-suggested code fix in production, according to the survey.
The only way to address that imbalance is for DevOps teams to rely more on AI platforms to review PRs. The challenge is that each production environment is different, so relying on an AI tool to review code that lacks any context about the production environment is not going to surface hidden issues that other testing tools are not going to be able to identify, said Blouvshtein.
Going forward, Lightrun will continue to expand the scope of the feedback it can, in near real time, provide DevOps teams with as more code gets checked into DevOps pipelines, he added.
Mitch Ashley, vice president and practice lead for software lifecycle engineering at the Futurum Group, said the constraint in software delivery has moved from writing code to verifying it, and verification is now a runtime problem. Teams shipping AI-generated code at volume cannot treat a passing test suite as proof a change is safe, he added.
Validation has to move earlier and account for live production behavior, or the gains from faster generation get spent on post-deployment remediation, said Ashley.
It’s not clear at what rate DevOps teams are embracing AI to help triage the onslaught of code now being generated by any number of readily available AI coding tools. The one thing that is certain, however, is that the question is no longer whether AI will be used to automate software engineering workflows, but to what degree.
from DevOps.com https://ift.tt/rX9O2P8
Comments
Post a Comment