Skip to content Skip to footer

Introducing GitHub Code Scanning Autofix: AI-Generated Security Fixes

In November 2023, GitHub announced the launch of code scanning autofix. This uses to suggest fixes for security vulnerabilities in users' codebases. The idea behind code scanning autofix is simple. When a code analysis tool such as CodeQL detects a problem, the affected code and a description of the problem are sent to a large language model (LLM). The LLM is then asked to suggest code edits to fix the problem without changing the code's functionality.

Code scanning autofix is a feature of GitHub code scanning, which analyzes the code in a repository to find security vulnerabilities and other errors. Scans can be triggered on a schedule or upon specified events, such as pushing to a branch or opening a pull request. When a problem is identified, an alert is presented to the user. Code scanning can be used with first- or third-party alerting tools, including open-source and private tools.

GitHub provides a first-party alerting tool powered by CodeQL. This semantic code analysis engine allows querying of a codebase as though it were data. GitHub's in-house security experts have developed a rich set of queries to detect security vulnerabilities across various popular languages and frameworks. Building on this detection capability, code scanning autofix takes security a step further by suggesting -generated alert fixes.

In its first iteration, code scanning autofix is enabled for CodeQL alerts detected in a pull request, beginning with JavaScript and TypeScript alerts. It explains the problem and its fix strategy in natural language, displays the suggested fix directly on the pull request page, and allows the developer to commit, dismiss, or edit the suggestion.

At the core of code scanning autofix lies a request to an LLM, expressed through an LLM prompt. CodeQL static analysis detects a , generating an alert referencing the problematic code location and any other relevant locations. The LLM prompt consists of general information about this type of , typically including a public example of the and how to fix it, extracted from the CodeQL query help, the source-code location and content of the alert message, relevant code snippets from the locations all along the flow path and any code locations referenced in the alert message, and a specification of the response expected.

The LLM is then asked to show how to edit the code to fix the vulnerability. The model outputs a Markdown consisting of detailed natural language instructions for fixing the vulnerability, a complete specification of the needed code edits following the format defined in the prompt, and a list of dependencies that should be added to the project, if applicable. This is required, for example, if the fix uses a third-party sanitization library on which the project does not already depend.

To support real-world complexity and overcome LLM limitations, code scanning autofix requires a combination of careful prompt crafting and post-processing heuristics. Selecting code to show the model involves using a set of heuristics to assign a surrounding region that provides the needed context while minimizing lines of code, leaving less relevant parts as necessary to achieve the target length. The region is designed to include the imports and definitions at the top of the file, which are often crucial for understanding the context.

In summary, code scanning autofix leverages to suggest fixes for security vulnerabilities in users' codebases. It is a feature of GitHub code scanning, which analyzes the code in a repository to find security vulnerabilities and other errors. By suggesting -generated fixes for alerts, code scanning autofix takes security a step further, making it easier for to fix vulnerabilities in their codebase.

Leave a comment

Newsletter Signup

The Grid —
The Matrix Has Me
Big Bear Lake, CA 92315

01010011 01111001 01110011 01110100 01100101 01101101 00100000
01000110 01100001 01101001 01101100 01110101 01110010 01100101

Never send a boy to do a woman's job.Kate

Deitasoft © 2024. All Rights Reserved.