Responsible use of GitHub Copilot code review

About GitHub Copilot code review

GitHub Copilot code review is an AI-powered feature that reviews code and provides feedback.

When a user requests a code review from Copilot, Copilot scans through the code changes, plus additional relevant context, and provides feedback on the code. As part of that feedback, it may also provide specific suggested code changes.

Copilot's review can be customized with coding guidelines, which are natural language descriptions of coding style and best practices. For more information, see "Configuring coding guidelines for GitHub Copilot code review."

The only supported language for GitHub Copilot code review is English.

GitHub Copilot code review inspects your code and provides feedback using a combination of natural language processing and machine learning. This process can be broken down into a number of steps.

Input processing

The code changes are combined with other relevant, contextual information (for example, the pull request’s title and body on GitHub) and your coding guidelines to form a prompt, and that prompt is sent to a large language model.

Language model analysis

The prompt is then passed through the Copilot language model, which is a neural network that has been trained on a large body of text data. The language model analyzes the input prompt.

Response generation

The language model generates a response based on its analysis of the input prompt. This response can take the form of natural language suggestions and code suggestions.

Output formatting

The response generated by Copilot is presented to the user directly in Visual Studio Code or as a pull request review on GitHub, providing code feedback linked to specific lines of specific files.

Where Copilot has provided a code suggestion, the suggestion is presented as a suggested change, which can be applied with a couple of clicks.

Use case for GitHub Copilot code review

The goal of GitHub Copilot code review is to quickly provide feedback on a developer’s code. This can enable developers to get code ready to merge more quickly and increase overall code quality.

Improving the performance of GitHub Copilot code review

Use Copilot code review to supplement human reviews, not to replace them

While GitHub Copilot code review can be a powerful tool for improving code quality, it is important to use it as a tool, rather than to replace human reviews.

You should always review and verify the feedback generated by Copilot code review, and supplement Copilot's feedback with careful human review to ensure your code meets your requirements.

Provide feedback

If you encounter any issues or limitations with Copilot code review, we recommend that you provide feedback by using the thumbs up and thumbs down buttons on Copilot's comments. This can help GitHub to improve the tool and address any concerns or limitations.

Configure coding guidelines

You can configure coding guidelines to help Copilot understand your coding style and best practices. For more information, see "Configuring coding guidelines for GitHub Copilot code review."

Limitations of GitHub Copilot code review

Depending on factors such as your codebase and programming language, you may encounter different levels of performance when using GitHub Copilot code review. The following information is designed to help you understand system limitations and key concepts about performance as they apply to GitHub Copilot code review.

Limited scope

Copilot code review only supports a specific set of programming languages, and it is only designed to identify a limited, fixed set of code quality issues. To ensure that all relevant problems are identified and corrected, Copilot code review should be supplemented with careful human code review.

Missed code quality problems

Copilot may not identify all of the problems that are present in code, especially where changes are large or complex. To ensure that all relevant problems are identified and corrected, Copilot code review should be supplemented with careful human code review.

False positives

Copilot code review has a risk of "hallucination" - that is, it may highlight problems in reviewed code that do not exist or are based on misunderstandings of the code. Comments generated by Copilot code review should be carefully reviewed and considered before taking action and making changes.

Inaccurate or insecure code

As part of its comments, Copilot code review may provide specific code suggestions. The code generated may appear to be valid, but may not actually be semantically or syntactically correct, or may not correctly resolve the problem identified in the comment. In addition, code generated by Copilot may contain security vulnerabilities or other issues. You should always carefully review and test code generated by Copilot.

Potential biases

Copilot's training data is drawn from existing code repositories, which may contain biases and errors that can be perpetuated by the tool. Additionally, Copilot code review may be biased toward certain programming languages or coding styles, which can lead to suboptimal or incomplete feedback.

Next steps

For details of how to use Copilot code review, see:

• "Using GitHub Copilot code review"