More and more developers across 黑料门 and at our clients are using GitHub Copilot for coding assistance. We have an internal guidance document that we are continuously updating as people are reporting back with their learnings and pitfalls. This is a high-level summary of that guidance document, focussed on the minimum a developer should know when they are getting started with an in-line coding assistance tool like GitHub Copilot.听
听
We are also publishing a series of more detailed memos about our learnings with AI-assisted software delivery in general over on .
Ground rule: You鈥檙e still responsible for your code. Copilot is your assistant.
How to prompt
Comments are part of the prompt.
Describe what you want the code to do, it鈥檚 not like prompting a chatbot.
听
group the customer list by the
鈥渃ity鈥 field
听
The more surrounding context, the better: Open files are part of the prompt. Whatever code you are typing is part of the prompt.
Descriptive function and variable names are part of the prompt.
groupListOfCustomers
ByCity()
听
Iterate on your prompt 鈥 provide examples, make the comment or function name more specific, reduce the scope of what you are trying to do.
Test descriptions, and test code, are part of the prompt. Have your test file open while you鈥檙e implementing.
describe(鈥渟hould group a customer
list by city鈥)
听
For changes to existing functions with good test coverage, consider deleting and regenerating the whole function
When it works better
听
These are some of the parameters under which it is expected to work better than in other situations.
Relatively straightforward or standard problems 鈥 repetitive, boilerplate, common logic like list processing, a repeating pattern, things you鈥檇 usually copy & paste.
Commonplace languages and frameworks.
Tasks you know how to do in theory, but you need assistance with the details. You should know enough to judge if Copilot鈥檚 suggestions makes sense.
For small chunks: About 10-15 lines of code, maximum, so that you can follow along, and don鈥檛 increase your cognitive load with lots of code review.
Beware: Copilot can amplify 鈥渂ad鈥 local code
Getting stuck in older patterns
After refactoring, or introducing new patterns into the codebase, you might find that Copilot is still stuck in 鈥渢he old ways鈥.
听
Proliferating bad practices
Copilot uses our existing code to make more suitable suggestions for our context, but depending on what it chooses as its example, it creates either the good or the bad kind of consistency.听
It鈥檚 a balance: Flow boost vs flow disruption听
Move on
Don鈥檛 spend too much time fixing things if the suggestions are obviously not useful. Two strikes and move on!听
Review fatigue
The larger the piece of code generated, the more code review you will have to do! Consider the balance between the 鈥渢yping鈥 boost and the energy needed for review.听
Prompt abstraction
If prompting takes more time than coding itself, consider if it鈥檚 a good use case for Copilot, or if you should discard the assistance and write the code yourself.听
Don鈥檛 become complacent
听
TLDR; Don鈥檛 become the person who drives into the lake because their navigation system tells them to.听
Automation bias
Once you have had some success with Copilot, be careful not to start over-trusting it.听
Sunk cost fallacy
Even though with Copilot, we did not invest time to write a suggestion ourself, we might still get over-attached to the supposed time saved of a multi-line suggestion.听
Anchoring bias
Be aware that once you have seen Copilot鈥檚 suggestions, you might have a harder time thinking about other solutions.听
Security vigilance
Copilot does some pre-filtering for vulnerable patterns, but you cannot rely on the code to be secure for your context. Especially watch out for hallucinated dependency names.听
Are you super excited about this?
听
Awesome, this is an important new area for developers to understand! But be careful not to overinvest into making it work all the time because you want it to work.听
Are you skeptical?
听
That鈥檚 a healthy attitude, there is a lot of hype at the moment and the tool is not perfect! Give it a chance, and try some things even when you think 鈥渢hat won鈥檛 work anyway鈥.听