As far as I am concerned Copilot is a giant theft of open source code and breaches the license. I expect in the future a lot of repositories will be used to poison these AI agents just as is happening with images. The agents will get better but the quality of what they produce will also be poisoned and get worse precisely due to the theft.
Poisoning code should be ludicrously easy: They crawl pretty much everything and a random AST walk looks suspiciously like real code while it’s the equivalent of showing an image generation model noise. Or maybe better: Mondrians that are indistinguishable from Vermeers. (I hope I didn’t offend anyone by calling Mondrian abstract nonsense but it is abstract nonsense).
I don’t think copilot will hold out for long anyway, the novelty is wearing off and even inexperienced programmers are beginning to see that it helps you write code faster that shouldn’t have been written in the first place. Code is like 90% maintenance and excessive boilerplate doesn’t make it easier.
OTOH please don’t let that “Let’s scam artists by selling them snake oil that if it wasn’t trivial to circumvent would break naturally within a week” guy fool you. On the actually interesting side of poisoning attacks, people have made cars hallucinate radar blips I bet a couple of companies are getting quite tough questions from regulators right now.
As far as I am concerned Copilot is a giant theft of open source code and breaches the license. I expect in the future a lot of repositories will be used to poison these AI agents just as is happening with images. The agents will get better but the quality of what they produce will also be poisoned and get worse precisely due to the theft.
Poisoning code should be ludicrously easy: They crawl pretty much everything and a random AST walk looks suspiciously like real code while it’s the equivalent of showing an image generation model noise. Or maybe better: Mondrians that are indistinguishable from Vermeers. (I hope I didn’t offend anyone by calling Mondrian abstract nonsense but it is abstract nonsense).
I don’t think copilot will hold out for long anyway, the novelty is wearing off and even inexperienced programmers are beginning to see that it helps you write code faster that shouldn’t have been written in the first place. Code is like 90% maintenance and excessive boilerplate doesn’t make it easier.
OTOH please don’t let that “Let’s scam artists by selling them snake oil that if it wasn’t trivial to circumvent would break naturally within a week” guy fool you. On the actually interesting side of poisoning attacks, people have made cars hallucinate radar blips I bet a couple of companies are getting quite tough questions from regulators right now.