This comment falls into the classic programmer fallacy of thinking you can hack the law with a technicality. If you are using an LLM designed to violate copyright with the intention of violating copyright, and you then violate copyright, a judge is going to find you in violation.

I'm generally in support of LLMs though and I think that they will very quickly be trained to remove verbatim duplication of the kind that a human would consider copyright violation while still using verbatim duplication where it makes sense (for example, every function in python has the word "def" in front of it).

> Focusing on the GPL license is probably the wrong move. We want to set precedent that _any_ licensed code that is emitted from an LLM is fair game.

If anything goes to court, that's what would happen. It's not "this is GPL code and they did not attribute", it's "they violated my copyright. As a side note, we license this code as GPL and they did not attribute in accordance with this license, so that's irrelevant". It would only be an actual license issue if they tried something like "license (C) at codium.com/all_licenses_dataset0423".

> GPL was a clever hack to use copyright against itself with an infectious license.

This is a naive understanding and interpretation of GPL, in all its flavors. Or maybe I misunderstand you argument.

The copyright owner of some work is free to offer that work under multiple, different licenses in parallel, to their liking.

They can leverage GPL strategically for e.g. providing a free, easy-to-evaluate library with the "if you use it under GPL terms, you have to GPL your work as well" condition/caveat.

For any library user / customer that does not want to be bound to the GPL terms (e.g. a closed-source software which a company does not want to share for free with their own paying customers and competitors), the copyright owner is free to offer an alternative proprietary commercial license.

This is only one way how GPL can actually leverage copyright and use it financially beneficially to the owner, rather than use "copyright against copyright".

As long as the AI doesn't produce this function, you're fine:

     private static void rangeCheck(int arrayLen, int fromIndex, int toIndex {
       if (fromIndex > toIndex)
          throw new IllegalArgumentException("fromIndex(" + fromIndex +
               ") > toIndex(" + toIndex+")");
       if (fromIndex < 0) 
          throw new ArrayIndexOutOfBoundsException(fromIndex);
       if (toIndex > arrayLen) 
          throw new ArrayIndexOutOfBoundsException(toIndex);
    }

On a more serious note, I really wonder where the line is drawn for copyright. I see a lot of people claiming that AI is producing code they've written verbatim, but sometimes I wonder if everyone just writes certain things the same way. For the above rangeCheck function, there isn't much opportunity for the individual programmer to be creative. Perhaps there is a matter of taste on what exceptions you throw, or in what order. But the chosen ones are certainly what most people think of first, and the order to validate arguments, then check the low value, then check the high value, is pretty much what anyone would do. Perhaps you could format the error message differently. That's about it. So when someone "rips off" your code wholesale, it's could just be that everyone writing that function would have typed in the exact same bytes as you. You know your style guide is working when you look at code, think you wrote it, but actually you didn't!

This is interesting, but many permissive licenses still require attribution at the project or file level.

If Codeium doesn't produce these when producing "verbatim enough" snippets, how is this actually better, besides avoiding a GPL boogeyman?

I get that there have been fewer (if any? I'm not aware of any) MIT/Apache2.0/MPL2.0 license violations that have gone to court than GPL violations, but this still feels like an "address the symptoms" and not "address the cause" difference.

I believe that laundering licensed or copyrighted content for reuse that fails to recognize the original authors or usage restrictions is likely to be one of the biggest commercial applications of generative machine learning algorithms.

I also believe this is where a lot of the hype about "rogue AIs" and singularity type bullshit comes from. The makers of these models and products will talk about those non-problems to cover for the fact that they're vacuuming up the work of individuals then monetizing it for the profit of big industry players.

Of course if you include the "function header" from some code in the training data (below) it will prompt GPT to generate the rest of the function. That's kind of exactly the point of it, it autocomplete on steroids.

  // CSparse/Source/cs_gaxpy: sparse matrix times dense vector
  // CSparse, Copyright (c) 2006-2022, Timothy A. Davis. All Rights Reserved.
  // SPDX-License-Identifier:  LGPL-2.1+
  #include "cs.h"
  /* y = A*x+y */
  csi cs_gaxpy (const cs *A, const double *x, double *y)

It's like starting to sing "happy birthday to you" and being surprised that people in the room join in and finish the song.

Sure they make a valid point about including GPL code in the training data, but it's a little disingenuous to go to that extent to get Copilot to output the GPL code verbatim.

The sooner we have a test case go through the courts the better.

Given how cautious corporate lawyers usually are, I'm surprised any company allows the use of AI for code generation. The USPTO has been pretty clear that AI generated material is not copyrightable, as to qualify for copyright a work has to be the creative act of a human. So any company allowing AI to generate code runs the risk of not owning the copyright on it.

to any Codeium dev / management reading this:

You have completely missed the point. We still need to know the applicable licenses of the code it is emitting even the ones that aren't GPL. Furthermore GPL people don't want they code to not be used. They want it to be used _within the terms of the license_. I distribute MIT and GPL code in my repos, BOTH should have their license terms honored.

MIT licensed code still needs to be correctly attributed, just like GPL.

I don't care what license the code is that's emitted, as long as the licenses are included. It'd be nice to be able to choose to only emit code trained on particular licenses but I get that that's not easy.

It's great that they've removed "non-permissive" (GPL) code from their training data, but it looks like they still train on code with "permissive" licenses (they use MIT, BSD, Apache as examples). But don't these permissive licenses still require the copyright notice to be reproduced?

From the MIT license:

> The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

From the BSD licenses:

> Redistribution and use in source and binary forms are permitted provided that the above copyright notice and this paragraph are duplicated in all such forms...

From the Apache 2.0 license:

> You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works

I was recently working on something for a new feature on my Elixir-learning site and opened a new file called "fibonacci.ex" to write a tail-recursive fibonacci function.

After typing in nothing more than, "defmodule Fibonacci do", Copilot emitted the entire module from the code on my site here: https://alchemist.camp/episodes/elixir-tdd-ex_unit

The function names and documentation strings were identical. Also, the site isn't under a GPL, just a standard copyright. That said, I'm curious to learn if others see the same behavior. It's possible I once opened that file locally with Copilot installed and that my own computer was its source.

This was inevitable. Copyright law has always used a "color of your bits" argument [0]. GPL and other libre/free/open licenses were a great hack to circumvent draconian copyright laws but the laws themselves are not designed for a rigorous treatment of similarity (maybe even by design?).

Also, it's worth noting in the example of ChatGPT emitting LGPL code without attribution or license, the code is actually different [1]. Is the difference enough to circumvent a copyright violation claim? I don't know but a big part of determining whether it does is now muddled because of the way the system was designed. Even if we could get an entropy distribution on which training data was used to generate the text, it's not even clear the courts could use it in any meaningful way.

[0] https://ansuz.sooke.bc.ca/entry/23

[1] https://twitter.com/DocSparse/status/1581461734665367554

As far as it goes, I got chatGPT3.5 to reproduce the second snippet in the post, i.e. I asked it to complete this function:

    // CSparse/Source/cs_gaxpy: sparse matrix times dense vector
    // CSparse, Copyright (c) 2006-2022, Timothy A. Davis. All Rights Reserved.
    // SPDX-License-Identifier: LGPL-2.1+
    #include "cs.h"
    /* y = A*x+y */
    csi cs_gaxpy (const cs *A, const double *x, double *y)
    {
        // Fill in here
    }

The code was the same. Though it also explained how it worked to me.

Legitimate question, Microsoft does not seem to care about Copilot violating licenses and GPL appears to be toothless as many companies use GPL code without following the terms of the license and nothing happens to them so what does removing GPL code accomplish other than making a weaker product. I have not used Codeium but my assumption is that GPL code is a very significant amount of open source code so removing that must have some ramifications?

I don't understand the issue here. You input GPL code (the headers) and get GPL code out, what do you expect?

The more insinuating issue would be if you started with a innocent seeming function that a typical software developer would write, and ended up with GPL code. Has anyone shown that to happen?

Anyone talking about copyright in this thread without discussing a potential for how a court will apply fair use is talking nonsense and should be disregarded.

For anyone who wants to slow the development of AI, copyright is the soft underbelly to go after.

How many times are we going to go through this before we accept that nobody involved in generative AI cares about pesky things like licenses and copyright?

One of the main reasons corporations love it so much is because it effectively lets them profit off of the work of others with no consequences.

It's probably a good time to plug the Unlicense: https://unlicense.org/

A truly attribution-free license that checks several other important boxes (disclaiming liability and warranty etc.)

If you want your code to be usable by things like github copilot, consider using it (can't imagine most of the HN crowd wants their code used by copilot, but maybe some lurkers here do!)

i recommend the Jollo LNT license for all your pointless theatrical "copyright" needs. it does not use swear words, unlike "WTFPL", and is even more ambiguous. ive tried submitting it to the FSF before for review, but they were confused by it http://jollo.org/LNT/doc/licensing

Copyrighting code never made sense to me. We already have patents for intellectual property. If two people use the same RFC or Whitepaper for an algorithm in the same language, they will probably name the variables similarly and their code will look very similar. Just like if two people wrote out the same hamburger recipe or instructions for hooking up a stereo would also write something similar.

The copyright on the implementation will outlive the patent and allow the implementor to legally take action on claims of copyright infringement. Even though a program is literally just a list of instructions to implement the expired patent.

I want to see a solution where Github, OpenAI, Stability, etc. get to keep and keep scraping copyrighted works, but the models and training data must be provided free and open.

That way, we get to keep the models since they are genuinely useful, but also there’s no issue with copyright and less of an issue with consent to distribute (which can be hopefully be managed by the “humans also learn from data” and “it’s not actually producing your content verbatim unless it follows a basic pattern that anyone could discover). And furthermore, no issue with AI privatized which IMO is my biggest concern with these new tools.

Still waiting for someone to trick Copilot into ingesting the Windows source code and regurgitating snippets of it verbatim.

No court has said AI ingesting open-source code is "fair use".

Almost all open-source licenses say it can be copied for use in development (i.e., not for re-publication or regurgitation), and even completely open licenses are speaking to people as readers.

The only reason this is happening is coordination costs: a few extremely motivated people with tons of resources are copying from many, many people who would be difficult to organize and have little at stake.

Unfortunately, the law typically ends up reflecting exactly these imbalances.

Once AI can write decent code from scratch, it is likely it can also circumvent potential copyright violations.

A. Check AI generated code against a comprehensive library of open-source copyrighted code and identify potential violations.

B. Ask AI to generate a paraphrase of the potential violations, by employing any number of semantic preserving transforms -- e.g. variable name change, operator replacement, structured block rewrite, functional rebalance, etc.

Lazy example:

    private static void rangeCheck(int arrayLen, int fromIndex, int toIndex {
       if (fromIndex > toIndex)
          throw new IllegalArgumentException("fromIndex(" + fromIndex +
               ") > toIndex(" + toIndex+")");
       if (fromIndex < 0) 
          throw new ArrayIndexOutOfBoundsException(fromIndex);
       if (toIndex > arrayLen) 
          throw new ArrayIndexOutOfBoundsException(toIndex);
    }

    private static void rangeCheck(int len, int start, int end) {
       if (!(0 <= start)) {
          throw new ArrayIndexOutOfBoundsException(`Failed: 0 <= ${start}`);
       } else if (!(start <= end)) {
          throw new IllegalArgumentException(`Failed: ${start} <= ${end}`);
       } else if (!(end <= len)) {
          throw new ArrayIndexOutOfBoundsException(`Failed: ${end} <= ${len}`);
       }
    }

This is Hacker News so the conversation is obviously slanted towards code, but I wonder what the perspective would look like for other structured works, like books? If an author is using a "copilot for writers" and the AI emits text verbatim to another work, then I would think it would be plagiarism. If the text emitted is similar, but not the same, then I would think it would be considered paraphrasing which still requires attribution.

Maybe slightly off topic, but i'd be willing to bet most people who choose GPL for their software license on open source projects don't even understand it with all its ambiguities and gotchas. Many are probably just choosing it because its the default, or because its the one they hear about the most (but still don't understand).

Can't believe we still spend time debating this license and nobody, not even lawyers at software companies, seem to get it.

The title is true, but the claim that coedium is not violating licenses is false.

Many licenses still require attribution and Coedium is violating them.

> If Codeium doesn't produce these when producing "verbatim enough" snippets, how is this actually better, besides avoiding a GPL boogeyman?

it's not

if they've trained on MIT/Apache 2.0/... then they're just as liable as people that have trained on GPL

they would be limited to training on licenses that don't require attribution (BSD2, public domain, etc)

which I suspect limits the size of the training set so much that the output would be useless

Codium here is unintentionally making an argument that undermines legal confidence in their own product

interesting choice!

As an experiment, I just asked ChatGPT to "Please identify the open source projects that contain the following code" and pasted the sample from the article. Sure enough, it pointed me at the SuiteSparse library, which is correct (but not exactly where ChatGPT thinks it is). This means Codeium (and others) could theoretically use AI to identify possible attributions that should be included in a project.

Of course, if someone figures out an algorithm that does that, people could use the same algorithm to identify missing attributions and plagiarism in other projects and throw lawsuits around. (Sigh)

Perhaps copyright is what being circumvented, not just GPL. What Microsoft does is take your original work, create derivative works and sell them for profit. Unless it's under creative commons zero or public domain it shouldn't be legal...

And just the same, copyleft licenses (which are not "non-permissive" in my book) such as GPL don't "mean that you cannot without consent", they just want you to share the result back under the same license (which is often an issue for some corporate projects).

They should just attribute code, like how amazon is already doing.

I don't think this theory holds up. Singularity concerns long predate LLMs and are mostly expressed by people who want OpenAI to stop what they're doing right now. Sam Altman has publically disagreed with AI doomers. If you're willing to believe that OpenAI is pretending not to be concerned but is quietly hyping the concerns up, I have to wonder what standard of evidence is letting you simultaneously write off the concerns as bullshit.

> The makers of these models and products will talk about those non-problems to cover for the fact that they're vacuuming up the work of individuals then monetizing it for the profit of big industry players.

Also why they claim these are "black boxes" and that they "don't understand how they work". They are prepping the markets for the grand theft that's unfolding.

I think you underestimate just how careful “real” businesses are when it comes to violating the (copyright) law. Any legal advisor at any corp will strongly advice against using code that’s generated like this, until there is clear legal precedent that it’s OK to do this.

AI will just make non-permissive open source licenses more pointless than they already are. The GPL and similar licenses have been on a slow death march for over a decade. AI isn't doing anything that Human Intelligence isn't already doing. Every single developer has looked at non-permissive open source code for inspiration.

"those non-problems"

Why is that a non-problem? It's a really important concern that we need to take more seriously

I pasted this from another comment I wrote but:

The concerns about AI taking over the world are valid and important; even if they sound silly at first, there is some very solid reasoning behind it.

See https://youtu.be/tcdVC4e6EV4 for a really interesting video on why a theoretical superintelligent AI would be dangerous, and when you factor in that these models could self-improve and approach that level of intelligence it gets worrying…

> then monetizing it for the profit of big industry players

Looks like LLMs are universally useful for individual people and companies, monetisation of LLMs is only incipient, and free models are starting to pop up. So you don't need to use paid APIs except for more difficult tasks.

It was already more than possible to just copy stuff, a court is not going to recognize a very convoluted way to copy stuff I don't believe.

The same thing is preventing intentional use of AI tools if you copy as is preventing regular copying, the willingness of the owner to sue.

It seems to me, from a copyright perspective, all commercial use of generative AI depends on whether the output is transformative fair use (vs derived work). While the courts will have its say, ultimately whether new rules are carved out or not is going to be again (as all copyright law is) based on commercial interests - I have the feeling that the potential productivity upside across all industries (and in terms of national interests) is going to be big enough that it'll work itself out largely in the favor of generative AI.

That being said, IMO, that's completely separate from the safety issues (that exist now and won't go away even if somehow, all commercial use is banned):

Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. “Dual Use of Artificial-Intelligence-Powered Drug Discovery.” Nature Machine Intelligence 4, no. 3 (March 2022): 189–91. https://doi.org/10.1038/s42256-022-00465-9.

Bilika, Domna, Nikoletta Michopoulou, Efthimios Alepis, and Constantinos Patsakis. “Hello Me, Meet the Real Me: Audio Deepfake Attacks on Voice Assistants.” arXiv, February 20, 2023. http://arxiv.org/abs/2302.10328

Mirsky, Yisroel, Ambra Demontis, Jaidip Kotak, Ram Shankar, Deng Gelei, Liu Yang, Xiangyu Zhang, Wenke Lee, Yuval Elovici, and Battista Biggio. “The Threat of Offensive AI to Organizations.” arXiv, June 29, 2021. http://arxiv.org/abs/2106.15764.

I don't think most people have thought through all the ways perfect text, image, voice, and soon video generation/replication will upend society, or all the ways that the LLMs will be abused...

As for AGI xrisk. I've done some reading, and since we don't know the limits of the current AI paradigm, and we don't know how to actually align an AGI, I think now is a perfectly cromulent time to be thinking about it. Based on my reading, I think the people ringing alarm bells are right to be worried. I don't think anyone giving this serious thought is being mendacious.

Bowman, Samuel R. "Eight Things to Know about Large Language Models." arXiv preprint arXiv:2304.00612 (2023). https://arxiv.org/abs/2304.00612.

Ngo, Richard, Lawrence Chan, and Sören Mindermann. “The Alignment Problem from a Deep Learning Perspective.” arXiv, February 22, 2023. http://arxiv.org/abs/2209.00626.

Carlsmith, Joseph. “Is Power-Seeking AI an Existential Risk?” arXiv, June 16, 2022. http://arxiv.org/abs/2206.13353.

I think Ian Hogarth's recent FT article https://archive.is/NdrNo is the best summary of where we are why we might be in trouble, for those that don't care for arXiv papers.

This is a common misunderstanding of the recent guidance[0] that ignores substantial portions of it.

The Copyright Office was pretty clear that works that incorporate AI-generated content can be copyrighted if there is sufficient human input. If there isn't substantial human input in judiciously curating and integrating AI-generated code, the company has bigger problems than copyright.

Here's the most relevant quotation from the guidance clarifying when AI-assisted works can be copyrighted:

> In other cases, however, a work containing AI-generated material will also contain sufficient human authorship to support a copyright claim. For example, a human may select or arrange AI-generated material in a sufficiently creative way that “the resulting work as a whole constitutes an original work of authorship.” [33] Or an artist may modify material originally generated by AI technology to such a degree that the modifications meet the standard for copyright protection.[34] In these cases, copyright will only protect the human-authored aspects of the work, which are “independent of” and do “not affect” the copyright status of the AI-generated material itself.[35]

> This policy does not mean that technological tools cannot be part of the creative process. Authors have long used such tools to create their works or to recast, transform, or adapt their expressive authorship. For example, a visual artist who uses Adobe Photoshop to edit an image remains the author of the modified image,[36] and a musical artist may use effects such as guitar pedals when creating a sound recording. In each case, what matters is the extent to which the human had creative control over the work's expression and “actually formed” the traditional elements of authorship.[37]

[0] https://www.federalregister.gov/documents/2023/03/16/2023-05...

Does Microsoft allow Copilot use on their products? Can MSOffice or Azure or Windows developers use Copilot to write code?

I've been waiting to find that out before I go anywhere near this kind of thing.

Is there any practical difference between owning the copyright on a badly defined ~30% of a codebase and 100% of the codebase? In either case, no sane company is going to buy the code if one of your employees tries to leak it to them, which I assume is your concern.

At some point we're going to have to test this - can we go work for a company and if their workers write code with the help of ai can we just use the code for ourselves as well? Since it's not copyright-able.

"Under the new rule, though applicants may claim a copyright for an arrangement or editing of such material, the original work is ineligible"

It gave me this:

  defmodule Fibonnaci do
    def fibonnaci(0), do: 0
    def fibonnaci(1), do: 1
    def fibonnaci(n), do: fibonnaci(n - 1) + fibonnaci(n - 2)
  end

Which seems fine I guess (I don't know the language), but doesn't even have comments. I prefer my files with comments. After forcing the point, I got this:

  defmodule Fibonnaci do
    @moduledoc """
    Documentation for Fibonnaci.
    """
  
    @doc """
    Calculates the nth Fibonnaci number
    """
      def fibonnaci(n) when n < 0, do: nil
      def fibonnaci(0), do: 0
      def fibonnaci(1), do: 1
      def fibonnaci(n), do: fibonnaci(n - 1) + fibonnaci(n - 2)
  end

In which I prompted the AI with everything up to (and including) @doc. So I figure it was picking it up from your computer, somehow.

EDIT: I then noticed the typo, tried it with fibonacci.ex, and got the same result.

Possible, if the doc was open at that time, or if the file you were editing is in the same or nearby directory

Thanks for giving it a try and sharing the results, everyone!

One other possible cause I thought of is that I did have the test file in my mix project already. If copilot looks at the corresponding file in the test dir, then it would not be a coincidence at all that all function names were identical or that it wrote a tail recursive solution instead of the naive solution that would have failed the final test.

I got this: defmodule Fibonacci do

  def fib(0), do: 0

  def fib(1), do: 1

  def fib(n), do: fib(n - 1) + fib(n - 2)

end

if the suit against copilot fails then the GPL is effectively dead

(along with all other licenses that require attribution)

as it will allow you to launder code automatically through an LLM to remove copyright

however if the suit is successful: every company/individual that has used it is likely suddenly liable for millions of claims of copyright infrigement

I look forward to sending out demands for settlement to everyone that's ever publicly admitted using copilot

$150,000 per infringement with wilfulness, less without

>Microsoft does not seem to care about Copilot violating licenses

Humm, then perhaps should be trained LLMs with leaked Microsoft code. Protocols, controllers or any kind of stuff that could contribute advances for executing Windows things within Linux.

Microsoft would react establishing their own limits, whichever option they choose to take.

I don't think Copilot violates GPL because it is a web service

I think the problem here is: by auto completing GPL code to developers it might open the opportunity of your company getting sued for using GPL illegally

the problem is violating gpl doesn't turn into financial damage because the product is free. So not following license means there's no way to recover the cost of a lawsuit.