Can an AI evolve through Darwinian evolution?

LLMs like ChatGPT can optimize code with some success. Today it is limited in what it can do and quite often the generated code is flawed. I expect that it will improve to a point where we can ask a LLM to optimize its own code, build itself and do subsequent training. If we then add a mechanism capable of rating the generated LLMs, we could introduce a variant of natural selection and Darwinian evolution.

If an AI evolves through Darwinian evolution, what is it then?

Imagine a world where AIs have become self-evolving? Their evolution might be controlled by humans, or they could have broken free of human intervention. How would we classify such an AI? Is it a machine or is it a new thing? I argue that a machine capable of replicating and improving itself, is something new?