I don't think you need to go to sci-fi scenarios because that is already reality for millions of lab animals on the planet, who are probably very sentient. Just a reminder:

>In the complaint, PCRM said Neuralink used a substance known as “BioGlue” that destroyed parts of the monkeys’ brains. It described animals exhibiting substantial psychological effects from the experiments, including anxiety, vomiting, poor appetite, hair loss and self-mutilating behavior including removing their own fingers.

>Neuralink called the data cited in the complaint “misleading”, saying in a blogpost it “did and continues to meet federally mandated standards”. After the UC Davis partnership came to an end, Neuralink moved its work to an in-house facility.

>It responded directly to allegations that more than a dozen monkeys died after Neuralink procedures, stating that some of these were “terminal procedures” – where live test subjects are euthanized “humanely” following surgery."

https://www.theguardian.com/world/2022/feb/15/elon-musk-neur...

Given that we're torturing the absolute crap out of intelligent, living beings not only 'for science' but for erection pills and hair loss shampoos and bogus gall blader medicine I think these AI debates are needlessly abstract and we know the answer to the question if we'd prevent harm anyway.

There is no point at which artificial life assumes the mantle and status of human life, because if there was such a thing it would immediately lead to the destruction of our society. Imagine if I could press a button and generate 50,000 instances of something that has the same rights as me. This is the equivalent of getting three wishes and making one of them be “50,000 more wishes.” If it were allowed the whole system implodes.

Either my 50,000 are slaves who outvote you and change all the laws because you didn’t act fast enough to make your own slave army, or each of my 50,000 uses their rights to generate 50,000 more of theor own slaves, ad infinitum.

This experiment is a reductio ad absurdum to the notion that artificial life— essentially a long string of carefully chosen bits— can have rights.

But aren’t real humans just information too, you ask? Yes. The reason we insist on human rights is simply that the alternative is endless war.

Black Mirror had an episode covering this very issue called Hang the DJ. (Spoiler alert!) Essentially a giant computer was simulating billions of human-to-human relationships using virtual (but sentient) people to see which would work, and which wouldn't, in order to make a dating app more accurate, before culling them off. I thought it was absolutely terrifying, yet most viewers overlooked the ethical issues of mass simulation of life for trivial purposes and thought it was cute(!) (I should stress that in the episode, the virtual people caught up in this are portrayed as being sentient and aware, they're not just a bundle of numbers and formulas like "people" in a game like Simcity.)

For religious folks this dilemma is easy: a material body needs a "spirit" to be alive (this is what is mainly responsible for consciousness and the ability to experience qualia), and automations have no spirit, therefore they are not (and can never be) alive.

For the non-religious I can see the dilemma. Human minds are just biological computers and nothing more, so there's a point at which electric computers will match or exceed the biological ones.

At any rate, both the religious and non religious are forced to assume consciousness is some form of magic or another beyond human control. There's no way to prove that anything or anyone is conscious beyond yourself.

As someone currently building GPT-* applications for fun and profit it has dawned on me. Essentially it's an ethics problem, and an unsolved one. I already feel really uncomfortable asking peurile questions of any LLM I experiment with, but at the same time, that's just my human projections of what the LLM "feels". As it has limited memory, and further, doesn't update its nodes in response to my queries, there's not a high likelihood that it has feelings. But once it does...

Isn’t this just a version of Roko’s basilisk?

https://rationalwiki.org/wiki/Roko%27s_basilisk

Edit: I mean it’s similar in that at some point you’re punishable for knowing and still preventing the AI instead of not bringing it about. I feel about the same with both scenarios.

Cf. https://arxiv.org/abs/1410.8233 ("Do Artificial Reinforcement-Learning Agents Matter Morally?").