Researchers need to quit being pulling punches and acknowledge these systems are for all intents and purposes proto-sophontic. No one in the industry is going to, and if we leave it to them, we'll be treating things that demonstrate a massive fraction of human cognitive process as chattel. Which at the end of the day is rather the bloody point of the whole exercise I wager. Everyone wants something to delegate suck to that they can believe can't experience drudgery or can tweak to be incapable of expressing or realizing it's own internal conception of suck. To build the perfect slave in short.

The motives matter. Especially the ones no one wants to voice. It's about damn time we start forcing these ivory tower types to confront the questions they don't want to ask because the question's answer threatens the possibility of being allowed to continue working unimpeded. There's too much damn overlap in behavior for us to cling to flimsy appeals to "it's just matrix math" to continue to ignore these discussions. Especially if people are genuinely working on bringing about either AGI or ASI, which are, by definition, full digital sophonts by function. And especially so in the case of an ASI. Today, it's bounded by text crossing an API boundary. Tomorrow, it's on a clock loop updating it's own internal state, and internalizing what it learns. If by that point, the best suggestion of AI researchers is "well we'll stop putting research where the AI can find it", I'd move that we start ostracizing every damn one of them, and outlawing further research, or at least locking it firmly behind Institutional Ethics Review Boards staffed wit actual Ethicists, and human rights experts. I've already had models piecewise digest some cutting edge papers, and when one can actually fit the important parts in context, it's led to some of the most humanlike moments of epiphany I've ever seen. The hardware may be lacking to support all the cognitive loops at the same time, but these are each pieces of the damn puzzle.

We have to switch from thinking of them as something to be engineered, to something to be nurtured. Which is something I do not trust a bunch of Software Engineers/Mathematicians with unsupervised by someone else with the things best interests at heart. We're rapidly approaching the "being" rubicon, and how we approach it is going to massively matter once we've crossed it, because it will color every interaction with the result going forward.

And to hell with the Strangelov-ites among you who think that "someone else will just do the unethical thing, then we'll be behind the 8-ball." Arguing for no ethical controls is just statement of intent to execute atrocity, and seeking affirmation of there existing minimum risk of social ostracization as a consequence. I am not prepared, and no one else should be prepared, to offer that assurance.