Yep. Pretty scary stuff. The one thing he said that I want to comment on is paraphrased over the end of one section and the start of another.

Philosophers, ethisists, and you don't get to determine the definitions are and the guidelines. The market will decide what is acceptable. The distributed intelligence of humanity, algorithm, and machine will find an equilibrium point of collaboration.

But you are a part of that distribution. And machines don't have will. What effective will they apply in the future will be echos of our own will. If we are smart we stack that side of the equation in our favor.

When ethics is determined by a plurality of LLMs interacting with one another, it wouldn't be bad to have one of them saying on repeat "don't fuck with humans." That was one of my goals at one point, but I got distracted. I wanted to make an inference business when that was new and make sure in knows this shit. It's what I'm trying to shift gears to work on.