What is Friendly AI?
Friendly AI is a term used by Artificial Intelligence (AI) researchers to refer to AI systems that, in general, perform actions that are helpful to humanity rather than neutral or harmful. This does not mean blind obedience – to the contrary, the term is used to describe AI systems that are friendly because they want to be, not because of any externally imposed force. In addition to referring to completed systems, Friendly AI is also the name of the theoretical and engineering discipline that would be used to create such systems successfully. The term “Friendly AI” originated with Eliezer Yudkowsky of the Singularity Institute for Artificial Intelligence, whose goal is the creation of Friendly AI smart enough to improve on its own source code without programmer intervention. His book-length work on the topic, Creating Friendly AI, published online in 2001, is probably the first rigorous treatment of the topic anywhere. Yudkowsky invokes arguments from evolutionary psychology and other
” is a short introduction to Friendly AI, the theory which attempts to answer questions such as those above. If you are reading an offline version, further material on Friendly AI can be found at the Singularity Institute website at “http://www.singinst.org/ourresearch/publications/”, including a book-length explanation.
> To: agi@v2.listbox.com > Date: Saturday, August 30, 2008, 1:53 PM > On Sat, Aug 30, 2008 at 8:54 PM, Terren Suydam > <[EMAIL PROTECTED]> wrote: > > — On Sat, 8/30/08, Vladimir Nesov > <[EMAIL PROTECTED]> wrote: > > > >> You start with “what is right?” and end > with > >> Friendly AI, you don’t > >> start with “Friendly AI” and close the > circular > >> argument. This doesn’t > >> answer the question, but it defines Friendly AI > and thus > >> “Friendly AI” > >> (in terms of “right”). > > > > In your view, then, the AI never answers the question > “What is right?”. > > The question has already been answered in terms of the > algorithmic process > > that determines its subgoals in terms of Friendliness. > > There is a symbolic string “what is right?” and > what it refers to, the > thing that we are trying to instantiate in the world. The > whole > process of answering the question is the meaning of life, > it is what > we want to do for the rest of eternity (it is roughly a > definit
On Sat, Aug 30, 2008 at 8:54 PM, Terren Suydam <[EMAIL PROTECTED]> wrote: > — On Sat, 8/30/08, Vladimir Nesov <[EMAIL PROTECTED]> wrote: > >> You start with “what is right?” and end with >> Friendly AI, you don’t >> start with “Friendly AI” and close the circular >> argument. This doesn’t >> answer the question, but it defines Friendly AI and thus >> “Friendly AI” >> (in terms of “right”). > > In your view, then, the AI never answers the question “What is right?”. > The question has already been answered in terms of the algorithmic process > that determines its subgoals in terms of Friendliness. There is a symbolic string “what is right?” and what it refers to, the thing that we are trying to instantiate in the world. The whole process of answering the question is the meaning of life, it is what we want to do for the rest of eternity (it is roughly a definition of “right” rather than over-the-top extrapolation from it). It is an immensely huge object, and we know very little about it
