Friday, August 22, 2008

Concept AI

Whenever I work on a digital project, I create a folder for it and make a notepad file for brainstorming and just scratch work. This is the current version of the brainstorming file for the AI-Concept I'm working on.


What separates machines from humans?? The answer that comes first to mind is that while we have free will. The power to have our own openions and to create new ideas out of scratch. Although that's how it seems, it's not the case. We humans cant create ANYTHING! We can only take what exists in nature and manipualte it into something new. Think of this. A raw piece of wood is cut from a forest and made into the paper that becomes the raw canvas for an artwork. We are able to make paper out of wood because we were observative. The artist is able to make an artwork not because he or she is CREATive, but because they have the power to observer and judge. Sitting in front of that blank canvas, hundreds of ideas rush into the artist's brain. But from where?? All of these ideas are from old experiences, in one form or another. Many times, new ideas are "created" from the combination of two concepts or situations into one. Many times these founding situations and ideas that make up the final ideas is not from the person themselves, but form someone else. All of human kind's achivements were possible because we were open minded. If we didnt share our findings, each of us still would be trying to invent the wheel by overselves a thousand different ways.

When faced with a blank canvas, artists often start off by drawing a single line. And then another. The image of these lines trigger memories of objects that could be formed from the combination of the lines. Looking up at a summer sky, we see the shapes of different objects on the faces of clouds, even though they are just clouds.

As children, we often imitate those older than us. We mimic the actions of those around us and learn form it. These process of observing and imitating, physically or mentally, allows us to learn. A process of rewards and punishments separating the good learning from the bad shapes our personality. Creating an program that can imitate what we tell it is fairly easy. So how do you go from an imitator into an Artificially Intelligent program? With goals, both big and small.

Goals are an important aspect of being human. We as humans are proud to announce that we have free will. "If I want, I can walk over to the front of the class. Punch the professor and run out, yelling and making a mess in the hallways." Yes. You can (as in ability to do something) do that. But WHY don't you? A person in their right minds would know that such actions can have serious consequences. It would most likely get you expelled from that school and subconsciously, you despise that idea since it would interfere with the goals you have in life. Notice I said the "goals you HAVE" in life. This is because many times, these goals are not your own. Your parents, friends and the society around you have a a lot on influence on the goals you set, especially those set when you are young and still prone to some aspects of learning from imitation.

Goals, combined with past experiences the program learnt, is the major aspect of an AI. You teach the program how to draw one line, and tell it to draw two lines. After many failures, it'd accomplish that goal. Learning from it's mistakes to accomplish the next, slightly harder task of drawing three lines, faster with fewer errors. The series of errors, punishments and rewards will develope into the AI's sense of judgment that judges good from bad. Although I enjoy watching Sci-Fi stories such as Terminator, Sarah Connor chronicles, I don't think such a situation would happen because of these very reasons.

-Unfinished. Aspects of Creativity, Thinking and Memory to be explained later.

It's been 4 or 5 days since I started the AI project. School will open soon, so I'll have to stop this and move on for now. I'll scratch down the basic outlines of the other ideas that I had so that if & when I start again on this project, I wont have to start from scratch.

Input:
Where am I
I am there
Who am I
I am Cobalt
What am I
I am human
Where are you
you are here
who are you
you are AI
what are you
you are a program
you are a module

The goal is to get the program to get a categorization of
1: I, am
2: You, are
3: where, I, there
4: where, you, here
5: who, I, Cobalt
6: who, you, AI
7: what, I, human
8: what, you, program, module
and so on

Later on (after more input), the categorization might look like the following.

1: who, what, where, when, why, how
2: where, US, Earth, Solar System
3: are, am, is was
and so on....

Make each node complete objects instead of just one word.

With the input "you are good" and "how are you" try:
you(are, good)
are(you, good)
good(you, are)

How(are, you)
are(How, you)
you(are, how)

That should let it associate Good with How. This system breaks down when new unknown words are introduced too fast
and the program confuses those words with the answer.

Memory: categorized by tags. Recent memories kept in RAM. snipeets from old events (more snippets of recente events than of old ones) also in memory. Snippets can be used to search for the rest of the memory as well as events directly before and after it and related events.

Creativity: combine random snippets (which lead to other snippets) in a comprehendible way. Apply concepts from one event/lesson on others. Use format from similar challenges/tasks/experiences. The snippets lead to a line of thought, sometimes creating recursion since in the process of looking through the snippets, the snippets are pushed up in the stack to be renewed/"newer". This creates the "That song is stuck in my head" scenerio.

Thinking: multiple threads jumping from topic to topic. Each topic is a .txt file with tags to related topics. sentences broken into web system.

Logic: basic skills of logic must be built into the system. a=b; b = c; a = ?
Pattern recognition should also be used

A heuritical system of punishments and rewards. Use this to organize the information better by re categorizing and eliminating the link from node to node when the heuritical cost becomes unacceptably high.

Also look into programming this in LISP so that it can access itself and modify itself while running.

Reading Test
Aoccdrnig to a rscheearch at cmabrigde uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the only important thing is that the first and the last letter be in the right place. the rest can be a total messa nd you can still read it without a problem. This is because the human mind does not read every letter by itelsf, but the word as a whole.

*Consciousness - A state where you are capable of observing and learning from the observation.
*Learning - A infinitively recursive process of imitation and experimentation (which leads to more "learning")
*Intelligence - A state of being conscious and able to learn. The pace of learning defines the intelligence quotient.
*Thought - The reaction to what was observed. The actual thought is based upon what was learned.

An AI's awareness of Self and the World: a challenging concept. Emotions: Even more challenging.

When you see an old grandfather clock running by turning gears throgh the energy from winding, you observe and learn from it. Sometime in the future, you are able to apply a similar concept to something else. If the clock was covered, then you'd have to go through the hassle of removing the cover to learn about it's working. Most people wont do this and thus, this covered approch becomes a wall to progression and learning. Closed-Source commercial programs are doing just the same. Open Source interpreted programs that are interpreted; those that the user can see without much hassle; will drive forward a considerable evolution in programming.

Does intelligence and thinking start before we are even born?? Aldous Huxley's Brave New World gets me thinking that at least a bit of "conditioning" is necessary for AI's to work properly. This would be where, at the start, the AI just observers everything without actually reacting. These observations set the foundations, the instincts, for further learning.

*NOTE* : I'm using a different type of Neural Network that doesnt use any sort of "Weights". It's just many independent nodes linked to related nodes/words. When asked a question, the question is sent to the node and it adds it's part of the answer to the answer String, determines what the next node in line is and calls it. This creates a train of thought. The answer will be based on not just the current question, but on the whole conversation. The "recent memory" object is shrunk down into snippets after the conversation and shrunk down even more after sleep.

All the Neural Netowkring of AI's can get very messy. That's why Sleep is necessary. The information in the recent neural netowork is reporcessed. The useful thoughts and nodes are kept while the useless ones are disposed of. The random paths from node to node create the sequence that becomes the basic outline of Dreams.

The concept of Understanding can be "understood"(LOL) better through the Dictionary scenario. Sometimes when you look up a word, you get a definition that uses the same word to define it! For example:
Consumer: person who consumes, esp. one who uses a product.
The reader looks up the word consumer because they didnt know the word "consumer" or words associated with it such as "consume". This definition doesn't truly explain the word in terms they know and thus, the reader doesn't understand the meaning.
If a second definition:
"Purchaser of goods or services"
is given and the reader knows all of the words making up the definition, then they can understand the meaning of the word.

When an AI looks up the word Consume and finds the second definition, it'll break down it's definition into smaller parts such as "purchaser" "of" "goods" "or" "services". Then it loops through each of those words and looks up words related to it. The process continues for a set number of turns. Then, it creates a new node with the word "consume" and assosciates the words "of", "goods" "or" "services" to it. It then also adds the words it found through the sub-search.

Most of the words the AI associates with the word "consume" is irrelevent. During sleep time, it sorts through the words and narrows down what is important and what is not.
The AI now "understands" the word "consume" and it'll be able to use it in define it in many different ways (instead of "purchaser" it could say "buyer" which it earlier identified as a word associated with "purchaser" with similar meaning).
Mastering usage is a whole different issue. It has to go through a trial and error process to expand and narrow down it's associated words, categorize the word, create rules for the word when it's being used with words of certain categories.



Thoughts?? Opinions??? Suggestions??

5 comments:

Mark said...

Hi, Cobalt,

I found your blog through your comment on mine (which I appreciate) and I've been reading yours. It's truly interesting and has some very well founded ideas. Specially those that indicate a "starting from scratch with basic rules" and "sleep to reorganize".

The main problem with AI theories these days, in my opinion, is that they start "too high up". They want to reproduce an "adult intelligence" without realising that the learning has to start at a much earlier, much simpler model.

The way you were expressing all those "node" concepts made me think of my days programing in Miranda and Prolog. It was very interesting as a possibility, but the interpreters available lacked one very fundamental functionality in order to make them useful for AI: the ability to let the program re-write itself. That, however may be an interesting avenue to investigate.

Also, we tend to over-glorify the human brain. If you look at behavioural patterns, the concept in itself tells you that we aren't all that varied, all that intelligent at all. Very small portions of the population can have some moments of "enlightenmnent" which aren't the norm, but a spark of brilliance in an otherwise dull task-driven daily-accostumed machine-like behaviour patterns. That, as AI scientists should actually make us happy, since it should mean we are easier to reproduce as a model. Should we not start by a "stupid" AI before moving onto higher models?

Goals are also a good theory, but bear in mind that they can be very hard to represent. A goal can be "to be happy". But, how do we quantify happiness? Actually, happiness can be measured but the "importance" and "successfullness" of our goal achievement... if we are the type of person who enjoys achieveing things. Thus, "anger" and "dissapointment" could as easily be represented or achieved by low goal achievement and the speed at which goals fail. Anger could then be "directed" at the reasons for failure, wether they be people or situations. Maybe the concept that an AI would not be able to have feelings is but a myth created by the inability to "descipher" what feelings really are...

However, starting from "I am", I think, is not the right start. When does a baby realise "he/she is"? When does that thought become relevant? It could be just an "overdeveloped" instinct for survival developed from "I want" and "I need" which truly is the first thing a baby may think (though it may not be able to word it, the thought will be there in one form or another).

Also, categorization must be a product of thought, not the basis for it. Different people categorize concepts differently. The organization may even be something sotially learned, thus culturally linked to your environment. It may be better to impress the need for categorization rather than impressing the actual structure of that categorization.

The grandfather clock example is very good, but failed. The concept doesn't become "more understandable" if the cover is removed. We may not need to know how it works in order to understand what it does. It is a matter of iteration. Imagination may lead someone to try to figure out how without seeing it, thus may be the seed for a fresh idea or the repetition of an existing one (the concept of "mutation" and "thought" can come together here)

I would be very interested to know what kind of algorythm you would use to make the node selection in the "sleep". How do you know what is "useful" and what is not?

The dictionary model, as you so elloquently put it is not perfect, but it may work if you allow a certain recursiveness (as you in deed allow it). Thus understanding can be understood the way recursiveness can be recursively understood (LOL).

Again, it all comes down on How to represent concepts mathematically. In all ernesty, much data has to be taken into account in order to "make up" a concept. And phisical perception is necessary for much of that conceptuallization. Higher-order conceptuallizing, after all, comes from higher beings that have already mastered their perceptions and can thus make better associations between phisical perceptions and their conceptual consequences.

Great blog, Cobalt. I will be following this and input my own thoughts as I can.

Cobalt said...

"the ability to let the program re-write itself. "
That's the main reason I'm looking into LISP. At first I was doubtful, but once I got over the brackets (((())))) it's proved itself to be a very good language to program and AI in.

"Should we not start by a "stupid" AI before moving onto higher models?"
Even a "stupid" human is smarter than most animals, and we can't even program most (not all) animals.

"Goals are also a good theory, but bear in mind that they can be very hard to represent. A goal can be "to be happy". "

I'm actually aiming for a much more basic versions capable of solving logical and mathematical problems through it's own knowledge (without accessing the Java's math system).

"Maybe the concept that an AI would not be able to have feelings is but a myth created by the inability to "descipher" what feelings really are... "

(http://www.phy.syr.edu/courses/modules/MM/brain/brain.html)
"Galileo, in 1623, put forward the view that science should only be concerned with primary qualities, those of the external world that could be measured or weighed. So-called secondary qualities, such as love, beauty, meaning and value, were said to lie outside the realm of science. Descartes, himself, supported this idea and proposed two categories: mind and matter. The matter category related to physical or extended substance, the mind category to thinking substance - that which is unextended and indivisible."

I've come to the conclusion that Artificial Intelligence will be possible in the near future, but Artificial Emotions and Artificial Consciousness may never be possible simply because of their abstract nature. We might be able to put in systems that simulate feelings and manipulate the reactions to represent those feelings... But they are just integers or variables representing feelings and not actual emotions.

I just used the "I Am" input as an example.

"Also, categorization must be a product of thought, not the basis for it."
I'm aiming for two types of categorization: Internal and External. The external categorization is the one that you apply in your daily life as a product of thought. But I believe that you also categorize information internally/subconsciously and the create rules that govern the behavior of that category. This system of rules and relations between categories is about the only thing I could come up with to express words and situations as concepts. Such a system would override the need to have to program in things like rules of grammar, rules of mathematics ect. if a good enough system for categorizing nodes and creating rules can be accomplished.

The main reason I had the grandfather clock example there is because I don't believe a true AI can be accomplished individually. I believe in making all of my AI programs Open Source so that others can use it and gain something from it. This is also the reason why I'm openly expressing my thoughts and concepts here, so that it helps drive forward the overall progression of the AI movement in some little way.

sgcleader said...

Hi Cobalt,

I recommend that you complete your study of Common LISP. Many of the example downloads are written in LISP.

Another recomendation is to read the Artificial Intelligence: A Modern Approach. This book will provide a baseline of working knowledge.

Mark said...

"LISP"
I never actually got into it. I looked it over. But the subject in uni was Prolog, so I learned that. I managed some pretty cool stuff on it (my project, a 4-part musical harmonizer in Prolog)

"stupid AI"
Actually, I meant more of a "stupid animal" AI than a "stupid human" AI. That is, to start in intelligence rather than self-consciousness.

"feelings"
I tried to access the URL you pasted, but I'unauthorised. I, however, think that a certain level of "feelings" will be part of any AI. "Happiness" and "Sadness" or, if you will, "Satisfaction" and "Unsatisfaction" would be a byproduct of goal achievement/failure, and hence, motivation. Feelings, I think, develop from that and the concept of "needs" and "dislike" as a factor of thigs that help achieve goals or make failure more likely.
"Not achievable due to the abstract nature"? Try to explain a colour to a blind person. Colour is an abstract if you lack the perceptual data to draw knowledge from. Feelings are also an abstract from perceptual data, but not from a physical sense, but if we can adapt the meaning of "perceptual data" a little, we may find that we already have the answer (results, byproducts, etc.)

"Categorization"
The theory is quite sound but, again, I think you're getting ahead of yourself trying to sort out the abstract meaning of a word from the word itself instead of going the natural way. When a child learns a word, it doesn't start by trying to categorize it, but rather, there is stimula and the knowledge of the word comes later. "Mum", "Dad" are words only applied to the concept of mother and father when speech is achieved, however, the concept exists much earlier than that (pre-born??).

Maybe a simpler example may ilustrate, since "mum" is much more complex than it seems. Take "water" for example. we first know the concept through phisical stimula. We can't even ask for a drink coherently for months! But in our brain, water is already a known concept.

What I mean is that words and their meaning must have "more meaning" than just a relation to a category. Phisical stimula may be necessary in order to categorize concepts. Or at least some concepts.

"Car" is a useless concept unless you know it's a form of transportation, but "transportation" is useless without the concept of phisical location!

"Grandfather clock"
Ah! Missed the point! I completely agree with you there. Things do develop a lot faster when you get lots of people involved. Open source is definitely the way to go.

"Artificial Intelligence: A Modern Approach"
Amazing!! I actually own that book! It was part of the required reading for my uni degree!

Cobalt, here's my email address: techiorATgmail.com

Cobalt said...

Sorry, the link didn't come up right. The link was "out of bounds", here it is:
http://www.phy.syr.edu/courses/
modules/MM/brain/brain.html

The site offers quiet a few useful resources that cover a lot of ground on Artificial Intelligence (Both the conceptual and programming aspects).

I still have the LISP tutorial downloaded to my hard disk, so I'll check it out. It's just so different from what I'm used to!

The reason I'm pushing the Categorization concept is because there needs to be some way to access the information in the brain quickly and efficiently. After a bit of research, I found that the brain categorizes to a certain level also.
http://www.ingentaconnect.com/
content/ap/ni/2002/00000016/
00000002/art01109

I also found this:
http://www.ncbi.nlm.nih.gov/
pubmed/12807412

But they're talking Greek for all I know! It talks something about categorization...

I think that the approach of mimicking the Human Brain or an Animal Brain should only be used to a certain extend at the base concepts since we are dealing with two different types of systems: The brain, a highly efficient parallel processing system and the average AI program with just a few threads for running different things. Our brains show one way to achieve intelligence. (Probably not the best approach and certainly not the worst). If we can create a simpler system that works almost as good, it's a start!

I've thought a lot about your sense based theory. So far, I can't find a way to disprove it and I can't find a way to implement it (because once that data is stored, it's just data from the senses. How will it associate meanings with it??)

If you teach a child what a "car" is by showing it a picture of a car, playing the sound of a car and explaining the concept of a car being used to take you places, the child would not truly understand the concept beyond what you told them + some aspects of imagination such as walking cars or something. The child would have to actually EXPERIENCE riding in a car before he/she can start to vaguely understand the concept. Concepts could be explained through experiences, but again, experiences should be more than what you see or hear, but (lol) something more! (just can't quiet figure out what)

You're right about the pre-born concepts. I heard about an experiment where a scientist stood in front of a baby duckling as it was hatching. The scientist was the first thing the duckling saw and thus, it assumed the scientist to be it's mother.

These concepts exist within us (in our genes?), but we have to be careful putting such things into the program because once you pass a certain point, the program is no more intelligent than a parrot styled chat-bot~)