Sunday, August 31, 2008

Learning to Teach

Developing AI's that can "Learn" has always been a challenge. One way to test if a program is truly "Learning" is to see if it can Teach a similar program the same material without "unlearning" the things it has learned.

The problem with most AI's is that they assume that the input being sent to them is from a reliable source. So if someone tells them "The world is flat", it'd believe it. If that program was to teach a second program, it'd get dumber by misinterpreting the learning process (asking questions, making both true and false assumptions) as valid input while the student program will get smarter learning from the input. At the end, both programs would end up wrong, thinking that the world is neither flat nor round, but instead a cube.

The question proposed here is whether the program can learn and then teach what it has learned effectively to a second program without affecting it's own knowledge.

Categorization and judgment based on past experiences could possibly be the solution. (See "Concept AI" & "Concept AI II")

Monday, August 25, 2008

Pattern Analysis

This is a basic Pattern Analysis program I wrote. It's just version 0.03 so it's capabilities are very limited. Sorry for the formatting :(

Usage: Just type in one number at a time, pressing enter after each number. Type ? to calculate the next item in the list. The program clear it's memory and start over with a new sequence.

Capabilities: I can find patterns such as:
1,1,1,1,1,? (1)
1,2,3,? (4)
1,4,9,16,25,36,49,64,81,? (100) Square of each number found using spacing pattern.
7,19,37,61,91(127)
1,8,27,64,125,216(343) Cube of each number

Right now, It's fairly bad at exponents and it cant figure out the Fibonacci sequence either :(


import java.util.Scanner;
import java.util.*;
import java.io.*;


public class PatternRecognition{

public static void main(String args[])
{
System.out.println("AI Pattern Analysis Module. V 0.03");
while(true){
ArrayList intSequence = new ArrayList();
String str = getInput();
while(!str.equals("?")){
int temp = (Integer) Integer.valueOf(str);
intSequence.add(temp);
str = getInput();
}
System.out.println("Next number will be "+nextInt(intSequence));
}
}
public static String getInput()
{
Scanner input = new Scanner(System.in);
System.out.print("# :");
String inputString = input.nextLine();
return inputString;
}

public static ArrayList nextInt(ArrayList list){
ArrayList nextInt = new ArrayList();
ArrayList spaces = getSpaces(list);
if(linear(spaces)){
nextInt.add(getNextLinear(list));
System.out.println("Linear pattern found");
}

if(true){
boolean done = false;
int counter = 0;
int lastAdd = 0;
while(!done){
counter++;
if(linear(spaces)){
done = true;
//return getNextLinear(list,spaces);
lastAdd+=(Integer) spaces.get(spaces.size()-1);
int lastInList = (Integer) list.get(list.size()-1);
nextInt.add(lastInList+=lastAdd);
System.out.println("Spacing pattern found");
}
else{
lastAdd+=(Integer) spaces.get(spaces.size()-1);
ArrayList sp2 = copyArray(spaces);
spaces = getSpaces(sp2);
}
}

}

int sequenceWidth = repeatingCheck(list);
if(sequenceWidth > 0){
// System.out.println("Sequence repeats every "+sequenceWidth+" digits.");
nextInt.add(getNextRepeating(list,sequenceWidth));
System.out.println("Sequential pattern found");
}

return nextInt;
}

public static ArrayList copyArray(ArrayList list){
ArrayList rtrn = new ArrayList();
for(int k = 0; k < list.size(); k++){
rtrn.add(list.get(k));
}
return rtrn;
}

public static ArrayList getSpaces(ArrayList list){
ArrayList spaces = new ArrayList();
int previous = 0;
for(int k = 0; k < list.size(); k++){
int i = (Integer) list.get(k);
if(k == 0){ previous = i;} // Initialize previous
else{
spaces.add(i-previous);
previous = i;
}
}
return spaces;
}

// Checks if it's a set of evenly spaced numbers
public static boolean linear(ArrayList list){
if(list.size() >= 1){
int reference = (Integer) list.get(0); // Initialize a default value for space
for(int k = 1; k < list.size(); k++){ // starts at 1 because 0 is init value
int i = (Integer) list.get(k);
if(i != reference){ return false;}
}
}
return true;
}

public static int getNextLinear(ArrayList list){
if(list.size() >= 2){
int i1 = (Integer) list.get(0);
int i2 = (Integer) list.get(1);
int difference = i2-i1;
int rtrn = (Integer) list.get(list.size()-1);
rtrn += difference;
return rtrn ;
}
else if(list.size() >= 1){return (Integer)list.get(0);}
else{return 0; }
}

public static int repeatingCheck(ArrayList list){
ArrayList sequence = new ArrayList();
boolean repeating = false;
boolean oneRepeat = true;

sequence.add(list.get(0));
for(int k = 1; k < list.size()-1; k++){
if(repeating(list,sequence)){
return k;
}
else{
sequence.add(list.get(k));
}
}
return 0;
}

public static boolean repeating(ArrayList list, ArrayList sequence){
int k2 = 0;
for(int k = 0; k < list.size(); k++){
int l = (Integer) list.get(k);
int s = (Integer) sequence.get(k2);

k2++;
if(k2 > sequence.size() -1){ k2=0; }

if(l != s){ return false; }
}
return true;
}

public static int getNextRepeating(ArrayList list, int sequenceWidth){

if(list.size() > sequenceWidth){
ArrayList sequence = new ArrayList();
for(int k = 0; k < sequenceWidth; k++){
sequence.add(list.get(k));
}

int k2 = 0;
for(int k = 0; k < list.size(); k++){
int l = (Integer) list.get(k);
int s = (Integer) sequence.get(k2);

k2++;
if(k2 > sequence.size() -1){ k2=0; }

}
return (Integer)sequence.get(k2);
}
return 0;
}
}



How can I improve this???

Concept AI (part 2)

This is the latest addition to the brainstorming file. Please not that the information not sited is just my concepts, they might not always be realistic or accurate.

http://www.phy.syr.edu/courses/modules/MM/brain/organizations/organizations.html#work
And sub/super site
"A simple explanation of the processing step is that the cell sums up the incoming signals and produces an output signal only if this sum exceeds some threshold; i.e. only if the total input signal is big enough will the cell `fire' an output signal to its neighbors.""new memories are stored not on individual neurons but by adjusting the strengths of connections between neurons.""the connection between two neurons will strengthen if more often than not the two neurons fire together. This is often called the Hebb rule.""The nodes in the network are vast simplifications of real neurons - they can only exist in one of two possible "states" - firing or not firing.""Certain nodes will be always "on" and others "off"."

Emotions affect the threshold. Causing "speechlessness" when too happy, lower threshold value for certain categories (associated with anger) when angry ect.

The connections between the neurons are also important. The higher the current that passes through the connection, the more it's strength.

A, B, C are neurons.
A is connected to b and c.
B is connected to a and c.
C is connected to a and b.
During sleep time, this system can be reworked to:
A is connected to b.
B is connected to c.
C is connected to b.

Once a neuron receives a signal, it waits to see if the signal will add up to the threshold. If not, it returns false, otherwise true. This feedback will help improve the heuristic algorithm that direct the connections.

Techior2
"cannot be limited to words, but must also extend to the concepts those words communicate"

Research "Hopfield Networks" for memory.

The input gates should use pattern recognition to simplify the complex input and direct the general flow of input to the right nodes.

Friday, August 22, 2008

Concept AI

Whenever I work on a digital project, I create a folder for it and make a notepad file for brainstorming and just scratch work. This is the current version of the brainstorming file for the AI-Concept I'm working on.


What separates machines from humans?? The answer that comes first to mind is that while we have free will. The power to have our own openions and to create new ideas out of scratch. Although that's how it seems, it's not the case. We humans cant create ANYTHING! We can only take what exists in nature and manipualte it into something new. Think of this. A raw piece of wood is cut from a forest and made into the paper that becomes the raw canvas for an artwork. We are able to make paper out of wood because we were observative. The artist is able to make an artwork not because he or she is CREATive, but because they have the power to observer and judge. Sitting in front of that blank canvas, hundreds of ideas rush into the artist's brain. But from where?? All of these ideas are from old experiences, in one form or another. Many times, new ideas are "created" from the combination of two concepts or situations into one. Many times these founding situations and ideas that make up the final ideas is not from the person themselves, but form someone else. All of human kind's achivements were possible because we were open minded. If we didnt share our findings, each of us still would be trying to invent the wheel by overselves a thousand different ways.

When faced with a blank canvas, artists often start off by drawing a single line. And then another. The image of these lines trigger memories of objects that could be formed from the combination of the lines. Looking up at a summer sky, we see the shapes of different objects on the faces of clouds, even though they are just clouds.

As children, we often imitate those older than us. We mimic the actions of those around us and learn form it. These process of observing and imitating, physically or mentally, allows us to learn. A process of rewards and punishments separating the good learning from the bad shapes our personality. Creating an program that can imitate what we tell it is fairly easy. So how do you go from an imitator into an Artificially Intelligent program? With goals, both big and small.

Goals are an important aspect of being human. We as humans are proud to announce that we have free will. "If I want, I can walk over to the front of the class. Punch the professor and run out, yelling and making a mess in the hallways." Yes. You can (as in ability to do something) do that. But WHY don't you? A person in their right minds would know that such actions can have serious consequences. It would most likely get you expelled from that school and subconsciously, you despise that idea since it would interfere with the goals you have in life. Notice I said the "goals you HAVE" in life. This is because many times, these goals are not your own. Your parents, friends and the society around you have a a lot on influence on the goals you set, especially those set when you are young and still prone to some aspects of learning from imitation.

Goals, combined with past experiences the program learnt, is the major aspect of an AI. You teach the program how to draw one line, and tell it to draw two lines. After many failures, it'd accomplish that goal. Learning from it's mistakes to accomplish the next, slightly harder task of drawing three lines, faster with fewer errors. The series of errors, punishments and rewards will develope into the AI's sense of judgment that judges good from bad. Although I enjoy watching Sci-Fi stories such as Terminator, Sarah Connor chronicles, I don't think such a situation would happen because of these very reasons.

-Unfinished. Aspects of Creativity, Thinking and Memory to be explained later.

It's been 4 or 5 days since I started the AI project. School will open soon, so I'll have to stop this and move on for now. I'll scratch down the basic outlines of the other ideas that I had so that if & when I start again on this project, I wont have to start from scratch.

Input:
Where am I
I am there
Who am I
I am Cobalt
What am I
I am human
Where are you
you are here
who are you
you are AI
what are you
you are a program
you are a module

The goal is to get the program to get a categorization of
1: I, am
2: You, are
3: where, I, there
4: where, you, here
5: who, I, Cobalt
6: who, you, AI
7: what, I, human
8: what, you, program, module
and so on

Later on (after more input), the categorization might look like the following.

1: who, what, where, when, why, how
2: where, US, Earth, Solar System
3: are, am, is was
and so on....

Make each node complete objects instead of just one word.

With the input "you are good" and "how are you" try:
you(are, good)
are(you, good)
good(you, are)

How(are, you)
are(How, you)
you(are, how)

That should let it associate Good with How. This system breaks down when new unknown words are introduced too fast
and the program confuses those words with the answer.

Memory: categorized by tags. Recent memories kept in RAM. snipeets from old events (more snippets of recente events than of old ones) also in memory. Snippets can be used to search for the rest of the memory as well as events directly before and after it and related events.

Creativity: combine random snippets (which lead to other snippets) in a comprehendible way. Apply concepts from one event/lesson on others. Use format from similar challenges/tasks/experiences. The snippets lead to a line of thought, sometimes creating recursion since in the process of looking through the snippets, the snippets are pushed up in the stack to be renewed/"newer". This creates the "That song is stuck in my head" scenerio.

Thinking: multiple threads jumping from topic to topic. Each topic is a .txt file with tags to related topics. sentences broken into web system.

Logic: basic skills of logic must be built into the system. a=b; b = c; a = ?
Pattern recognition should also be used

A heuritical system of punishments and rewards. Use this to organize the information better by re categorizing and eliminating the link from node to node when the heuritical cost becomes unacceptably high.

Also look into programming this in LISP so that it can access itself and modify itself while running.

Reading Test
Aoccdrnig to a rscheearch at cmabrigde uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the only important thing is that the first and the last letter be in the right place. the rest can be a total messa nd you can still read it without a problem. This is because the human mind does not read every letter by itelsf, but the word as a whole.

*Consciousness - A state where you are capable of observing and learning from the observation.
*Learning - A infinitively recursive process of imitation and experimentation (which leads to more "learning")
*Intelligence - A state of being conscious and able to learn. The pace of learning defines the intelligence quotient.
*Thought - The reaction to what was observed. The actual thought is based upon what was learned.

An AI's awareness of Self and the World: a challenging concept. Emotions: Even more challenging.

When you see an old grandfather clock running by turning gears throgh the energy from winding, you observe and learn from it. Sometime in the future, you are able to apply a similar concept to something else. If the clock was covered, then you'd have to go through the hassle of removing the cover to learn about it's working. Most people wont do this and thus, this covered approch becomes a wall to progression and learning. Closed-Source commercial programs are doing just the same. Open Source interpreted programs that are interpreted; those that the user can see without much hassle; will drive forward a considerable evolution in programming.

Does intelligence and thinking start before we are even born?? Aldous Huxley's Brave New World gets me thinking that at least a bit of "conditioning" is necessary for AI's to work properly. This would be where, at the start, the AI just observers everything without actually reacting. These observations set the foundations, the instincts, for further learning.

*NOTE* : I'm using a different type of Neural Network that doesnt use any sort of "Weights". It's just many independent nodes linked to related nodes/words. When asked a question, the question is sent to the node and it adds it's part of the answer to the answer String, determines what the next node in line is and calls it. This creates a train of thought. The answer will be based on not just the current question, but on the whole conversation. The "recent memory" object is shrunk down into snippets after the conversation and shrunk down even more after sleep.

All the Neural Netowkring of AI's can get very messy. That's why Sleep is necessary. The information in the recent neural netowork is reporcessed. The useful thoughts and nodes are kept while the useless ones are disposed of. The random paths from node to node create the sequence that becomes the basic outline of Dreams.

The concept of Understanding can be "understood"(LOL) better through the Dictionary scenario. Sometimes when you look up a word, you get a definition that uses the same word to define it! For example:
Consumer: person who consumes, esp. one who uses a product.
The reader looks up the word consumer because they didnt know the word "consumer" or words associated with it such as "consume". This definition doesn't truly explain the word in terms they know and thus, the reader doesn't understand the meaning.
If a second definition:
"Purchaser of goods or services"
is given and the reader knows all of the words making up the definition, then they can understand the meaning of the word.

When an AI looks up the word Consume and finds the second definition, it'll break down it's definition into smaller parts such as "purchaser" "of" "goods" "or" "services". Then it loops through each of those words and looks up words related to it. The process continues for a set number of turns. Then, it creates a new node with the word "consume" and assosciates the words "of", "goods" "or" "services" to it. It then also adds the words it found through the sub-search.

Most of the words the AI associates with the word "consume" is irrelevent. During sleep time, it sorts through the words and narrows down what is important and what is not.
The AI now "understands" the word "consume" and it'll be able to use it in define it in many different ways (instead of "purchaser" it could say "buyer" which it earlier identified as a word associated with "purchaser" with similar meaning).
Mastering usage is a whole different issue. It has to go through a trial and error process to expand and narrow down it's associated words, categorize the word, create rules for the word when it's being used with words of certain categories.



Thoughts?? Opinions??? Suggestions??

Why

Why go through all the hassel of writing a blog? Well, first of all, I want to document my thoughts and ideas so that I can look back on them someday. Secondly, to share knowledge. Under my desk at home, I have a cardboard box filled to the brink with old bits and pieces of paper. They are all that remains of my many different projects/ideas about many different things. Most, never got off the drawing board. Others are still "under construction", and probably will remain without much changes until the paper starts to fade away. I've realized that many of these projects will never get done by me, simply because of the lack of time. Other than letting these concepts go to waste, I'd like to publish them here so that someone somewhere may benefit off of it. Let me know if you're working on something similar. I'm curious to see if they'll actually work!