A working AI: Mission Accompished??

I'd taken a break from all the AI business for a few weeks. I'm surprised at how much I got done working with a clear head. I've run some initial tests through my concept for the Biological Based but not Weights Based Neural Network. The results look unbelievably amazing. The program starts out with absolutely no info and then learns everything quickly and efficiently. Here's what I've added to the log (read Concept AI part 1 and 2 first).

******

Miller's Law: A person can only keep 7 plus or minus 2 items in mind at one time.

http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html#Introduction%20to%20neural%20networks

Firing rules to figure out weather a synapse fires based upon the input.

http://www.spiegel.de/international/spiegel/0,1518,466789,00.html

The nerve cells open or close it's ion channels before it fires.

*Going back to some more basic research about the fundamentals of the Brain itself.

http://www.enchantedlearning.com/subjects/anatomy/brain/Neuron.shtml
The brain has about 100 billion neurons. Then there are glial cells, which provides support (what kind?). The sizes of the Neurons differ.
Neruron: Cell Body, Dentrites (signal receivers), projections (axon, which conduct the nerve signal). Axon Terminals transmit electro-chemical signals over the gap known as the synapse.
Dentrites bring info to the cell body. The axon takes information away from the cell body. Bundles of them are called nerves or as nerve tracts or pathways. Dendrites branch from the cell body and receive message.
One neuron has about 1000-10,000 synapses, meaning it communicates with that many other neurons.

Glial cells: Nerve cells that don't carry nerve impulses. They basically manufucture stuff for the neurons.

Basically:
Dendrites: Input Connections (one or more)
Axon: Output Connection (only one). So it's like a linked list where the last elements connects to the first element. One axon can speak to multiple dentrites on the reveiver.
Axon Terminals: (more than one)
Synapse: Space between input & output connection

Hebbs Rule: Cells that fire together, wire together.
A synapse's strength depends on the number of Ion-Channels it has.

9-26-08
6:47

It works! I went through the previously non-working AI program and reworked the whole thing. It's version 0.03 now. I'm amazed that it was able to work so efficiently when I put in the charges concept into it. There are still many lil bugs to work out. I've just tested it with some lettter comparisons. The current data set is just 2 sets of 3 letters. It goes like this:
a b
b c
/end
d e
e f
/end

Then I can go in and reset the charges and add another connection linking the two node-network segments toghether into one.

/rnodecharges
c d

Before those statements, a connection to node a would activate node b and c, but after the statements, it activates b, c, d, e and f (with smaller charges as you go futher down the line.)

The main bug now is that the connection between c d is looking too strong. IDK. Maybey it's supposed to be like that. D is emitting a charge similar to that of a.

10:59. After examining it a bit more, I've found and fixed some more bugs. I'm using a new data set now. It goes like this:

Greek is spoken in Greece
/end
Greece is in England
/end
The Greeks created the Olympics
/end
Athens is the capital of Greece
/end
Sparta is the enemy of Greece
/end
Where refers to the places like Greece Athens and Sparta
/end

After clearing the charges, I ask it:
where is Greek spoken
/end

And I got the result:
/end
/charges
greek 1.0
is 1.0
spoken 1.0
in 2.4935020205414458
greece 3.4648806640625
england 1.4565926569239502
the 0.9368278391792345
greeks 0.43610654105691715
created 0.6253771713864278
olympics 0.6719345918743638
athens 1.8224265625
capital 1.4442937950131418
of 1.4180175011313327
sparta 1.6934484375000003
enemy 1.3801804402274132
about 1.2319124453341073
spartans 1.3034834714624521
where 1.0
refers 0.55
to 0.72875
places 0.75366328125
like 0.7785765625000001
and 0.85331640625

This CLEARLY gives you Greece as the answer to the question. It's amazing how quickly it can learn. All of this was achieved with a minimal amount of processing and with NO INFORMATION (except the /end and the other /commands) hard-coded in. Such clear results with just a single repetition!
I think I'm onto something here. I'm now trying to incorporate larger and larger datasets to see how it'll handle it. Oh, by the way: The dataset is completly made up. I have no clue what the capital of Greece is. This is just an example.

9/27/08
11-34 AM
I'm now working with a data set of 40 Nodes creating 221 Connections (Which becomes 185 when sleep is applied with a 0.001 charge). A basic version of the Sleep method has been created to clean up the weak nodes to increase efficiency. I've also put in a Feedback system (+ or - depending on the answer) that strengthens or weakens the node based upon the feedback. It's still looking amazingly good. Will keep updating!
1:47 PM
Grammar is the real trouble here. grammar words like Of, and, or, an, a, the ect. is connected to a whole bunch of different nodes, so it creates some unnecessary charges.

An Idea for improvement.
Greece refers to a place
Where refers to place and Greece, China, England, America, India
Place refers to Greece
So rework the system so
Where refers to place
place refers to India, Greece, china, england, America ect.

This reduces the number of connections. Adding this to the sleep system will allow you to delete the unnecessary (unacceptably weak) connections and the looping connections.
I'm still amazed at it's efficiency though. It's able to comprehend simple inputs like:
My name is Cobalt
Your name is Ron New
What refers to Name
What is My name
?
Cobalt
What is your name
?
Ron New

Ron New is New Ron spelled backwards and with a w instead of a u.
I've now moved it up a notch to read files. Each sentence must be put in a different line, or else it gets REALLY slow.
The current data set is:
184 Nodes with 1678 connections!
The system resources taken up (At the start of the program):
Java.exe PID: 1472 CPU: 00% CPU Time: 00:00:03 Mem Usage: 348 K

This fluctuates between as low as 384 KB to as high as 10,000 KB. System.gc is called a few times in the program so that the memory is used wisely.

The network is working, but it still needs work. I'm going back to more research to apply new concepts into the network to improve it.

http://www.learnartificialneuralnetworks.com/
Action potentials are the electric signals that neurons use to convey information to the brain. All these signals are identical. Therefore, the brain determines what type of information is being received based on the path that the signal took. The brain analyzes the patterns of signals being sent and from that information it can interpret the type of information being received.

It turns out one aspect of my program, the weakening of signals, is not consistent with the biological system. "There are uninsulated parts of the axon. These areas are called Nodes of Ranvier. At these nodes, the signal traveling down the axon is regenerated. This ensures that the signal traveling down the axon travels fast and remains constant (i.e. very short propagation delay and no weakening of the signal)."

Instead of letting one node send pretty much equal amount of strength to it's connected nodes, how about a system where the signal strength is determined by percentage. This would mean that all charges must originate from the initial charges. The output would then be smaller numbers. This might be good or this might be bad. Let's find out!

Nop, that doesn't work. I don't know why, but it doesn't work. I think I was just lucky hitting the target on that first try after the break. Some small, ever so minute changes can sometimes throw the system off.

*********************

Mission Accomplished?? Not exactly. Answering one question (will this work) leads to a thousand new questions. I'm creating a hand-made knowledge base for it to work out off (some basic grammar concepts and general knowledge). The program can handle any information in any language that you teach it! The charges concept has been accomplished (sorta), so now I have to work on some other concepts to make the system independent. It has to take the answer and then figure out how it's going to be applied or presented as an answer. There are a hundreds of applications for the charge-based neural network concept. I assure you, as soon as I'm done with a stable version of the program, it'll be released as an Open Source Program :) I'm planning on releasing it as part of my other project known as OSCEAN (Open Source Content Environment and Abstract Network) that will put the AI to work in a few logical application. But until then, the actual code will have to be kept under wraps. Just for the thrill of suspense when it's finally released!

But now, please reply with some other types of tests that you want me to run on the system and I'll post you the results. Wish me & Ron (NewRon get it?) luck!

Levels Of Thinking

Levels of Thinking
1. When your environment is unfavorable, you spend most of your time thinking about the basic needs for survival: food, water, air.

2. When you have food, water and air, you think about other things that make life easier: Electricity, Technology, Science, Mathematics

Suppose that that goal has been achieved as well, and somehow, everyday-life was made as easy as it possibly could be. Let's say, we have robots that cook for us, clean for us, farm for us ect. We won't have to do anything ourselves.
3. This will lead us to a new level of thinking that'll bring forth a new era in Human History. I'm not sure what that will be though.

But, will we want to be just idle humans?? Will we want to waste our lives consuming without producing? I believe that Boredom makes us human. We are not satisfied by what we have. We feel that we are endowed with a higher purpose to do something else. Always, something else. So it just may be that we humans will never reach that 3rd level of thinking. But an Artificial Intelligent program, with no needs, goals or desires, might be able to...

Still here!

Yep, Still Here! I've been caught up with school the for the past few weeks and Hurricane IKE left us without power for 7 days! I also had school off the entire week, but without power there was nothing else to do except think (like the good ol` days. Actually think for a while!). I'll be posting all about it in my next post. Stay put!

Learning to Teach

Developing AI's that can "Learn" has always been a challenge. One way to test if a program is truly "Learning" is to see if it can Teach a similar program the same material without "unlearning" the things it has learned.

The problem with most AI's is that they assume that the input being sent to them is from a reliable source. So if someone tells them "The world is flat", it'd believe it. If that program was to teach a second program, it'd get dumber by misinterpreting the learning process (asking questions, making both true and false assumptions) as valid input while the student program will get smarter learning from the input. At the end, both programs would end up wrong, thinking that the world is neither flat nor round, but instead a cube.

The question proposed here is whether the program can learn and then teach what it has learned effectively to a second program without affecting it's own knowledge.

Categorization and judgment based on past experiences could possibly be the solution. (See "Concept AI" & "Concept AI II")

Pattern Analysis

This is a basic Pattern Analysis program I wrote. It's just version 0.03 so it's capabilities are very limited. Sorry for the formatting :(

Usage: Just type in one number at a time, pressing enter after each number. Type ? to calculate the next item in the list. The program clear it's memory and start over with a new sequence.

Capabilities: I can find patterns such as:
1,1,1,1,1,? (1)
1,2,3,? (4)
1,4,9,16,25,36,49,64,81,? (100) Square of each number found using spacing pattern.
7,19,37,61,91(127)
1,8,27,64,125,216(343) Cube of each number

Right now, It's fairly bad at exponents and it cant figure out the Fibonacci sequence either :(


import java.util.Scanner;
import java.util.*;
import java.io.*;


public class PatternRecognition{

public static void main(String args[])
{
System.out.println("AI Pattern Analysis Module. V 0.03");
while(true){
ArrayList intSequence = new ArrayList();
String str = getInput();
while(!str.equals("?")){
int temp = (Integer) Integer.valueOf(str);
intSequence.add(temp);
str = getInput();
}
System.out.println("Next number will be "+nextInt(intSequence));
}
}
public static String getInput()
{
Scanner input = new Scanner(System.in);
System.out.print("# :");
String inputString = input.nextLine();
return inputString;
}

public static ArrayList nextInt(ArrayList list){
ArrayList nextInt = new ArrayList();
ArrayList spaces = getSpaces(list);
if(linear(spaces)){
nextInt.add(getNextLinear(list));
System.out.println("Linear pattern found");
}

if(true){
boolean done = false;
int counter = 0;
int lastAdd = 0;
while(!done){
counter++;
if(linear(spaces)){
done = true;
//return getNextLinear(list,spaces);
lastAdd+=(Integer) spaces.get(spaces.size()-1);
int lastInList = (Integer) list.get(list.size()-1);
nextInt.add(lastInList+=lastAdd);
System.out.println("Spacing pattern found");
}
else{
lastAdd+=(Integer) spaces.get(spaces.size()-1);
ArrayList sp2 = copyArray(spaces);
spaces = getSpaces(sp2);
}
}

}

int sequenceWidth = repeatingCheck(list);
if(sequenceWidth > 0){
// System.out.println("Sequence repeats every "+sequenceWidth+" digits.");
nextInt.add(getNextRepeating(list,sequenceWidth));
System.out.println("Sequential pattern found");
}

return nextInt;
}

public static ArrayList copyArray(ArrayList list){
ArrayList rtrn = new ArrayList();
for(int k = 0; k < list.size(); k++){
rtrn.add(list.get(k));
}
return rtrn;
}

public static ArrayList getSpaces(ArrayList list){
ArrayList spaces = new ArrayList();
int previous = 0;
for(int k = 0; k < list.size(); k++){
int i = (Integer) list.get(k);
if(k == 0){ previous = i;} // Initialize previous
else{
spaces.add(i-previous);
previous = i;
}
}
return spaces;
}

// Checks if it's a set of evenly spaced numbers
public static boolean linear(ArrayList list){
if(list.size() >= 1){
int reference = (Integer) list.get(0); // Initialize a default value for space
for(int k = 1; k < list.size(); k++){ // starts at 1 because 0 is init value
int i = (Integer) list.get(k);
if(i != reference){ return false;}
}
}
return true;
}

public static int getNextLinear(ArrayList list){
if(list.size() >= 2){
int i1 = (Integer) list.get(0);
int i2 = (Integer) list.get(1);
int difference = i2-i1;
int rtrn = (Integer) list.get(list.size()-1);
rtrn += difference;
return rtrn ;
}
else if(list.size() >= 1){return (Integer)list.get(0);}
else{return 0; }
}

public static int repeatingCheck(ArrayList list){
ArrayList sequence = new ArrayList();
boolean repeating = false;
boolean oneRepeat = true;

sequence.add(list.get(0));
for(int k = 1; k < list.size()-1; k++){
if(repeating(list,sequence)){
return k;
}
else{
sequence.add(list.get(k));
}
}
return 0;
}

public static boolean repeating(ArrayList list, ArrayList sequence){
int k2 = 0;
for(int k = 0; k < list.size(); k++){
int l = (Integer) list.get(k);
int s = (Integer) sequence.get(k2);

k2++;
if(k2 > sequence.size() -1){ k2=0; }

if(l != s){ return false; }
}
return true;
}

public static int getNextRepeating(ArrayList list, int sequenceWidth){

if(list.size() > sequenceWidth){
ArrayList sequence = new ArrayList();
for(int k = 0; k < sequenceWidth; k++){
sequence.add(list.get(k));
}

int k2 = 0;
for(int k = 0; k < list.size(); k++){
int l = (Integer) list.get(k);
int s = (Integer) sequence.get(k2);

k2++;
if(k2 > sequence.size() -1){ k2=0; }

}
return (Integer)sequence.get(k2);
}
return 0;
}
}



How can I improve this???

Concept AI (part 2)

This is the latest addition to the brainstorming file. Please not that the information not sited is just my concepts, they might not always be realistic or accurate.

http://www.phy.syr.edu/courses/modules/MM/brain/organizations/organizations.html#work
And sub/super site
"A simple explanation of the processing step is that the cell sums up the incoming signals and produces an output signal only if this sum exceeds some threshold; i.e. only if the total input signal is big enough will the cell `fire' an output signal to its neighbors.""new memories are stored not on individual neurons but by adjusting the strengths of connections between neurons.""the connection between two neurons will strengthen if more often than not the two neurons fire together. This is often called the Hebb rule.""The nodes in the network are vast simplifications of real neurons - they can only exist in one of two possible "states" - firing or not firing.""Certain nodes will be always "on" and others "off"."

Emotions affect the threshold. Causing "speechlessness" when too happy, lower threshold value for certain categories (associated with anger) when angry ect.

The connections between the neurons are also important. The higher the current that passes through the connection, the more it's strength.

A, B, C are neurons.
A is connected to b and c.
B is connected to a and c.
C is connected to a and b.
During sleep time, this system can be reworked to:
A is connected to b.
B is connected to c.
C is connected to b.

Once a neuron receives a signal, it waits to see if the signal will add up to the threshold. If not, it returns false, otherwise true. This feedback will help improve the heuristic algorithm that direct the connections.

Techior2
"cannot be limited to words, but must also extend to the concepts those words communicate"

Research "Hopfield Networks" for memory.

The input gates should use pattern recognition to simplify the complex input and direct the general flow of input to the right nodes.

Concept AI

Whenever I work on a digital project, I create a folder for it and make a notepad file for brainstorming and just scratch work. This is the current version of the brainstorming file for the AI-Concept I'm working on.


What separates machines from humans?? The answer that comes first to mind is that while we have free will. The power to have our own openions and to create new ideas out of scratch. Although that's how it seems, it's not the case. We humans cant create ANYTHING! We can only take what exists in nature and manipualte it into something new. Think of this. A raw piece of wood is cut from a forest and made into the paper that becomes the raw canvas for an artwork. We are able to make paper out of wood because we were observative. The artist is able to make an artwork not because he or she is CREATive, but because they have the power to observer and judge. Sitting in front of that blank canvas, hundreds of ideas rush into the artist's brain. But from where?? All of these ideas are from old experiences, in one form or another. Many times, new ideas are "created" from the combination of two concepts or situations into one. Many times these founding situations and ideas that make up the final ideas is not from the person themselves, but form someone else. All of human kind's achivements were possible because we were open minded. If we didnt share our findings, each of us still would be trying to invent the wheel by overselves a thousand different ways.

When faced with a blank canvas, artists often start off by drawing a single line. And then another. The image of these lines trigger memories of objects that could be formed from the combination of the lines. Looking up at a summer sky, we see the shapes of different objects on the faces of clouds, even though they are just clouds.

As children, we often imitate those older than us. We mimic the actions of those around us and learn form it. These process of observing and imitating, physically or mentally, allows us to learn. A process of rewards and punishments separating the good learning from the bad shapes our personality. Creating an program that can imitate what we tell it is fairly easy. So how do you go from an imitator into an Artificially Intelligent program? With goals, both big and small.

Goals are an important aspect of being human. We as humans are proud to announce that we have free will. "If I want, I can walk over to the front of the class. Punch the professor and run out, yelling and making a mess in the hallways." Yes. You can (as in ability to do something) do that. But WHY don't you? A person in their right minds would know that such actions can have serious consequences. It would most likely get you expelled from that school and subconsciously, you despise that idea since it would interfere with the goals you have in life. Notice I said the "goals you HAVE" in life. This is because many times, these goals are not your own. Your parents, friends and the society around you have a a lot on influence on the goals you set, especially those set when you are young and still prone to some aspects of learning from imitation.

Goals, combined with past experiences the program learnt, is the major aspect of an AI. You teach the program how to draw one line, and tell it to draw two lines. After many failures, it'd accomplish that goal. Learning from it's mistakes to accomplish the next, slightly harder task of drawing three lines, faster with fewer errors. The series of errors, punishments and rewards will develope into the AI's sense of judgment that judges good from bad. Although I enjoy watching Sci-Fi stories such as Terminator, Sarah Connor chronicles, I don't think such a situation would happen because of these very reasons.

-Unfinished. Aspects of Creativity, Thinking and Memory to be explained later.

It's been 4 or 5 days since I started the AI project. School will open soon, so I'll have to stop this and move on for now. I'll scratch down the basic outlines of the other ideas that I had so that if & when I start again on this project, I wont have to start from scratch.

Input:
Where am I
I am there
Who am I
I am Cobalt
What am I
I am human
Where are you
you are here
who are you
you are AI
what are you
you are a program
you are a module

The goal is to get the program to get a categorization of
1: I, am
2: You, are
3: where, I, there
4: where, you, here
5: who, I, Cobalt
6: who, you, AI
7: what, I, human
8: what, you, program, module
and so on

Later on (after more input), the categorization might look like the following.

1: who, what, where, when, why, how
2: where, US, Earth, Solar System
3: are, am, is was
and so on....

Make each node complete objects instead of just one word.

With the input "you are good" and "how are you" try:
you(are, good)
are(you, good)
good(you, are)

How(are, you)
are(How, you)
you(are, how)

That should let it associate Good with How. This system breaks down when new unknown words are introduced too fast
and the program confuses those words with the answer.

Memory: categorized by tags. Recent memories kept in RAM. snipeets from old events (more snippets of recente events than of old ones) also in memory. Snippets can be used to search for the rest of the memory as well as events directly before and after it and related events.

Creativity: combine random snippets (which lead to other snippets) in a comprehendible way. Apply concepts from one event/lesson on others. Use format from similar challenges/tasks/experiences. The snippets lead to a line of thought, sometimes creating recursion since in the process of looking through the snippets, the snippets are pushed up in the stack to be renewed/"newer". This creates the "That song is stuck in my head" scenerio.

Thinking: multiple threads jumping from topic to topic. Each topic is a .txt file with tags to related topics. sentences broken into web system.

Logic: basic skills of logic must be built into the system. a=b; b = c; a = ?
Pattern recognition should also be used

A heuritical system of punishments and rewards. Use this to organize the information better by re categorizing and eliminating the link from node to node when the heuritical cost becomes unacceptably high.

Also look into programming this in LISP so that it can access itself and modify itself while running.

Reading Test
Aoccdrnig to a rscheearch at cmabrigde uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the only important thing is that the first and the last letter be in the right place. the rest can be a total messa nd you can still read it without a problem. This is because the human mind does not read every letter by itelsf, but the word as a whole.

*Consciousness - A state where you are capable of observing and learning from the observation.
*Learning - A infinitively recursive process of imitation and experimentation (which leads to more "learning")
*Intelligence - A state of being conscious and able to learn. The pace of learning defines the intelligence quotient.
*Thought - The reaction to what was observed. The actual thought is based upon what was learned.

An AI's awareness of Self and the World: a challenging concept. Emotions: Even more challenging.

When you see an old grandfather clock running by turning gears throgh the energy from winding, you observe and learn from it. Sometime in the future, you are able to apply a similar concept to something else. If the clock was covered, then you'd have to go through the hassle of removing the cover to learn about it's working. Most people wont do this and thus, this covered approch becomes a wall to progression and learning. Closed-Source commercial programs are doing just the same. Open Source interpreted programs that are interpreted; those that the user can see without much hassle; will drive forward a considerable evolution in programming.

Does intelligence and thinking start before we are even born?? Aldous Huxley's Brave New World gets me thinking that at least a bit of "conditioning" is necessary for AI's to work properly. This would be where, at the start, the AI just observers everything without actually reacting. These observations set the foundations, the instincts, for further learning.

*NOTE* : I'm using a different type of Neural Network that doesnt use any sort of "Weights". It's just many independent nodes linked to related nodes/words. When asked a question, the question is sent to the node and it adds it's part of the answer to the answer String, determines what the next node in line is and calls it. This creates a train of thought. The answer will be based on not just the current question, but on the whole conversation. The "recent memory" object is shrunk down into snippets after the conversation and shrunk down even more after sleep.

All the Neural Netowkring of AI's can get very messy. That's why Sleep is necessary. The information in the recent neural netowork is reporcessed. The useful thoughts and nodes are kept while the useless ones are disposed of. The random paths from node to node create the sequence that becomes the basic outline of Dreams.

The concept of Understanding can be "understood"(LOL) better through the Dictionary scenario. Sometimes when you look up a word, you get a definition that uses the same word to define it! For example:
Consumer: person who consumes, esp. one who uses a product.
The reader looks up the word consumer because they didnt know the word "consumer" or words associated with it such as "consume". This definition doesn't truly explain the word in terms they know and thus, the reader doesn't understand the meaning.
If a second definition:
"Purchaser of goods or services"
is given and the reader knows all of the words making up the definition, then they can understand the meaning of the word.

When an AI looks up the word Consume and finds the second definition, it'll break down it's definition into smaller parts such as "purchaser" "of" "goods" "or" "services". Then it loops through each of those words and looks up words related to it. The process continues for a set number of turns. Then, it creates a new node with the word "consume" and assosciates the words "of", "goods" "or" "services" to it. It then also adds the words it found through the sub-search.

Most of the words the AI associates with the word "consume" is irrelevent. During sleep time, it sorts through the words and narrows down what is important and what is not.
The AI now "understands" the word "consume" and it'll be able to use it in define it in many different ways (instead of "purchaser" it could say "buyer" which it earlier identified as a word associated with "purchaser" with similar meaning).
Mastering usage is a whole different issue. It has to go through a trial and error process to expand and narrow down it's associated words, categorize the word, create rules for the word when it's being used with words of certain categories.



Thoughts?? Opinions??? Suggestions??

Why

Why go through all the hassel of writing a blog? Well, first of all, I want to document my thoughts and ideas so that I can look back on them someday. Secondly, to share knowledge. Under my desk at home, I have a cardboard box filled to the brink with old bits and pieces of paper. They are all that remains of my many different projects/ideas about many different things. Most, never got off the drawing board. Others are still "under construction", and probably will remain without much changes until the paper starts to fade away. I've realized that many of these projects will never get done by me, simply because of the lack of time. Other than letting these concepts go to waste, I'd like to publish them here so that someone somewhere may benefit off of it. Let me know if you're working on something similar. I'm curious to see if they'll actually work!