MACHINE LEARNING

Subscribe to An Interview with Udi Schlessinger (ML + Evolutionary Computation) 5 posts, 4 voices
Forums » Introduce Yourself » An Interview with Udi Schlessinger (M...

AN INTERVIEW WITH UDI SCHLESSINGER (ML + EVOLUTIONARY COMPUTATION)

  • Easily the favorite thing about being part of a network like this is meeting amazing people like Dr. Udi Schlessinger. Udi is an incredibly hard-working individual with an interesting story blending machine learning with evolutionary computation. Below is part one of my interview with Udi. Take this chance to connect with him through this thread.

    IAN: How did you first become involved in machine learning?

    UDI: Back in 1997 I read this science fiction book by Dan Simmons, "Rise of Endymion". In the book Dan mentions a guy by the name of Tom Ray who created a virtual environment and programmed a virtual organism. This organism was basically a few lines of assembly code. All it could do is reproduce by copying itself to a memory location. Every time the organism reproduced, there was a chance of a small mutation occurring which would usually break the code and cause it to be unusable. Tom left it running for a while, and when he came back he was *shocked* to discover that his virtual organisms not only thrived, but optimized themselves far beyond his original programming. Some even mutated enough that they became parasites to the original organisms, which learned how to fight back. When I read this I thought "wow, what a cool idea". And for years afterwards had this idea in the back of my mind that I should try and program something like this on my own. 

    In late 2002 I had a lot of free time, so I started thinking: why don't I try to program that thing I was excited about in 1997? But just before I started, I did a quick Google search. And was completely surprised! Even though the Rise of Endymion is a fictional story, the part about Tom Ray was not - in fact, Dan described *exactly* what Tom Ray, a real scientist, did. Apparently Tom belonged to a small, but thriving community of people who try to replicate biological means in a machine for various reasons: some to create 'life', others to create intelligence that can solve problems for us, without us having to do anything except for give the problem and allow the code to evolve. This field is called Biologically inspired computing (and consists of many areas, such as evolutionary computation, certain subsets of machine learning, artificial life, etc).

    This was a life changing moment for me. I simply had to learn this. At that moment I decided that I will and start a Ph.D. in this subject. This is how I got interested in evolutionary algorithms.

    As part of my preparation for a Ph.D I began researching everything that is involved. Although I was familiar with neural networks before (just the general term), I never used them or knew exactly their potential. So during the course of these investigations I became extremely interesting in neural networks, and as I stated before, specifically, the usage of evolutionary algorithms to train them instead of standard ML techniques. To me it seemed like the best of both worlds. Many people agree with this approach, while others do not.

    IAN: Tell us about your Ph.D thesis on evolutionary algorithms and how machine learning played a part in it.

    The UK (or at least, my university) has an interesting rule: you must have two supervisors (equivalent to US advisors). This was meant to protect the student in case his advisor is problematic, something which happens *very* frequently. One of my supervisors was a computer scientist (biologically inspired computing specialist). The other was an optical-illusion specialist. The resulting project - my project - started as a one that will interest both:

    Basically, my goal was to create an environment and populate it with agents, each could reproduce, it could die, it would need to obtain resources. It would also have a small neural network controlling it (which would be random when the agent is first created), and every time the agent reproduced, a slightly mutation to the network would occur: either its contents, its topology/structure, or other parameters. Note that this is something you can't do with standard ML techniques, change the topology (layers, connectivity) of neural networks in real time, however, if you evolve them - you can!

    Because of the nature of the environment, my neural networks were unique: they were based on a standard feedforward network, but we devised a set of inputs that served as 'eyes' for my agents which emulated the way the human eyes work. The agents' eyes would evolve as well.

    This framework would be flexible enough that I could pose questions to it, and by leaving it for a few hours/days, I could come back and see whether they agents have been able to get results for the problems. Because we were dealing with optical illusions, many of the problems were visually based, but were not limited to that.

    So I had two primary goals with my project

    1. Find a way to get the evolution process to be effective. This was my primary goal, and the reason I went to do a Ph.D. I strongly believe in this field, and think that if we can increase its potency, we can solve many problems that until now are impossible.

    2. By analyzing the evolved agents, we may learn something about the real world. This was of lesser interest to me, but the more I worked on this, the more fascinated I became.

    It turns out my agents evolved eyes that - in a very simplified version - are used in a similar way that ours are. They learned how to solve most problems that I gave them.

    At this point I - from various reasons - took my project in a different direction. I grew fascinated with the idea of multiple agents working together. You all have heard of nanotechnology (futuristic technology, where billions of microscopic robots could be put in our bloodstream and basically fix any health problem we have). Science is nowhere near that goal. However, a secondary associated problem is controlling a billion agents concurrently to solve this problem. This is a lesser known issue, but as important. Think about it: we can barely create operating systems that doesn't crash, so have *billions* of different components that work harmoniously inside the human body... is kind of way beyond us at the moment.

    So I thought that my work could eventually be combined with that. I thought that if I could get my agents to cooperate, this may provide one of the necessary steps.

    The first thing I did was to explore the agents' evolvability, basically, what makes them evolve better. Or in other words, what makes the evolutionary process - the learning process - effective. I found several ways that in fact significantly improve the agents' evolvability. Interestingly, by reading a lot of biological literature, these are all methods which are thoerized by biologists to affect the course of evolution in our planet. Is that not cool?

    The challenge I faced by doing this, is that at least in the domain I was working, *very* few people have worked on this issue until that point. Maybe a handful. So a few times I literally had to invent my own measures to quantify improvement. My work - which was somewhat controversial because of that - is still cited and some people have actually built on it. 

    Then I continued to the second stage. Instead of each of my agents having one neural network (brain), I thought of expanding it. I based my design on the mixtures-of-expert method (a ML construct where a gating network controls a number of neural networks).  So I enabled each agent to have one 'master' neural network which controlled up to 8 neural networks. The goal was that the master network learn to recognize the problem the agent is facing, and will direct the input the agent receives into one of the subordinate networks. Basically, allowing for specialization. This worked phenomenally well, and to my surprise (I didn't anticipate this), I actually found out that here too the way this works is a lot like biology. My most successful agents learned to assign tasks to different subordinate networks: so some of these would specialize in movement, others in recognizing resources, others in reproduction. I was nominated for best paper for this work.

    Then I took it a degree further. I thought: won't it be useful if my agents could combine and form the equivalent of multicellular organisms? Basically a group of them would work as a whole to solve a problem? Based on what I mentioned earlier, I'm sure you see my goal. If I can do this with 20 agents.. maybe in the future I can do this with 20 billion agents... and this could really be a step towards Nanotechnology. 

    This turned out to be much harder than I thought it would be. But I was eventually able to do this. My agents would occasionally combine and start forming these multi-agent virtual .. agents. The end result was that each agent was controlled by - at times - hundreds of neural networks working together, and this was a multi-layer construct (network -> hierarchical network -> multicellular agent). 

    I've done several additional things, but they are not directly related to ML or Evolutionary Computation, so I won't elaborate.  

    IAN: What is your favorite story during your Ph.D education that involved machine learning?

    It relates to my last issue: to my surprise, I found out that my multicellular agents started piding labor amongst their components: some agents lost the ability to see, others lost the ability to move. After all, our eyes can't survive on their own, and neither can our hands or feet. It was truly like a very simplified version of a biological creature. Again, when I compared this with biological literature, I discovered that many theories with regards to primordial life turning from unicellular to multicellular manifested themselves in my simulations (Just in case you don't know: Earth had - for 1 or 2 billion years - *only* single celled organisms in it. No plants. No animals. No dinosaurs. Nothing. This worked very well for billions of years, what made it change?) 

    Personally I am very excited by the concept of numerous agents, each a unit of knowledge, working together for one goal. Although I evolved my networks, I believe this can be done using standard Machine Learning algorithms, though it would probably be harder to do.

     

    I highly encourage you to connect with Udi through this thread. However, you can reach him through these means too:

      Blog: http://www.IndustryReview.org
      Twitter: http://www.twitter.com/uschles
      LinkedIn: http://www.linkedin.com/pub/udi-schlessinger/0/734/911
      Facebook: www.facebook.com/uschles

    Posted by Ian Ma 5 months ago

  • Hi Udi,

         Your research sounds really interesting!  Do you have any references to papers, or any other publications showing your results?  Do you plan on doing more work in this domain?  Its surprising to me that you say that not many people had done research similiar to yours when you began.  Has that changed much since then?

     

    -Sidney

     

    Posted by Sidney Burks 5 months ago

  • Hi Sidney,


    Sorry for the delay in my response, wasn't aware that a question has been posted.


    Of course! Unfortunately, I published roughly only a third of my research. However, that is all available online.

    My academic publications page:

    http://www.cs.ucl.ac.uk/external/U.Schlessinger/publications.html

    Another related article:

    http://www.cs.ucl.ac.uk/external/U.Schlessinger/research.html


    There are more articles (academic & non-academic) online, just search for my last name and "Mosaic World". But the publications are really the best place to start.

    I *definitely* intend to do more research in this domain, though not specifically continue what I've done before. That was too much effort that could've
    been demonstrated in far simpler means.

    I didn't say not many people did research similar to mine: that's actually not true. Although the community is small, *many* people have done similar work. However, the investigation of evolvability of dynamic agents is something that's really not been been barely done. Both before I did it, and even now too (though clearly more work has been accomplished - I can see that if only because some papers continue my own work).

    I'll be glad to answer more questions,

    Udi



    Posted by Udi Schlessinger 5 months ago

  • hi udi

    your reasearch is awesome ! but i felt this question revelant, in case ur simulating multiagent about thousands of em, each performing their routines parallely , it would then call for a massive computational infrastructure, right ?. Please correct me if im wrong !

    Posted by sree ram 5 months ago

  • Hi Sree,

     

    Sorry for the delay in response, and thanks for the kind words!

     

    Your question is good. My framework was indeed very computationally intensive. It's not just in terms of agents but also in terms of the size of the environment - one which constantly shifts and changes. For example, I normally used 100x100 grids, and found out that when I increase it to 150x150, often I get solutions much faster (in terms of agent generations), but it would take much longer in "real time". So it was a constant dilemma.

    Fortunately, I could easily balance the size of the environment, resources, which in turn would affect the number of agents, which in turn would affect computational necessities. I could also directly limit it, but I usually chose not to do that (since it's an artificial constraint).

    Roughly when I was halfway through my studies we considered using our university's cluster (I don't remember how many they had, but quite a lot), but because of various reasons, some of which - though not the major one - I was already halfway through, we never went ahead with it. I Imagine if I started it this way it's possible I would've obtained many more interesting results...

     

    Udi

    Posted by Udi Schlessinger 4 months ago

Reply to this thread

Your Reply
Captcha
simple_captcha.jpg

RECENT TOPICS