Episode 51: Neuromorphic Engineering

Lisa Peña (LP): Welcome to a new year of listening and learning. We're heading into 2023 discussing the mind-bending, fascinating field of neuromorphic engineering. It's next-level artificial intelligence developing computer systems that work like the human brain. How can a computer think like a person? Plenty of food for thought next on this first Technology Today episode of 2023.

[MUSIC PLAYING]

We live with technology, science, engineering, and the results of innovative research every day. Now, let's understand it better. You're listening to the Technology Today podcast presented by Southwest Research Institute. From deep sea to deep space, we develop solutions to benefit humankind. Transcripts and photos for this episode and all episodes are available at podcast.swri.org. Share the podcast and hit that Subscribe button on your favorite podcast platform.

Hello and welcome to our first Technology Today episode of the new year. I'm Lisa Peña. We begin 2023 with a brain-teasing topic as we learn about SwRI's new neuromorphic engineering capabilities. We're talking about building biologically-inspired technology, computer systems that behave like the human brain. And these systems can be used in multiple industries for multiple purposes including aerospace, space science, autonomous driving, and even wearable smart devices like your watch. Our guest today is neuromorphic engineering expert and SwRI Staff Engineer, Dr. Steven Harbour, who is leading SwRI's neuromorphic research. Welcome, and thank you for being here, Steve.

The team from the SwRI Dayton Engineering Advanced Projects Lab for Neuromorphic Engineering and Spiking Neural Networks Research works on setting up their artificial intelligence unmanned aerial system, equipped with a neuromorphic processor and controlled entirely by spiking neural networks. Pictured above from left to right: Sam Bryan, Alex Henderson, Stephen Schlager, Dr. Steven Harbour and Ashlie Abballe.

Dr. Steven Harbour (SH): Well, thank you so much, Lisa. It's an honor and a privilege to be able to be on your podcast.

LP: Well, this is such an exciting topic, a fascinating topic to launch the year with. So to start this discussion, let's define neuromorphic engineering. What is it? How do you define it?

SH: Certainly. Well, neuromorphic engineering is a relatively young field that attempts to build both hardware and software realizations of biologically-realistic models of the brain, our neurons, and neural systems, using electronic circuits, algorithms, specifically, something called SNNs, which we'll describe a little bit later, implemented in very large-scale integration technology.

While originally focusing on models implemented using mainly analog circuits, the field has grown and expanded to include the modeling of neural processing systems that incorporate the computational role the brain that model learning and cognitive processes and that implement large distributed spiking neural networks using a variety of design techniques and technologies. This emerging field is characterized by its multidisciplinary nature and is focused on the physics and math and computation, cell biology, theoretical neuroscience, artificial intelligence and machine learning, device physics, electrical engineering, and computer and data science.

LP: OK, so essentially, you are building networks, building technology to emulate some of the functions of the human brain. So what is superior about our brains? Why do we want to create technology that functions like the human brain?

SH: Certainly, the human brain is far superior to the ordinary computer driven by what's called von Neumann architecture processors, which is what we use today. All our computers use the von Neumann architecture processors today, and our brain is far superior to those processors. CPUs and GPUs, central processing units, and graphical processing units are what you'll find in our computers and also computing systems, and automobiles, self-driving automobiles, aerospace, space, that's our current bread and butter today. And our brain is far better than they are.

Now, I know that those computers, when combined with a lot of GPUs, they can compute numbers fast, but what they're not good at is our brain is better at reasoning, interpreting the outside world, coming up with new ideas, and dealing with the unexpected query. The brain is capable of imagination, for example. The human brain only uses 20 watts of power, which is less than a light bulb, and it has 100 billion neurons. Our memory is co-located with and throughout our brain where it's not on the current von Neumann architecture that we use.

Our cognition uses parallel computing. Our brains can learn on the fly. Current computers cannot do that. Our brains use asynchronous event-based spikes with sparse information. Our brains are better at learning new things and then changing as required in a new environment. Therefore, in order to improve how our computers currently operate, which they pretty much peaked, so to speak, we need to turn to the human brain as our model and guide and create technology that functions like the human brain.

LP: So the brain is in so many ways, superior to the computer systems we're currently using. It's really fascinating. I keep using that, coming back to that word, but it really is think of a computer kind of being able to do it all, but it still can't accomplish what our brains accomplish and in such an efficient manner. So one of the main components that you are working on building are spiking neural networks that perform like the brain. Spiking neural networks, you mentioned that term a while ago. So let's talk about that. What are SNNs, spiking neural networks?
 

Like the human brain, artificial intelligence is progressing towards continual learning, utilizing spiking neural networks, mimicking human memory.

SH: Certainly, certainly. So they are what we call in the field as third-generation artificial intelligence and machine learning networks. We'll start with that. SNNs mimic human brain activity and can be accomplished in software, or hardware, or both at the same time. They mimic information in the brain that are action potentials, neuron spikes, which may be grouped into spike trains, or even coordinated waves of brain activity between neurons through synapse. Those are connections.

These are an effect of an ion reaction caused by a thought or thoughts or vice versa. So we have this electrical activity occurring in our brain and they happen in spikes. When a voltage threshold is crossed within a cell, then a voltage spike occurs, and then that message, if you will, is transferred to the next neuron, so on and so forth. And so we then emulate that either electronically and/or through software or both, so our electrical circuits communicate like the human brain.

LP: So that message being transferred from neuron to neuron, there's a spike there, which is what gives us the name?

SH: That's correct. That is correct. And the next neuron either impedes that spike based on what it is, what that message is if you will, or it continues it along to the next neuron, and so on and so forth. There's something that occurs in the brain called long-term potentiation. And if a spike is generated from one neuron and is passed to another neuron if that spike is repeated relatively quickly, it reinforces that information. It also strengthens the connection between the two neurons, and I'm sure everyone has heard of neurons that fire together wire together.

Well, actually a memory is formed that way in the brain. That's how we form memories. And so we then emulate that in the computing system to form memories on the same processor, which is completely different than how we do it today. Long-term depression controls that, so you don't want your mind to go with several spikes, and then you would have overriding of memory occur. And so to avoid that, another process called long-term depression occurs in our brain and to control how memories are formed, and we can emulate that in the computer as well.

LP: So the spiking neural network, what is better about that system versus what we currently have in a computer? I know you mentioned there's memory there. I mean, does the computer actually remember past experiences, so to speak?

SH: Yes, it does. So currently today with the current computer architecture that we use today with a processor, the memory is off-chip, it's off the processor, so there are other devices where the memory is stored and/or manipulated. So the processor has to take the information off-board of the chip and save it to a memory device and then also retrieve it when necessary. Well, that takes extra time.

Even though things are happening very fast in computers, there's a distance involved there and also a bottleneck, so there's one pathway to that memory device going and coming and the traffic has to be controlled, so that slows things down. It also eats up a lot of extra energy that's not necessary to go back and forth. Whereas, when it's on the processor, the memory is actually on the processor, through this similarity to a long-term potentiation, long-term depression methodology, it then is stored actually on the processor and remembered on the processor, which makes retrieving it near-simultaneous as writing the memory, if you will, if you look at it from a writing perspective.
 

SwRI Engineer and Neuroscientist Dr. Steven Harbour is leading the Institute’s neuromorphic engineering research and development. Here, Harbour is pictured presenting on neuromorphic developments at SwRI’s fall Tom Talks, a recurring speaker series named for the Institute’s founder and inspired by the popular TED Talks.

The spike and why our brains are far better at being able to comprehend something quicker or make an inference faster is that the spikes can be more sparse than a continuous-type data information, so we can use less information and recognize something. A perfect example is let's say you were going to have a drone at a ranch to fly and look for coyotes to alert the rancher or farmer of that ranch, you'd have to train traditionally today with first and second-generation AIML, you'd have to train it with several images of various coyotes, 1,000 plus images for it to be accurate enough to be able to identify a coyote.

Whereas as a human, we clearly have some experience with animals, however, you don't have to show us 1,000 pictures of a coyote. You can show us one or two pictures of a coyote and we've got it. We know that's a coyote and we'll recognize that coyote rather quickly, much better than a computer will. And it goes back to the spikes and the sparsity of events that we need, all we need to understand what that is, and the memory is very rapidly processed.

LP: OK, so we're talking about retaining memory. So when we say we're building computers to be like the human brain, how far does this go? Is this a processing function only? I guess, what are the limits here? Are we anywhere near like a computer having original thoughts or feelings?

SH: Certainly, certainly, a very good question and something that we need to be very aware of ethically as we proceed. So yeah, this is an architecture that we can go very far in improving the computer's ability to think deeper, and so we have to be very careful with that. However, it's, again, we're modeling the computer to be like the human brain and nervous system and that focuses around silicon, right? A person-made hardware, software.

And it performs far better than traditional von Neumann systems do with less power and they perform better, but it's still just artificial intelligence. It's not artificial general intelligence, which is what you're referring to there a little bit. And the system is not self-aware. So these neuromorphic processors don't have the ability to be self-aware, so there shouldn't be any concerns with that kind of stuff occurring.

LP: So it sounds silly, right? But I think it's important to get that message out there because when we're talking about building something like the human brain, well, we want to know how much like the brain is it. So good to know, not self-aware. So let's talk about how long we've been working on these types of technologies. SwRI, this is sort of a recent development for us. So tell us about how long we've been working on neuromorphic solutions, and what neuromorphic technology is the Institute currently working on? I with our defense and intelligence work, we can't always disclose projects, but what can you talk about today?

SH: Certainly, absolutely. So since about 2020, Southwest Research Institute, we've been involved in fundamental basic and applied research when it comes to neuromorphics. And I've got to give a shoutout to three gentlemen that I think are visionaries, and that would be Chris Camargo, Nils Smith, and Walt Downing. They had the vision to approve and fund the IRs and PDIRs for neuromorphic research which is now growing into prototypes and ultimately, products. And so we've begun and are furthering that research and defense intelligence solutions first. It's going to grow elsewhere. It has to. The projects currently, like neuromorphic pilot, Amelia aircraft and drone mission computers, and Loyal Wingman. So they're not self-aware. I'll restate that. But well, this enables us.

LP: It's important.

SH: Yeah, it's very important, very important. This enables us, though, to get to the next level of AIML, and we can have systems that, for example, can behave more like the human pilot and start to make some decisions, so that enables us to do that. And so it's pretty neat to be able to have a pilot that might be a year at Loyal Wingman, but it may be a neuromorphic system that's over there flying next to you.

However, I will also say that this will be what's called pilot on the loop, which is a little bit different than pilot in the loop. But pilot on the loop can always take over or take control of the neuromorphic system, so they're not going to be out there flying on their own, doing their own thing. There's always going to be a human that can take over control of that system.

And so recent discoveries made through our research that's part of what we're talking about as far as prototypes and products is that we've done several investigative type testing with these different prototype devices, and we've discovered that this technology, neuromorphics implements low SWaP. Low SWaP is low size, weight, and power while outperforming the GPU and CPU in results. It's pretty amazing.

For example, currently, we have what I will call GPU farms, server farms out there to do processing, maybe even machine-learning processing, and they use simply way too much power. They consume way too much power. In fact, they can equivalently produce as much CO₂ in two weeks as 1,300 cars and most folks don't realize that. And so it's not very wise to continue doing more GPU, GPU, GPU. We're going to have to look to this alternative. And so the question could be, well, we do that, is this neuromorphic processor going to perform well enough?

Well, as I've been saying earlier, it performs better, and so it does. When you compare the neuromorphic preprocessor to the von Neumann architectures of GPU and CPUs, they do perform far better. They train faster in a AIML-type environment. They need less epochs to train and that's another way they consume less energy. They also arrive at inference much faster which is a performance measurement. And they are more accurate than the CPU and GPU, so they perform better, and they do it for less power like our brains. We just need four bananas a day for our brain and we're good to go.

LP: But the computers don't accept bananas.

[LAUGHTER]

SH: No, no, that's a good point. Yeah, no, we haven't imitated that one yet, no.

LP: All right, so we're looking at better processing, more accuracy, less energy usage. So all around, it sounds like the neuromorphic processors are outperforming the traditional processors, and I wanted to get a little bit more into the technology. You mentioned the neuromorphic pilot, Amelia. How soon until, well, first of all, there are already systems that can fly planes, correct?

SH: Surely.

LP: But this is a little bit different, and it has the, as we mentioned, some of the better components of neuromorphic processing. So can you talk about that a little bit?

SH: Sure, absolutely.

LP: And how soon can we see it in use?

SH: OK, so basically, yes, we've had we'll say autopilots on aircraft for a very long time. I probably have it memorized what year the autopilot started. It goes back a long, long time, but nonetheless, that's automation. It's not artificial intelligence. And so we even today are starting to use Gen 1 Gen 2 AIML to do some autopilot flying and some lower-level decision-making. But if you want to get to that approaching human-level-type thinking and decision-making, neuromorphics is where you need to go. That gives you that ability you didn't have before.

And so literally, the system can behave more closely to the actual pilot, and this includes decision-making, you know, what should I do next. And that's the advantage you get with the neuromorphic technology. Along with low SWaP, you get that ability as well. And so we're working on that right now. I think you could see this pretty rapidly, anywhere between one year from now to no further out than a couple of years away using that in the way that we're planning to use it.

LP: OK, so this was kind of an in-depth discussion about how the aerospace industry could use neuromorphic technology, but there are other examples, other industries that could benefit from this type of technology. Can you tell us a little bit more about other industries and other uses for neuromorphic technology?

SH: Absolutely, absolutely. So as you know, in SwRI, we're using this currently in aerospace for defense in Division 16. Neuromorphic systems, again, provide low SWaP. And so, again, we've talked about the inference time being much quicker and the accuracy being much better, and 10, 20, 30 times less power than other processors. So they're perfect for air transportation and defense, but also for health wearables, so wearable devices, watches that can sense the various health of a person, and other type devices that can be worn.

And this can be done on the edge per se and I'll describe what that is. So currently, I've just started research with Prativa Hartnett and Division 10 on neuromorphics involving wearable devices, you know, being able to process it on the spot, faster, without having to go to the cloud to process the information. That's another thing that neuromorphics does for you. Also, space science, avionics for long-endurance probe, and onboard processing, neuromorphics is the place to go to ensure that happens.

LP: OK, so many up-and-coming uses for this truly amazing technology. So as I mentioned, this is next-level artificial intelligence, so how is it different from the traditional AI we've come to know?

SH: It emulates how the human brain interacts with the world to deliver capabilities closer to human cognition. Biologically-inspired and plausible, artificial spiking neural networks is what we call them and we talked about them, these are novel models that simulate natural learning by dynamically remapping neural networks and are used in neuromorphic computing to make decisions in response to learned patterns over time.

Neuromorphic processors leverage these asynchronous event-based SNNs to achieve orders of magnitude gains and performance with far less power required over conventional architectures. So yeah, the sky is the limit. I also see these being, our motto is deep sea to deep space, I see this also in submersibles, submarines, and ships as well, and those various uses.

LP: So there really is no limit to how we can apply neuromorphic solutions, which is, again, my buzzword, my favorite word of the day, fascinating. So well, let's talk about you mentioned this is an emerging field. So what are some of the challenges of being on the ground floor of developing this technology?

SH: Certainly, certainly. So there are in the very beginning like when you get started with anything that's difficult and a lot of unknowns, there were a lot of unknowns when first getting started. And there can be some false starts and stumbles. And some people look at this and go, especially when I started doing research in this area that, hey, this is crazy, and you just have to stick with it. I started early research in the very beginning probably in 2014, and then obviously, continued that with Southwest Research in 2020, and we have come a long way.

It's a long road, but it's very important. And any time that you do research I think that's worth doing, it's not going to be easy. There's going to be an element of folks that think that maybe that's a little bit kooky, but you just keep going, and you may find out that that path is leading you down the wrong way. You could try a different path and continue on, and you may find out that it's going to be a dead end. But clearly, I think we're past those wrong turns and dead ends, and we're going we're going down the right path.

LP: All right, and if you are new to this topic, as you said, some might find it kooky. It sounds slightly scary, computers with human minds. It sounds a little bit like a sci-fi movie, but, again, that's not really what we're talking about here, and we've reviewed this through this entire conversation, but I think it's worth repeating. Can you tell us about the benefits of engineering technology to emulate the brain? Why push forward?

SH: Certainly, absolutely. So this shouldn't be scary. It's not meant to be scary. This is not going to replace the human mind, but it will help humans and society. So from robotics and artificial limbs to possible brain-computer interaction, to help people with various disabilities, or to perform medical treatments, so view this as a good thing and not a bad thing. These aren't when you think of all the sci-fi movies, that's not what this is. It's not science fiction.

It just seemed rather a good idea if you want to make computers better the human brain does a pretty good job of it, a very good job of it let's emulate the human brain to get to the next level, the next generation, and that's what we've done. But clearly, we have not built a human brain. And when we talk about, say a million neurons on a neuromorphic processor, that may seem impressive, and it is for today's technology, but our human brain has 100 billion neurons. So yeah, I think we're in good shape. And this is meant to help humans, not take over the world.

LP: So that's great that you pointed out that one day, this could be used in medical treatments. Could you walk us through a medical treatment scenario for neuromorphic engineering?

SH: In severe PTSD cases, the hippocampus actually shrinks. That's where a lot of our learning occurs and/or neurogenesis occurs, and so it actually has a medical effect both physiologically and psychologically. And so if there's a way to utilize a neuromorphic processor at some degree to aid with that with the human, either through computerized cognitive training or actually a little bit more evasive, it might be able to help that situation.

Or someone who has brain damage in an automobile accident, or some kind of cognitive disability, there might be a way to interface the neuromorphic processor with the person to actually help them. And so, yeah, I do think that's a follow-on research area that I would love to also pursue. So I do think this, whereas traditional processors really couldn't do that. They just weren't structured [INAUDIBLE]. Go ahead.

LP: Something like a device to pick up where the mental disability leaves off to help a person make decisions.

SH: Exactly right.

LP: So that's really interesting. Does not exist currently, but that's sort of your vision. That's one of your many visions for this technology of what it could be.

SH: Right.

LP: So as you advance this technology as we said, there are some critics, so any pushback? Are you experiencing that? And what do you say to critics of this type of work?

SH: Certainly. So at first, there were a lot of critics. So you know, I remember and when I talk about 2014, very, very extremely fundamental early stages, right? And you're right, a lot of folks were like you're crazy. And then 2016, and then certainly, as it evolves, a few more acceptors and things are proven more in the laboratory. And then 2020, again, I go back to those gentleman I mentioned earlier There was a lot of folks that I talked to at first that, you know, again, get out of here. But then they saw the possibilities and were willing to take a small risk and go further with it.

And so from that, the proof's in the pudding. So as you do more experimentation, the critics start to melt away and it gets brought out by the scientific work that we do I call them naysayers but they'll start to understand and it goes away. So it kind of occurs naturally. It's just part of research. There'll always be folks that will not be for this probably, but I think that the results are going to show time and time again that it's just too good of a, at some point, products that folks will accept it, and so that's what I found.

LP: So you obviously have a lot of enthusiasm for this field as you push forward and push past the critics, and you've been doing it for many years now. So what do you enjoy about this field? Why do you do it? What's your motivation?

SH: Absolutely, certainly. So I find it to be absolutely fascinating, in general, to perform scientific inquiry and discovery, especially in the field of neuromorphics. This is truly a disruptive technology. I know it's going to help humans, the environment, and also the defense of this great nation, and we need that. Also, it's right up my alley being both that I'm an electrical engineer and a neuroscientist, and pretty much this technology combines both of those fields. So it was a perfect fit for me.

LP: All amazing reasons. Disruptive technology that is, as you said, it uses less energy, so it's going to help the environment. It's going to help humans and lead to much-needed solutions for our defense and intelligence. You know, I have to ask. How does that happen? How does one become an electrical engineer and a neuroscientist? Did you see this field coming to be? How did you merge both of those fields for yourself? How did you make that decision?

SH: Sure, sure, absolutely. So yeah, I think that I had, having an electrical engineering background, and it was I had the desire to continue further and do the Ph.D. I had done a lot of research before entering any kind of PhD, applying to any kind of PhD program. And yes, I was doing some reading on neuromorphic-type stuff, and it seemed like a very good thing to look into. Emulating the human brain, you know, that seemed like to me a good pathway to take.

SH: Sure, sure, absolutely. So yeah, I think that I had, having an electrical engineering background, and it was I had the desire to continue further and do the Ph.D. I had done a lot of research before entering any kind of PhD, applying to any kind of PhD program. And yes, I was doing some reading on neuromorphic-type stuff, and it seemed like a very good thing to look into. Emulating the human brain, you know, that seemed like to me a good pathway to take.

Well, there are some skill sets I had to have. Being a double E, the human brain, neuroscience, that seems to be kind of an odd match, but actually, it's not. And so it's a good combination to have to go in this type of research, to get that PhD, for example, the way that I did it in neuroscience. And my neuroscience is in something called two different type of neurosciences, computational neuroscience, which involves a lot of math and theoretical-type stuff. But then also I included biological neuroscience, which is actually how our brains exactly function on a biological level, and being neuroscience, it's mixed with, of course, a psychology as well, our mind and how we think as well as how we're biologically put together.

And so that seemed like the right thing to do, and so I started doing research in neuromorphics that way. And it seemed like to be a real good degree combination, and I had seen that others in the field had also gone on that pathway. And so that's why I chose it.

LP: I know you've mentioned a few of your plans already and you have quite a few, but what do you see for the future of this technology? Will it be everywhere?

SH: Absolutely, absolutely. So I think in 5 to 10 years, we're going to see neuromorphic processors to some degree in our home personal computers. We're going to see them in our cars, especially cars that are purely electric and need every bit of energy they can have saved up on that battery and not expended on GPU systems. And we're going to see it in non-electric cars. It's perfect for autonomous driving. We're going to see it in our airplanes, in our space, in outer space, on our ships, and in our subs, along with those wearable and smart devices, and also, in the medical arena as well. It will change the way we live for the better.

LP: All right, so essentially, we could see neuromorphic technology just about everywhere. And I really enjoyed this look ahead. I love letting our listeners get a glimpse of what our engineers and scientists are working on and giving them that heads up, that little preview of what's to come. So thank you for such, again, fascinating information, just the best word for everything we've learned today. The neuromorphic field is booming, and thank you for giving us insight on what you're working on and what's next.

SH: Absolutely, and I want to thank y'all. I want to thank your whole crew, Lisa, and what you're doing so the word can get out. And I also want to give a huge shoutout to Southwest Research Institute for enabling this research to occur, to have the vision to kind of go where the water is kind of unknown, but willing to take a chance based on the potential benefits, and now that we find they are the real benefits. I'm going to give it all due to Southwest Research Institute. It's a great place to work and It's just wonderful.

And thank you to our listeners for learning along with us today. You can hear all of our Technology Today episodes and see photos and complete transcripts at podcast.swri.org. Remember to share our podcast and subscribe on your favorite podcast platform.

Want to see what else we're up to? Connect with Southwest Research Institute on Facebook, Instagram, Twitter, LinkedIn, and YouTube. Check out the Technology Today Magazine at technologytoday.swri.org. And now is a great time to become an SwRI problem solver. Visit our career page at SwRI.jobs.

Ian McKinney and Bryan Ortiz are the podcast audio engineers and editors. I am producer and host, Lisa Peña.

Thanks for listening.

[MUSIC PLAYING]