Resolving vision problems by configuring FPGAs to spike the brain’s neural networks

Dr. Frédéric Mailhot, graduate student Gullaume, and Dr. Jean Rouat (right), Université de Sherbrooke
As microelectronic technology has become more advanced, so have the ambitions of researchers who would like to apply this technology to solve the physiological problems that afflict us. A particularly appealing goal is to help blind people better navigate the world by providing them with a system that converts video input into specific arrays of sound that they can interpret. Achieving this worthwhile objective, however, requires confronting the intricate complexity of the brain’s operation.

“If you look at the neural connections in the brain, we cannot do the same thing in electronic hardware,” says Dr. Frédéric Mailhot, Director of Electrical and Computer Engineering at the Université de Sherbrooke. He points to the dizzying network made up of billions of cellular connections, which would be all but impossible to replicate in a portable device. “There are too many. We cannot create this scale of interconnection.”

Nevertheless, he has spent the past decade examining how Field-Programmable Gate Array (FPGA) circuitry can be configured to provide a practical way to replicate the brain’s handling of visual information. “It is a daunting task, but a fascinating one to work on,” he admits.

He credits his interest in this subject to a dynamic collaboration with his departmental colleague, Dr. Jean Rouat, who is also an adjunct professor in the Department of Biological Sciences at the Université de Montréal. Dr. Rouat has been exploring ways of representing brain activity in a microelectronic setting and has led a number of projects based on the observation that the brain takes in information through spiking neural networks, a design strategy that transmits signals between nodes when certain activation levels are achieved.

This model, which has been internationally patented, is also part of the Neurocomputational and Intelligent Signal Processing Research Group (NECOTIS) at the Université de Sherbrooke. According to Mailhot and Rouat, spiking neural networks make it possible to capture the synthetic activity that takes place when the brain recognizes a particular visual item.

“If we look at how we learn from a neurological point of view, it is due to the changing connectivity between the neurons,” they explain, noting that the electronic model can perform image segmentation that begins to build a visual vocabulary for future reference. Once a pattern or object has been identified, that information can then be transferred from a visual input to auditory input using the same type of spiking network connections. “Ultimately it’s a data compression system, which condenses the data in such a way that you can recognize simpler objects,” says Mailhot.

As this technology moves into testing with laboratory subjects by the end of 2014, the researchers are counting on another remarkable feature of our brains—neuroplasticity. Rouat points out that individuals undergoing rehabilitation regularly demonstrate the brain’s ability to overcome injury or disease by establishing new pathways for carrying out physical and mental functions. One of Rouat’s mentors, the late American neuroscientist Paul Bach-y-Rita, demonstrated how electronic stimulation in key areas like the tongue could accelerate the pace of this rehabilitation. Rouat is optimistic about achieving similar success with this pioneering investigation into how spiking neural networks can create auditory signals that blind people can subsequently process in their brain’s visual cortex.

“We believe that we can use the plasticity of the brain to reeducate or modify an area of the brain,” Rouat says, adding that even more might be possible. In the meantime, he and his colleagues will have their hands full just tackling the challenge of blindness. CMC has played a central part in helping them do so, starting with the Virtex-6 FPGA ML605 development board from Xilinx and accompanying software. “Access to this equipment would not have been possible without the support of CMC,” he says. “It allowed us to create unique work that led to publication and patent application.”

Guillaume Séguin-Godin, a graduate student working on this project with Rouat and Mailhot, was especially pleased by the FPGA boards system that came with a preconfigured workstation, tools, libraries, documentation and associated IP already installed and ready to use. “I found that very impressive,” he recalls, adding that the work benefitted immensely from access to this hardware. “Without them we wouldn’t be able to do the same kind of research.”

That sentiment is shared by Mailhot, who began his career with the EDA software vendor Synopsys in California, and regards hardware considerations as essential to making this research relevant. He anticipates that at some point the team will request help from CMC to acquire an application-specific integrated circuit as the next step, but for now the flexibility of the FPGA is ideally suited to their purposes for neural networking. “It’s a very good way to prototype,” he concludes.

Printable version