Towards More Energy Efficient Machine Learning Models

https://odsc.com/california/#register

Inspired by the human brain

Spiking Neural Networks

import os

from torchvision import transforms

from bindsnet.datasets import MNIST
from bindsnet.encoding import PoissonEncoder

# Load MNIST dataset with the Poisson encoding scheme
# time: Length of Poisson spike train per input variable.
# dt: Simulation time step.
dataset = MNIST(
PoissonEncoder(time=250, dt=50),
label_encoder=None,
download=True,
root=os.path.join("path_to_data", "MNIST"),
transform=transforms.Compose(
[transforms.ToTensor(), transforms.Lambda(lambda x: x * intensity)]
),
)

Frameworks

from bindsnet.models import DiehlAndCook2015

# build Diehl & Cook network
network = DiehlAndCook2015(
n_inpt=784, # number of input neurons
n_neurons=100, # Number of excitatory, inhibitory neurons
exc=22.5, # Strength of synapse weights from excitatory to inhibitory layer
inh=17.5, # Strength of synapse weights from inhibitory to excitatory layer
dt=1.0, # Simulation time step
nu=[1e-10, 1e-3], # pair of learning rates for pre- and post-synaptic events, resp.
norm=78.4, # Input to excitatory layer connection weights normalization constant
inpt_shape=(1, 28, 28))
network.connections
{('X', 'Ae'): Connection(
(source): Input()
(target): DiehlAndCookNodes()
), ('Ae', 'Ai'): Connection(
(source): DiehlAndCookNodes()
(target): LIFNodes()
), ('Ai', 'Ae'): Connection(
(source): LIFNodes()
(target): DiehlAndCookNodes()
)}
network.connections[("Ae", "Ai")].w
Parameter containing:
tensor([[22.5000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.0000],
[ 0.0000, 22.5000, 0.0000, ..., 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 22.5000, ..., 0.0000, 0.0000, 0.0000],
...,
[ 0.0000, 0.0000, 0.0000, ..., 22.5000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000, ..., 0.0000, 22.5000, 0.0000],
[ 0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 22.5000]])
import torch

# Create a dataloader to iterate and batch data
dataloader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=True)
for (i, d) in enumerate(dataloader):
if i > n_train:
break

image = d["encoded_image"]
label = d["label"]
# Get next input sample.
inputs = {"X": image.view(time, 1, 1, 28, 28)}
# Run the network on the input.
network.run(inputs=inputs, time=time, input_time_dim=1)

References

--

--

--

Our passion is bringing thousands of the best and brightest data scientists together under one roof for an incredible learning and networking experience.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Tensor+Flow=TensorFlow

Face Recognition System using VGG16

Dirty Jobs with NLP

Transforming the World Into Paintings with CycleGAN

Machine learning for automatic video summary generation

ODSC West 2021 Preview: Practical Reinforcement Learning for Data Scientists

Where should you learn AutoML?

TensorFlow Furthers the Development of Machine Learning

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
ODSC - Open Data Science

ODSC - Open Data Science

Our passion is bringing thousands of the best and brightest data scientists together under one roof for an incredible learning and networking experience.

More from Medium

Multichannel Speech Enhancement Using Deep Neural Networks

An amazon device called Alexa that allows you to send command voice orders. This device has several microphones and performs multichannel speech enhancement

#HealthDataStories: Building A Machine Learning Model To Predict Medical Appointment Attendance

Deep Learning Shapes Machine Learning Technology

Machine Learning Technology

Age prediction of a speaker’s voice