Lecture 4 min.
The physical carriers of artificial neural networks are the hardware or physical medium where the network's weights are actually stored and where neuron computations and signal transmission are actually carried out.
A neural network can run on an ordinary central processor of a computer.
Examples:
A CPU is suitable for small models, program logic and data preparation, but for large neural networks it is usually slower than a GPU.
The most widespread carrier for training and running large neural networks.
Examples:
A GPU is well suited because neural networks require a huge number of parallel matrix computations.
Specialized chips for neural networks, especially from Google.
They are optimized specifically for machine learning operations: matrices, tensors, multiplications, transformers.
These are special AI blocks inside phones, laptops and microcontrollers.
Examples:
They are used for speech recognition, photo processing, local LLMs, cameras, AR and generative features.
Programmable chips that can be configured for a specific neural network.
Pro: you can build a very efficient circuit for a specific task.
Con: harder to program than a GPU.
These are specialized microchips built only for AI.
Examples:
An ASIC is faster and more energy-efficient than general-purpose chips, but less flexible.
Fig. The complexity and benefits of using different types of carriers of artificial neural networks
These are chips that try to imitate the work of the brain more physically: pulses, events, spikes.
Examples:
They are used for spiking neural networks. This is closer to biological neurons, but for now less widespread than GPUs.
Instead of electrons, photons (light) are used.
The idea:
Potential advantages: high speed and low power consumption.
For now these are mostly experimental and specialized solutions.
Computations are performed not with digital 0s and 1s, but with physical quantities:
For example, the weights of a neural network can be stored as the conductance level of an element.
A memristor can store a neural network "weight" as a physical resistance.
Roughly speaking:
This is promising for hardware neural networks, where memory and computation are located almost in the same place.
Qubits and quantum states are used.
Quantum neural networks are for now mostly a research area, not a mass technology.
Experimental systems where computations are performed on living neural cultures or "brain + computer" hybrids.
Examples of ideas:
This is not ordinary AI in the industrial sense, but rather the field of biocomputing.
Today most artificial neural networks physically exist as:
weight numbers in computer memory + computations on a GPU / TPU / NPU
That is, a modern neural network is not a "brain in a box", but a huge set of numbers that is stored in memory and processed by specialized processors.
Comments