Skip to main content
Bluecoders
← Tech glossary

Transformer

TechTerm

Transformer = the architecture behind LLMs

The transformer is a neural network architecture introduced by Google researchers in 2017 in the paper Attention Is All You Need, which underpins nearly all large language models and many image, audio and multimodal models.

Its central idea is the attention mechanism: to process each element in a sequence, the model weighs the importance of all the other elements. Unlike the recurrent networks that came before it, it processes the sequence in one block rather than word by word, which makes it possible to train very large models in parallel on thousands of GPUs.

GPT, BERT, Llama, Claude, Gemini and Mistral all rely on variants of the transformer. Its weak point: the cost of attention grows steeply with text length. Alternative or hybrid architectures (state space models such as Mamba) aim to reduce it.

Why it matters when hiring

The transformer separates two families of profiles. A Research Scientist or Research Engineer needs to understand how it works internally and know how to modify it. An AI Engineer who builds applications from existing models does not need the same depth, and requiring it drives away good candidates. Clarify which of the two you are looking for before writing the job ad.

A recruitment need?

Describe what you need. A recruiter from your sector will call you back within 24 business hours.