LLM Internal Simulator | The AI Codex

User prompt

Userinput

Explain in 5 points how self-attention works and add one simple example.

Assistantsimulated output

Ready when you are. I will simulate: tokenization → embeddings → N layers → logits → sampling → text.

Execution and connections

idle

Pipeline tokenization → embeddings → attention → logits → sampling

Run idle

Llama 3 70BGQA · RoPE

Decoder-only transformer con cache KV, atención causal y sampler autoregresivo.

01 input

Tokens

BPE ids + mask

02 residual

Embedding

d_model stream

03 attention

Q/K/V

heads + causal mask

04 transform

MLP / MoE

feature routing

05 sampler

Logits

temp + top-p

06 output

Guarded text

stream + checks

tuneParámetros

Arquitectura Llama 3 70B

Decoder-only + GQA: explora el flujo de tokens, residual stream, atención, MLP, KV cache, sampling y verificación de salida.

70B parameters

Model

Modelos reducidos a tres perfiles: uno moderno denso, uno sparse MoE y uno pequeño para comparar escala.

Arquitectura

Decoder-only

causal LM

Contexto

1,024 tok

KV cache: 12 capas

Text Input

Configuration

Capas 6

Attention heads 8

Speed 5x

Temperatura 0.70

Top-p 0.90

Metrics

Tokens0

0.0 tok/s

Layer0/6

–

Attention0%

entropy 0.00

Top‑1 Prob.0%

latency 0 ms

Context budget 0%

0 / 1024 tokens

Guardrail score ok

policy checks pending

KV cache

0 MB

Serving cost

$0.0000

prefill + decode estimate

Latency timeline

0 ms

Measures per-stage time (simulated) and keeps realistic proportions between tokenization, attention, and sampling.

Top‑K probabilities

k=8

Simulates logits → softmax → top-k. Useful for seeing confidence and alternatives during sampling.

Attention heatmap

Token-by-token map by layer and head. Useful for seeing patterns (diagonal, jumps, focus).

Keyboard Shortcuts

1-4 Go to stage

Space Ejecutar

R Reset

T Guided tour