Master LLM and Gen AI with 600+ Real Interview Questions

Question: How would you explain the significance of positional encoding in a Transformer model?

A) It eliminates the need for token embeddings by encoding word positions directly.
B) It enables the model to process sequences in a random order without losing context.
C) It provides the model with information about the order of tokens in the input sequence.
D) It optimizes memory usage during training by removing redundant sequences.

Correct Answer:
C) It provides the model with information about the order of tokens in the input sequence.

Explanation:
Since Transformers process tokens in parallel rather than sequentially, they lack inherent knowledge about token order. Positional encodings inject this information, enabling the model to differentiate between positions and process sequential dependencies.