How would you describe the function of the feed-forward neural network (FFN) in the Transformer architecture? A) To perform the embedding of input tokens before applying attention. B) To apply non-linear transformations to the attention outputs for richer representations. C) To normalize the inputs and outputs of each layer for consistent scaling. D) To combine attention outputs from all heads into a single representation.
Correct Answer: B) To apply non-linear transformations to the attention outputs for richer representations.
Explanation: The FFN processes the outputs of the attention mechanism by applying a non-linear transformation. This enhances the model’s ability to learn complex patterns and representations beyond linear relationships.
