Understanding Linear Algebra in Rasa and Linguistics

How Does This Knowledge Align With Linguistics?

In computational linguistics, vector representations of words (embeddings) and sequences allow us to model language mathematically. Rasa uses vector spaces for comparing user utterances and predicting intents and actions. Linear algebra helps in building these spaces, understanding distances (e.g., cosine similarity), and transforming the data through matrix operations.

Linear Algebra Practice in Rasa Contexts

You can apply matrix multiplication when transforming word embeddings through layers (e.g., in DIETClassifier). You might calculate dot products to evaluate attention mechanisms. A practice example would be:

• Compute cosine similarity between two intent vectors
• Project a high-dimensional user utterance vector into a lower-dimensional intent space using matrix multiplication

Doing It By Hand (No Code)

Suppose you have word vectors:

        vector('hello') = [1, 2]
        vector('hi') = [2, 3]
        

        dot product = (1)(2) + (2)(3) = 2 + 6 = 8
        norm('hello') = sqrt(1^2 + 2^2) = sqrt(5)
        norm('hi') = sqrt(2^2 + 3^2) = sqrt(13)
        similarity = 8 / (sqrt(5) * sqrt(13)) ≈ 0.992
        

What Are the Analogies Comparing?

Analogies compare the *relationship* between pairs. For instance:

king - man ≈ queen - woman