Training Skip-Gram Model

This is a sanitized, read-only preview. Nothing executes in this page.

This activity has instructions but no notebook preview.

Record your practice

Optional self-reporting helps you remember what you practiced and never gates course completion.

Back to lesson

Training Skip-Gram Model

Use this guided lab to apply the lesson concepts.

Task

In this lab, you we challenge you to train Word2Vec model using Skip-Gram technique. Train a network with embedding to predict neighboring words in $N$ -tokens-wide Skip-Gram window. You can use the code from this lesson, and slightly modify it.

The Dataset

You are welcome to use any book. You can find a lot of free texts at Project Gutenberg, for example, here is a direct link to Alice's Adventures in Wonderland) by Lewis Carroll. Or, you can use Shakespeare's plays, which you can get using the following code:

path_to_file = tf.keras.utils.get_file(
   'shakespeare.txt',
   'https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt')
text = open(path_to_file, 'rb').read.decode(encoding='utf-8')

Explore!

If you have time and want to get deeper into the subject, try to explore several things:

How does embedding size affects the results?
How does different text styles affect the result?
Take several very different types of words and their synonyms, obtain their vector representations, apply PCA to reduce dimensions to 2, and plot them in 2D space. Do you see any patterns?

Lab instructions

Record your practice

Lab instructions

Record your practice

Training Skip-Gram Model

Task

The Dataset

Explore!