Safe lab preview

GPT-Py Torch

This is a sanitized, read-only preview. Nothing executes in this page.

Read-only

Notebook preview

GPT-Py Torch

Download notebook

Markdown cell[1]

> **Integrated runtime note:** This preview uses a small deterministic, rights-safe offline fixture for reproducible learning. Full-scale results require the lesson's documented dataset or model in an approved external environment. ## Experimenting with OpenAI GPT This notebook is part of AI for Beginners. In this notebook, we will explore how we can play with OpenAI-GPT model using Hugging Face `transformers` library. Without further ado, let's instantiate text generating pipeline and start generating!

Code cell[2]

# Course-owned tiny PyTorch language model for deterministic offline decoding.
import hashlib
import re
import torch

_COURSE_LANGUAGE_CORPUS = """
artificial intelligence helps people learn solve problems and build useful tools
neural networks learn patterns from examples while responsible teams test their limits
students explore language models by comparing greedy top k and sampled decoding
clear prompts include context constraints examples and a concrete desired outcome
safe systems protect privacy document uncertainty and keep people in control
cats are curious animals dogs are loyal companions and students practice with both
bonjour means hello in french and etudiant means student
science fiction films often explore identity technology society and imagination
"""

class CourseOfflineTextGenerator:
    def __init__(self, corpus):
        tokens = re.findall(r"[a-z]+", corpus.lower())
        self.vocabulary = sorted(set(tokens))
        self.index = {token: position for position, token in enumerate(self.vocabulary)}
        counts = torch.full((len(self.vocabulary), len(self.vocabulary)), 0.05, dtype=torch.float64)
        for current, following in zip(tokens, tokens[1:]):
            counts[self.index[current], self.index[following]] += 1.0
        self.transitions = counts / counts.sum(dim=1, keepdim=True)

    @staticmethod
    def _would_repeat(tokens, candidate, width):
        if not width or width < 2 or len(tokens) < width - 1:
            return False
        proposed = tuple(tokens[-(width - 1):] + [candidate])
        return any(tuple(tokens[index:index + width]) == proposed for index in range(len(tokens) - width + 1))

    def __call__(self, prompt, max_length=40, num_return_sequences=1, top_k=None,
                 do_sample=False, temperature=1.0, num_beams=None,
                 no_repeat_ngram_size=None, **_kwargs):
        prompt_tokens = re.findall(r"[a-z]+", str(prompt).lower())
        sequence_count = min(max(int(num_return_sequences), 1), 5)
        target_length = min(max(int(max_length), len(prompt_tokens) + 1), len(prompt_tokens) + 24)
        results = []
        for sequence_index in range(sequence_count):
            generated = list(prompt_tokens)
            seed_material = f"{prompt}|{sequence_index}|{top_k}|{temperature}|{num_beams}|{do_sample}"
            seed = int.from_bytes(hashlib.sha256(seed_material.encode("utf-8")).digest()[:8], "big")
            rng = torch.Generator().manual_seed(seed % (2**63 - 1))
            while len(generated) < target_length:
                previous = generated[-1] if generated else "artificial"
                row = self.transitions[self.index.get(previous, 0)].clone()
                if temperature and float(temperature) != 1.0:
                    row = torch.softmax(torch.log(row.clamp_min(1e-12)) / max(float(temperature), 0.1), dim=0)
                if top_k:
                    keep = min(max(int(top_k), 1), len(self.vocabulary))
                    values, indices = torch.topk(row, keep)
                    candidate_index = int(indices[torch.multinomial(values / values.sum(), 1, generator=rng)])
                elif num_beams and not do_sample:
                    candidate_index = int(torch.argmax(row))
                else:
                    candidate_index = int(torch.multinomial(row, 1, generator=rng))
                candidate = self.vocabulary[candidate_index]
                if self._would_repeat(generated, candidate, no_repeat_ngram_size):
                    candidate = self.vocabulary[(candidate_index + sequence_index + 1) % len(self.vocabulary)]
                generated.append(candidate)
            continuation = " ".join(generated[len(prompt_tokens):])
            results.append({"generated_text": f"{str(prompt).strip()} {continuation}".strip()})
        return results

generator = CourseOfflineTextGenerator(_COURSE_LANGUAGE_CORPUS)

Markdown cell[3]

## Prompt Engineering In some of the problems, you can use openai-gpt generation right away by designing correct prompts. Have a look at the examples below:

Code cell[4]

generator("Synonyms of a word cat:", max_length=20, num_return_sequences=5)

Code cell[5]

generator("I love when you say this -> Positive\nI have myself -> Negative\nThis is awful for you to say this ->", max_length=40, num_return_sequences=5)

Code cell[6]

generator("Translate English to French: cat => chat, dog => chien, student => ", top_k=50, max_length=30, num_return_sequences=3)

Code cell[7]

generator("People who liked the movie The Matrix also liked ", max_length=40, num_return_sequences=5)

Markdown cell[8]

## Text Sampling Strategies So far we have been using simple **greedy** sampling strategy, when we selected next word based on the highest probability. Here is how it works:

Code cell[9]

prompt = "It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw"
generator(prompt,max_length=100,num_return_sequences=5)

Markdown cell[10]

**Beam Search** allows the generator to explore several directions (*beams*) of text generation, and select the ones with highers overall score. You can do beam search by providing `num_beams` parameter. You can also specify `no_repeat_ngram_size` to penalize the model for repeating n-grams of a given size:

Code cell[11]

prompt = "It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw"
generator(prompt,max_length=100,num_return_sequences=5,num_beams=10,no_repeat_ngram_size=2)

Markdown cell[12]

**Sampling** selects the next word non-deterministically, using the probability distribution returned by the model. You turn on sampling using `do_sample=True` parameter. You can also specify `temperature`, to make the model more or less deterministic.

Code cell[13]

prompt = "It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw"
generator(prompt,max_length=100,do_sample=True,temperature=0.8)

Markdown cell[14]

We can also provide to additional parameters to sampling: * `top_k` specifies the number of word options to consider when using sampling. This minimizes the chance of getting weird (low-probability) words in our text. * `top_p` is similar, but we chose the smallest subset of most probable words, whose total probability is larger than p. Feel free to experiment with adding those parameters in.

Markdown cell[15]

## Fine-Tuning your models You can also [fine-tune your model](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/fine-tuning?pivots=programming-language-studio) on your own dataset. This will allow you to adjust the style of text, while keeping the major part of language model.

Outputs, execution counts, widgets, and active content were removed during import. Run notebooks only in an external environment you trust.

Record your practice

Optional self-reporting helps you remember what you practiced and never gates course completion.

Back to lesson

# Course-owned tiny PyTorch language model for deterministic offline decoding. import hashlib import re import torch _COURSE_LANGUAGE_CORPUS = """ artificial intelligence helps people learn solve problems and build useful tools neural networks learn patterns from examples while responsible teams test their limits students explore language models by comparing greedy top k and sampled decoding clear prompts include context constraints examples and a concrete desired outcome safe systems protect privacy document uncertainty and keep people in control cats are curious animals dogs are loyal companions and students practice with both bonjour means hello in french and etudiant means student science fiction films often explore identity technology society and imagination """ class CourseOfflineTextGenerator: def __init__(self, corpus): tokens = re.findall(r"[a-z]+", corpus.lower()) self.vocabulary = sorted(set(tokens)) self.index = {token: position for position, token in enumerate(self.vocabulary)} counts = torch.full((len(self.vocabulary), len(self.vocabulary)), 0.05, dtype=torch.float64) for current, following in zip(tokens, tokens[1:]): counts[self.index[current], self.index[following]] += 1.0 self.transitions = counts / counts.sum(dim=1, keepdim=True) @staticmethod def _would_repeat(tokens, candidate, width): if not width or width < 2 or len(tokens) < width - 1: return False proposed = tuple(tokens[-(width - 1):] + [candidate]) return any(tuple(tokens[index:index + width]) == proposed for index in range(len(tokens) - width + 1)) def __call__(self, prompt, max_length=40, num_return_sequences=1, top_k=None, do_sample=False, temperature=1.0, num_beams=None, no_repeat_ngram_size=None, **_kwargs): prompt_tokens = re.findall(r"[a-z]+", str(prompt).lower()) sequence_count = min(max(int(num_return_sequences), 1), 5) target_length = min(max(int(max_length), len(prompt_tokens) + 1), len(prompt_tokens) + 24) results = [] for sequence_index in range(sequence_count): generated = list(prompt_tokens) seed_material = f"{prompt}|{sequence_index}|{top_k}|{temperature}|{num_beams}|{do_sample}" seed = int.from_bytes(hashlib.sha256(seed_material.encode("utf-8")).digest()[:8], "big") rng = torch.Generator().manual_seed(seed % (2**63 - 1)) while len(generated) < target_length: previous = generated[-1] if generated else "artificial" row = self.transitions[self.index.get(previous, 0)].clone() if temperature and float(temperature) != 1.0: row = torch.softmax(torch.log(row.clamp_min(1e-12)) / max(float(temperature), 0.1), dim=0) if top_k: keep = min(max(int(top_k), 1), len(self.vocabulary)) values, indices = torch.topk(row, keep) candidate_index = int(indices[torch.multinomial(values / values.sum(), 1, generator=rng)]) elif num_beams and not do_sample: candidate_index = int(torch.argmax(row)) else: candidate_index = int(torch.multinomial(row, 1, generator=rng)) candidate = self.vocabulary[candidate_index] if self._would_repeat(generated, candidate, no_repeat_ngram_size): candidate = self.vocabulary[(candidate_index + sequence_index + 1) % len(self.vocabulary)] generated.append(candidate) continuation = " ".join(generated[len(prompt_tokens):]) results.append({"generated_text": f"{str(prompt).strip()} {continuation}".strip()}) return results generator = CourseOfflineTextGenerator(_COURSE_LANGUAGE_CORPUS)

prompt = "It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw" generator(prompt,max_length=100,num_return_sequences=5,num_beams=10,no_repeat_ngram_size=2)

prompt = "It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw" generator(prompt,max_length=100,do_sample=True,temperature=0.8)