From Words to Meaning: Implementing Word2Vec from Scratch

Word embeddings are one of the most transformative developments in Natural Language Processing (NLP). They solve a fundamental problem: how can we represent words as numerical vectors that capture their meaning and relationships?

Why We Need Numerical Word Representations

Machine learning models, including neural networks, classifiers, and other algorithms, operate on numerical data. They can’t directly process raw text like “cat” or “dog”. To use words in these models, we must convert them into numerical vectors.

This conversion is crucial because it enables:

Mathematical operations: Computing similarities, distances, and transformations
Learning from data: Training models to recognize patterns and relationships
Generalization: Understanding semantic relationships beyond the training examples

The challenge is finding a representation that not only converts words to numbers, but also preserves and captures their semantic meaning and relationships.

The Problem with Traditional Representations

One common approach is one-hot encoding, where each word is represented by a sparse vector with a single 1 and all other elements as 0.

For example, if we have a vocabulary of 5 words ["dog", "cat", "car", "truck", "bird"], the one-hot encoding for “cat” would be:

[0, 1, 0, 0, 0]

This representation has critical limitations:

No semantic relationships: The vectors for “cat” and “dog” are as different from each other as “cat” and “car”, even though cats and dogs are semantically similar. The distance between any two different words is always the same.
High dimensionality: For a vocabulary of size V, each word requires a V-dimensional vector. With large vocabularies (tens or hundreds of thousands of words), this becomes computationally expensive.
Sparsity: Each vector is mostly zeros, which wastes memory and computation.

The Solution: Word Embeddings

Word embeddings solve these problems by learning dense, low-dimensional vector representations where:

Similar words have similar vectors: Words that share meaning or context are positioned close together in the vector space. For example, “cat” and “dog” will have vectors that are close to each other, while “cat” and “car” will be farther apart.
Semantic relationships are captured: The vector space encodes relationships like analogies (e.g., “king” - “man” + “woman” ≈ “queen”) and semantic similarity.
Efficient representation: Instead of V dimensions (where V is vocabulary size), we use a fixed, much smaller number of dimensions (typically 50-few thousands), making the representation both dense and computationally efficient.

Word2Vec: A Breakthrough in Static Embeddings

Word2Vec, introduced by Mikolov et al. in 2013, was a breakthrough that made high-quality word embeddings accessible and practical. It learns static embeddings where each word gets a single, fixed vector representation regardless of context.

While modern transformer models (like BERT and GPT) learn contextual embeddings (where the same word can have different vectors depending on context), Word2Vec-style static embeddings remain valuable for many applications and provide an excellent foundation for understanding how neural networks can learn meaningful word representations.

In this post, we will implement Word2Vec from scratch using Python and PyTorch, learning how to generate word embeddings that capture semantic relationships from raw text.

The Idea Behind Word2Vec

“You shall know a word by the company it keeps” - J.R. Firth

The fundamental insight behind Word2Vec is the distributional hypothesis: words that appear in similar contexts tend to have similar meanings. If two words frequently appear near the same surrounding words, they likely share semantic properties.

For example, consider the sentences:

“The cat sat on the mat”
“The dog sat on the mat”

Since “cat” and “dog” appear in similar contexts (both before “sat on the mat”), they should have similar vector representations. This is exactly what Word2Vec learns to do.

How Word2Vec Works

Word2Vec learns word embeddings by training a neural network to predict words from their context. The key insight is that by learning to predict context words, the model must learn meaningful word representations such that words that need to predict similar contexts will naturally develop similar vectors.

Word2Vec offers two architectures:

Skip-Gram: Given a target word, predict the surrounding context words. For example, given “cat”, predict words like “the”, “sat”, “on”, “mat” that appear nearby.
CBOW (Continuous Bag of Words): Given the surrounding context words, predict the target word. For example, given “the”, “sat”, “on”, “mat”, predict “cat”.

Both approaches learn embeddings that capture semantic relationships, but they optimize from different directions. We will implement the Skip-Gram model, which is often preferred for larger datasets and tends to produce better embeddings for infrequent words.

Math Behind Word2Vec

Let’s build up the mathematical formulation of Word2Vec step by step, starting with the intuitive goal and working our way to the optimization objective.

The Intuitive Goal

For Skip-Gram, we want to maximize the probability of correctly predicting context words given a target word. If we see the word “cat” in our text, we want our model to assign high probability to words like “the”, “sat”, “on”, “mat” that typically appear near “cat”.

The Objective Function

Given a sequence of training words $w_1, w_2, w_3, \ldots, w_T$, for each target word $w_t$, we want to maximize the probability of predicting all words within a context window of size $c$.

For a single target word $w_t$, we want to maximize: $\prod_{-c \leq j \leq c, j \neq 0} p(w_{t+j} \mid w_t)$

This is the probability of predicting all context words $w_{t-c}, \ldots, w_{t-1}, w_{t+1}, \ldots, w_{t+c}$ given the target word $w_t$.

Taking the logarithm (which converts products to sums and is numerically more stable), we get: $\sum_{-c \leq j \leq c, j \neq 0} \log p(w_{t+j} \mid w_t)$

Averaging over all positions in the training sequence, our objective becomes: $\frac{1}{T} \sum_{t=1}^{T} \sum_{-c\leq j\leq c, j\neq 0} \log p(w_{t+j} \mid w_t)$

Goal: Maximize this quantity (or equivalently, minimize its negative, which becomes our loss function).

Modeling the Conditional Probability

Now we need to define $p(w_O \mid w_I)$ : The probability of seeing output word $w_O$ given input word $w_I$.

Word2Vec uses a simple but powerful approach: it represents each word with two vectors:

Input embedding $v_{w_I}$: used when the word is the target (input)
Output embedding $v’_{w_O}$: used when the word is in the context (output)

The similarity between a target word and a context word is measured by their dot product: $(v’{w_O})^{\top} v{w_I}$. Higher dot product means higher similarity, which should correspond to higher probability.

To convert these similarity scores into probabilities (which must sum to 1 over all possible context words), we use the softmax function:

\[p(w_O \mid w_I) = \frac{\exp((v'_{w_O})^{\top} v_{w_I})}{\sum_{w=1}^V \exp((v'_{w})^{\top} v_{w_I})}\]

The numerator $\exp((v'_{w_O})^{\top} v_{w_I})$ gives higher probability to words with high similarity. The denominator $\sum_{w=1}^V \exp((v'_{w})^{\top} v_{w_I})$ normalizes over all $V$ words in the vocabulary, ensuring probabilities sum to 1.

The Computational Challenge

Here’s the problem: computing the softmax is expensive. This is fundamentally a multi-class classification problem: given a target word, we need to assign higher probability to the actual context words and lower probability to all other words in the vocabulary.

For each training example, computing the softmax requires:

Compute dot products with all $V$ words in the vocabulary
Exponentiate all $V$ values
Sum them for normalization

With a vocabulary of 50,000 words and millions of training examples, this becomes computationally prohibitive. Each gradient update would require $O(V)$ operations, making training extremely slow.

Negative Sampling

Instead of comparing the target word against all $V$ words every time (expensive), we can approximate this by sampling. Over many training iterations, we’ll eventually compare against most words, but we only need to compute a few at a time.

Negative sampling transforms the multi-class classification problem into a simpler binary classification task:

Positive examples: (target word, actual context word) pairs that appear together in the text
Negative examples: (target word, random word) pairs that don’t appear together

Instead of computing probabilities over all $V$ words, we:

Maximize the probability of the actual context word (positive example)
Minimize the probability of $k$ randomly sampled words (negative examples)

Mathematical Formulation: We model this as binary classification where $D$ is a binary indicator:

$D=1$ means words $w_I$ and $w_O$ appear together (positive pair)
$D=0$ means they don’t appear together (negative pair)

The probability of a positive pair is: $p(D=1 \mid w_I, w_O) = \sigma((v'_{w_O})^{\top} v_{w_I})$

The probability of a negative pair is: $p(D=0 \mid w_I, w_O) = \sigma(-(v'_{w_O})^{\top} v_{w_I})$

where $\sigma(x) = \frac{1}{1 + e^{-x}}$ is the sigmoid function. The negative sign in $p(D=0)$ ensures that words with low similarity (low dot product) get low probability.

Why This Approximates Softmax:

The softmax compares the target word against all $V$ words simultaneously
Negative sampling compares against $k$ random words at a time
Over many training iterations with different random samples, we effectively compare against most words in the vocabulary
This stochastic approximation achieves similar learning while being orders of magnitude faster

For a target word $w_I$ and context word $w_O$, the objective becomes:

\[\log \sigma((v'_{w_O})^{\top} v_{w_I}) + \sum_{i=1}^{k} \log \sigma(-(v'_{w_i})^{\top} v_{w_I})\]

where:

The first term $\log \sigma((v’{w_O})^{\top} v{w_I})$ maximizes the probability that the actual context word $w_O$ appears with $w_I$
The second term $\sum_{i=1}^{k} \log \sigma(-(v’{w_i})^{\top} v{w_I})$ minimizes the probability that $k$ randomly sampled words $w_i$ appear with $w_I$

Efficiency Gain: Instead of computing over all $V$ words, we only need to compute $k+1$ dot products (typically $k=5$ to $20$), reducing complexity from $O(V)$ to $O(k)$, which is orders of magnitude faster!

Step 6: Why This Works

By learning to distinguish real context words from random words, the model must learn embeddings where:

Words that appear together have high dot products (similar vectors)
Words that don’t appear together have low dot products (dissimilar vectors)

This naturally leads to the semantic clustering we want: similar words end up with similar embeddings. The stochastic sampling of negative words over many iterations ensures that the model eventually learns to distinguish the target word from most other words in the vocabulary, approximating the effect of the full softmax.

Alternative: Hierarchical Softmax

Another approach to avoid the full softmax is hierarchical softmax, which uses a binary tree structure over the vocabulary. Instead of computing over all words, it follows a path through the tree (requiring only $\log_2(V)$ operations). However, negative sampling is simpler to implement and often performs better in practice, which is why we’ll use it in our implementation.

Setup

Let’s start by importing the necessary libraries and loading the data.

import os
import re
import math
import random
import urllib.request
import zipfile
import numpy as np
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, Dataset
from torch import optim
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from nltk.tokenize import word_tokenize
from tqdm import tqdm
import collections

import nltk
nltk.download('punkt_tab')

[nltk_data] Downloading package punkt_tab to
[nltk_data]     /Users/ravi.mandliya/nltk_data...
[nltk_data]   Package punkt_tab is already up-to-date!

True

Download and Prepare the Data

We will use the text8 dataset, which is a preprocessed version of the Wikipedia corpus.

DATA_URL = 'https://mattmahoney.net/dc/text8.zip'
DATA_DIR = './data/'
ZIP_FILE_NAME = 'text8.zip'
TEXT_FILE_NAME = 'text8'

ZIP_FILE_PATH = os.path.join(DATA_DIR, ZIP_FILE_NAME)
TEXT_FILE_PATH = os.path.join(DATA_DIR, TEXT_FILE_NAME)

# Create data directory if it doesn't exist
os.makedirs(DATA_DIR, exist_ok=True)

# Download and extract if the text file doesn't exist
if not os.path.exists(TEXT_FILE_PATH):
    print(f"Downloading {ZIP_FILE_NAME}...")
    urllib.request.urlretrieve(DATA_URL, ZIP_FILE_PATH)
    
    print(f"Extracting {ZIP_FILE_NAME}...")
    with zipfile.ZipFile(ZIP_FILE_PATH, 'r') as zip_ref:
        zip_ref.extractall(DATA_DIR)
    
    print(f"Data extracted to {DATA_DIR}")
else:
    print(f"Text file already exists at {TEXT_FILE_PATH}")

# Read and tokenize the text
print(f"Reading text from {TEXT_FILE_PATH}...")
with open(TEXT_FILE_PATH, 'r', encoding='utf-8') as f:
    text = f.read()
    words = word_tokenize(text)

print(f"Total words: {len(words)}")
print(f"First 100 words: {words[:100]}")

Text file already exists at ./data/text8
Reading text from ./data/text8...
Total words: 17007698
First 100 words: ['anarchism', 'originated', 'as', 'a', 'term', 'of', 'abuse', 'first', 'used', 'against', 'early', 'working', 'class', 'radicals', 'including', 'the', 'diggers', 'of', 'the', 'english', 'revolution', 'and', 'the', 'sans', 'culottes', 'of', 'the', 'french', 'revolution', 'whilst', 'the', 'term', 'is', 'still', 'used', 'in', 'a', 'pejorative', 'way', 'to', 'describe', 'any', 'act', 'that', 'used', 'violent', 'means', 'to', 'destroy', 'the', 'organization', 'of', 'society', 'it', 'has', 'also', 'been', 'taken', 'up', 'as', 'a', 'positive', 'label', 'by', 'self', 'defined', 'anarchists', 'the', 'word', 'anarchism', 'is', 'derived', 'from', 'the', 'greek', 'without', 'archons', 'ruler', 'chief', 'king', 'anarchism', 'as', 'a', 'political', 'philosophy', 'is', 'the', 'belief', 'that', 'rulers', 'are', 'unnecessary', 'and', 'should', 'be', 'abolished', 'although', 'there', 'are', 'differing']

Build the Vocabulary

We’ll create mappings between words and integer IDs and remove rare words to keep vocabulary manageable. The code is below followed by the explanation.

class Word2VecDataset(Dataset):
  def __init__(self, words, word_to_idx, word_freqs, window_size=5,
               negative_sample_counts=5, subsample_threshold=1e-3):
    # words: already tokenized list of words (from Word2Vec class)
    # word_to_idx: {word: idx} mapping from Word2Vec class
    # word_freqs: {word: frequency} mapping from Word2Vec class (already computed)
    self.words = words
    self.word_to_idx = word_to_idx
    self.word_freqs = word_freqs  # Use precomputed frequencies from Word2Vec
    self.vocab_size = len(word_to_idx)  # Vocabulary size for negative sampling
    self.window_size = window_size
    self.negative_sample_counts = negative_sample_counts
    self.subsample_threshold = subsample_threshold

    # Convert words to indices, filtering out words not in vocabulary
    self.encoded_text = [self.word_to_idx[word] for word in words if word in self.word_to_idx]

    # Compute word frequencies by index for subsampling
    # word_freqs is {word: frequency}, we need {idx: frequency}
    self.word_freqs_by_idx = {word_to_idx[word]: freq for word, freq in word_freqs.items() if word in word_to_idx}

    # subsample probability: probability to keep each word
    # Formula: P(keep) = min(1.0, sqrt(t / f(w))) where t is threshold, f(w) is frequency
    # Rare words (f < t): P(keep) = 1.0 (always kept)
    # Frequent words (f > t): P(keep) = sqrt(t/f) < 1.0 (discarded more often)
    self.subsample_probs = {
      idx: min(1.0, np.sqrt(self.subsample_threshold / freq)) if freq > 0 else 1.0
      for idx, freq in self.word_freqs_by_idx.items()
    }
    
    self.pairs = []
    np.random.seed(42)  # Fixed seed for deterministic subsampling
    random.seed(42)
    
    for i, center_word in enumerate(self.encoded_text):
      # Apply subsampling deterministically
      if center_word not in self.subsample_probs:
        continue
      if np.random.rand() >= self.subsample_probs[center_word]:
        continue
      
      # Generate all context pairs for this center word
      # Use random window size for variety (but deterministic with fixed seed)
      context_window = random.randint(1, self.window_size)
      start = max(0, i - context_window)
      end = min(len(self.encoded_text), i + context_window + 1)
      
      for j in range(start, end):
        if i != j:
          self.pairs.append((center_word, self.encoded_text[j]))
    
    # Reset random seed for training randomness
    np.random.seed()
    random.seed()
    
    self.negative_sample_weights = self._generate_negative_sample_weights()

  def _generate_negative_sample_weights(self):
    # Use vocab_size to ensure correct array size for negative sampling
    # Convert word frequencies to counts for negative sampling distribution
    # Only use words that are in the vocabulary
    total_count = len(self.words)
    freq = np.zeros(self.vocab_size)
    for word in self.word_to_idx.keys():
      if word in self.word_freqs:
        idx = self.word_to_idx[word]
        word_freq = self.word_freqs[word]
        # Convert frequency back to count
        freq[idx] = word_freq * total_count
      else:
        # If word is in vocab but not in word_freqs (shouldn't happen, but handle gracefully)
        idx = self.word_to_idx[word]
        freq[idx] = 0.0
    
    # Apply power law: freq^0.75 (standard Word2Vec approach)
    # This raises each word's count to the 3/4 power, which helps balance
    # the distribution between frequent and rare words
    freq = np.power(freq, 0.75)
    # Normalize to get probability distribution
    freq = freq / np.sum(freq)
    return torch.FloatTensor(freq)

  def __len__(self):
    return len(self.pairs)

  def __getitem__(self, idx):
    # Simple O(1) lookup - pairs are already generated
    target, context = self.pairs[idx]
    
    # Generate negative samples on-the-fly
    negs = torch.multinomial(
      self.negative_sample_weights,
      self.negative_sample_counts,
      replacement=True
    )
    
    return torch.tensor(target, dtype=torch.long), torch.tensor(context, dtype=torch.long), negs

Building the Word2Vec Dataset

The Word2VecDataset class transforms raw text into training examples for the Skip-Gram model. This process involves several key steps, each with important mathematical foundations. Let’s break down how the dataset is constructed:

Step 1: Encoding Text to Indices

First, we convert words to integer indices for efficient processing:

self.encoded_text = [self.word_to_idx[word] for word in text if word in self.word_to_idx]

This creates a sequence of integer IDs, filtering out words not in our vocabulary.

Step 2: Subsampling Frequent Words

Word2Vec uses subsampling to balance the training data. Very frequent words (like “the”, “a”, “of”) provide less semantic information but dominate the training signal. Subsampling randomly discards these words with probability proportional to their frequency.

Mathematical Formulation: For a word $w$ with frequency $f(w)$ in the corpus, the probability of keeping the word is:

\[P(\text{keep } w) = \min\left(1.0, \sqrt{\frac{t}{f(w)}}\right)\]

where $t$ is the subsampling threshold (typically $10^{-3}$ to $10^{-5}$).

Intuition:

Rare words ($f(w) < t$): $P(\text{keep}) = 1.0$ : Always kept, preserving important semantic information
Frequent words ($f(w) > t$): $P(\text{keep}) = \sqrt{t/f(w)} < 1.0$ : Discarded more often, reducing their dominance

For example, if $t = 0.001$ and a word appears with frequency $f(w) = 0.01$ (1% of all words):

$P(\text{keep}) = \sqrt{0.001/0.01} = \sqrt{0.1} \approx 0.316$

This means we keep this word only about 31.6% of the time, effectively downweighting it in training.

Why $\sqrt{t/f(w)}$? This formula ensures that:

Words with frequency exactly equal to the threshold are kept with probability 1.0
The probability decreases smoothly as frequency increases
Very frequent words (like “the”) are heavily downweighted

Step 3: Generating Training Pairs

For each center word $w_t$ at position $t$ in the (subsampled) text, we generate context pairs:

Random Context Window: Instead of a fixed window size, we use a random window size $c \in [1, \text{window_size}]$ for each center word. This adds variety to the training data.
Context Extraction: For a center word at position $i$ with window size $c$, we extract context words from positions $[i-c, i-1]$ and $[i+1, i+c]$.

Pair Generation: Each context word $w_{t+j}$ (where $j \neq 0$ and $

\leq c$) is paired with the center word $w_t$ to create a training example $(w_t, w_{t+j})$.

Example: For the sentence “the quick brown fox jumps” with window_size=2:

Center word “brown” at position 2
Random window $c=2$ selected
Context words: “the” (position 0), “quick” (position 1), “fox” (position 3), “jumps” (position 4)
Generated pairs: ("brown", "the"), ("brown", "quick"), ("brown", "fox"), ("brown", "jumps")

Step 4: Negative Sampling Distribution

Negative sampling requires a probability distribution over the vocabulary to sample “negative” (non-context) words. The original Word2Vec implementation uses a unigram distribution raised to the 3/4 power.

Mathematical Formulation: For each word $w$ with count $c(w)$ in the corpus, we compute:

\[P_n(w) = \frac{c(w)^{3/4}}{\sum_{w' \in V} c(w')^{3/4}}\]

where $P_n(w)$ is the probability of sampling word $w$ as a negative example, and $V$ is the vocabulary.

Why $c(w)^{3/4}$? The 3/4 power (0.75) is a heuristic that:

Reduces the dominance of very frequent words: Without the power, words like “the” would be sampled too often as negatives
Increases the probability of moderately frequent words: This helps the model learn better distinctions
Still favors frequent words over rare ones: Rare words are less useful as negative examples

Example: If “the” appears 100,000 times and “cat” appears 1,000 times:

Without power: $P_n(\text{the}) / P_n(\text{cat}) = 100,000 / 1,000 = 100$
With 3/4 power: $P_n(\text{the}) / P_n(\text{cat}) = 100,000^{0.75} / 1,000^{0.75} \approx 31,623 / 178 \approx 178$

The ratio is still large (178:1), but much smaller than 100:1, giving “cat” a better chance of being sampled.

Step 5: On-the-Fly Negative Sampling

During training, for each positive pair $(w_t, w_c)$, we sample $k$ negative words from $P_n(w)$:

negs = torch.multinomial(self.negative_sample_weights, self.negative_samples, replacement=True)

This creates $k$ negative examples $(w_t, w_{\text{neg}1}), \ldots, (w_t, w{\text{neg}k})$ where each $w{\text{neg}_i}$ is sampled according to $P_n(w)$.

Why on-the-fly? Generating negative samples during __getitem__ is efficient because:

It avoids storing millions of pre-generated negative samples
torch.multinomial is highly optimized
The randomness helps with generalization

We created a dataset class that generates pairs of target and context words. It also generates negative samples on-the-fly.

Define the Model

We’ll use PyTorch to define the Word2Vec model. We will have two embedding matrices:

Input embedding matrix $V \times d$: maps target words to their embeddings
Output embedding matrix $V \times d$: maps context words to their embeddings

The model will learn two embedding matrices, one for the target words and one for the context words. We’ll use uniform initialization for the embeddings to ease the computation of the loss function.

class SkipGramModel(nn.Module):
  def __init__(self, vocab_size, embedding_dim):
    super().__init__()
    self.in_embed = nn.Embedding(vocab_size, embedding_dim)
    self.out_embed = nn.Embedding(vocab_size, embedding_dim)
    self._init_weights()

  def _init_weights(self):
    initrange = 0.5 / self.in_embed.embedding_dim
    self.in_embed.weight.data.uniform_(-initrange, initrange)
    self.out_embed.weight.data.uniform_(-initrange, initrange)

  def forward(self, center_words, context_words, negative_samples):
    center_embeds = self.in_embed(center_words) # (batch_size, embedding_dim)
    context_embeds = self.out_embed(context_words) # (batch_size, embedding_dim)
    negative_embeds = self.out_embed(negative_samples) # (batch_size, negative_samples, embedding_dim)

    # Compute the loss
    pos_score = torch.sum(center_embeds * context_embeds, dim=1) # (batch_size,)
    pos_score = torch.sigmoid(pos_score)

    # Compute dot products: (batch_size, negative_samples, embedding_dim) @ (batch_size, embedding_dim, 1)
    # center_embeds.unsqueeze(2) adds dimension: (batch_size, embedding_dim) -> (batch_size, embedding_dim, 1)
    neg_score = torch.bmm(negative_embeds, center_embeds.unsqueeze(2)) # (batch_size, negative_samples, 1)
    neg_score = torch.sigmoid(neg_score).squeeze(2) # (batch_size, negative_samples)

    return pos_score, neg_score

  def get_embedding(self, word_idx):
    return self.in_embed(torch.LongTensor([word_idx])).detach()

  def get_all_center_embeddings(self):
    return self.in_embed.weight.data

Understanding the Skip-Gram Model Architecture

The SkipGramModel implements the core Word2Vec architecture using PyTorch. Let’s break down how it works:

Dual Embedding Matrices

Word2Vec uses two separate embedding matrices for each word:

in_embed: Input embeddings used when a word is the center/target word (the word we’re predicting context for)
out_embed: Output embeddings used when a word is a context word (or negative sample)

This dual representation is a key design choice. While it doubles the number of parameters, it often leads to better embeddings because the model can learn different representations for the same word depending on its role (center vs. context).

Weight Initialization

The embeddings are initialized uniformly in the range $[-0.5/d, 0.5/d]$ where $d$ is the embedding dimension:

initrange = 0.5 / embedding_dim

This small initialization range helps with:

Stable training: Prevents gradients from exploding early in training
Symmetry breaking: Small random values ensure different words start with different embeddings
Numerical stability: Keeps initial dot products in a reasonable range for the sigmoid function

Forward Pass: Computing Similarity Scores

The forward method computes how well the model predicts context words:

Embedding Lookup:
- center_embeds: Gets embeddings for target words (shape: (batch_size, embedding_dim))
- context_embeds: Gets embeddings for actual context words (shape: (batch_size, embedding_dim))
- negative_embeds: Gets embeddings for negative samples (shape: (batch_size, negative_samples, embedding_dim))
Positive Score: Measures similarity between center and context words:
```
pos_score = torch.sum(center_embeds * context_embeds, dim=1)
pos_score = torch.sigmoid(pos_score)
```
- Element-wise multiplication followed by sum computes the dot product (cosine similarity when embeddings are normalized)
- The sigmoid converts the dot product into a probability $p \in (0, 1)$
- Higher dot product → higher probability → model is confident these words appear together
Negative Score: Measures similarity between center words and negative samples:
```
neg_score = torch.bmm(negative_embeds, center_embeds.unsqueeze(2))
neg_score = torch.sigmoid(neg_score).squeeze(2)
```
- torch.bmm performs batch matrix multiplication: for each sample in the batch, it computes dot products between the center embedding and all negative embeddings
- unsqueeze(2) adds a dimension to center_embeds: (batch_size, embedding_dim) → (batch_size, embedding_dim, 1) to make it compatible with batch matrix multiplication
- The matrix multiplication: (batch_size, negative_samples, embedding_dim) @ (batch_size, embedding_dim, 1) = (batch_size, negative_samples, 1)
- We squeeze(2) to remove the last dimension, getting (batch_size, negative_samples)
- Each value represents how similar the center word is to a negative sample (we want this to be low)

The Learning Objective

The model returns pos_score and neg_score, which will be used in the loss function:

\[\mathcal{L} = -\log(\text{pos\_score}) - \sum_{i=1}^{k} \log(1 - \text{neg\_score}_i)\]

This loss function:

Maximizes pos_score (probability that real context words appear with the center word)
Minimizes neg_score (probability that random words appear with the center word)

By optimizing this objective, the model learns embeddings where:

Words that appear together in text have high dot products (similar vectors)
Words that don’t appear together have low dot products (dissimilar vectors)

Training the Word2Vec Model

Now in this section, we will train the Word2Vec model. We will use the Word2Vec class to train the model. The code is below followed by the explanation.

class Word2Vec:
  def __init__(self, embedding_dim=128, window_size=5, negative_samples_counts=5, min_count=5, learning_rate=0.001, num_epochs=5):
    self.embedding_dim = embedding_dim
    self.window_size = window_size
    self.negative_samples_counts = negative_samples_counts
    self.min_count = min_count
    self.learning_rate = learning_rate
    self.num_epochs = num_epochs

    self.word_to_idx = None
    self.idx_to_word = None
    self.dataset = None

    self.model = None
    self.optimizer = None
    
  def _preprocess_text_and_build_vocab(self, text):
    # Tokenize the text once
    self.words = word_tokenize(text)
    # Count words for vocabulary building and frequency calculations
    word_counts = collections.Counter(self.words)
    # Filter words that meet min_count, then assign contiguous indices 0, 1, 2, ...
    # This ensures indices are in range [0, vocab_size-1]
    words_meeting_min_count = [(word, count) for word, count in word_counts.items() if count >= self.min_count]
    self.word_to_idx = {word: idx for idx, (word, count) in enumerate(words_meeting_min_count)}
    # Build reverse mapping: {idx: word}
    self.idx_to_word = {idx: word for word, idx in self.word_to_idx.items()}
    # Compute word frequencies for subsampling and negative sampling
    # Only compute frequencies for words in vocabulary (meets min_count)
    total_count = len(self.words)
    self.word_freqs = {}
    for word, count in word_counts.items():
      if word in self.word_to_idx:  # Only include words in vocabulary
        self.word_freqs[word] = count / total_count
    print(f"Vocabulary size: {len(self.word_to_idx)}")

  def _compute_loss(self, pos_score, neg_score):
    # pos_score: (batch_size,)
    # neg_score: (batch_size, negative_samples)
    # Loss formula: -log(pos_score) - sum_{i=1}^k log(1 - neg_score_i)
    pos_loss = -torch.log(pos_score + 1e-10).mean()
    # Sum over negative_samples dimension first, then average over batch
    neg_loss = -torch.log(1 - neg_score + 1e-10).sum(dim=1).mean()
    return pos_loss + neg_loss

  def train(self, text, batch_size=128):
    self._preprocess_text_and_build_vocab(text)
    # Pass preprocessed data to the dataset
    # Dataset only handles training example generation, not text processing
    self.dataset = Word2VecDataset(self.words, self.word_to_idx, self.word_freqs,
      self.window_size, self.negative_samples_counts)
    dataloader = DataLoader(self.dataset, batch_size=batch_size, shuffle=True)
    self.model = SkipGramModel(len(self.word_to_idx), self.embedding_dim)
    self.optimizer = optim.Adam(self.model.parameters(), lr=self.learning_rate)


    print(f"Training for {self.num_epochs} epochs...")
    for epoch in range(self.num_epochs):
      epoch_loss = 0
      # Create tqdm wrapper inside the loop for each epoch
      tqdm_dataloader = tqdm(dataloader, total=len(dataloader), desc=f"Epoch {epoch+1}/{self.num_epochs}")
      for idx, (target, context, negs) in enumerate(tqdm_dataloader):
        self.optimizer.zero_grad()
        pos_score, neg_score = self.model(target, context, negs)
        loss = self._compute_loss(pos_score, neg_score)
        loss.backward()
        self.optimizer.step()
        epoch_loss += loss.item()
        if (idx + 1) % 100 == 0:
          print(f"Epoch {epoch+1} Batch {idx+1} loss: {loss.item()}")
          # checkpointing
          torch.save({
            "epoch": epoch,
            "model_state_dict": self.model.state_dict(),
            "optimizer_state_dict": self.optimizer.state_dict(),
            "loss": loss.item()
          }, f"data/checkpoint_{epoch+1}_{idx+1}.pth")
          # print closest words to the center word
          print(f"Closest words to the center word {self.idx_to_word[target[0].item()]}: {self.most_similar(self.idx_to_word[target[0].item()])}")
      avg_loss = epoch_loss / len(dataloader)
      print(f"Epoch {epoch+1} loss: {avg_loss}")

    print("Training complete!")

  def most_similar(self, word, top_k=10):
    word_idx = self.word_to_idx[word]
    embedding = self.model.get_embedding(word_idx)
    all_embeddings = self.model.get_all_center_embeddings()
    similarities = torch.matmul(embedding, all_embeddings.T) # (1, vocab_size)
    similarities = similarities.squeeze(0) # (vocab_size,)
    _, indices = torch.topk(similarities, top_k)
    return [self.idx_to_word[idx.item()] for idx in indices]

  
  def get_embedding(self, word):
    word_idx = self.word_to_idx[word]
    return self.model.get_embedding(word_idx).detach()

if __name__ == "__main__":
  word2vec = Word2Vec(embedding_dim=128, window_size=5,
    negative_samples_counts=5, min_count=5, learning_rate=0.001, num_epochs=2)

word2vec.train(text, batch_size=2048)

Vocabulary size: 71286
Training for 2 epochs...

Epoch 1/2:   0%|          | 99/34343 [01:33<2:15:28,  4.21it/s] 

Epoch 1 Batch 100 loss: 3.774440050125122

Epoch 1/2:   0%|          | 100/34343 [01:34<3:06:45,  3.06it/s]

Closest words to the center word stones: ['the', 'of', 'and', 'in', 'a', 'to', 'as', 'by', 'that', 'at']

Epoch 1/2:   1%|          | 199/34343 [03:11<38:21:02,  4.04s/it]

Epoch 1 Batch 200 loss: 3.144181489944458

Epoch 1/2:   1%|          | 200/34343 [03:12<28:15:59,  2.98s/it]

Closest words to the center word through: ['the', 'in', 'a', 'of', 'zero', 'on', 'at', 'and', 'first', 'is']

Epoch 1/2:   1%|          | 299/34343 [04:30<5:17:29,  1.79it/s] 

Epoch 1 Batch 300 loss: 2.9165852069854736

Epoch 1/2:   1%|          | 300/34343 [04:31<5:19:34,  1.78it/s]

Closest words to the center word that: ['part', 'number', 'time', 'used', 'be', 'same', 'state', 'use', 'such', 'known']

Epoch 1/2:   1%|          | 399/34343 [05:48<2:37:18,  3.60it/s] 

Epoch 1 Batch 400 loss: 2.7336912155151367

Epoch 1/2:   1%|          | 400/34343 [05:48<3:10:16,  2.97it/s]

Closest words to the center word president: ['nine', 'eight', 'seven', 'zero', 'six', 'one', 'four', 'five', 'two', 'three']

Epoch 1/2:   1%|▏         | 500/34343 [06:59<2:40:33,  3.51it/s] 

Epoch 1 Batch 500 loss: 2.6643283367156982
Closest words to the center word special: ['eight', 'nine', 'four', 'seven', 'case', 'zero', 'name', 'republic', 'isbn', 'referred']

Epoch 1/2:   2%|▏         | 599/34343 [08:10<18:33:53,  1.98s/it]

Epoch 1 Batch 600 loss: 2.6351046562194824

Epoch 1/2:   2%|▏         | 600/34343 [08:11<14:17:31,  1.52s/it]

Closest words to the center word zero: ['isbn', 'eight', 'seven', 'nine', 'zero', 'six', 'four', 'five', 'one', 'three']

Epoch 1/2:   2%|▏         | 699/34343 [09:21<3:55:13,  2.38it/s] 

Epoch 1 Batch 700 loss: 2.578791856765747

Epoch 1/2:   2%|▏         | 700/34343 [09:22<4:28:48,  2.09it/s]

Closest words to the center word dharma: ['isbn', 'nine', 'eight', 'six', 'zero', 'seven', 'creation', 'four', 'july', 'one']

Epoch 1/2:   2%|▏         | 799/34343 [10:30<2:06:13,  4.43it/s] 

Epoch 1 Batch 800 loss: 2.5281713008880615

Epoch 1/2:   2%|▏         | 800/34343 [10:30<3:08:00,  2.97it/s]

Closest words to the center word victor: ['isbn', 'nine', 'seven', 'zero', 'eight', 'six', 'births', 'three', 'km', 'four']

Epoch 1/2:   3%|▎         | 899/34343 [11:28<1:57:59,  4.72it/s] 

Epoch 1 Batch 900 loss: 2.517162799835205

Epoch 1/2:   3%|▎         | 900/34343 [11:29<3:03:03,  3.04it/s]

Closest words to the center word radiation: ['isbn', 'nine', 'seven', 'births', 'ability', 'rest', 'six', 'result', 'zero', 'eight']

Epoch 1/2:   3%|▎         | 999/34343 [12:52<16:58:02,  1.83s/it]

Epoch 1 Batch 1000 loss: 2.5241129398345947

Epoch 1/2:   3%|▎         | 1000/34343 [12:52<13:35:01,  1.47s/it]

Closest words to the center word precocious: ['isbn', 'nine', 'zero', 'km', 'seven', 'june', 'december', 'births', 'eight', 'july']

Epoch 1/2:   3%|▎         | 1099/34343 [13:57<2:48:34,  3.29it/s] 

Epoch 1 Batch 1100 loss: 2.4730119705200195

Epoch 1/2:   3%|▎         | 1100/34343 [13:58<3:36:10,  2.56it/s]

Closest words to the center word a: ['isbn', 'not', 'zero', 'births', 'december', 'refer', 'referred', 'nine', 'used', 'be']

Epoch 1/2:   3%|▎         | 1199/34343 [15:00<2:00:17,  4.59it/s] 

Epoch 1 Batch 1200 loss: 2.4821088314056396

Epoch 1/2:   3%|▎         | 1200/34343 [15:00<3:06:05,  2.97it/s]

Closest words to the center word justice: ['isbn', 'births', 'september', 'nine', 'eight', 'december', 'km', 'zero', 'june', 'july']

Epoch 1/2:   4%|▍         | 1299/34343 [16:03<1:55:52,  4.75it/s] 

Epoch 1 Batch 1300 loss: 2.4716808795928955

Epoch 1/2:   4%|▍         | 1300/34343 [16:04<3:02:40,  3.01it/s]

Closest words to the center word least: ['isbn', 'births', 'december', 'june', 'km', 'january', 'july', 'november', 'zero', 'september']

Epoch 1/2:   4%|▍         | 1399/34343 [17:11<6:05:14,  1.50it/s] 

Epoch 1 Batch 1400 loss: 2.4694762229919434

Epoch 1/2:   4%|▍         | 1400/34343 [17:12<6:05:56,  1.50it/s]

Closest words to the center word gettier: ['isbn', 'births', 'december', 'nine', 'km', 'july', 'eight', 'january', 'seven', 'six']

Epoch 1/2:   4%|▍         | 1499/34343 [18:19<2:48:39,  3.25it/s] 

Epoch 1 Batch 1500 loss: 2.460878849029541

Epoch 1/2:   4%|▍         | 1500/34343 [18:19<3:38:57,  2.50it/s]

Closest words to the center word music: ['isbn', 'births', 'importance', 'km', 'november', 'february', 'july', 'pp', 'eight', 'basis']

Epoch 1/2:   5%|▍         | 1599/34343 [19:22<1:59:59,  4.55it/s] 

Epoch 1 Batch 1600 loss: 2.453460931777954

Epoch 1/2:   5%|▍         | 1600/34343 [19:22<3:04:47,  2.95it/s]

Closest words to the center word popular: ['referred', 'births', 'isbn', 'true', 'believed', 'rest', 'importance', 'idea', 'result', 'absence']

Epoch 1/2:   5%|▍         | 1699/34343 [20:20<1:59:22,  4.56it/s] 

Epoch 1 Batch 1700 loss: 2.4053971767425537

Epoch 1/2:   5%|▍         | 1700/34343 [20:31<31:43:42,  3.50s/it]

Closest words to the center word old: ['births', 'isbn', 'km', 'january', 'november', 'zero', 'june', 'eight', 'pp', 'th']

Epoch 1/2:   5%|▌         | 1799/34343 [21:30<4:53:54,  1.85it/s] 

Epoch 1 Batch 1800 loss: 2.4356462955474854

Epoch 1/2:   5%|▌         | 1800/34343 [21:31<5:09:49,  1.75it/s]

Closest words to the center word in: ['births', 'isbn', 'pp', 'february', 'november', 'inducted', 'june', 'eight', 'km', 'nine']

Epoch 1/2:   6%|▌         | 1899/34343 [22:38<1:57:59,  4.58it/s] 

Epoch 1 Batch 1900 loss: 2.4106228351593018

Epoch 1/2:   6%|▌         | 1900/34343 [22:38<3:03:19,  2.95it/s]

Closest words to the center word bradley: ['births', 'isbn', 'pp', 'km', 'nine', 'february', 'november', 'laureate', 'inducted', 'eight']

Epoch 1/2:   6%|▌         | 1999/34343 [23:47<1:55:14,  4.68it/s] 

Epoch 1 Batch 2000 loss: 2.4157354831695557

Epoch 1/2:   6%|▌         | 2000/34343 [23:48<2:59:00,  3.01it/s]

Closest words to the center word fermentation: ['births', 'isbn', 'pp', 'qur', 'referred', 'importance', 'rest', 'refer', 'difficult', 'easier']

Epoch 1/2:   6%|▌         | 2099/34343 [25:11<30:41:01,  3.43s/it]

Epoch 1 Batch 2100 loss: 2.407923698425293

Epoch 1/2:   6%|▌         | 2100/34343 [25:12<23:08:32,  2.58s/it]

Closest words to the center word doing: ['able', 'believed', 'know', 'referred', 'expected', 'want', 'do', 'seen', 'necessary', 'difficult']

Epoch 1/2:   6%|▋         | 2199/34343 [26:20<2:31:25,  3.54it/s] 

Epoch 1 Batch 2200 loss: 2.4096288681030273

Epoch 1/2:   6%|▋         | 2200/34343 [26:20<3:25:12,  2.61it/s]

Closest words to the center word may: ['may', 'births', 'can', 'inducted', 'there', 'must', 'would', 'than', 'pp', 'isbn']

Epoch 1/2:   7%|▋         | 2299/34343 [27:29<2:08:58,  4.14it/s] 

Epoch 1 Batch 2300 loss: 2.3787875175476074

Epoch 1/2:   7%|▋         | 2300/34343 [27:30<3:14:13,  2.75it/s]

Closest words to the center word outside: ['qur', 'rest', 'origin', 'importance', 'happens', 'prepare', 'variety', 'idea', 'collapse', 'consequence']

Epoch 1/2:   7%|▋         | 2399/34343 [28:38<1:53:21,  4.70it/s] 

Epoch 1 Batch 2400 loss: 2.392396926879883

Epoch 1/2:   7%|▋         | 2400/34343 [28:38<2:52:15,  3.09it/s]

Closest words to the center word added: ['believed', 'able', 'referred', 'inducted', 'fact', 'difficult', 'rest', 'possible', 'stored', 'know']

Epoch 1/2:   7%|▋         | 2499/34343 [29:53<6:49:19,  1.30it/s] 

Epoch 1 Batch 2500 loss: 2.417815685272217

Epoch 1/2:   7%|▋         | 2500/34343 [29:53<6:20:36,  1.39it/s]

Closest words to the center word private: ['difficult', 'referred', 'stored', 'remove', 'beings', 'able', 'quantity', 'regard', 'violation', 'sizes']

Epoch 1/2:   8%|▊         | 2599/34343 [31:01<2:18:07,  3.83it/s] 

Epoch 1 Batch 2600 loss: 2.3879668712615967

Epoch 1/2:   8%|▊         | 2600/34343 [31:02<3:13:45,  2.73it/s]

Closest words to the center word estate: ['births', 'isbn', 'inducted', 'pp', 'th', 'december', 'june', 'km', 'nine', 'january']

Epoch 1/2:   8%|▊         | 2699/34343 [32:11<1:54:11,  4.62it/s] 

Epoch 1 Batch 2700 loss: 2.4030895233154297

Epoch 1/2:   8%|▊         | 2700/34343 [32:11<2:55:18,  3.01it/s]

Closest words to the center word helmets: ['isbn', 'births', 'inducted', 'pp', 'km', 'pmid', 'laureate', 'july', 'agave', 'rfc']

Epoch 1/2:   8%|▊         | 2799/34343 [33:34<43:04:08,  4.92s/it]

Epoch 1 Batch 2800 loss: 2.378714084625244

Epoch 1/2:   8%|▊         | 2800/34343 [33:34<31:51:41,  3.64s/it]

Closest words to the center word computation: ['know', 'determine', 'does', 'stored', 'referred', 'difficult', 'argue', 'reason', 'believe', 'belief']

Epoch 1/2:   8%|▊         | 2899/34343 [34:30<3:25:10,  2.55it/s] 

Epoch 1 Batch 2900 loss: 2.374361276626587

Epoch 1/2:   8%|▊         | 2900/34343 [34:31<4:04:08,  2.15it/s]

Closest words to the center word mostly: ['births', 'fact', 'inducted', 'isbn', 'pp', 'referred', 'considered', 'treated', 'argue', 'understood']

Epoch 1/2:   9%|▊         | 2999/34343 [35:16<2:00:09,  4.35it/s] 

Epoch 1 Batch 3000 loss: 2.3956961631774902

Epoch 1/2:   9%|▊         | 3000/34343 [35:16<3:06:41,  2.80it/s]

Closest words to the center word relationship: ['lack', 'distinguish', 'suited', 'fact', 'belief', 'treated', 'regard', 'stored', 'easy', 'kind']

Epoch 1/2:   9%|▉         | 3099/34343 [36:00<1:51:54,  4.65it/s] 

Epoch 1 Batch 3100 loss: 2.348349094390869

Epoch 1/2:   9%|▉         | 3100/34343 [36:01<3:00:04,  2.89it/s]

Closest words to the center word absent: ['births', 'inducted', 'isbn', 'years', 'december', 'pp', 'gwh', 'know', 'expected', 'september']

Epoch 1/2:   9%|▉         | 3199/34343 [37:16<30:25:10,  3.52s/it]

Epoch 1 Batch 3200 loss: 2.376477003097534

Epoch 1/2:   9%|▉         | 3200/34343 [37:17<22:53:02,  2.65s/it]

Closest words to the center word kahn: ['laureate', 'births', 'isbn', 'inducted', 'pp', 'gwh', 'pmid', 'nobel', 'actress', 'nine']

Epoch 1/2:  10%|▉         | 3299/34343 [38:26<4:09:28,  2.07it/s] 

Epoch 1 Batch 3300 loss: 2.341855764389038

Epoch 1/2:  10%|▉         | 3300/34343 [38:26<4:33:14,  1.89it/s]

Closest words to the center word lit: ['frac', 'laureate', 'mathbf', 'x', 'y', 'births', 'isbn', 'inducted', 'you', 'f']

Epoch 1/2:  10%|▉         | 3399/34343 [39:36<1:57:18,  4.40it/s] 

Epoch 1 Batch 3400 loss: 2.33795166015625

Epoch 1/2:  10%|▉         | 3400/34343 [39:36<2:58:23,  2.89it/s]

Closest words to the center word switched: ['births', 'laureate', 'inducted', 'isbn', 'able', 'pmid', 'remove', 'january', 'february', 'know']

Epoch 1/2:  10%|█         | 3499/34343 [40:45<1:49:59,  4.67it/s] 

Epoch 1 Batch 3500 loss: 2.36179780960083

Epoch 1/2:  10%|█         | 3500/34343 [40:45<2:49:20,  3.04it/s]

Closest words to the center word makes: ['know', 'difficult', 'does', 'must', 'can', 'should', 'want', 'need', 'you', 'might']

Epoch 1/2:  10%|█         | 3599/34343 [42:09<21:06:08,  2.47s/it]

Epoch 1 Batch 3600 loss: 2.352299690246582

Epoch 1/2:  10%|█         | 3600/34343 [42:10<16:24:09,  1.92s/it]

Closest words to the center word with: ['inducted', 'births', 'isbn', 'pmid', 'gwh', 'had', 'will', 'into', 'explain', 'than']

Epoch 1/2:  11%|█         | 3699/34343 [43:19<3:12:54,  2.65it/s] 

Epoch 1 Batch 3700 loss: 2.3344993591308594

Epoch 1/2:  11%|█         | 3700/34343 [43:20<3:48:33,  2.23it/s]

Closest words to the center word frankie: ['births', 'laureate', 'inducted', 'isbn', 'gwh', 'pmid', 'frac', 'sep', 'pp', 'oct']

Epoch 1/2:  11%|█         | 3799/34343 [44:28<1:51:11,  4.58it/s] 

Epoch 1 Batch 3800 loss: 2.371368408203125

Epoch 1/2:  11%|█         | 3800/34343 [44:28<2:48:44,  3.02it/s]

Closest words to the center word eight: ['births', 'laureate', 'isbn', 'inducted', 'gwh', 'pmid', 'sep', 'nine', 'pp', 'seven']

Epoch 1/2:  11%|█▏        | 3899/34343 [45:26<1:47:46,  4.71it/s] 

Epoch 1 Batch 3900 loss: 2.3470311164855957

Epoch 1/2:  11%|█▏        | 3900/34343 [45:26<2:48:37,  3.01it/s]

Closest words to the center word ratification: ['inducted', 'births', 'pmid', 'midst', 'gregorian', 'laureate', 'united', 'republic', 'february', 'gwh']

Epoch 1/2:  12%|█▏        | 3999/34343 [46:48<11:21:33,  1.35s/it]

Epoch 1 Batch 4000 loss: 2.332225799560547

Epoch 1/2:  12%|█▏        | 4000/34343 [46:49<9:31:30,  1.13s/it] 

Closest words to the center word ends: ['inducted', 'pmid', 'births', 'years', 'gwh', 'isbn', 'june', 'february', 'ft', 'sep']

Epoch 1/2:  12%|█▏        | 4099/34343 [47:58<2:22:11,  3.54it/s] 

Epoch 1 Batch 4100 loss: 2.3461973667144775

Epoch 1/2:  12%|█▏        | 4100/34343 [47:58<3:15:17,  2.58it/s]

Closest words to the center word modern: ['spoken', 'languages', 'eastern', 'countries', 'areas', 'european', 'europe', 'nations', 'context', 'western']

Epoch 1/2:  12%|█▏        | 4199/34343 [49:05<1:46:33,  4.71it/s] 

Epoch 1 Batch 4200 loss: 2.3501133918762207

Epoch 1/2:  12%|█▏        | 4200/34343 [49:06<2:45:06,  3.04it/s]

Closest words to the center word nine: ['births', 'inducted', 'laureate', 'gwh', 'isbn', 'pmid', 'sep', 'nine', 'pp', 'oct']

Epoch 1/2:  13%|█▎        | 4299/34343 [50:14<1:45:16,  4.76it/s] 

Epoch 1 Batch 4300 loss: 2.3361902236938477

Epoch 1/2:  13%|█▎        | 4300/34343 [50:15<2:46:05,  3.01it/s]

Closest words to the center word for: ['for', 'frac', 'than', 'if', 'own', 'likely', 'there', 'during', 'ago', 'term']

Epoch 1/2:  13%|█▎        | 4399/34343 [51:39<6:07:27,  1.36it/s] 

Epoch 1 Batch 4400 loss: 2.3040339946746826

Epoch 1/2:  13%|█▎        | 4400/34343 [51:40<5:52:04,  1.42it/s]

Closest words to the center word three: ['births', 'inducted', 'gwh', 'isbn', 'pmid', 'sep', 'unpaved', 'laureate', 'sq', 'est']

Epoch 1/2:  13%|█▎        | 4499/34343 [52:49<1:50:00,  4.52it/s] 

Epoch 1 Batch 4500 loss: 2.3336434364318848

Epoch 1/2:  13%|█▎        | 4500/34343 [52:49<2:52:41,  2.88it/s]

Closest words to the center word shutout: ['gwh', 'births', 'inducted', 'laureate', 'isbn', 'pmid', 'sep', 'unpaved', 'pp', 'oct']

Epoch 1/2:  13%|█▎        | 4599/34343 [53:58<1:47:07,  4.63it/s] 

Epoch 1 Batch 4600 loss: 2.305166482925415

Epoch 1/2:  13%|█▎        | 4600/34343 [53:59<2:56:24,  2.81it/s]

Closest words to the center word episode: ['inducted', 'gregorian', 'pmid', 'years', 'gwh', 'sep', 'january', 'february', 'june', 'laureate']

Epoch 1/2:  14%|█▎        | 4699/34343 [55:22<39:56:33,  4.85s/it]

Epoch 1 Batch 4700 loss: 2.291905403137207

Epoch 1/2:  14%|█▎        | 4700/34343 [55:23<29:29:32,  3.58s/it]

Closest words to the center word have: ['have', 'has', 'had', 'be', 'are', 'know', 'argue', 'been', 'were', 'want']

Epoch 1/2:  14%|█▍        | 4799/34343 [56:31<2:32:50,  3.22it/s] 

Epoch 1 Batch 4800 loss: 2.3587260246276855

Epoch 1/2:  14%|█▍        | 4800/34343 [56:32<3:19:07,  2.47it/s]

Closest words to the center word disk: ['mathbf', 'frac', 'insubstantial', 'x', 'mbox', 'cdot', 'import', 'function', 'disk', 'file']

Epoch 1/2:  14%|█▍        | 4899/34343 [57:39<1:52:51,  4.35it/s] 

Epoch 1 Batch 4900 loss: 2.3159313201904297

Epoch 1/2:  14%|█▍        | 4900/34343 [57:39<2:52:15,  2.85it/s]

Closest words to the center word years: ['years', 'gregorian', 'leap', 'million', 'ago', 'year', 'days', 'hours', 'months', 'minutes']

Epoch 1/2:  15%|█▍        | 4999/34343 [58:46<1:43:51,  4.71it/s] 

Epoch 1 Batch 5000 loss: 2.2998197078704834

Epoch 1/2:  15%|█▍        | 5000/34343 [58:47<2:45:45,  2.95it/s]

Closest words to the center word the: ['the', 'states', 'this', 'its', 'kingdom', 'united', 'war', 'th', 'nations', 'a']

Epoch 1/2:  15%|█▍        | 5099/34343 [1:00:10<8:10:56,  1.01s/it] 

Epoch 1 Batch 5100 loss: 2.3164725303649902

Epoch 1/2:  15%|█▍        | 5100/34343 [1:00:11<7:21:15,  1.10it/s]

Closest words to the center word that: ['that', 'not', 'we', 'know', 'what', 'necessarily', 'you', 'why', 'if', 'never']

Epoch 1/2:  15%|█▌        | 5199/34343 [1:01:20<2:16:53,  3.55it/s] 

Epoch 1 Batch 5200 loss: 2.3047921657562256

Epoch 1/2:  15%|█▌        | 5200/34343 [1:01:21<3:05:59,  2.61it/s]

Closest words to the center word theories: ['argue', 'materials', 'practices', 'properties', 'topics', 'aspects', 'particles', 'beings', 'knowledge', 'types']

Epoch 1/2:  15%|█▌        | 5299/34343 [1:02:30<1:45:58,  4.57it/s] 

Epoch 1 Batch 5300 loss: 2.3116066455841064

Epoch 1/2:  15%|█▌        | 5300/34343 [1:02:30<2:45:16,  2.93it/s]

Closest words to the center word book: ['book', 'wife', 'pmid', 'son', 'laureate', 'gwh', 'frac', 'inducted', 'father', 'feb']

Epoch 1/2:  16%|█▌        | 5399/34343 [1:03:39<1:43:14,  4.67it/s] 

Epoch 1 Batch 5400 loss: 2.2891645431518555

Epoch 1/2:  16%|█▌        | 5400/34343 [1:03:56<40:44:25,  5.07s/it]

Closest words to the center word owned: ['gwh', 'pmid', 'sep', 'elected', 'united', 'asia', 'east', 'union', 'afc', 'territories']

Epoch 1/2:  16%|█▌        | 5499/34343 [1:05:04<6:11:32,  1.29it/s] 

Epoch 1 Batch 5500 loss: 2.292771577835083

Epoch 1/2:  16%|█▌        | 5500/34343 [1:05:05<5:45:38,  1.39it/s]

Closest words to the center word real: ['frac', 'mathbf', 'x', 'finite', 'cdot', 'phi', 'sum', 'y', 'mathrm', 'rangle']

Epoch 1/2:  16%|█▋        | 5599/34343 [1:06:13<2:01:59,  3.93it/s] 

Epoch 1 Batch 5600 loss: 2.282719135284424

Epoch 1/2:  16%|█▋        | 5600/34343 [1:06:13<2:51:15,  2.80it/s]

Closest words to the center word cgi: ['mathbf', 'frac', 'mbox', 'cdot', 'algebra', 'http', 'laureate', 'www', 'linear', 'programming']

Epoch 1/2:  17%|█▋        | 5699/34343 [1:07:23<1:43:02,  4.63it/s] 

Epoch 1 Batch 5700 loss: 2.2954509258270264

Epoch 1/2:  17%|█▋        | 5700/34343 [1:07:24<2:52:42,  2.76it/s]

Closest words to the center word blues: ['laureate', 'pmid', 'actress', 'actor', 'gwh', 'inducted', 'births', 'comedian', 'physicist', 'musician']

Epoch 1/2:  17%|█▋        | 5799/34343 [1:08:47<38:01:39,  4.80s/it]

Epoch 1 Batch 5800 loss: 2.3222241401672363

Epoch 1/2:  17%|█▋        | 5800/34343 [1:08:48<28:04:33,  3.54s/it]

Closest words to the center word escape: ['bring', 'return', 'returned', 'throne', 'decided', 'want', 'keep', 'able', 'him', 'refused']

Epoch 1/2:  17%|█▋        | 5899/34343 [1:09:56<4:51:39,  1.63it/s] 

Epoch 1 Batch 5900 loss: 2.214003324508667

Epoch 1/2:  17%|█▋        | 5900/34343 [1:09:57<4:51:12,  1.63it/s]

Closest words to the center word ions: ['mathbf', 'frac', 'molecules', 'acids', 'carbon', 'finite', 'algebra', 'dioxide', 'properties', 'linear']

Epoch 1/2:  17%|█▋        | 5999/34343 [1:10:53<1:50:19,  4.28it/s] 

Epoch 1 Batch 6000 loss: 2.3203701972961426

Epoch 1/2:  17%|█▋        | 6000/34343 [1:10:54<2:45:13,  2.86it/s]

Closest words to the center word water: ['est', 'km', 'water', 'carbon', 'dioxide', 'capita', 'energy', 'molecules', 'gwh', 'temperature']

Epoch 1/2:  18%|█▊        | 6099/34343 [1:12:02<1:40:58,  4.66it/s] 

Epoch 1 Batch 6100 loss: 2.3139994144439697

Epoch 1/2:  18%|█▊        | 6100/34343 [1:12:03<2:42:57,  2.89it/s]

Closest words to the center word lm: ['gwh', 'pmid', 'frac', 'mathbf', 'grt', 'inducted', 'cdot', 'mbox', 'cdots', 'unpaved']

Epoch 1/2:  18%|█▊        | 6199/34343 [1:13:22<28:33:27,  3.65s/it]

Epoch 1 Batch 6200 loss: 2.263488531112671

Epoch 1/2:  18%|█▊        | 6200/34343 [1:13:23<21:26:26,  2.74s/it]

Closest words to the center word town: ['nfc', 'gregorian', 'city', 'town', 'wales', 'south', 'located', 'york', 'county', 'afc']

Epoch 1/2:  18%|█▊        | 6299/34343 [1:14:32<3:08:11,  2.48it/s] 

Epoch 1 Batch 6300 loss: 2.2832627296447754

Epoch 1/2:  18%|█▊        | 6300/34343 [1:14:32<4:03:19,  1.92it/s]

Closest words to the center word west: ['nfc', 'east', 'south', 'west', 'north', 'afc', 'ocean', 'coast', 'southeast', 'atlantic']

Epoch 1/2:  19%|█▊        | 6399/34343 [1:15:37<1:43:55,  4.48it/s] 

Epoch 1 Batch 6400 loss: 2.263874053955078

Epoch 1/2:  19%|█▊        | 6400/34343 [1:15:38<2:34:04,  3.02it/s]

Closest words to the center word his: ['his', 'her', 'him', 'brother', 'wife', 'my', 'father', 'she', 'son', 'shortly']

Epoch 1/2:  19%|█▉        | 6499/34343 [1:16:45<1:37:36,  4.75it/s] 

Epoch 1 Batch 6500 loss: 2.2927496433258057

Epoch 1/2:  19%|█▉        | 6500/34343 [1:16:45<2:33:43,  3.02it/s]

Closest words to the center word seven: ['gwh', 'pmid', 'births', 'inducted', 'kwh', 'sep', 'isbn', 'feb', 'laureate', 'jul']

Epoch 1/2:  19%|█▉        | 6599/34343 [1:18:10<14:51:21,  1.93s/it]

Epoch 1 Batch 6600 loss: 2.261687994003296

Epoch 1/2:  19%|█▉        | 6600/34343 [1:18:11<11:51:08,  1.54s/it]

Closest words to the center word renamed: ['pmid', 'inducted', 'gwh', 'sep', 'afc', 'nfc', 'nfl', 'gregorian', 'feb', 'jul']

Epoch 1/2:  20%|█▉        | 6699/34343 [1:19:21<2:23:05,  3.22it/s] 

Epoch 1 Batch 6700 loss: 2.2591235637664795

Epoch 1/2:  20%|█▉        | 6700/34343 [1:19:22<3:03:21,  2.51it/s]

Closest words to the center word greek: ['roman', 'greek', 'orthodox', 'ancient', 'catholic', 'mythology', 'century', 'spoken', 'indo', 'eastern']

Epoch 1/2:  20%|█▉        | 6799/34343 [1:20:30<1:39:12,  4.63it/s] 

Epoch 1 Batch 6800 loss: 2.2623863220214844

Epoch 1/2:  20%|█▉        | 6800/34343 [1:20:30<2:33:05,  3.00it/s]

Closest words to the center word than: ['than', 'much', 'less', 'est', 'expensive', 'efficient', 'gwh', 'faster', 'km', 'rate']

Epoch 1/2:  20%|██        | 6899/34343 [1:21:39<1:37:05,  4.71it/s] 

Epoch 1 Batch 6900 loss: 2.270664691925049

Epoch 1/2:  20%|██        | 6900/34343 [1:21:40<2:32:36,  3.00it/s]

Closest words to the center word easier: ['easier', 'difficult', 'enough', 'prove', 'anything', 'sure', 'necessary', 'understand', 'impossible', 'interact']

Epoch 1/2:  20%|██        | 6999/34343 [1:23:04<7:41:38,  1.01s/it] 

Epoch 1 Batch 7000 loss: 2.2632012367248535

Epoch 1/2:  20%|██        | 7000/34343 [1:23:05<6:48:42,  1.12it/s]

Closest words to the center word flying: ['gwh', 'pmid', 'cdot', 'frac', 'inducted', 'cdots', 'jul', 'mbox', 'grt', 'unpaved']

Epoch 1/2:  21%|██        | 7099/34343 [1:24:13<1:44:05,  4.36it/s] 

Epoch 1 Batch 7100 loss: 2.292957305908203

Epoch 1/2:  21%|██        | 7100/34343 [1:24:14<2:33:31,  2.96it/s]

Closest words to the center word solemnly: ['distinguish', 'necessarily', 'know', 'prove', 'understood', 'perceive', 'verify', 'beings', 'believe', 'understand']

Epoch 1/2:  21%|██        | 7199/34343 [1:25:21<1:36:40,  4.68it/s] 

Epoch 1 Batch 7200 loss: 2.2485849857330322

Epoch 1/2:  21%|██        | 7200/34343 [1:25:22<2:32:12,  2.97it/s]

Closest words to the center word discovered: ['inducted', 'gregorian', 'published', 'discovered', 'pmid', 'century', 'superseded', 'believed', 'leap', 'buried']

Epoch 1/2:  21%|██▏       | 7299/34343 [1:26:31<1:39:22,  4.54it/s] 

Epoch 1 Batch 7300 loss: 2.3111839294433594

Epoch 1/2:  21%|██▏       | 7300/34343 [1:26:47<37:32:27,  5.00s/it]

Closest words to the center word his: ['his', 'her', 'him', 'brother', 'my', 'wife', 'eldest', 'jesus', 'she', 'father']

Epoch 1/2:  22%|██▏       | 7399/34343 [1:27:57<2:35:28,  2.89it/s] 

Epoch 1 Batch 7400 loss: 2.267235279083252

Epoch 1/2:  22%|██▏       | 7400/34343 [1:27:57<3:10:57,  2.35it/s]

Closest words to the center word principia: ['laureate', 'cdots', 'gwh', 'frac', 'cdot', 'mathbf', 'isbn', 'mbox', 'pmid', 'mathrm']

Epoch 1/2:  22%|██▏       | 7499/34343 [1:29:06<1:39:41,  4.49it/s] 

Epoch 1 Batch 7500 loss: 2.2627291679382324

Epoch 1/2:  22%|██▏       | 7500/34343 [1:29:06<2:30:43,  2.97it/s]

Closest words to the center word object: ['subset', 'vector', 'mathbf', 'finite', 'algebra', 'euclidean', 'integer', 'topological', 'function', 'boolean']

Epoch 1/2:  22%|██▏       | 7599/34343 [1:30:14<1:34:52,  4.70it/s] 

Epoch 1 Batch 7600 loss: 2.241898536682129

Epoch 1/2:  22%|██▏       | 7600/34343 [1:30:15<2:27:03,  3.03it/s]

Closest words to the center word aesthetic: ['cognitive', 'processes', 'topics', 'disorders', 'genetic', 'behaviors', 'computational', 'cognition', 'ethical', 'aspects']

Epoch 1/2:  22%|██▏       | 7699/34343 [1:31:41<10:01:32,  1.35s/it]

Epoch 1 Batch 7700 loss: 2.234434127807617

Epoch 1/2:  22%|██▏       | 7700/34343 [1:31:41<8:23:50,  1.13s/it] 

Closest words to the center word battleship: ['gwh', 'pmid', 'nfc', 'afc', 'gregorian', 'capita', 'median', 'cyg', 'championship', 'grt']

Epoch 1/2:  23%|██▎       | 7799/34343 [1:32:52<2:18:28,  3.19it/s] 

Epoch 1 Batch 7800 loss: 2.2651569843292236

Epoch 1/2:  23%|██▎       | 7800/34343 [1:32:53<2:57:01,  2.50it/s]

Closest words to the center word george: ['laureate', 'politician', 'physicist', 'cricketer', 'ois', 'footballer', 'jr', 'actress', 'pmid', 'earl']

Epoch 1/2:  23%|██▎       | 7899/34343 [1:34:01<1:37:20,  4.53it/s] 

Epoch 1 Batch 7900 loss: 2.230804443359375

Epoch 1/2:  23%|██▎       | 7900/34343 [1:34:01<2:31:54,  2.90it/s]

Closest words to the center word occasionally: ['understood', 'difficult', 'interpreted', 'easier', 'interact', 'willing', 'treat', 'understand', 'verify', 'durable']

Epoch 1/2:  23%|██▎       | 7999/34343 [1:35:12<1:36:30,  4.55it/s] 

Epoch 1 Batch 8000 loss: 2.285022258758545

Epoch 1/2:  23%|██▎       | 8000/34343 [1:35:12<2:30:09,  2.92it/s]

Closest words to the center word english: ['english', 'laureate', 'nobel', 'physicist', 'prize', 'languages', 'language', 'spoken', 'footballer', 'french']

Epoch 1/2:  24%|██▎       | 8099/34343 [1:36:37<7:26:38,  1.02s/it] 

Epoch 1 Batch 8100 loss: 2.292471408843994

Epoch 1/2:  24%|██▎       | 8100/34343 [1:36:38<6:28:18,  1.13it/s]

Closest words to the center word western: ['eastern', 'asia', 'caribbean', 'western', 'nfc', 'southeast', 'populous', 'bordering', 'south', 'africa']

Epoch 1/2:  24%|██▍       | 8199/34343 [1:37:49<2:03:11,  3.54it/s] 

Epoch 1 Batch 8200 loss: 2.2767786979675293

Epoch 1/2:  24%|██▍       | 8200/34343 [1:37:49<2:48:22,  2.59it/s]

Closest words to the center word united: ['united', 'federated', 'states', 'nfc', 'senate', 'republic', 'commonwealth', 'canada', 'nations', 'micronesia']

Epoch 1/2:  24%|██▍       | 8299/34343 [1:39:00<1:33:34,  4.64it/s] 

Epoch 1 Batch 8300 loss: 2.2504992485046387

Epoch 1/2:  24%|██▍       | 8300/34343 [1:39:00<2:27:19,  2.95it/s]

Closest words to the center word commonly: ['commonly', 'widely', 'referred', 'spoken', 'regarded', 'used', 'understood', 'compounds', 'sometimes', 'often']

Epoch 1/2:  24%|██▍       | 8399/34343 [1:40:10<1:32:05,  4.69it/s] 

Epoch 1 Batch 8400 loss: 2.262690544128418

Epoch 1/2:  24%|██▍       | 8400/34343 [1:40:27<37:48:40,  5.25s/it]

Closest words to the center word series: ['series', 'game', 'television', 'animated', 'video', 'episode', 'fantasy', 'games', 'cdots', 'album']

Epoch 1/2:  25%|██▍       | 8499/34343 [1:41:37<5:26:37,  1.32it/s] 

Epoch 1 Batch 8500 loss: 2.2831146717071533

Epoch 1/2:  25%|██▍       | 8500/34343 [1:41:37<5:09:21,  1.39it/s]

Closest words to the center word network: ['intelsat', 'gwh', 'internet', 'software', 'directory', 'isps', 'gnu', 'computer', 'protocol', 'server']

Epoch 1/2:  25%|██▌       | 8599/34343 [1:42:46<1:44:06,  4.12it/s] 

Epoch 1 Batch 8600 loss: 2.2123119831085205

Epoch 1/2:  25%|██▌       | 8600/34343 [1:42:47<2:34:09,  2.78it/s]

Closest words to the center word those: ['those', 'denominations', 'themselves', 'individuals', 'kinds', 'christians', 'citizens', 'muslims', 'births', 'speakers']

Epoch 1/2:  25%|██▌       | 8699/34343 [1:44:00<1:31:31,  4.67it/s] 

Epoch 1 Batch 8700 loss: 2.2346959114074707

Epoch 1/2:  25%|██▌       | 8700/34343 [1:44:00<2:25:05,  2.95it/s]

Closest words to the center word catalogue: ['gwh', 'pmid', 'cyg', 'twh', 'laureate', 'jul', 'jun', 'oct', 'kwh', 'runways']

Epoch 1/2:  26%|██▌       | 8799/34343 [1:45:25<37:38:29,  5.30s/it]

Epoch 1 Batch 8800 loss: 2.221660852432251

Epoch 1/2:  26%|██▌       | 8800/34343 [1:45:26<27:53:43,  3.93s/it]

Closest words to the center word or: ['than', 'or', 'molecules', 'frac', 'amino', 'mathbf', 'membrane', 'atoms', 'dioxide', 'lossy']

Epoch 1/2:  26%|██▌       | 8899/34343 [1:46:35<3:24:55,  2.07it/s] 

Epoch 1 Batch 8900 loss: 2.273404359817505

Epoch 1/2:  26%|██▌       | 8900/34343 [1:46:36<3:53:33,  1.82it/s]

Closest words to the center word publish: ['publish', 'parents', 'convince', 'respond', 'devote', 'compelled', 'desire', 'understand', 'him', 'pursue']

Epoch 1/2:  26%|██▌       | 8999/34343 [1:47:34<1:34:00,  4.49it/s] 

Epoch 1 Batch 9000 loss: 2.2859272956848145

Epoch 1/2:  26%|██▌       | 9000/34343 [1:47:35<2:24:46,  2.92it/s]

Closest words to the center word born: ['laureate', 'born', 'actress', 'actor', 'inducted', 'pmid', 'prize', 'nobel', 'births', 'cricketer']

Epoch 1/2:  26%|██▋       | 9099/34343 [1:48:34<1:31:28,  4.60it/s] 

Epoch 1 Batch 9100 loss: 2.2506296634674072

Epoch 1/2:  26%|██▋       | 9100/34343 [1:48:34<2:22:49,  2.95it/s]

Closest words to the center word databases: ['lossy', 'databases', 'cognitive', 'boolean', 'computational', 'dynamical', 'applications', 'algebra', 'processes', 'spectroscopy']

Epoch 1/2:  27%|██▋       | 9199/34343 [1:49:56<13:42:11,  1.96s/it]

Epoch 1 Batch 9200 loss: 2.211230516433716

Epoch 1/2:  27%|██▋       | 9200/34343 [1:49:57<10:55:05,  1.56s/it]

Closest words to the center word references: ['www', 'icrm', 'links', 'http', 'isbn', 'ifrcs', 'org', 'edu', 'encyclopedia', 'laureate']

Epoch 1/2:  27%|██▋       | 9299/34343 [1:50:57<2:17:39,  3.03it/s] 

Epoch 1 Batch 9300 loss: 2.2425570487976074

Epoch 1/2:  27%|██▋       | 9300/34343 [1:50:57<2:58:55,  2.33it/s]

Closest words to the center word root: ['mathbf', 'frac', 'infty', 'cdot', 'rangle', 'vector', 'mathrm', 'langle', 'inverse', 'euclidean']

Epoch 1/2:  27%|██▋       | 9399/34343 [1:51:59<1:30:32,  4.59it/s] 

Epoch 1 Batch 9400 loss: 2.238952398300171

Epoch 1/2:  27%|██▋       | 9400/34343 [1:52:00<2:23:05,  2.91it/s]

Closest words to the center word of: ['of', 'est', 'expectancy', 'populous', 'ottoman', 'icrm', 'catholic', 'judicial', 'same', 'holy']

Epoch 1/2:  28%|██▊       | 9499/34343 [1:53:02<1:28:45,  4.67it/s] 

Epoch 1 Batch 9500 loss: 2.2426815032958984

Epoch 1/2:  28%|██▊       | 9500/34343 [1:53:03<2:21:21,  2.93it/s]

Closest words to the center word asw: ['gwh', 'runways', 'laureate', 'pmid', 'cdot', 'cyg', 'jul', 'twh', 'icrm', 'grt']

Epoch 1/2:  28%|██▊       | 9599/34343 [1:54:22<9:16:01,  1.35s/it] 

Epoch 1 Batch 9600 loss: 2.2299399375915527

Epoch 1/2:  28%|██▊       | 9600/34343 [1:54:22<7:52:19,  1.15s/it]

Closest words to the center word etherboot: ['gwh', 'cyg', 'pmid', 'cdot', 'twh', 'runways', 'jun', 'kwh', 'grt', 'jul']

Epoch 1/2:  28%|██▊       | 9699/34343 [1:55:19<1:36:25,  4.26it/s] 

Epoch 1 Batch 9700 loss: 2.273279905319214

Epoch 1/2:  28%|██▊       | 9700/34343 [1:55:19<2:25:33,  2.82it/s]

Closest words to the center word dvd: ['windows', 'cdot', 'pc', 'gwh', 'floppy', 'dvd', 'mbox', 'video', 'os', 'mac']

Epoch 1/2:  29%|██▊       | 9799/34343 [1:56:14<1:26:32,  4.73it/s] 

Epoch 1 Batch 9800 loss: 2.206517219543457

Epoch 1/2:  29%|██▊       | 9800/34343 [1:56:14<2:17:25,  2.98it/s]

Closest words to the center word behaviour: ['cognitive', 'beings', 'processes', 'behaviors', 'perception', 'symptoms', 'subjective', 'phenomena', 'belief', 'ideas']

Epoch 1/2:  29%|██▉       | 9899/34343 [1:57:03<1:25:38,  4.76it/s] 

Epoch 1 Batch 9900 loss: 2.2395756244659424

Epoch 1/2:  29%|██▉       | 9900/34343 [1:57:04<2:15:31,  3.01it/s]

Closest words to the center word area: ['km', 'capita', 'kilometers', 'area', 'coastline', 'sq', 'runways', 'meters', 'metropolitan', 'nfc']

Epoch 1/2:  29%|██▉       | 9999/34343 [1:58:10<2:44:55,  2.46it/s] 

Epoch 1 Batch 10000 loss: 2.2685863971710205

Epoch 1/2:  29%|██▉       | 10000/34343 [1:58:11<3:13:13,  2.10it/s]

Closest words to the center word crime: ['sexual', 'communism', 'health', 'political', 'courts', 'elections', 'jurisdiction', 'economic', 'proponent', 'institutions']

Epoch 1/2:  29%|██▉       | 10099/34343 [1:59:15<1:30:44,  4.45it/s] 

Epoch 1 Batch 10100 loss: 2.2157480716705322

Epoch 1/2:  29%|██▉       | 10100/34343 [1:59:15<2:23:56,  2.81it/s]

Closest words to the center word serves: ['serves', 'referred', 'known', 'regarded', 'became', 'insofar', 'is', 'served', 'allows', 'refers']

Epoch 1/2:  30%|██▉       | 10199/34343 [2:00:03<1:26:28,  4.65it/s] 

Epoch 1 Batch 10200 loss: 2.212567090988159

Epoch 1/2:  30%|██▉       | 10200/34343 [2:00:04<2:18:40,  2.90it/s]

Closest words to the center word for: ['for', 'manpower', 'gwh', 'cdots', 'cdot', 'lossy', 'gregorian', 'capita', 'best', 'pmid']

Epoch 1/2:  30%|██▉       | 10299/34343 [2:01:07<7:14:17,  1.08s/it] 

Epoch 1 Batch 10300 loss: 2.2562217712402344

Epoch 1/2:  30%|██▉       | 10300/34343 [2:01:08<6:20:08,  1.05it/s]

Closest words to the center word generation: ['cdots', 'mathbf', 'macintosh', 'frac', 'generation', 'cdot', 'median', 'mac', 'os', 'infty']

Epoch 1/2:  30%|███       | 10399/34343 [2:02:11<2:01:47,  3.28it/s] 

Epoch 1 Batch 10400 loss: 2.2322494983673096

Epoch 1/2:  30%|███       | 10400/34343 [2:02:12<2:42:17,  2.46it/s]

Closest words to the center word density: ['density', 'capita', 'runways', 'gwh', 'median', 'km', 'meters', 'gdp', 'unpaved', 'est']

Epoch 1/2:  31%|███       | 10499/34343 [2:03:19<1:41:39,  3.91it/s] 

Epoch 1 Batch 10500 loss: 2.2334342002868652

Epoch 1/2:  31%|███       | 10500/34343 [2:03:19<2:27:20,  2.70it/s]

Closest words to the center word president: ['president', 'minister', 'elected', 'appointed', 'governor', 'secretary', 'appoints', 'chairman', 'presidential', 'vice']

Epoch 1/2:  31%|███       | 10599/34343 [2:04:07<1:23:37,  4.73it/s] 

Epoch 1 Batch 10600 loss: 2.2524309158325195

Epoch 1/2:  31%|███       | 10600/34343 [2:04:08<2:13:14,  2.97it/s]

Closest words to the center word marie: ['laureate', 'footballer', 'ois', 'actress', 'cricketer', 'fran', 'pngimage', 'pmid', 'actor', 'chemist']

Epoch 1/2:  31%|███       | 10699/34343 [2:05:01<5:32:27,  1.19it/s] 

Epoch 1 Batch 10700 loss: 2.226073741912842

Epoch 1/2:  31%|███       | 10700/34343 [2:05:02<5:08:25,  1.28it/s]

Closest words to the center word slightly: ['slightly', 'inches', 'grt', 'meters', 'than', 'median', 'capita', 'temperatures', 'unpaved', 'faster']

Epoch 1/2:  31%|███▏      | 10799/34343 [2:06:02<1:50:07,  3.56it/s] 

Epoch 1 Batch 10800 loss: 2.213536024093628

Epoch 1/2:  31%|███▏      | 10800/34343 [2:06:02<2:24:40,  2.71it/s]

Closest words to the center word who: ['who', 'whom', 'married', 'younger', 'births', 'actors', 'killed', 'singers', 'son', 'accused']

Epoch 1/2:  32%|███▏      | 10899/34343 [2:07:03<1:24:56,  4.60it/s] 

Epoch 1 Batch 10900 loss: 2.2501258850097656

Epoch 1/2:  32%|███▏      | 10900/34343 [2:07:03<2:09:57,  3.01it/s]

Closest words to the center word calendar: ['gregorian', 'calendar', 'leap', 'day', 'lunisolar', 'year', 'month', 'bce', 'observances', 'nfc']

Epoch 1/2:  32%|███▏      | 10999/34343 [2:08:06<1:23:34,  4.66it/s] 

Epoch 1 Batch 11000 loss: 2.234565019607544

Epoch 1/2:  32%|███▏      | 11000/34343 [2:08:07<2:12:19,  2.94it/s]

Closest words to the center word communications: ['ifrcs', 'intelsat', 'communications', 'demographics', 'telephones', 'opcw', 'iom', 'ifad', 'transportation', 'isps']

Epoch 1/2:  32%|███▏      | 11099/34343 [2:09:00<3:11:12,  2.03it/s] 

Epoch 1 Batch 11100 loss: 2.1692628860473633

Epoch 1/2:  32%|███▏      | 11100/34343 [2:09:01<3:32:05,  1.83it/s]

Closest words to the center word names: ['languages', 'dialects', 'names', 'words', 'alphabet', 'cdots', 'verbs', 'nouns', 'phrases', 'speakers']

Epoch 1/2:  33%|███▎      | 11199/34343 [2:09:45<1:27:29,  4.41it/s] 

Epoch 1 Batch 11200 loss: 2.212488889694214

Epoch 1/2:  33%|███▎      | 11200/34343 [2:09:46<2:17:52,  2.80it/s]

Closest words to the center word denmark: ['denmark', 'republic', 'slovenia', 'serbia', 'austria', 'observances', 'poland', 'hungary', 'lithuania', 'montenegro']

Epoch 1/2:  33%|███▎      | 11299/34343 [2:10:34<1:21:01,  4.74it/s] 

Epoch 1 Batch 11300 loss: 2.241395950317383

Epoch 1/2:  33%|███▎      | 11300/34343 [2:10:34<2:13:54,  2.87it/s]

Closest words to the center word proved: ['gwh', 'grt', 'proved', 'lasted', 'twh', 'vote', 'cyg', 'election', 'householder', 'speculated']

Epoch 1/2:  33%|███▎      | 11399/34343 [2:11:38<31:19:09,  4.91s/it]

Epoch 1 Batch 11400 loss: 2.2049591541290283

Epoch 1/2:  33%|███▎      | 11400/34343 [2:11:39<23:23:50,  3.67s/it]

Closest words to the center word food: ['commodities', 'imports', 'textiles', 'ifad', 'exports', 'fuels', 'machinery', 'ifrcs', 'vegetables', 'pollution']

Epoch 1/2:  33%|███▎      | 11499/34343 [2:12:49<3:07:23,  2.03it/s] 

Epoch 1 Batch 11500 loss: 2.1915111541748047

Epoch 1/2:  33%|███▎      | 11500/34343 [2:12:49<3:28:27,  1.83it/s]

Closest words to the center word opposed: ['opposed', 'referred', 'republics', 'conservative', 'wipo', 'regarded', 'communist', 'socialist', 'conservatives', 'appointed']

Epoch 1/2:  34%|███▍      | 11599/34343 [2:13:44<1:24:20,  4.49it/s] 

Epoch 1 Batch 11600 loss: 2.1804370880126953

Epoch 1/2:  34%|███▍      | 11600/34343 [2:13:45<2:18:10,  2.74it/s]

Closest words to the center word leadership: ['leadership', 'socialist', 'privy', 'communist', 'democratic', 'liberties', 'overthrow', 'ministers', 'republics', 'party']

Epoch 1/2:  34%|███▍      | 11699/34343 [2:14:38<1:20:35,  4.68it/s] 

Epoch 1 Batch 11700 loss: 2.2079873085021973

Epoch 1/2:  34%|███▍      | 11700/34343 [2:14:38<2:05:47,  3.00it/s]

Closest words to the center word two: ['gwh', 'cyg', 'twh', 'pmid', 'grt', 'kwh', 'unpaved', 'jun', 'sep', 'runways']

Epoch 1/2:  34%|███▍      | 11799/34343 [2:15:43<9:35:59,  1.53s/it] 

Epoch 1 Batch 11800 loss: 2.204357385635376

Epoch 1/2:  34%|███▍      | 11800/34343 [2:15:44<8:02:17,  1.28s/it]

Closest words to the center word upgrade: ['gwh', 'kwh', 'unpaved', 'runways', 'grt', 'capita', 'twh', 'isps', 'cyg', 'mhz']

Epoch 1/2:  35%|███▍      | 11899/34343 [2:16:33<1:30:12,  4.15it/s] 

Epoch 1 Batch 11900 loss: 2.221238136291504

Epoch 1/2:  35%|███▍      | 11900/34343 [2:16:33<2:19:56,  2.67it/s]

Closest words to the center word chesterton: ['cdot', 'cdots', 'rightarrow', 'rangle', 'mathbf', 'otimes', 'frac', 'rang', 'qquad', 'cos']

Epoch 1/2:  35%|███▍      | 11999/34343 [2:17:19<1:25:11,  4.37it/s] 

Epoch 1 Batch 12000 loss: 2.1798417568206787

Epoch 1/2:  35%|███▍      | 12000/34343 [2:17:20<2:14:34,  2.77it/s]

Closest words to the center word they: ['they', 'we', 'you', 'happen', 'not', 'he', 'decide', 'themselves', 'occur', 'able']

Epoch 1/2:  35%|███▌      | 12099/34343 [2:18:06<1:21:18,  4.56it/s] 

Epoch 1 Batch 12100 loss: 2.215872287750244

Epoch 1/2:  35%|███▌      | 12100/34343 [2:18:06<2:10:05,  2.85it/s]

Closest words to the center word informal: ['empirical', 'implications', 'ethical', 'grammatical', 'cognitive', 'anarcho', 'rabbinic', 'computational', 'grammar', 'verbs']

Epoch 1/2:  36%|███▌      | 12199/34343 [2:19:00<2:28:31,  2.48it/s] 

Epoch 1 Batch 12200 loss: 2.1946401596069336

Epoch 1/2:  36%|███▌      | 12200/34343 [2:19:01<3:02:58,  2.02it/s]

Closest words to the center word games: ['games', 'game', 'consoles', 'championship', 'video', 'playstation', 'console', 'football', 'baseball', 'kart']

Epoch 1/2:  36%|███▌      | 12299/34343 [2:19:44<1:22:36,  4.45it/s] 

Epoch 1 Batch 12300 loss: 2.2030444145202637

Epoch 1/2:  36%|███▌      | 12300/34343 [2:19:45<2:12:41,  2.77it/s]

Closest words to the center word reach: ['reach', 'grt', 'meters', 'nfc', 'kilometers', 'km', 'lose', 'playoffs', 'afc', 'kilometres']

Epoch 1/2:  36%|███▌      | 12399/34343 [2:20:19<1:21:03,  4.51it/s] 

Epoch 1 Batch 12400 loss: 2.1845791339874268

Epoch 1/2:  36%|███▌      | 12400/34343 [2:20:20<2:04:08,  2.95it/s]

Closest words to the center word mary: ['mary', 'eldest', 'jesus', 'nephew', 'son', 'aragon', 'brother', 'grandson', 'earl', 'duke']

Epoch 1/2:  36%|███▋      | 12499/34343 [2:21:26<17:57:48,  2.96s/it]

Epoch 1 Batch 12500 loss: 2.248018980026245

Epoch 1/2:  36%|███▋      | 12500/34343 [2:21:26<13:45:35,  2.27s/it]

Closest words to the center word mean: ['mathbf', 'frac', 'infty', 'cdots', 'inverse', 'cdot', 'mathrm', 'rangle', 'mbox', 'mean']

Epoch 1/2:  37%|███▋      | 12599/34343 [2:22:10<2:21:06,  2.57it/s] 

Epoch 1 Batch 12600 loss: 2.2393078804016113

Epoch 1/2:  37%|███▋      | 12600/34343 [2:22:11<2:49:14,  2.14it/s]

Closest words to the center word method: ['boolean', 'entropy', 'vector', 'mechanics', 'differential', 'quantum', 'dynamical', 'equations', 'coding', 'lossy']

Epoch 1/2:  37%|███▋      | 12699/34343 [2:22:57<1:19:39,  4.53it/s] 

Epoch 1 Batch 12700 loss: 2.1995980739593506

Epoch 1/2:  37%|███▋      | 12700/34343 [2:22:58<2:22:52,  2.52it/s]

Closest words to the center word subsequent: ['reign', 'subsequent', 'semitism', 'political', 'economic', 'reforms', 'lasted', 'stalin', 'abuses', 'napoleonic']

Epoch 1/2:  37%|███▋      | 12799/34343 [2:23:49<1:17:18,  4.64it/s] 

Epoch 1 Batch 12800 loss: 2.1995325088500977

Epoch 1/2:  37%|███▋      | 12800/34343 [2:23:50<2:00:18,  2.98it/s]

Closest words to the center word contingent: ['pluriform', 'socio', 'desertification', 'vested', 'judiciary', 'monetary', 'economic', 'bordering', 'judicial', 'coasts']

Epoch 1/2:  38%|███▊      | 12899/34343 [2:24:42<7:27:26,  1.25s/it] 

Epoch 1 Batch 12900 loss: 2.247729539871216

Epoch 1/2:  38%|███▊      | 12900/34343 [2:24:43<6:31:54,  1.10s/it]

Closest words to the center word british: ['british', 'laureate', 'politician', 'american', 'canadian', 'nobel', 'ifrcs', 'footballer', 'ifad', 'actor']

Epoch 1/2:  38%|███▊      | 12999/34343 [2:25:31<2:17:46,  2.58it/s] 

Epoch 1 Batch 13000 loss: 2.226405620574951

Epoch 1/2:  38%|███▊      | 13000/34343 [2:25:32<2:43:28,  2.18it/s]

Closest words to the center word battles: ['battles', 'gregorian', 'wars', 'ottoman', 'reign', 'war', 'outbreak', 'napoleonic', 'fought', 'invaded']

Epoch 1/2:  38%|███▊      | 13099/34343 [2:26:19<1:20:39,  4.39it/s] 

Epoch 1 Batch 13100 loss: 2.228623628616333

Epoch 1/2:  38%|███▊      | 13100/34343 [2:26:19<2:01:37,  2.91it/s]

Closest words to the center word ring: ['mathbf', 'cdots', 'frac', 'rightarrow', 'rangle', 'cdot', 'rang', 'langle', 'infty', 'mathcal']

Epoch 1/2:  38%|███▊      | 13199/34343 [2:27:01<1:14:37,  4.72it/s] 

Epoch 1 Batch 13200 loss: 2.221473217010498

Epoch 1/2:  38%|███▊      | 13200/34343 [2:27:01<1:57:55,  2.99it/s]

Closest words to the center word jeane: ['laureate', 'cyg', 'gwh', 'comedian', 'actor', 'footballer', 'grt', 'actress', 'inducted', 'pmid']

Epoch 1/2:  39%|███▊      | 13299/34343 [2:27:58<6:38:36,  1.14s/it] 

Epoch 1 Batch 13300 loss: 2.1854989528656006

Epoch 1/2:  39%|███▊      | 13300/34343 [2:27:59<5:47:55,  1.01it/s]

Closest words to the center word v: ['cdots', 'cdot', 'frac', 'rangle', 'rightarrow', 'mathbf', 'qquad', 'infty', 'rang', 'leq']

Epoch 1/2:  39%|███▉      | 13399/34343 [2:28:42<1:34:43,  3.69it/s] 

Epoch 1 Batch 13400 loss: 2.2130494117736816

Epoch 1/2:  39%|███▉      | 13400/34343 [2:28:43<2:11:23,  2.66it/s]

Closest words to the center word type: ['vector', 'electromagnetic', 'covalent', 'subset', 'morphism', 'ions', 'type', 'hydrogen', 'lossy', 'euclidean']

Epoch 1/2:  39%|███▉      | 13499/34343 [2:29:30<1:16:09,  4.56it/s] 

Epoch 1 Batch 13500 loss: 2.2179203033447266

Epoch 1/2:  39%|███▉      | 13500/34343 [2:29:30<1:57:49,  2.95it/s]

Closest words to the center word freemasons: ['wftu', 'wipo', 'unido', 'wmo', 'upu', 'ifrcs', 'ifad', 'wtoo', 'tribes', 'aryans']

Epoch 1/2:  40%|███▉      | 13599/34343 [2:30:30<1:12:31,  4.77it/s] 

Epoch 1 Batch 13600 loss: 2.207820177078247

Epoch 1/2:  40%|███▉      | 13600/34343 [2:30:31<1:51:01,  3.11it/s]

Closest words to the center word for: ['for', 'manpower', 'capita', 'parity', 'lossy', 'gwh', 'gregorian', 'cdot', 'copyleft', 'newnode']

Epoch 1/2:  40%|███▉      | 13699/34343 [2:31:16<2:26:15,  2.35it/s] 

Epoch 1 Batch 13700 loss: 2.229189395904541

Epoch 1/2:  40%|███▉      | 13700/34343 [2:31:17<2:50:38,  2.02it/s]

Closest words to the center word ale: ['ifad', 'ifrcs', 'icrm', 'callithrix', 'fricative', 'classis', 'ifc', 'sauce', 'agave', 'aloe']

Epoch 1/2:  40%|████      | 13799/34343 [2:31:59<1:18:16,  4.37it/s] 

Epoch 1 Batch 13800 loss: 2.2239222526550293

Epoch 1/2:  40%|████      | 13800/34343 [2:31:59<2:00:38,  2.84it/s]

Closest words to the center word stored: ['stored', 'storage', 'cathode', 'disk', 'lossy', 'floppy', 'input', 'vector', 'circuits', 'combustion']

Epoch 1/2:  40%|████      | 13899/34343 [2:32:36<1:13:06,  4.66it/s] 

Epoch 1 Batch 13900 loss: 2.2027957439422607

Epoch 1/2:  40%|████      | 13900/34343 [2:32:37<1:55:49,  2.94it/s]

Closest words to the center word will: ['will', 'shall', 'would', 'must', 'can', 'should', 'could', 'does', 'might', 'doesn']

Epoch 1/2:  41%|████      | 13999/34343 [2:33:13<1:10:47,  4.79it/s] 

Epoch 1 Batch 14000 loss: 2.2237448692321777

Epoch 1/2:  41%|████      | 14000/34343 [2:33:19<10:25:14,  1.84s/it]

Closest words to the center word changed: ['changed', 'gregorian', 'sixteenth', 'traced', 'lasted', 'weakened', 'ratified', 'ottoman', 'postponed', 'adopted']

Epoch 1/2:  41%|████      | 14099/34343 [2:33:56<2:02:07,  2.76it/s] 

Epoch 1 Batch 14100 loss: 2.222094774246216

Epoch 1/2:  41%|████      | 14100/34343 [2:33:56<2:29:22,  2.26it/s]

Closest words to the center word wages: ['manpower', 'wages', 'expenditures', 'income', 'males', 'revenues', 'investment', 'prices', 'mortality', 'rates']

Epoch 1/2:  41%|████▏     | 14199/34343 [2:34:34<1:13:00,  4.60it/s] 

Epoch 1 Batch 14200 loss: 2.184338092803955

Epoch 1/2:  41%|████▏     | 14200/34343 [2:34:35<1:52:56,  2.97it/s]

Closest words to the center word a: ['a', 'any', 'mathbf', 'every', 'cdot', 'an', 'infty', 'mathrm', 'countable', 'another']

Epoch 1/2:  42%|████▏     | 14299/34343 [2:35:09<1:09:43,  4.79it/s] 

Epoch 1 Batch 14300 loss: 2.216555595397949

Epoch 1/2:  42%|████▏     | 14300/34343 [2:35:09<1:49:01,  3.06it/s]

Closest words to the center word same: ['same', 'gregorian', 'polynomial', 'frac', 'cardinality', 'mathbf', 'inverse', 'discrete', 'exact', 'focal']

Epoch 1/2:  42%|████▏     | 14399/34343 [2:35:52<8:07:24,  1.47s/it] 

Epoch 1 Batch 14400 loss: 2.2166624069213867

Epoch 1/2:  42%|████▏     | 14400/34343 [2:35:53<6:44:04,  1.22s/it]

Closest words to the center word user: ['user', 'findable', 'interface', 'gnu', 'server', 'lossy', 'browser', 'browsers', 'dns', 'graphical']

Epoch 1/2:  42%|████▏     | 14499/34343 [2:36:33<1:19:21,  4.17it/s] 

Epoch 1 Batch 14500 loss: 2.2113637924194336

Epoch 1/2:  42%|████▏     | 14500/34343 [2:36:34<2:10:10,  2.54it/s]

Closest words to the center word fine: ['sauce', 'beverages', 'nickel', 'vegetables', 'liquid', 'hydroxide', 'dairy', 'minerals', 'metal', 'fine']

Epoch 1/2:  43%|████▎     | 14599/34343 [2:37:09<1:10:03,  4.70it/s] 

Epoch 1 Batch 14600 loss: 2.209588050842285

Epoch 1/2:  43%|████▎     | 14600/34343 [2:37:10<1:50:39,  2.97it/s]

Closest words to the center word for: ['for', 'manpower', 'newnode', 'lossy', 'parity', 'insubstantial', 'capita', 'lossless', 'median', 'copyleft']

Epoch 1/2:  43%|████▎     | 14699/34343 [2:37:46<1:11:10,  4.60it/s] 

Epoch 1 Batch 14700 loss: 2.2102251052856445

Epoch 1/2:  43%|████▎     | 14700/34343 [2:37:46<1:50:45,  2.96it/s]

Closest words to the center word following: ['gregorian', 'following', 'calendar', 'leap', 'punic', 'lunisolar', 'napoleonic', 'factbook', 'ottoman', 'feast']

Epoch 1/2:  43%|████▎     | 14799/34343 [2:38:31<3:12:17,  1.69it/s] 

Epoch 1 Batch 14800 loss: 2.21830677986145

Epoch 1/2:  43%|████▎     | 14800/34343 [2:38:32<3:19:50,  1.63it/s]

Closest words to the center word dancing: ['piano', 'jazz', 'solo', 'singers', 'dancing', 'genres', 'musical', 'singing', 'folk', 'violin']

Epoch 1/2:  43%|████▎     | 14899/34343 [2:39:29<1:22:30,  3.93it/s] 

Epoch 1 Batch 14900 loss: 2.1905465126037598

Epoch 1/2:  43%|████▎     | 14900/34343 [2:39:30<2:00:53,  2.68it/s]

Closest words to the center word gorges: ['km', 'unpaved', 'runways', 'humid', 'irrigated', 'temperate', 'pastures', 'arable', 'coastline', 'elevation']

Epoch 1/2:  44%|████▎     | 14999/34343 [2:40:11<1:08:21,  4.72it/s] 

Epoch 1 Batch 15000 loss: 2.2109179496765137

Epoch 1/2:  44%|████▎     | 15000/34343 [2:40:11<1:51:52,  2.88it/s]

Closest words to the center word compiling: ['lossy', 'browsers', 'input', 'lossless', 'boolean', 'bilinear', 'formats', 'dynamical', 'homomorphism', 'applications']

Epoch 1/2:  44%|████▍     | 15099/34343 [2:41:10<22:00:38,  4.12s/it]

Epoch 1 Batch 15100 loss: 2.195054054260254

Epoch 1/2:  44%|████▍     | 15100/34343 [2:41:11<16:38:17,  3.11s/it]

Closest words to the center word amd: ['gwh', 'kwh', 'cyg', 'twh', 'rfc', 'pmid', 'intel', 'isps', 'unpaved', 'ieee']

Epoch 1/2:  44%|████▍     | 15199/34343 [2:41:56<2:02:27,  2.61it/s] 

Epoch 1 Batch 15200 loss: 2.172524929046631

Epoch 1/2:  44%|████▍     | 15200/34343 [2:41:57<2:30:38,  2.12it/s]

Closest words to the center word al: ['al', 'ibn', 'abu', 'abd', 'agave', 'bin', 'ifrcs', 'unctad', 'wahhab', 'unido']

Epoch 1/2:  45%|████▍     | 15299/34343 [2:43:09<1:19:00,  4.02it/s] 

Epoch 1 Batch 15300 loss: 2.2157206535339355

Epoch 1/2:  45%|████▍     | 15300/34343 [2:43:10<1:54:51,  2.76it/s]

Closest words to the center word most: ['most', 'more', 'many', 'less', 'largest', 'earliest', 'best', 'populous', 'widely', 'among']

Epoch 1/2:  45%|████▍     | 15399/34343 [2:44:20<1:07:51,  4.65it/s] 

Epoch 1 Batch 15400 loss: 2.1843748092651367

Epoch 1/2:  45%|████▍     | 15400/34343 [2:44:21<1:44:43,  3.01it/s]

Closest words to the center word estonian: ['laureate', 'footballer', 'finalist', 'ivoire', 'estonian', 'ibrd', 'slovak', 'icrm', 'czech', 'physiologist']

Epoch 1/2:  45%|████▌     | 15499/34343 [2:45:47<19:14:30,  3.68s/it]

Epoch 1 Batch 15500 loss: 2.217909336090088

Epoch 1/2:  45%|████▌     | 15500/34343 [2:45:47<14:24:56,  2.75s/it]

Closest words to the center word portions: ['portions', 'denominations', 'ghats', 'basins', 'deserts', 'peoples', 'uplands', 'deposits', 'tributaries', 'shores']

Epoch 1/2:  45%|████▌     | 15599/34343 [2:46:44<1:55:41,  2.70it/s] 

Epoch 1 Batch 15600 loss: 2.2558064460754395

Epoch 1/2:  45%|████▌     | 15600/34343 [2:46:45<2:19:36,  2.24it/s]

Closest words to the center word by: ['by', 'agave', 'ifrcs', 'ifc', 'unctad', 'icrm', 'ifad', 'gwh', 'cyg', 'newly']

Epoch 1/2:  46%|████▌     | 15699/34343 [2:47:36<1:11:29,  4.35it/s] 

Epoch 1 Batch 15700 loss: 2.2247748374938965

Epoch 1/2:  46%|████▌     | 15700/34343 [2:47:37<1:46:19,  2.92it/s]

Closest words to the center word sade: ['sade', 'anh', 'baptiste', 'ois', 'biography', 'cegep', 'medici', 'johann', 'renoir', 'laureate']

Epoch 1/2:  46%|████▌     | 15799/34343 [2:48:44<1:05:14,  4.74it/s] 

Epoch 1 Batch 15800 loss: 2.1729252338409424

Epoch 1/2:  46%|████▌     | 15800/34343 [2:48:45<1:42:05,  3.03it/s]

Closest words to the center word emerging: ['ifrcs', 'desertification', 'ifad', 'ilo', 'unctad', 'faire', 'socio', 'oau', 'iom', 'ifc']

Epoch 1/2:  46%|████▋     | 15899/34343 [2:49:45<9:27:05,  1.84s/it] 

Epoch 1 Batch 15900 loss: 2.181105613708496

Epoch 1/2:  46%|████▋     | 15900/34343 [2:49:45<7:33:59,  1.48s/it]

Closest words to the center word in: ['in', 'throughout', 'pmid', 'nfc', 'kwh', 'gregorian', 'during', 'until', 'gwh', 'cyg']

Epoch 1/2:  47%|████▋     | 15999/34343 [2:50:37<1:35:13,  3.21it/s] 

Epoch 1 Batch 16000 loss: 2.2073497772216797

Epoch 1/2:  47%|████▋     | 16000/34343 [2:50:38<2:01:31,  2.52it/s]

Closest words to the center word groups: ['groups', 'ethnic', 'denominations', 'religions', 'minorities', 'parties', 'dialects', 'genders', 'minority', 'faiths']

Epoch 1/2:  47%|████▋     | 16099/34343 [2:51:19<1:09:39,  4.36it/s] 

Epoch 1 Batch 16100 loss: 2.1873230934143066

Epoch 1/2:  47%|████▋     | 16100/34343 [2:51:19<1:46:05,  2.87it/s]

Closest words to the center word india: ['timor', 'tajikistan', 'namibia', 'lanka', 'asia', 'swaziland', 'zambia', 'indonesia', 'india', 'nepal']

Epoch 1/2:  47%|████▋     | 16199/34343 [2:52:04<1:04:15,  4.71it/s] 

Epoch 1 Batch 16200 loss: 2.1928420066833496

Epoch 1/2:  47%|████▋     | 16200/34343 [2:52:05<1:43:11,  2.93it/s]

Closest words to the center word coaches: ['coach', 'playoffs', 'afc', 'nfc', 'gwh', 'kwh', 'runways', 'finalist', 'basketball', 'footballer']

Epoch 1/2:  47%|████▋     | 16299/34343 [2:53:12<4:34:32,  1.10it/s] 

Epoch 1 Batch 16300 loss: 2.2128429412841797

Epoch 1/2:  47%|████▋     | 16300/34343 [2:53:13<4:09:27,  1.21it/s]

Closest words to the center word found: ['found', 'mentioned', 'buried', 'agave', 'soluble', 'located', 'explained', 'shown', 'discovered', 'traced']

Epoch 1/2:  48%|████▊     | 16399/34343 [2:54:06<1:17:20,  3.87it/s] 

Epoch 1 Batch 16400 loss: 2.2028470039367676

Epoch 1/2:  48%|████▊     | 16400/34343 [2:54:07<1:49:27,  2.73it/s]

Closest words to the center word officer: ['officer', 'secretary', 'commander', 'deputy', 'unido', 'minister', 'appoints', 'marshal', 'politician', 'unctad']

Epoch 1/2:  48%|████▊     | 16499/34343 [2:55:03<1:02:54,  4.73it/s] 

Epoch 1 Batch 16500 loss: 2.228391408920288

Epoch 1/2:  48%|████▊     | 16500/34343 [2:55:03<1:40:02,  2.97it/s]

Closest words to the center word nero: ['tiberius', 'emperor', 'elector', 'grandson', 'throne', 'alexius', 'constantius', 'eldest', 'maximilian', 'aurelius']

Epoch 1/2:  48%|████▊     | 16599/34343 [2:55:51<1:05:26,  4.52it/s] 

Epoch 1 Batch 16600 loss: 2.1958260536193848

Epoch 1/2:  48%|████▊     | 16600/34343 [2:55:51<1:37:40,  3.03it/s]

Closest words to the center word europe: ['asia', 'populous', 'europe', 'scandinavia', 'nfc', 'bordering', 'annexed', 'subcontinent', 'timor', 'eurasia']

Epoch 1/2:  49%|████▊     | 16699/34343 [2:56:46<2:28:17,  1.98it/s] 

Epoch 1 Batch 16700 loss: 2.1679584980010986

Epoch 1/2:  49%|████▊     | 16700/34343 [2:56:46<2:39:58,  1.84it/s]

Closest words to the center word the: ['the', 'nfc', 'vernal', 'populous', 'afc', 'solar', 'scrimmage', 'its', 'frac', 'ecliptic']

Epoch 1/2:  49%|████▉     | 16799/34343 [2:57:45<1:04:42,  4.52it/s] 

Epoch 1 Batch 16800 loss: 2.1783878803253174

Epoch 1/2:  49%|████▉     | 16800/34343 [2:57:46<1:40:17,  2.92it/s]

Closest words to the center word order: ['order', 'differential', 'attempt', 'node', 'alphabetical', 'relation', 'accordance', 'topological', 'predicate', 'isomorphic']

Epoch 1/2:  49%|████▉     | 16899/34343 [2:58:33<1:01:27,  4.73it/s] 

Epoch 1 Batch 16900 loss: 2.2228479385375977

Epoch 1/2:  49%|████▉     | 16900/34343 [2:58:33<1:39:04,  2.93it/s]

Closest words to the center word included: ['included', 'published', 'playstation', 'appeared', 'featured', 'twh', 'bwv', 'released', 'gregorian', 'cyg']

Epoch 1/2:  49%|████▉     | 16999/34343 [2:59:30<14:07:21,  2.93s/it]

Epoch 1 Batch 17000 loss: 2.196394205093384

Epoch 1/2:  50%|████▉     | 17000/34343 [2:59:30<10:56:10,  2.27s/it]

Closest words to the center word scottish: ['laureate', 'footballer', 'theologian', 'politician', 'cricketer', 'scottish', 'physiologist', 'statesman', 'novelists', 'chemist']

Epoch 1/2:  50%|████▉     | 17099/34343 [3:00:18<1:39:05,  2.90it/s] 

Epoch 1 Batch 17100 loss: 2.187544584274292

Epoch 1/2:  50%|████▉     | 17100/34343 [3:00:18<2:03:07,  2.33it/s]

Closest words to the center word pilots: ['grt', 'playoffs', 'missiles', 'pilots', 'aircraft', 'manpower', 'runways', 'ships', 'gwh', 'helicopters']

Epoch 1/2:  50%|█████     | 17199/34343 [3:01:01<1:01:09,  4.67it/s] 

Epoch 1 Batch 17200 loss: 2.1696689128875732

Epoch 1/2:  50%|█████     | 17200/34343 [3:01:01<1:42:26,  2.79it/s]

Closest words to the center word comstock: ['gwh', 'twh', 'ifrcs', 'ifad', 'icrm', 'ifc', 'cyg', 'jul', 'agave', 'pngimage']

Epoch 1/2:  50%|█████     | 17299/34343 [3:01:58<1:00:05,  4.73it/s] 

Epoch 1 Batch 17300 loss: 2.1982107162475586

Epoch 1/2:  50%|█████     | 17300/34343 [3:01:59<1:34:10,  3.02it/s]

Closest words to the center word and: ['ifad', 'ifrcs', 'ifc', 'icrm', 'icftu', 'gwh', 'kwh', 'unctad', 'and', 'unido']

Epoch 1/2:  51%|█████     | 17399/34343 [3:03:02<3:11:06,  1.48it/s] 

Epoch 1 Batch 17400 loss: 2.195021629333496

Epoch 1/2:  51%|█████     | 17400/34343 [3:03:03<3:19:05,  1.42it/s]

Closest words to the center word during: ['during', 'period', 'manpower', 'war', 'outbreak', 'napoleonic', 'triassic', 'cretaceous', 'nfc', 'ottoman']

Epoch 1/2:  51%|█████     | 17499/34343 [3:03:46<1:07:06,  4.18it/s] 

Epoch 1 Batch 17500 loss: 2.182107925415039

Epoch 1/2:  51%|█████     | 17500/34343 [3:03:47<1:39:42,  2.82it/s]

Closest words to the center word line: ['scrimmage', 'line', 'mjs', 'cdots', 'perpendicular', 'node', 'pngimage', 'cue', 'playoffs', 'ecliptic']

Epoch 1/2:  51%|█████     | 17599/34343 [3:04:33<59:36,  4.68it/s]   

Epoch 1 Batch 17600 loss: 2.21285080909729

Epoch 1/2:  51%|█████     | 17600/34343 [3:04:33<1:32:42,  3.01it/s]

Closest words to the center word or: ['or', 'ifad', 'nucleic', 'reactive', 'carboxylic', 'ifrcs', 'covalent', 'than', 'carbohydrates', 'ions']

Epoch 1/2:  52%|█████▏    | 17699/34343 [3:05:15<1:00:43,  4.57it/s] 

Epoch 1 Batch 17700 loss: 2.1553187370300293

Epoch 1/2:  52%|█████▏    | 17700/34343 [3:05:30<21:04:49,  4.56s/it]

Closest words to the center word plateau: ['plateau', 'bordering', 'mountainous', 'humid', 'temperate', 'hilly', 'subtropical', 'southwest', 'basin', 'coastal']

Epoch 1/2:  52%|█████▏    | 17799/34343 [3:06:38<3:27:30,  1.33it/s] 

Epoch 1 Batch 17800 loss: 2.158411979675293

Epoch 1/2:  52%|█████▏    | 17800/34343 [3:06:39<3:15:00,  1.41it/s]

Closest words to the center word name: ['name', 'alphabet', 'word', 'title', 'derives', 'surname', 'derived', 'testament', 'hebrew', 'names']

Epoch 1/2:  52%|█████▏    | 17899/34343 [3:07:28<1:07:02,  4.09it/s] 

Epoch 1 Batch 17900 loss: 2.1678919792175293

Epoch 1/2:  52%|█████▏    | 17900/34343 [3:07:29<1:40:59,  2.71it/s]

Closest words to the center word tr: ['cdots', 'rightarrow', 'cdot', 'leq', 'otimes', 'qquad', 'equiv', 'aq', 'mbox', 'rang']

Epoch 1/2:  52%|█████▏    | 17999/34343 [3:08:10<58:23,  4.66it/s]   

Epoch 1 Batch 18000 loss: 2.214918375015259

Epoch 1/2:  52%|█████▏    | 18000/34343 [3:08:11<1:31:00,  2.99it/s]

Closest words to the center word one: ['gwh', 'cyg', 'twh', 'kwh', 'jul', 'grt', 'pmid', 'unpaved', 'lup', 'pngimage']

Epoch 1/2:  53%|█████▎    | 18099/34343 [3:09:12<11:17:49,  2.50s/it]

Epoch 1 Batch 18100 loss: 2.172701597213745

Epoch 1/2:  53%|█████▎    | 18100/34343 [3:09:12<8:46:37,  1.95s/it] 

Closest words to the center word khartoum: ['opcw', 'iom', 'rupee', 'swaziland', 'nfc', 'tajikistan', 'barbuda', 'zambia', 'ifc', 'bordering']

Epoch 1/2:  53%|█████▎    | 18199/34343 [3:09:55<1:38:37,  2.73it/s] 

Epoch 1 Batch 18200 loss: 2.181840658187866

Epoch 1/2:  53%|█████▎    | 18200/34343 [3:09:55<2:00:25,  2.23it/s]

Closest words to the center word mitsubishi: ['gwh', 'kwh', 'unpaved', 'twh', 'runways', 'cyg', 'mjs', 'dwt', 'pngimage', 'telephones']

Epoch 1/2:  53%|█████▎    | 18299/34343 [3:10:52<1:03:13,  4.23it/s] 

Epoch 1 Batch 18300 loss: 2.1708874702453613

Epoch 1/2:  53%|█████▎    | 18300/34343 [3:10:53<1:33:06,  2.87it/s]

Closest words to the center word apple: ['macintosh', 'apple', 'amiga', 'ibm', 'intel', 'microsoft', 'atari', 'pc', 'mac', 'playstation']

Epoch 1/2:  54%|█████▎    | 18399/34343 [3:11:50<56:32,  4.70it/s]   

Epoch 1 Batch 18400 loss: 2.180054187774658

Epoch 1/2:  54%|█████▎    | 18400/34343 [3:11:51<1:28:37,  3.00it/s]

Closest words to the center word prior: ['prior', 'grt', 'gwh', 'twh', 'spend', 'manpower', 'devote', 'kwh', 'leap', 'resolve']

Epoch 1/2:  54%|█████▍    | 18499/34343 [3:12:47<7:25:49,  1.69s/it] 

Epoch 1 Batch 18500 loss: 2.1845858097076416

Epoch 1/2:  54%|█████▍    | 18500/34343 [3:12:47<6:00:32,  1.37s/it]

Closest words to the center word horse: ['horse', 'sox', 'callithrix', 'humid', 'playoffs', 'grt', 'equus', 'playoff', 'tamarin', 'wild']

Epoch 1/2:  54%|█████▍    | 18599/34343 [3:13:35<1:22:50,  3.17it/s] 

Epoch 1 Batch 18600 loss: 2.209273099899292

Epoch 1/2:  54%|█████▍    | 18600/34343 [3:13:36<1:46:07,  2.47it/s]

Closest words to the center word different: ['different', 'distinct', 'variety', 'countably', 'eukaryotic', 'abelian', 'grammatical', 'various', 'finite', 'multicellular']

Epoch 1/2:  54%|█████▍    | 18699/34343 [3:14:18<1:03:57,  4.08it/s] 

Epoch 1 Batch 18700 loss: 2.2018415927886963

Epoch 1/2:  54%|█████▍    | 18700/34343 [3:14:18<1:32:36,  2.82it/s]

Closest words to the center word catastrophic: ['humid', 'ottoman', 'outbreak', 'rainfall', 'ottomans', 'recession', 'manpower', 'grt', 'erosion', 'humidity']

Epoch 1/2:  55%|█████▍    | 18799/34343 [3:15:04<54:35,  4.75it/s]   

Epoch 1 Batch 18800 loss: 2.192572593688965

Epoch 1/2:  55%|█████▍    | 18800/34343 [3:15:05<1:23:57,  3.09it/s]

Closest words to the center word m: ['cdots', 'cdot', 'rightarrow', 'qquad', 'mathbf', 'leq', 'cyg', 'otimes', 'frac', 'mathrm']

Epoch 1/2:  55%|█████▌    | 18899/34343 [3:16:10<4:38:27,  1.08s/it] 

Epoch 1 Batch 18900 loss: 2.1832780838012695

Epoch 1/2:  55%|█████▌    | 18900/34343 [3:16:10<4:07:20,  1.04it/s]

Closest words to the center word dawlah: ['gwh', 'cyg', 'twh', 'pngimage', 'jul', 'kwh', 'grt', 'pmid', 'finalist', 'icrm']

Epoch 1/2:  55%|█████▌    | 18999/34343 [3:17:00<1:09:28,  3.68it/s] 

Epoch 1 Batch 19000 loss: 2.15401029586792

Epoch 1/2:  55%|█████▌    | 19000/34343 [3:17:00<1:35:46,  2.67it/s]

Closest words to the center word runs: ['nfc', 'runs', 'afc', 'playoffs', 'runways', 'unpaved', 'divisional', 'grt', 'playoff', 'kilometers']

Epoch 1/2:  56%|█████▌    | 19099/34343 [3:17:46<53:35,  4.74it/s]   

Epoch 1 Batch 19100 loss: 2.178741455078125

Epoch 1/2:  56%|█████▌    | 19100/34343 [3:17:47<1:24:18,  3.01it/s]

Closest words to the center word word: ['word', 'verb', 'alphabet', 'nouns', 'hebrew', 'infinitive', 'derives', 'adjectives', 'declension', 'aramaic']

Epoch 1/2:  56%|█████▌    | 19199/34343 [3:18:34<53:49,  4.69it/s]   

Epoch 1 Batch 19200 loss: 2.2004618644714355

Epoch 1/2:  56%|█████▌    | 19200/34343 [3:18:35<1:23:38,  3.02it/s]

Closest words to the center word coherent: ['topological', 'priori', 'homomorphism', 'eukaryotic', 'pluriform', 'countably', 'coherent', 'holomorphic', 'irreducibly', 'thermodynamic']

Epoch 1/2:  56%|█████▌    | 19299/34343 [3:19:39<4:12:36,  1.01s/it] 

Epoch 1 Batch 19300 loss: 2.1489410400390625

Epoch 1/2:  56%|█████▌    | 19300/34343 [3:19:40<3:45:58,  1.11it/s]

Closest words to the center word national: ['national', 'unctad', 'ifc', 'ifrcs', 'opcw', 'icrm', 'iom', 'unicameral', 'ifad', 'wto']

Epoch 1/2:  56%|█████▋    | 19399/34343 [3:20:36<55:27,  4.49it/s]   

Epoch 1 Batch 19400 loss: 2.1447086334228516

Epoch 1/2:  56%|█████▋    | 19400/34343 [3:20:36<1:24:31,  2.95it/s]

Closest words to the center word nine: ['gwh', 'kwh', 'twh', 'cyg', 'pngimage', 'jul', 'grt', 'pmid', 'births', 'nfc']

Epoch 1/2:  57%|█████▋    | 19499/34343 [3:21:32<52:41,  4.70it/s]   

Epoch 1 Batch 19500 loss: 2.176616668701172

Epoch 1/2:  57%|█████▋    | 19500/34343 [3:21:32<1:23:06,  2.98it/s]

Closest words to the center word scripture: ['scripture', 'jesus', 'testament', 'judaism', 'scriptures', 'prophet', 'christianity', 'baptism', 'tanakh', 'teachings']

Epoch 1/2:  57%|█████▋    | 19599/34343 [3:22:07<51:53,  4.74it/s]  

Epoch 1 Batch 19600 loss: 2.1506729125976562

Epoch 1/2:  57%|█████▋    | 19600/34343 [3:22:12<7:41:34,  1.88s/it]

Closest words to the center word catholic: ['catholic', 'orthodox', 'lutheran', 'anglican', 'churches', 'church', 'episcopal', 'communion', 'protestant', 'denominations']

Epoch 1/2:  57%|█████▋    | 19699/34343 [3:22:52<58:26,  4.18it/s]   

Epoch 1 Batch 19700 loss: 2.2006030082702637

Epoch 1/2:  57%|█████▋    | 19700/34343 [3:22:53<1:30:42,  2.69it/s]

Closest words to the center word term: ['term', 'noun', 'pejorative', 'adjective', 'word', 'usage', 'derogatory', 'declension', 'synonym', 'verb']

Epoch 1/2:  58%|█████▊    | 19799/34343 [3:23:37<52:12,  4.64it/s]   

Epoch 1 Batch 19800 loss: 2.1924495697021484

Epoch 1/2:  58%|█████▊    | 19800/34343 [3:23:37<1:28:34,  2.74it/s]

Closest words to the center word exchange: ['kwh', 'exchange', 'gwh', 'unpaved', 'runways', 'expenditures', 'imports', 'telephones', 'capita', 'income']

Epoch 1/2:  58%|█████▊    | 19899/34343 [3:24:21<53:40,  4.49it/s]   

Epoch 1 Batch 19900 loss: 2.175184965133667

Epoch 1/2:  58%|█████▊    | 19900/34343 [3:24:22<1:23:48,  2.87it/s]

Closest words to the center word immigrant: ['immigrant', 'novelists', 'ethnic', 'entertainers', 'naturalized', 'hispanic', 'indigenous', 'immigrants', 'minority', 'lithuanians']

Epoch 1/2:  58%|█████▊    | 19999/34343 [3:25:13<1:38:46,  2.42it/s] 

Epoch 1 Batch 20000 loss: 2.192317008972168

Epoch 1/2:  58%|█████▊    | 20000/34343 [3:25:14<1:53:43,  2.10it/s]

Closest words to the center word downtown: ['downtown', 'township', 'devry', 'erie', 'urbana', 'nfc', 'lansing', 'county', 'afc', 'abet']

Epoch 1/2:  59%|█████▊    | 20099/34343 [3:25:57<1:00:47,  3.91it/s]

Epoch 1 Batch 20100 loss: 2.1599984169006348

Epoch 1/2:  59%|█████▊    | 20100/34343 [3:25:58<1:27:51,  2.70it/s]

Closest words to the center word four: ['gwh', 'cyg', 'grt', 'kwh', 'twh', 'unpaved', 'pngimage', 'pmid', 'jul', 'runways']

Epoch 1/2:  59%|█████▉    | 20199/34343 [3:26:42<49:56,  4.72it/s]   

Epoch 1 Batch 20200 loss: 2.2375783920288086

Epoch 1/2:  59%|█████▉    | 20200/34343 [3:26:42<1:20:05,  2.94it/s]

Closest words to the center word after: ['after', 'before', 'months', 'shortly', 'grt', 'during', 'lasted', 'leap', 'thereafter', 'ottoman']

Epoch 1/2:  59%|█████▉    | 20299/34343 [3:27:25<49:14,  4.75it/s]   

Epoch 1 Batch 20300 loss: 2.1636946201324463

Epoch 1/2:  59%|█████▉    | 20300/34343 [3:27:26<1:19:07,  2.96it/s]

Closest words to the center word layer: ['layer', 'intelsat', 'ifrcs', 'iom', 'icrm', 'ozone', 'ifc', 'ifad', 'tanker', 'gwh']

Epoch 1/2:  59%|█████▉    | 20399/34343 [3:28:18<2:50:54,  1.36it/s] 

Epoch 1 Batch 20400 loss: 2.1505420207977295

Epoch 1/2:  59%|█████▉    | 20400/34343 [3:28:19<3:12:03,  1.21it/s]

Closest words to the center word automobile: ['kwh', 'gwh', 'icrm', 'ifrcs', 'ifc', 'finalist', 'iom', 'pngimage', 'ifad', 'mjs']

Epoch 1/2:  60%|█████▉    | 20499/34343 [3:29:02<55:40,  4.14it/s]   

Epoch 1 Batch 20500 loss: 2.1120963096618652

Epoch 1/2:  60%|█████▉    | 20500/34343 [3:29:03<1:23:43,  2.76it/s]

Closest words to the center word wives: ['wives', 'daughters', 'eldest', 'sons', 'prophecies', 'pregnant', 'niece', 'elector', 'heirs', 'remarried']

Epoch 1/2:  60%|█████▉    | 20599/34343 [3:29:44<48:07,  4.76it/s]   

Epoch 1 Batch 20600 loss: 2.2029638290405273

Epoch 1/2:  60%|█████▉    | 20600/34343 [3:29:44<1:17:44,  2.95it/s]

Closest words to the center word set: ['topological', 'morphism', 'set', 'morphisms', 'countably', 'finite', 'homomorphism', 'subset', 'infty', 'countable']

Epoch 1/2:  60%|██████    | 20699/34343 [3:30:34<47:55,  4.74it/s]   

Epoch 1 Batch 20700 loss: 2.179687261581421

Epoch 1/2:  60%|██████    | 20700/34343 [3:30:44<12:17:34,  3.24s/it]

Closest words to the center word classical: ['classical', 'baroque', 'analytic', 'liberalism', 'composers', 'philosophers', 'renaissance', 'individualist', 'genres', 'physics']

Epoch 1/2:  61%|██████    | 20799/34343 [3:31:28<2:23:32,  1.57it/s] 

Epoch 1 Batch 20800 loss: 2.1586945056915283

Epoch 1/2:  61%|██████    | 20800/34343 [3:31:28<2:26:01,  1.55it/s]

Closest words to the center word main: ['main', 'telephones', 'demographics', 'factbook', 'bissau', 'faso', 'verde', 'windward', 'federated', 'comoros']

Epoch 1/2:  61%|██████    | 20899/34343 [3:32:08<49:56,  4.49it/s]  

Epoch 1 Batch 20900 loss: 2.179399013519287

Epoch 1/2:  61%|██████    | 20900/34343 [3:32:09<1:20:35,  2.78it/s]

Closest words to the center word st: ['st', 'nd', 'saint', 'afc', 'nfc', 'earl', 'agave', 'rd', 'th', 'yoannis']

Epoch 1/2:  61%|██████    | 20999/34343 [3:32:47<47:55,  4.64it/s]   

Epoch 1 Batch 21000 loss: 2.194061279296875

Epoch 1/2:  61%|██████    | 21000/34343 [3:32:48<1:24:07,  2.64it/s]

Closest words to the center word entirely: ['entirely', 'evenly', 'completely', 'exclusively', 'supplanted', 'universally', 'intelligible', 'falsifiable', 'axiomatic', 'identical']

Epoch 1/2:  61%|██████▏   | 21099/34343 [3:33:47<9:11:50,  2.50s/it] 

Epoch 1 Batch 21100 loss: 2.1672215461730957

Epoch 1/2:  61%|██████▏   | 21100/34343 [3:33:48<7:13:04,  1.96s/it]

Closest words to the center word griffin: ['footballer', 'cricketer', 'politician', 'actor', 'songwriter', 'wrestler', 'philanthropist', 'pianist', 'actress', 'swimmer']

Epoch 1/2:  62%|██████▏   | 21199/34343 [3:34:28<1:18:32,  2.79it/s] 

Epoch 1 Batch 21200 loss: 2.168684720993042

Epoch 1/2:  62%|██████▏   | 21200/34343 [3:34:29<1:40:58,  2.17it/s]

Closest words to the center word but: ['but', 'falsifiable', 'universally', 'though', 'necessarily', 'countably', 'happen', 'supplanted', 'gregorian', 'because']

Epoch 1/2:  62%|██████▏   | 21299/34343 [3:35:07<48:27,  4.49it/s]   

Epoch 1 Batch 21300 loss: 2.2020010948181152

Epoch 1/2:  62%|██████▏   | 21300/34343 [3:35:08<1:14:56,  2.90it/s]

Closest words to the center word genres: ['genres', 'singers', 'composers', 'songwriters', 'styles', 'westerns', 'jazz', 'compositions', 'musical', 'themes']

Epoch 1/2:  62%|██████▏   | 21399/34343 [3:35:48<45:23,  4.75it/s]   

Epoch 1 Batch 21400 loss: 2.167635202407837

Epoch 1/2:  62%|██████▏   | 21400/34343 [3:35:48<1:14:18,  2.90it/s]

Closest words to the center word both: ['both', 'various', 'faiths', 'many', 'several', 'vested', 'pluriform', 'intelligible', 'unicellular', 'incomprehensible']

Epoch 1/2:  63%|██████▎   | 21499/34343 [3:36:29<4:33:00,  1.28s/it]

Epoch 1 Batch 21500 loss: 2.1763830184936523

Epoch 1/2:  63%|██████▎   | 21500/34343 [3:36:30<3:49:15,  1.07s/it]

Closest words to the center word anka: ['pngimage', 'footballer', 'cyg', 'cricketer', 'jul', 'swimmer', 'finalist', 'laureate', 'bwv', 'eug']

Epoch 1/2:  63%|██████▎   | 21599/34343 [3:37:11<56:37,  3.75it/s]  

Epoch 1 Batch 21600 loss: 2.194138526916504

Epoch 1/2:  63%|██████▎   | 21600/34343 [3:37:11<1:19:39,  2.67it/s]

Closest words to the center word however: ['gwh', 'argue', 'however', 'kwh', 'believe', 'lifes', 'falsifiable', 'though', 'necessarily', 'vested']

Epoch 1/2:  63%|██████▎   | 21699/34343 [3:37:54<45:08,  4.67it/s]   

Epoch 1 Batch 21700 loss: 2.184887409210205

Epoch 1/2:  63%|██████▎   | 21700/34343 [3:37:55<1:08:40,  3.07it/s]

Closest words to the center word than: ['than', 'lifes', 'slightly', 'less', 'expectancy', 'millimeters', 'gwh', 'more', 'faster', 'considerably']

Epoch 1/2:  63%|██████▎   | 21799/34343 [3:38:42<44:36,  4.69it/s]   

Epoch 1 Batch 21800 loss: 2.1667675971984863

Epoch 1/2:  63%|██████▎   | 21800/34343 [3:38:42<1:08:38,  3.05it/s]

Closest words to the center word even: ['even', 'lifes', 'expensive', 'noticeable', 'worse', 'perceive', 'consuming', 'grt', 'occur', 'tolerated']

Epoch 1/2:  64%|██████▍   | 21899/34343 [3:39:35<2:30:40,  1.38it/s] 

Epoch 1 Batch 21900 loss: 2.1998085975646973

Epoch 1/2:  64%|██████▍   | 21900/34343 [3:39:36<2:21:39,  1.46it/s]

Closest words to the center word most: ['most', 'more', 'less', 'lifes', 'many', 'earliest', 'largest', 'highly', 'populous', 'especially']

Epoch 1/2:  64%|██████▍   | 21999/34343 [3:40:12<45:34,  4.51it/s]  

Epoch 1 Batch 22000 loss: 2.1783924102783203

Epoch 1/2:  64%|██████▍   | 22000/34343 [3:40:13<1:12:41,  2.83it/s]

Closest words to the center word land: ['arable', 'land', 'irrigated', 'pastures', 'km', 'sq', 'unpaved', 'runways', 'kilometers', 'hydropower']

Epoch 1/2:  64%|██████▍   | 22099/34343 [3:40:54<43:41,  4.67it/s]   

Epoch 1 Batch 22100 loss: 2.199709415435791

Epoch 1/2:  64%|██████▍   | 22100/34343 [3:40:55<1:07:33,  3.02it/s]

Closest words to the center word descended: ['descended', 'aryans', 'indo', 'slavic', 'assimilated', 'germanic', 'borrowed', 'aramaic', 'inhabited', 'migrated']

Epoch 1/2:  65%|██████▍   | 22199/34343 [3:41:38<43:12,  4.68it/s]  

Epoch 1 Batch 22200 loss: 2.1871800422668457

Epoch 1/2:  65%|██████▍   | 22200/34343 [3:41:38<1:07:47,  2.99it/s]

Closest words to the center word then: ['otimes', 'infty', 'cdots', 'rangle', 'operatorname', 'qquad', 'leq', 'ldots', 'cdot', 'bigg']

Epoch 1/2:  65%|██████▍   | 22299/34343 [3:42:30<1:14:40,  2.69it/s] 

Epoch 1 Batch 22300 loss: 2.1602892875671387

Epoch 1/2:  65%|██████▍   | 22300/34343 [3:42:31<1:29:12,  2.25it/s]

Closest words to the center word th: ['th', 'nd', 'nfc', 'nineteenth', 'rd', 'afc', 'gwh', 'twentieth', 'seventeenth', 'sixteenth']

Epoch 1/2:  65%|██████▌   | 22399/34343 [3:43:19<59:56,  3.32it/s]   

Epoch 1 Batch 22400 loss: 2.1497557163238525

Epoch 1/2:  65%|██████▌   | 22400/34343 [3:43:19<1:20:30,  2.47it/s]

Closest words to the center word broadly: ['broadly', 'nouns', 'linguists', 'goidelic', 'fundamentalist', 'heretical', 'dialects', 'phonology', 'dravidian', 'urdu']

Epoch 1/2:  66%|██████▌   | 22499/34343 [3:44:05<41:26,  4.76it/s]   

Epoch 1 Batch 22500 loss: 2.182007312774658

Epoch 1/2:  66%|██████▌   | 22500/34343 [3:44:05<1:04:32,  3.06it/s]

Closest words to the center word nation: ['populous', 'republic', 'nation', 'legislature', 'bicameral', 'timor', 'territory', 'subcontinent', 'factbook', 'macedonia']

Epoch 1/2:  66%|██████▌   | 22599/34343 [3:44:57<1:35:55,  2.04it/s] 

Epoch 1 Batch 22600 loss: 2.1575498580932617

Epoch 1/2:  66%|██████▌   | 22600/34343 [3:44:58<1:46:26,  1.84it/s]

Closest words to the center word all: ['all', 'abelian', 'morphisms', 'integers', 'identical', 'reals', 'various', 'finite', 'many', 'countably']

Epoch 1/2:  66%|██████▌   | 22699/34343 [3:45:41<1:04:36,  3.00it/s] 

Epoch 1 Batch 22700 loss: 2.166353702545166

Epoch 1/2:  66%|██████▌   | 22700/34343 [3:45:41<1:21:13,  2.39it/s]

Closest words to the center word n: ['cdots', 'cdot', 'otimes', 'rightarrow', 'qquad', 'leq', 'mathrm', 'frac', 'ldots', 'operatorname']

Epoch 1/2:  66%|██████▋   | 22799/34343 [3:46:24<41:33,  4.63it/s]   

Epoch 1 Batch 22800 loss: 2.1599996089935303

Epoch 1/2:  66%|██████▋   | 22800/34343 [3:46:24<1:04:53,  2.96it/s]

Closest words to the center word century: ['century', 'centuries', 'nfc', 'afc', 'bc', 'nineteenth', 'bce', 'dynasty', 'philosophers', 'th']

Epoch 1/2:  67%|██████▋   | 22899/34343 [3:47:20<40:37,  4.69it/s]   

Epoch 1 Batch 22900 loss: 2.165294885635376

Epoch 1/2:  67%|██████▋   | 22900/34343 [3:47:20<1:02:53,  3.03it/s]

Closest words to the center word qquad: ['qquad', 'cdot', 'cdots', 'otimes', 'frac', 'mathbf', 'leq', 'rangle', 'ldots', 'mathrm']

Epoch 1/2:  67%|██████▋   | 22999/34343 [3:48:13<2:27:26,  1.28it/s]

Epoch 1 Batch 23000 loss: 2.1681160926818848

Epoch 1/2:  67%|██████▋   | 23000/34343 [3:48:14<2:22:11,  1.33it/s]

Closest words to the center word organization: ['ifrcs', 'ifc', 'iom', 'unido', 'unctad', 'ifad', 'opcw', 'ilo', 'upu', 'iho']

Epoch 1/2:  67%|██████▋   | 23099/34343 [3:49:02<46:25,  4.04it/s]  

Epoch 1 Batch 23100 loss: 2.163196325302124

Epoch 1/2:  67%|██████▋   | 23100/34343 [3:49:02<1:10:34,  2.66it/s]

Closest words to the center word could: ['could', 'can', 'should', 'would', 'might', 'must', 'shall', 'will', 'wouldn', 'did']

Epoch 1/2:  68%|██████▊   | 23199/34343 [3:49:45<41:47,  4.44it/s]  

Epoch 1 Batch 23200 loss: 2.1257312297821045

Epoch 1/2:  68%|██████▊   | 23200/34343 [3:49:46<1:03:32,  2.92it/s]

Closest words to the center word situation: ['situation', 'pluriform', 'vested', 'recession', 'impose', 'coercion', 'coup', 'laissez', 'regimes', 'restructuring']

Epoch 1/2:  68%|██████▊   | 23299/34343 [3:50:34<38:57,  4.72it/s]   

Epoch 1 Batch 23300 loss: 2.1690590381622314

Epoch 1/2:  68%|██████▊   | 23300/34343 [3:50:34<1:00:40,  3.03it/s]

Closest words to the center word has: ['has', 'had', 'have', 'having', 'pluriform', 'ifrcs', 'been', 'ifc', 'ifad', 'enjoys']

Epoch 1/2:  68%|██████▊   | 23399/34343 [3:51:32<2:22:20,  1.28it/s] 

Epoch 1 Batch 23400 loss: 2.1786019802093506

Epoch 1/2:  68%|██████▊   | 23400/34343 [3:51:33<2:12:50,  1.37it/s]

Closest words to the center word see: ['see', 'list', 'disambiguation', 'newnode', 'est', 'topics', 'wtoo', 'redirects', 'wmo', 'ifrcs']

Epoch 1/2:  68%|██████▊   | 23499/34343 [3:52:33<43:18,  4.17it/s]   

Epoch 1 Batch 23500 loss: 2.1621744632720947

Epoch 1/2:  68%|██████▊   | 23500/34343 [3:52:34<1:04:50,  2.79it/s]

Closest words to the center word heard: ['heard', 'sorry', 'wouldn', 'loved', 'tonight', 'couldn', 'cried', 'replied', 'll', 'hadn']

Epoch 1/2:  69%|██████▊   | 23599/34343 [3:53:22<38:29,  4.65it/s]  

Epoch 1 Batch 23600 loss: 2.185946464538574

Epoch 1/2:  69%|██████▊   | 23600/34343 [3:53:23<59:35,  3.00it/s]

Closest words to the center word observations: ['observations', 'relativity', 'assumptions', 'aether', 'planets', 'gravitational', 'galaxies', 'diffraction', 'findable', 'mechanics']

Epoch 1/2:  69%|██████▉   | 23699/34343 [3:54:27<14:50:07,  5.02s/it]

Epoch 1 Batch 23700 loss: 2.178269386291504

Epoch 1/2:  69%|██████▉   | 23700/34343 [3:54:27<10:55:07,  3.69s/it]

Closest words to the center word people: ['people', 'births', 'albanians', 'jews', 'refugees', 'americans', 'nationality', 'immigrants', 'africans', 'citizens']

Epoch 1/2:  69%|██████▉   | 23799/34343 [3:55:17<1:07:02,  2.62it/s] 

Epoch 1 Batch 23800 loss: 2.169729709625244

Epoch 1/2:  69%|██████▉   | 23800/34343 [3:55:18<1:30:42,  1.94it/s]

Closest words to the center word game: ['game', 'games', 'console', 'nintendo', 'playstation', 'championship', 'consoles', 'playoff', 'nfc', 'afc']

Epoch 1/2:  70%|██████▉   | 23899/34343 [3:56:25<38:16,  4.55it/s]   

Epoch 1 Batch 23900 loss: 2.174274444580078

Epoch 1/2:  70%|██████▉   | 23900/34343 [3:56:26<57:46,  3.01it/s]

Closest words to the center word minister: ['minister', 'prime', 'deputy', 'ministers', 'cdu', 'chancellor', 'boutros', 'attlee', 'secretary', 'president']

Epoch 1/2:  70%|██████▉   | 23999/34343 [3:57:19<37:32,  4.59it/s]   

Epoch 1 Batch 24000 loss: 2.1914472579956055

Epoch 1/2:  70%|██████▉   | 24000/34343 [3:57:20<59:25,  2.90it/s]

Closest words to the center word known: ['known', 'referred', 'regarded', 'agave', 'classified', 'suited', 'classed', 'described', 'abbreviated', 'argumentum']

Epoch 1/2:  70%|███████   | 24099/34343 [3:58:33<7:27:52,  2.62s/it] 

Epoch 1 Batch 24100 loss: 2.1623618602752686

Epoch 1/2:  70%|███████   | 24100/34343 [3:58:34<5:44:44,  2.02s/it]

Closest words to the center word the: ['pluriform', 'the', 'nfc', 'ifrcs', 'vernal', 'icrm', 'ifc', 'afc', 'intelsat', 'desertification']

Epoch 1/2:  70%|███████   | 24199/34343 [3:59:34<43:35,  3.88it/s]   

Epoch 1 Batch 24200 loss: 2.130772352218628

Epoch 1/2:  70%|███████   | 24200/34343 [3:59:34<1:01:35,  2.74it/s]

Closest words to the center word left: ['left', 'rangle', 'otimes', 'cdots', 'cdot', 'frac', 'mathbf', 'operatorname', 'rang', 'right']

Epoch 1/2:  71%|███████   | 24299/34343 [4:00:27<37:45,  4.43it/s]   

Epoch 1 Batch 24300 loss: 2.1426546573638916

Epoch 1/2:  71%|███████   | 24300/34343 [4:00:28<1:06:34,  2.51it/s]

Closest words to the center word grover: ['township', 'footballer', 'earl', 'leiserson', 'cricketer', 'viscount', 'finalist', 'ssn', 'laureate', 'cyg']

Epoch 1/2:  71%|███████   | 24399/34343 [4:01:32<35:22,  4.68it/s]   

Epoch 1 Batch 24400 loss: 2.164360523223877

Epoch 1/2:  71%|███████   | 24400/34343 [4:01:33<57:52,  2.86it/s]

Closest words to the center word catholics: ['catholics', 'orthodox', 'denominations', 'christians', 'churches', 'theologians', 'anglicans', 'protestants', 'protestant', 'bishops']

Epoch 1/2:  71%|███████▏  | 24499/34343 [4:02:38<1:34:36,  1.73it/s] 

Epoch 1 Batch 24500 loss: 2.15714693069458

Epoch 1/2:  71%|███████▏  | 24500/34343 [4:02:39<1:36:26,  1.70it/s]

Closest words to the center word after: ['after', 'before', 'shortly', 'months', 'during', 'afterwards', 'lasted', 'thereafter', 'grt', 'abdur']

Epoch 1/2:  72%|███████▏  | 24599/34343 [4:03:26<37:17,  4.35it/s]  

Epoch 1 Batch 24600 loss: 2.1690213680267334

Epoch 1/2:  72%|███████▏  | 24600/34343 [4:03:27<1:00:02,  2.70it/s]

Closest words to the center word others: ['others', 'agnostics', 'psychologists', 'scholars', 'neopagans', 'masculists', 'atheists', 'sects', 'unido', 'feminists']

Epoch 1/2:  72%|███████▏  | 24699/34343 [4:04:25<34:29,  4.66it/s]   

Epoch 1 Batch 24700 loss: 2.1629137992858887

Epoch 1/2:  72%|███████▏  | 24700/34343 [4:04:26<57:08,  2.81it/s]

Closest words to the center word history: ['history', 'demographics', 'allafrica', 'factbook', 'geography', 'federated', 'timeline', 'barbuda', 'topics', 'archaeology']

Epoch 1/2:  72%|███████▏  | 24799/34343 [4:05:37<8:46:35,  3.31s/it] 

Epoch 1 Batch 24800 loss: 2.1606433391571045

Epoch 1/2:  72%|███████▏  | 24800/34343 [4:05:38<6:41:02,  2.52s/it]

Closest words to the center word tao: ['tao', 'tiberian', 'otimes', 'ching', 'landsmannschaft', 'nevi', 'factum', 'ctus', 'infinitive', 'polskiej']

Epoch 1/2:  73%|███████▎  | 24899/34343 [4:06:36<1:05:32,  2.40it/s] 

Epoch 1 Batch 24900 loss: 2.2006969451904297

Epoch 1/2:  73%|███████▎  | 24900/34343 [4:06:37<1:13:43,  2.13it/s]

Closest words to the center word criminal: ['criminal', 'enforcement', 'tribunal', 'unido', 'crimes', 'jurisdiction', 'appellate', 'liability', 'defendant', 'judicial']

Epoch 1/2:  73%|███████▎  | 24999/34343 [4:07:33<34:59,  4.45it/s]   

Epoch 1 Batch 25000 loss: 2.1721980571746826

Epoch 1/2:  73%|███████▎  | 25000/34343 [4:07:34<57:53,  2.69it/s]

Closest words to the center word and: ['ifrcs', 'ifad', 'kwh', 'gwh', 'ifc', 'icrm', 'unctad', 'ibrd', 'opcw', 'icftu']

Epoch 1/2:  73%|███████▎  | 25099/34343 [4:08:22<32:50,  4.69it/s]  

Epoch 1 Batch 25100 loss: 2.1649208068847656

Epoch 1/2:  73%|███████▎  | 25100/34343 [4:08:22<53:48,  2.86it/s]

Closest words to the center word vfl: ['nfc', 'afc', 'playoffs', 'championship', 'nfl', 'divisional', 'playoff', 'finals', 'steelers', 'broncos']

Epoch 1/2:  73%|███████▎  | 25199/34343 [4:09:34<6:38:34,  2.62s/it] 

Epoch 1 Batch 25200 loss: 2.140643358230591

Epoch 1/2:  73%|███████▎  | 25200/34343 [4:09:34<5:11:34,  2.04s/it]

Closest words to the center word one: ['gwh', 'kwh', 'cyg', 'grt', 'twh', 'pngimage', 'jul', 'unpaved', 'pmid', 'hbk']

Epoch 1/2:  74%|███████▎  | 25299/34343 [4:10:41<57:17,  2.63it/s]   

Epoch 1 Batch 25300 loss: 2.167630910873413

Epoch 1/2:  74%|███████▎  | 25300/34343 [4:10:41<1:11:59,  2.09it/s]

Closest words to the center word some: ['some', 'many', 'various', 'several', 'certain', 'anecdotal', 'hundreds', 'lifes', 'most', 'faiths']

Epoch 1/2:  74%|███████▍  | 25399/34343 [4:11:50<33:00,  4.52it/s]   

Epoch 1 Batch 25400 loss: 2.109294891357422

Epoch 1/2:  74%|███████▍  | 25400/34343 [4:11:50<53:27,  2.79it/s]

Closest words to the center word added: ['added', 'bits', 'gwh', 'kwh', 'telephones', 'twh', 'unpaved', 'cyg', 'grt', 'demographics']

Epoch 1/2:  74%|███████▍  | 25499/34343 [4:12:56<31:12,  4.72it/s]   

Epoch 1 Batch 25500 loss: 2.1761088371276855

Epoch 1/2:  74%|███████▍  | 25500/34343 [4:12:57<52:49,  2.79it/s]

Closest words to the center word beginner: ['beginner', 'findable', 'tutorials', 'karaoke', 'brainfuck', 'venues', 'downloads', 'databases', 'lossless', 'lossy']

Epoch 1/2:  75%|███████▍  | 25599/34343 [4:14:13<4:34:06,  1.88s/it] 

Epoch 1 Batch 25600 loss: 2.1636462211608887

Epoch 1/2:  75%|███████▍  | 25600/34343 [4:14:14<3:43:12,  1.53s/it]

Closest words to the center word the: ['the', 'nfc', 'pluriform', 'vernal', 'populous', 'desertification', 'vested', 'scrimmage', 'bicameral', 'ifrcs']

Epoch 1/2:  75%|███████▍  | 25699/34343 [4:15:16<39:02,  3.69it/s]   

Epoch 1 Batch 25700 loss: 2.1753110885620117

Epoch 1/2:  75%|███████▍  | 25700/34343 [4:15:16<53:50,  2.68it/s]

Closest words to the center word property: ['property', 'pluriform', 'ethical', 'monetary', 'welfare', 'equality', 'liability', 'intellectual', 'rights', 'profit']

Epoch 1/2:  75%|███████▌  | 25799/34343 [4:16:05<30:36,  4.65it/s]  

Epoch 1 Batch 25800 loss: 2.143805503845215

Epoch 1/2:  75%|███████▌  | 25800/34343 [4:16:06<46:22,  3.07it/s]

Closest words to the center word six: ['gwh', 'kwh', 'grt', 'cyg', 'pngimage', 'twh', 'unpaved', 'mjs', 'pmid', 'jul']

Epoch 1/2:  75%|███████▌  | 25899/34343 [4:17:04<29:39,  4.74it/s]   

Epoch 1 Batch 25900 loss: 2.1542530059814453

Epoch 1/2:  75%|███████▌  | 25900/34343 [4:17:05<46:21,  3.04it/s]

Closest words to the center word were: ['were', 'are', 'camps', 'persecuted', 'romans', 'been', 'serbs', 'brutally', 'greeks', 'mongols']

Epoch 1/2:  76%|███████▌  | 25999/34343 [4:18:06<1:58:38,  1.17it/s]

Epoch 1 Batch 26000 loss: 2.15213680267334

Epoch 1/2:  76%|███████▌  | 26000/34343 [4:18:07<1:49:01,  1.28it/s]

Closest words to the center word reject: ['reject', 'laissez', 'ethical', 'egoism', 'reconstructionist', 'fundamentalists', 'naturalism', 'deny', 'relativism', 'anarcho']

Epoch 1/2:  76%|███████▌  | 26099/34343 [4:19:15<35:07,  3.91it/s]   

Epoch 1 Batch 26100 loss: 2.172351837158203

Epoch 1/2:  76%|███████▌  | 26100/34343 [4:19:16<51:11,  2.68it/s]

Closest words to the center word supported: ['unido', 'supported', 'unctad', 'pluriform', 'upu', 'wmo', 'wipo', 'criticized', 'exercised', 'vested']

Epoch 1/2:  76%|███████▋  | 26199/34343 [4:20:09<28:37,  4.74it/s]  

Epoch 1 Batch 26200 loss: 2.1398873329162598

Epoch 1/2:  76%|███████▋  | 26200/34343 [4:20:10<46:27,  2.92it/s]

Closest words to the center word often: ['often', 'commonly', 'sometimes', 'interchangeably', 'widely', 'generally', 'frequently', 'synonymously', 'usually', 'chemically']

Epoch 1/2:  77%|███████▋  | 26299/34343 [4:21:13<28:15,  4.74it/s]   

Epoch 1 Batch 26300 loss: 2.1213347911834717

Epoch 1/2:  77%|███████▋  | 26300/34343 [4:21:30<11:31:49,  5.16s/it]

Closest words to the center word noblemen: ['noblemen', 'nobles', 'brutally', 'bolsheviks', 'vassals', 'persecuted', 'fled', 'persecutions', 'pashtun', 'raped']

Epoch 1/2:  77%|███████▋  | 26399/34343 [4:22:28<1:00:32,  2.19it/s] 

Epoch 1 Batch 26400 loss: 2.1519837379455566

Epoch 1/2:  77%|███████▋  | 26400/34343 [4:22:29<1:09:46,  1.90it/s]

Closest words to the center word at: ['at', 'devry', 'graduate', 'abet', 'near', 'ssn', 'utc', 'hochschule', 'attended', 'urbana']

Epoch 1/2:  77%|███████▋  | 26499/34343 [4:23:38<28:52,  4.53it/s]   

Epoch 1 Batch 26500 loss: 2.1253271102905273

Epoch 1/2:  77%|███████▋  | 26500/34343 [4:23:39<45:35,  2.87it/s]

Closest words to the center word k: ['otimes', 'cdot', 'rightarrow', 'cdots', 'leq', 'operatorname', 'qquad', 'ldots', 'rangle', 'mathrm']

Epoch 1/2:  77%|███████▋  | 26599/34343 [4:24:44<27:19,  4.72it/s]   

Epoch 1 Batch 26600 loss: 2.144522190093994

Epoch 1/2:  77%|███████▋  | 26600/34343 [4:24:45<46:49,  2.76it/s]

Closest words to the center word rex: ['tamarin', 'callithrix', 'marmoset', 'leiserson', 'cegep', 'wco', 'sieur', 'eulemur', 'eug', 'agave']

Epoch 1/2:  78%|███████▊  | 26699/34343 [4:26:10<7:26:30,  3.50s/it] 

Epoch 1 Batch 26700 loss: 2.166740655899048

Epoch 1/2:  78%|███████▊  | 26700/34343 [4:26:11<5:37:01,  2.65s/it]

Closest words to the center word air: ['air', 'ifrcs', 'icrm', 'ifad', 'navy', 'ifc', 'opcw', 'missiles', 'pollution', 'iom']

Epoch 1/2:  78%|███████▊  | 26799/34343 [4:27:08<30:46,  4.09it/s]   

Epoch 1 Batch 26800 loss: 2.105173110961914

Epoch 1/2:  78%|███████▊  | 26800/34343 [4:27:09<43:43,  2.88it/s]

Closest words to the center word word: ['word', 'kanji', 'declension', 'pronoun', 'sanskrit', 'alphabet', 'verb', 'hebrew', 'tiberian', 'transliteration']

Epoch 1/2:  78%|███████▊  | 26899/34343 [4:28:17<26:42,  4.64it/s]   

Epoch 1 Batch 26900 loss: 2.165642023086548

Epoch 1/2:  78%|███████▊  | 26900/34343 [4:28:18<1:08:59,  1.80it/s]

Closest words to the center word its: ['its', 'their', 'ifrcs', 'telephones', 'ifad', 'our', 'depends', 'vested', 'iho', 'varies']

Epoch 1/2:  79%|███████▊  | 26999/34343 [4:29:10<26:29,  4.62it/s]  

Epoch 1 Batch 27000 loss: 2.178239583969116

Epoch 1/2:  79%|███████▊  | 27000/34343 [4:29:10<41:04,  2.98it/s]

Closest words to the center word accessible: ['accessible', 'findable', 'lossy', 'apis', 'gprs', 'expensive', 'accessing', 'available', 'browsers', 'cheaper']

Epoch 1/2:  79%|███████▉  | 27099/34343 [4:30:30<1:32:48,  1.30it/s] 

Epoch 1 Batch 27100 loss: 2.155221462249756

Epoch 1/2:  79%|███████▉  | 27100/34343 [4:30:30<1:27:15,  1.38it/s]

Closest words to the center word clergyman: ['clergyman', 'politician', 'statesman', 'theologian', 'novelist', 'laureate', 'philanthropist', 'cricketer', 'footballer', 'dramatist']

Epoch 1/2:  79%|███████▉  | 27199/34343 [4:31:32<29:56,  3.98it/s]   

Epoch 1 Batch 27200 loss: 2.1478168964385986

Epoch 1/2:  79%|███████▉  | 27200/34343 [4:31:32<42:45,  2.78it/s]

Closest words to the center word as: ['as', 'argumentum', 'clavier', 'regarded', 'colloquially', 'agave', 'hominem', 'boutros', 'insofar', 'popularly']

Epoch 1/2:  79%|███████▉  | 27299/34343 [4:32:43<25:35,  4.59it/s]   

Epoch 1 Batch 27300 loss: 2.148089647293091

Epoch 1/2:  79%|███████▉  | 27300/34343 [4:32:43<39:47,  2.95it/s]

Closest words to the center word rogers: ['comedian', 'hammett', 'actress', 'bandleader', 'lesh', 'mcvie', 'ronnie', 'coach', 'comedienne', 'actor']

Epoch 1/2:  80%|███████▉  | 27399/34343 [4:33:58<9:54:46,  5.14s/it]

Epoch 1 Batch 27400 loss: 2.1383790969848633

Epoch 1/2:  80%|███████▉  | 27400/34343 [4:33:58<7:17:52,  3.78s/it]

Closest words to the center word material: ['material', 'findable', 'cvd', 'rna', 'amplification', 'solvents', 'dioxide', 'molecules', 'eukaryotic', 'tissues']

Epoch 1/2:  80%|████████  | 27499/34343 [4:34:57<1:08:43,  1.66it/s]

Epoch 1 Batch 27500 loss: 2.1313536167144775

Epoch 1/2:  80%|████████  | 27500/34343 [4:34:57<1:09:04,  1.65it/s]

Closest words to the center word calls: ['newnode', 'findable', 'calls', 'asks', 'prev', 'asking', 'unido', 'wants', 'thank', 'requests']

Epoch 1/2:  80%|████████  | 27599/34343 [4:36:03<27:07,  4.14it/s]   

Epoch 1 Batch 27600 loss: 2.1406517028808594

Epoch 1/2:  80%|████████  | 27600/34343 [4:36:04<40:19,  2.79it/s]

Closest words to the center word production: ['kwh', 'gwh', 'production', 'twh', 'commodities', 'textiles', 'imports', 'fuels', 'exports', 'electricity']

Epoch 1/2:  81%|████████  | 27699/34343 [4:37:07<23:49,  4.65it/s]  

Epoch 1 Batch 27700 loss: 2.1324963569641113

Epoch 1/2:  81%|████████  | 27700/34343 [4:37:08<41:34,  2.66it/s]

Closest words to the center word chronicle: ['chronicle', 'nevi', 'mamre', 'mechon', 'hammadi', 'afc', 'nfc', 'dramatists', 'encyclop', 'tanakh']

Epoch 1/2:  81%|████████  | 27799/34343 [4:38:15<3:53:54,  2.14s/it]

Epoch 1 Batch 27800 loss: 2.1638875007629395

Epoch 1/2:  81%|████████  | 27800/34343 [4:38:16<3:06:28,  1.71s/it]

Closest words to the center word spoken: ['spoken', 'dialects', 'urdu', 'languages', 'slavic', 'speakers', 'bangla', 'afrikaans', 'dialect', 'hindi']

Epoch 1/2:  81%|████████  | 27899/34343 [4:39:18<51:57,  2.07it/s]  

Epoch 1 Batch 27900 loss: 2.159383535385132

Epoch 1/2:  81%|████████  | 27900/34343 [4:39:18<59:29,  1.80it/s]

Closest words to the center word animation: ['animation', 'animated', 'anime', 'consoles', 'codecs', 'machinima', 'lossy', 'graphics', 'karaoke', 'lossless']

Epoch 1/2:  82%|████████▏ | 27999/34343 [4:40:27<24:06,  4.39it/s]  

Epoch 1 Batch 28000 loss: 2.138845920562744

Epoch 1/2:  82%|████████▏ | 28000/34343 [4:40:27<38:53,  2.72it/s]

Closest words to the center word beginning: ['gregorian', 'beginning', 'cretaceous', 'leap', 'end', 'proleptic', 'triassic', 'outbreak', 'vernal', 'nineteenth']

Epoch 1/2:  82%|████████▏ | 28099/34343 [4:41:37<22:21,  4.66it/s]  

Epoch 1 Batch 28100 loss: 2.157573699951172

Epoch 1/2:  82%|████████▏ | 28100/34343 [4:41:38<35:55,  2.90it/s]

Closest words to the center word k: ['otimes', 'cdot', 'cdots', 'leq', 'rightarrow', 'qquad', 'ldots', 'operatorname', 'infty', 'rangle']

Epoch 1/2:  82%|████████▏ | 28199/34343 [4:42:49<3:50:50,  2.25s/it]

Epoch 1 Batch 28200 loss: 2.166411876678467

Epoch 1/2:  82%|████████▏ | 28200/34343 [4:42:49<3:03:17,  1.79s/it]

Closest words to the center word super: ['afc', 'nfc', 'super', 'playoffs', 'playoff', 'divisional', 'kart', 'pngimage', 'bowl', 'steelers']

Epoch 1/2:  82%|████████▏ | 28299/34343 [4:43:49<35:29,  2.84it/s]  

Epoch 1 Batch 28300 loss: 2.1673877239227295

Epoch 1/2:  82%|████████▏ | 28300/34343 [4:43:50<45:45,  2.20it/s]

Closest words to the center word by: ['by', 'ifrcs', 'ifc', 'icrm', 'ifad', 'agave', 'unctad', 'unido', 'ibrd', 'xaver']

Epoch 1/2:  83%|████████▎ | 28399/34343 [4:45:00<21:53,  4.53it/s]  

Epoch 1 Batch 28400 loss: 2.122408390045166

Epoch 1/2:  83%|████████▎ | 28400/34343 [4:45:01<36:13,  2.73it/s]

Closest words to the center word out: ['out', 'down', 'up', 'off', 'away', 'scrimmage', 'prev', 'onto', 'reins', 'couldn']

Epoch 1/2:  83%|████████▎ | 28499/34343 [4:46:09<20:33,  4.74it/s]  

Epoch 1 Batch 28500 loss: 2.1728084087371826

Epoch 1/2:  83%|████████▎ | 28500/34343 [4:46:09<34:36,  2.81it/s]

Closest words to the center word death: ['death', 'resurrection', 'expectancy', 'infant', 'tenji', 'imprisonment', 'mortality', 'birth', 'jehoram', 'sentenced']

Epoch 1/2:  83%|████████▎ | 28599/34343 [4:47:33<2:05:07,  1.31s/it]

Epoch 1 Batch 28600 loss: 2.1430716514587402

Epoch 1/2:  83%|████████▎ | 28600/34343 [4:47:34<1:48:30,  1.13s/it]

Closest words to the center word masters: ['nfc', 'masters', 'championships', 'championship', 'winners', 'champions', 'afc', 'champion', 'finals', 'wimbledon']

Epoch 1/2:  84%|████████▎ | 28699/34343 [4:48:45<26:15,  3.58it/s]  

Epoch 1 Batch 28700 loss: 2.115452289581299

Epoch 1/2:  84%|████████▎ | 28700/34343 [4:48:45<38:24,  2.45it/s]

Closest words to the center word children: ['children', 'householder', 'couples', 'males', 'females', 'female', 'daughters', 'parents', 'male', 'adults']

Epoch 1/2:  84%|████████▍ | 28799/34343 [4:49:54<19:54,  4.64it/s]  

Epoch 1 Batch 28800 loss: 2.215231418609619

Epoch 1/2:  84%|████████▍ | 28800/34343 [4:49:55<33:19,  2.77it/s]

Closest words to the center word iraq: ['iraq', 'saudi', 'syria', 'iraqi', 'kuwait', 'dinar', 'peacekeeping', 'gaza', 'hussein', 'lebanon']

Epoch 1/2:  84%|████████▍ | 28899/34343 [4:51:03<19:08,  4.74it/s]  

Epoch 1 Batch 28900 loss: 2.143759250640869

Epoch 1/2:  84%|████████▍ | 28900/34343 [4:51:03<32:34,  2.78it/s]

Closest words to the center word salt: ['beets', 'broadleaf', 'cassava', 'salt', 'humid', 'limestone', 'subtropical', 'arable', 'citrus', 'gravel']

Epoch 1/2:  84%|████████▍ | 28999/34343 [4:52:27<1:09:25,  1.28it/s]

Epoch 1 Batch 29000 loss: 2.1738033294677734

Epoch 1/2:  84%|████████▍ | 29000/34343 [4:52:28<1:06:59,  1.33it/s]

Closest words to the center word the: ['the', 'nfc', 'pluriform', 'ifrcs', 'vernal', 'ifc', 'newnode', 'afc', 'scrimmage', 'nicaea']

Epoch 1/2:  85%|████████▍ | 29099/34343 [4:53:37<19:26,  4.50it/s]  

Epoch 1 Batch 29100 loss: 2.130866289138794

Epoch 1/2:  85%|████████▍ | 29100/34343 [4:53:37<31:46,  2.75it/s]

Closest words to the center word whether: ['whether', 'falsifiable', 'subjective', 'ethical', 'countably', 'causal', 'cardinality', 'verifiable', 'normative', 'disprove']

Epoch 1/2:  85%|████████▌ | 29199/34343 [4:54:46<18:11,  4.71it/s]  

Epoch 1 Batch 29200 loss: 2.167146682739258

Epoch 1/2:  85%|████████▌ | 29200/34343 [4:54:47<29:43,  2.88it/s]

Closest words to the center word wear: ['wear', 'pants', 'eat', 'wears', 'throw', 'tamarin', 'wore', 'wearing', 'dresses', 'dressing']

Epoch 1/2:  85%|████████▌ | 29299/34343 [4:56:10<6:56:10,  4.95s/it]

Epoch 1 Batch 29300 loss: 2.1464898586273193

Epoch 1/2:  85%|████████▌ | 29300/34343 [4:56:11<5:09:19,  3.68s/it]

Closest words to the center word canaan: ['assyria', 'canaan', 'persia', 'constantinople', 'ceded', 'conquered', 'judah', 'heraclius', 'sumer', 'flee']

Epoch 1/2:  86%|████████▌ | 29399/34343 [4:57:19<26:18,  3.13it/s]  

Epoch 1 Batch 29400 loss: 2.1150240898132324

Epoch 1/2:  86%|████████▌ | 29400/34343 [4:57:20<34:01,  2.42it/s]

Closest words to the center word principal: ['principal', 'largest', 'ifad', 'vested', 'administrative', 'gangetic', 'main', 'leeward', 'primary', 'ifc']

Epoch 1/2:  86%|████████▌ | 29499/34343 [4:58:28<18:08,  4.45it/s]  

Epoch 1 Batch 29500 loss: 2.1666815280914307

Epoch 1/2:  86%|████████▌ | 29500/34343 [4:58:28<30:55,  2.61it/s]

Closest words to the center word of: ['of', 'ifrcs', 'ifad', 'ifc', 'akan', 'icrm', 'wtoo', 'rajonas', 'nazarene', 'iom']

Epoch 1/2:  86%|████████▌ | 29599/34343 [4:59:38<16:56,  4.67it/s]  

Epoch 1 Batch 29600 loss: 2.1527047157287598

Epoch 1/2:  86%|████████▌ | 29600/34343 [4:59:38<28:13,  2.80it/s]

Closest words to the center word a: ['a', 'pluriform', 'automorphism', 'morphism', 'cardinality', 'every', 'otimes', 'newnode', 'another', 'any']

Epoch 1/2:  86%|████████▋ | 29699/34343 [5:01:03<1:19:02,  1.02s/it]

Epoch 1 Batch 29700 loss: 2.139249324798584

Epoch 1/2:  86%|████████▋ | 29700/34343 [5:01:03<1:09:47,  1.11it/s]

Closest words to the center word canadian: ['canadian', 'footballer', 'american', 'ifrcs', 'cricketer', 'australian', 'kwh', 'ifc', 'laureate', 'comedian']

Epoch 1/2:  87%|████████▋ | 29799/34343 [5:02:10<20:47,  3.64it/s]  

Epoch 1 Batch 29800 loss: 2.1743788719177246

Epoch 1/2:  87%|████████▋ | 29800/34343 [5:02:11<28:17,  2.68it/s]

Closest words to the center word metres: ['grt', 'gwh', 'metres', 'inches', 'meters', 'unpaved', 'twh', 'runways', 'ft', 'kilometers']

Epoch 1/2:  87%|████████▋ | 29899/34343 [5:03:19<17:31,  4.22it/s]  

Epoch 1 Batch 29900 loss: 2.123094081878662

Epoch 1/2:  87%|████████▋ | 29900/34343 [5:03:20<26:14,  2.82it/s]

Closest words to the center word aka: ['pngimage', 'ifrcs', 'icrm', 'mjs', 'ifc', 'ifad', 'agave', 'icftu', 'ibrd', 'tamarin']

Epoch 1/2:  87%|████████▋ | 29999/34343 [5:04:28<15:29,  4.67it/s]  

Epoch 1 Batch 30000 loss: 2.1571402549743652

Epoch 1/2:  87%|████████▋ | 30000/34343 [5:04:45<6:17:40,  5.22s/it]

Closest words to the center word singapore: ['opcw', 'chungcheong', 'barbuda', 'swaziland', 'icrm', 'busan', 'kwh', 'ifrcs', 'singapore', 'rupee']

Epoch 1/2:  88%|████████▊ | 30099/34343 [5:05:54<55:07,  1.28it/s]  

Epoch 1 Batch 30100 loss: 2.1652884483337402

Epoch 1/2:  88%|████████▊ | 30100/34343 [5:05:55<52:38,  1.34it/s]

Closest words to the center word a: ['a', 'pluriform', 'automorphism', 'cardinality', 'every', 'morphism', 'otimes', 'monoid', 'another', 'newnode']

Epoch 1/2:  88%|████████▊ | 30199/34343 [5:07:02<17:32,  3.94it/s]  

Epoch 1 Batch 30200 loss: 2.124663829803467

Epoch 1/2:  88%|████████▊ | 30200/34343 [5:07:03<25:40,  2.69it/s]

Closest words to the center word record: ['record', 'playoffs', 'grammy', 'nfl', 'mvp', 'singles', 'batting', 'playoff', 'championship', 'billboard']

Epoch 1/2:  88%|████████▊ | 30299/34343 [5:08:11<14:42,  4.58it/s]  

Epoch 1 Batch 30300 loss: 2.1587517261505127

Epoch 1/2:  88%|████████▊ | 30300/34343 [5:08:12<23:10,  2.91it/s]

Closest words to the center word b: ['b', 'd', 'cyg', 'laureate', 'pngimage', 'footballer', 'physiologist', 'pmid', 'gwh', 'cdots']

Epoch 1/2:  89%|████████▊ | 30399/34343 [5:09:35<5:00:57,  4.58s/it]

Epoch 1 Batch 30400 loss: 2.1257636547088623

Epoch 1/2:  89%|████████▊ | 30400/34343 [5:09:35<3:43:47,  3.41s/it]

Closest words to the center word philo: ['yoannis', 'nevi', 'philo', 'patriarch', 'palaeologus', 'barnabas', 'epiphanius', 'ezekiel', 'isaiah', 'jesu']

Epoch 1/2:  89%|████████▉ | 30499/34343 [5:10:44<38:26,  1.67it/s]  

Epoch 1 Batch 30500 loss: 2.1307451725006104

Epoch 1/2:  89%|████████▉ | 30500/34343 [5:10:45<41:19,  1.55it/s]

Closest words to the center word began: ['began', 'continued', 'started', 'helped', 'went', 'came', 'attempted', 'saw', 'coincided', 'culminated']

Epoch 1/2:  89%|████████▉ | 30599/34343 [5:11:52<14:43,  4.24it/s]  

Epoch 1 Batch 30600 loss: 2.1571314334869385

Epoch 1/2:  89%|████████▉ | 30600/34343 [5:11:53<22:46,  2.74it/s]

Closest words to the center word field: ['field', 'magnetic', 'mathbf', 'dipole', 'topological', 'electromagnetic', 'vector', 'gravitational', 'automorphism', 'subfield']

Epoch 1/2:  89%|████████▉ | 30699/34343 [5:13:00<13:04,  4.65it/s]  

Epoch 1 Batch 30700 loss: 2.15496826171875

Epoch 1/2:  89%|████████▉ | 30700/34343 [5:13:01<21:34,  2.81it/s]

Closest words to the center word fiction: ['fiction', 'fantasy', 'horror', 'novels', 'dystopian', 'fandom', 'science', 'illustrators', 'anthologies', 'mythos']

Epoch 1/2:  90%|████████▉ | 30799/34343 [5:14:25<3:27:37,  3.52s/it]

Epoch 1 Batch 30800 loss: 2.1584253311157227

Epoch 1/2:  90%|████████▉ | 30800/34343 [5:14:26<2:36:34,  2.65s/it]

Closest words to the center word ii: ['ii', 'iii', 'iv', 'palaeologus', 'yoannis', 'vii', 'anastasius', 'comnenus', 'jumaada', 'viii']

Epoch 1/2:  90%|████████▉ | 30899/34343 [5:15:33<23:17,  2.46it/s]  

Epoch 1 Batch 30900 loss: 2.1477136611938477

Epoch 1/2:  90%|████████▉ | 30900/34343 [5:15:34<27:38,  2.08it/s]

Closest words to the center word held: ['held', 'elected', 'elections', 'vested', 'referendum', 'election', 'nfc', 'nominated', 'appointed', 'ecumenical']

Epoch 1/2:  90%|█████████ | 30999/34343 [5:16:41<12:17,  4.53it/s]  

Epoch 1 Batch 31000 loss: 2.171220302581787

Epoch 1/2:  90%|█████████ | 31000/34343 [5:16:42<18:59,  2.93it/s]

Closest words to the center word consisted: ['consisted', 'consists', 'pluriform', 'consisting', 'consist', 'dwt', 'comprised', 'regiments', 'battalions', 'composed']

Epoch 1/2:  91%|█████████ | 31099/34343 [5:17:49<11:20,  4.76it/s]  

Epoch 1 Batch 31100 loss: 2.148926258087158

Epoch 1/2:  91%|█████████ | 31100/34343 [5:17:50<25:37,  2.11it/s]

Closest words to the center word antonio: ['antonio', 'anh', 'cegep', 'universidad', 'bwv', 'footballer', 'icrm', 'eug', 'lez', 'rovere']

Epoch 1/2:  91%|█████████ | 31199/34343 [5:19:16<1:36:34,  1.84s/it]

Epoch 1 Batch 31200 loss: 2.142033338546753

Epoch 1/2:  91%|█████████ | 31200/34343 [5:19:17<1:18:24,  1.50s/it]

Closest words to the center word the: ['the', 'pluriform', 'ifrcs', 'vernal', 'nfc', 'newnode', 'ghats', 'outskirts', 'holocene', 'nabla']

Epoch 1/2:  91%|█████████ | 31299/34343 [5:20:25<16:37,  3.05it/s]  

Epoch 1 Batch 31300 loss: 2.155806064605713

Epoch 1/2:  91%|█████████ | 31300/34343 [5:20:25<21:34,  2.35it/s]

Closest words to the center word is: ['is', 'exists', 'bijective', 'satisfies', 'contains', 'refers', 'depends', 'serves', 'approximates', 'was']

Epoch 1/2:  91%|█████████▏| 31399/34343 [5:21:33<10:29,  4.67it/s]  

Epoch 1 Batch 31400 loss: 2.1606945991516113

Epoch 1/2:  91%|█████████▏| 31400/34343 [5:21:34<16:30,  2.97it/s]

Closest words to the center word camel: ['callithrix', 'equus', 'stenella', 'ferus', 'camel', 'ursus', 'mentha', 'tamarin', 'hamster', 'leontopithecus']

Epoch 1/2:  92%|█████████▏| 31499/34343 [5:22:41<09:57,  4.76it/s]  

Epoch 1 Batch 31500 loss: 2.0854337215423584

Epoch 1/2:  92%|█████████▏| 31500/34343 [5:22:42<16:08,  2.94it/s]

Closest words to the center word occurred: ['occurred', 'lasted', 'erupted', 'outbreak', 'eruption', 'lasts', 'resulted', 'happened', 'pleistocene', 'existed']

Epoch 1/2:  92%|█████████▏| 31599/34343 [5:24:05<46:46,  1.02s/it]  

Epoch 1 Batch 31600 loss: 2.163994073867798

Epoch 1/2:  92%|█████████▏| 31600/34343 [5:24:06<41:22,  1.11it/s]

Closest words to the center word for: ['for', 'ifrcs', 'ifad', 'newnode', 'iom', 'ifc', 'lossy', 'prev', 'lastnode', 'provide']

Epoch 1/2:  92%|█████████▏| 31699/34343 [5:25:14<10:00,  4.40it/s]  

Epoch 1 Batch 31700 loss: 2.14137601852417

Epoch 1/2:  92%|█████████▏| 31700/34343 [5:25:15<15:09,  2.91it/s]

Closest words to the center word form: ['form', 'adjoint', 'subgroup', 'groupoid', 'pluriform', 'bilinear', 'mathfrak', 'approximant', 'covalent', 'conjugate']

Epoch 1/2:  93%|█████████▎| 31799/34343 [5:26:22<09:10,  4.62it/s]  

Epoch 1 Batch 31800 loss: 2.1413497924804688

Epoch 1/2:  93%|█████████▎| 31800/34343 [5:26:23<14:10,  2.99it/s]

Closest words to the center word separate: ['separate', 'distinct', 'homomorphism', 'pluriform', 'different', 'functor', 'morphisms', 'homomorphisms', 'grouped', 'subnational']

Epoch 1/2:  93%|█████████▎| 31899/34343 [5:27:30<08:50,  4.61it/s]  

Epoch 1 Batch 31900 loss: 2.151773691177368

Epoch 1/2:  93%|█████████▎| 31900/34343 [5:27:48<3:42:49,  5.47s/it]

Closest words to the center word ideological: ['ideological', 'egalitarianism', 'leninist', 'authoritarianism', 'authoritarian', 'ideology', 'marxism', 'socio', 'laissez', 'leninism']

Epoch 1/2:  93%|█████████▎| 31999/34343 [5:28:56<13:27,  2.90it/s]  

Epoch 1 Batch 32000 loss: 2.17047119140625

Epoch 1/2:  93%|█████████▎| 32000/34343 [5:28:57<16:36,  2.35it/s]

Closest words to the center word their: ['their', 'its', 'your', 'our', 'her', 'his', 'respective', 'gesserit', 'themselves', 'my']

Epoch 1/2:  93%|█████████▎| 32099/34343 [5:30:04<08:23,  4.46it/s]  

Epoch 1 Batch 32100 loss: 2.138092041015625

Epoch 1/2:  93%|█████████▎| 32100/34343 [5:30:05<13:01,  2.87it/s]

Closest words to the center word regime: ['regime', 'bolsheviks', 'coup', 'gorbachev', 'reforms', 'pluriform', 'commissar', 'junta', 'communist', 'perestroika']

Epoch 1/2:  94%|█████████▍| 32199/34343 [5:31:13<07:55,  4.51it/s]  

Epoch 1 Batch 32200 loss: 2.1601781845092773

Epoch 1/2:  94%|█████████▍| 32200/34343 [5:31:13<12:08,  2.94it/s]

Closest words to the center word time: ['time', 'period', 'utc', 'distances', 'elapsed', 'moment', 'vernal', 'mhz', 'speeds', 'cardinality']

Epoch 1/2:  94%|█████████▍| 32299/34343 [5:32:37<44:22,  1.30s/it]  

Epoch 1 Batch 32300 loss: 2.141155242919922

Epoch 1/2:  94%|█████████▍| 32300/34343 [5:32:37<37:25,  1.10s/it]

Closest words to the center word he: ['he', 'she', 'clytemnestra', 'gehrig', 'aegisthus', 'remarried', 'bogart', 'tenji', 'ribbentrop', 'bacall']

Epoch 1/2:  94%|█████████▍| 32399/34343 [5:33:46<09:52,  3.28it/s]  

Epoch 1 Batch 32400 loss: 2.1982171535491943

Epoch 1/2:  94%|█████████▍| 32400/34343 [5:33:47<12:53,  2.51it/s]

Closest words to the center word secret: ['secret', 'wipo', 'unido', 'wmo', 'wtoo', 'wco', 'fbi', 'upu', 'wftu', 'defendant']

Epoch 1/2:  95%|█████████▍| 32499/34343 [5:34:55<06:45,  4.55it/s]  

Epoch 1 Batch 32500 loss: 2.177432060241699

Epoch 1/2:  95%|█████████▍| 32500/34343 [5:34:56<10:21,  2.97it/s]

Closest words to the center word traditionally: ['traditionally', 'widely', 'popularly', 'historically', 'interchangeably', 'commonly', 'ethnically', 'heretical', 'fundamentalist', 'universally']

Epoch 1/2:  95%|█████████▍| 32599/34343 [5:36:02<06:10,  4.71it/s]  

Epoch 1 Batch 32600 loss: 2.1100528240203857

Epoch 1/2:  95%|█████████▍| 32600/34343 [5:36:03<09:31,  3.05it/s]

Closest words to the center word immediately: ['immediately', 'shortly', 'resign', 'shuja', 'vowed', 'odrade', 'withdraw', 'aegisthus', 'clytemnestra', 'matres']

Epoch 1/2:  95%|█████████▌| 32699/34343 [5:37:27<28:55,  1.06s/it]  

Epoch 1 Batch 32700 loss: 2.1485962867736816

Epoch 1/2:  95%|█████████▌| 32700/34343 [5:37:28<25:11,  1.09it/s]

Closest words to the center word isbn: ['isbn', 'gwh', 'kwh', 'icrm', 'pp', 'ibrd', 'hbk', 'pmid', 'twh', 'jul']

Epoch 1/2:  96%|█████████▌| 32799/34343 [5:38:36<07:21,  3.50it/s]  

Epoch 1 Batch 32800 loss: 2.1603503227233887

Epoch 1/2:  96%|█████████▌| 32800/34343 [5:38:36<09:55,  2.59it/s]

Closest words to the center word cost: ['gwh', 'cost', 'expenditures', 'kwh', 'gdp', 'speeds', 'grt', 'bandwidth', 'tons', 'rate']

Epoch 1/2:  96%|█████████▌| 32899/34343 [5:39:45<05:07,  4.69it/s]  

Epoch 1 Batch 32900 loss: 2.1025540828704834

Epoch 1/2:  96%|█████████▌| 32900/34343 [5:39:45<09:00,  2.67it/s]

Closest words to the center word john: ['john', 'earl', 'cricketer', 'leiserson', 'george', 'cormen', 'yoannis', 'wadsworth', 'francis', 'viscount']

Epoch 1/2:  96%|█████████▌| 32999/34343 [5:40:54<04:47,  4.68it/s]  

Epoch 1 Batch 33000 loss: 2.1741445064544678

Epoch 1/2:  96%|█████████▌| 33000/34343 [5:41:10<1:55:06,  5.14s/it]

Closest words to the center word road: ['road', 'nfc', 'afc', 'commuter', 'oakland', 'piccadilly', 'ferry', 'playoffs', 'divisional', 'erie']

Epoch 1/2:  96%|█████████▋| 33099/34343 [5:42:18<16:19,  1.27it/s]  

Epoch 1 Batch 33100 loss: 2.1251473426818848

Epoch 1/2:  96%|█████████▋| 33100/34343 [5:42:18<15:43,  1.32it/s]

Closest words to the center word that: ['that', 'falsifiable', 'priori', 'empirically', 'testable', 'countably', 'divinely', 'verifiable', 'conclusive', 'hadn']

Epoch 1/2:  97%|█████████▋| 33199/34343 [5:43:26<04:39,  4.09it/s]  

Epoch 1 Batch 33200 loss: 2.127079486846924

Epoch 1/2:  97%|█████████▋| 33200/34343 [5:43:26<07:11,  2.65it/s]

Closest words to the center word lady: ['lady', 'boleyn', 'duchess', 'wife', 'granddaughter', 'niece', 'princess', 'daughter', 'heiress', 'aunt']

Epoch 1/2:  97%|█████████▋| 33299/34343 [5:44:35<03:44,  4.65it/s]  

Epoch 1 Batch 33300 loss: 2.1511149406433105

Epoch 1/2:  97%|█████████▋| 33300/34343 [5:44:36<06:12,  2.80it/s]

Closest words to the center word connected: ['connected', 'isomorphic', 'homeomorphic', 'automorphism', 'perpendicular', 'homomorphism', 'connecting', 'bijective', 'bounded', 'normed']

Epoch 1/2:  97%|█████████▋| 33399/34343 [5:45:59<1:17:31,  4.93s/it]

Epoch 1 Batch 33400 loss: 2.126316547393799

Epoch 1/2:  97%|█████████▋| 33400/34343 [5:46:00<57:26,  3.66s/it]  

Closest words to the center word spanish: ['spanish', 'portuguese', 'french', 'polynesia', 'blica', 'italian', 'argentine', 'austro', 'dutch', 'basque']

Epoch 1/2:  98%|█████████▊| 33499/34343 [5:47:07<06:49,  2.06it/s]  

Epoch 1 Batch 33500 loss: 2.1518990993499756

Epoch 1/2:  98%|█████████▊| 33500/34343 [5:47:08<07:43,  1.82it/s]

Closest words to the center word split: ['split', 'divided', 'pluriform', 'inducted', 'factions', 'merged', 'subdivided', 'nfc', 'afc', 'coalitions']

Epoch 1/2:  98%|█████████▊| 33599/34343 [5:48:18<04:14,  2.93it/s]  

Epoch 1 Batch 33600 loss: 2.1346943378448486

Epoch 1/2:  98%|█████████▊| 33600/34343 [5:48:19<05:38,  2.19it/s]

Closest words to the center word basal: ['phosphorylation', 'basal', 'citric', 'cvd', 'secreted', 'hormones', 'pyruvate', 'glycogen', 'anterior', 'clotting']

Epoch 1/2:  98%|█████████▊| 33699/34343 [5:49:28<02:18,  4.67it/s]

Epoch 1 Batch 33700 loss: 2.154817819595337

Epoch 1/2:  98%|█████████▊| 33700/34343 [5:49:29<04:06,  2.60it/s]

Closest words to the center word land: ['arable', 'land', 'irrigated', 'pastures', 'hydropower', 'km', 'sq', 'runways', 'crops', 'navigable']

Epoch 1/2:  98%|█████████▊| 33799/34343 [5:50:52<22:53,  2.52s/it]

Epoch 1 Batch 33800 loss: 2.142815589904785

Epoch 1/2:  98%|█████████▊| 33800/34343 [5:50:53<17:53,  1.98s/it]

Closest words to the center word african: ['african', 'saharan', 'chungcheong', 'asian', 'ifrcs', 'oau', 'wtoo', 'wto', 'oecs', 'afdb']

Epoch 1/2:  99%|█████████▊| 33899/34343 [5:52:00<02:33,  2.90it/s]

Epoch 1 Batch 33900 loss: 2.1455960273742676

Epoch 1/2:  99%|█████████▊| 33900/34343 [5:52:01<03:13,  2.28it/s]

Closest words to the center word chess: ['chess', 'championships', 'finalist', 'badminton', 'tennis', 'handball', 'championship', 'uefa', 'hockey', 'prix']

Epoch 1/2:  99%|█████████▉| 33999/34343 [5:53:08<01:13,  4.68it/s]

Epoch 1 Batch 34000 loss: 2.185729503631592

Epoch 1/2:  99%|█████████▉| 34000/34343 [5:53:08<02:02,  2.80it/s]

Closest words to the center word known: ['known', 'referred', 'regarded', 'agave', 'described', 'refered', 'classified', 'remembered', 'argumentum', 'understood']

Epoch 1/2:  99%|█████████▉| 34099/34343 [5:54:18<00:58,  4.19it/s]

Epoch 1 Batch 34100 loss: 2.1650309562683105

Epoch 1/2:  99%|█████████▉| 34100/34343 [5:54:18<01:28,  2.76it/s]

Closest words to the center word html: ['iom', 'ftp', 'pngimage', 'html', 'opcw', 'nonsignatory', 'http', 'webelements', 'icrm', 'www']

Epoch 1/2: 100%|█████████▉| 34199/34343 [5:55:40<03:06,  1.30s/it]

Epoch 1 Batch 34200 loss: 2.1328563690185547

Epoch 1/2: 100%|█████████▉| 34200/34343 [5:55:41<02:38,  1.11s/it]

Closest words to the center word thus: ['falsifiable', 'causal', 'thus', 'bijective', 'pluriform', 'priori', 'rationality', 'reactivity', 'integrable', 'axiom']

Epoch 1/2: 100%|█████████▉| 34299/34343 [5:56:48<00:10,  4.28it/s]

Epoch 1 Batch 34300 loss: 2.1766209602355957

Epoch 1/2: 100%|█████████▉| 34300/34343 [5:56:49<00:15,  2.73it/s]

Closest words to the center word animals: ['animals', 'organisms', 'multicellular', 'species', 'mammals', 'fungi', 'cassava', 'invertebrates', 'beets', 'shellfish']

Epoch 1/2: 100%|██████████| 34343/34343 [5:57:16<00:00,  1.60it/s]

Epoch 1 loss: 2.234515273133304

Epoch 2/2:   0%|          | 99/34343 [01:23<9:13:17,  1.03it/s] 

Epoch 2 Batch 100 loss: 2.1427221298217773

Epoch 2/2:   0%|          | 100/34343 [01:24<8:27:31,  1.12it/s]

Closest words to the center word people: ['people', 'refugees', 'albanians', 'births', 'persons', 'americans', 'politicians', 'nationality', 'immigrants', 'lgbt']

Epoch 2/2:   1%|          | 199/34343 [02:31<2:27:51,  3.85it/s] 

Epoch 2 Batch 200 loss: 2.13472843170166

Epoch 2/2:   1%|          | 200/34343 [02:32<3:40:42,  2.58it/s]

Closest words to the center word been: ['been', 'become', 'fallen', 'arisen', 'be', 'existed', 'harshly', 'recently', 'agave', 'householder']

Epoch 2/2:   1%|          | 299/34343 [03:39<2:02:12,  4.64it/s] 

Epoch 2 Batch 300 loss: 2.078275442123413

Epoch 2/2:   1%|          | 300/34343 [03:40<3:24:26,  2.78it/s]

Closest words to the center word governor: ['governor', 'governors', 'minister', 'lieutenant', 'bailiff', 'secretary', 'deputy', 'appointed', 'premiers', 'earl']

Epoch 2/2:   1%|          | 399/34343 [04:47<1:59:58,  4.72it/s] 

Epoch 2 Batch 400 loss: 2.158493757247925

Epoch 2/2:   1%|          | 400/34343 [05:04<48:47:35,  5.18s/it]

Closest words to the center word marry: ['marry', 'remarried', 'married', 'grandmother', 'clytemnestra', 'seduce', 'tenji', 'aegisthus', 'daughter', 'confess']

Epoch 2/2:   1%|▏         | 499/34343 [06:13<5:49:12,  1.62it/s] 

Epoch 2 Batch 500 loss: 2.12949538230896

Epoch 2/2:   1%|▏         | 500/34343 [06:13<6:04:33,  1.55it/s]

Closest words to the center word with: ['with', 'ifrcs', 'between', 'close', 'ifad', 'insubstantial', 'tamarin', 'icrm', 'ifc', 'positively']

Epoch 2/2:   2%|▏         | 599/34343 [07:21<2:04:04,  4.53it/s] 

Epoch 2 Batch 600 loss: 2.130528450012207

Epoch 2/2:   2%|▏         | 600/34343 [07:22<3:16:58,  2.85it/s]

Closest words to the center word bj: ['bj', 'finalist', 'wagoner', 'ulvaeus', 'discography', 'rgen', 'rk', 'composer', 'householder', 'elke']

Epoch 2/2:   2%|▏         | 699/34343 [08:29<1:58:28,  4.73it/s] 

Epoch 2 Batch 700 loss: 2.119719982147217

Epoch 2/2:   2%|▏         | 700/34343 [08:30<3:14:13,  2.89it/s]

Closest words to the center word lowlands: ['lowlands', 'subtropical', 'humid', 'semiarid', 'mountainous', 'uplands', 'cordillera', 'hilly', 'arid', 'broadleaf']

Epoch 2/2:   2%|▏         | 799/34343 [09:58<33:12:40,  3.56s/it]

Epoch 2 Batch 800 loss: 2.059316396713257

Epoch 2/2:   2%|▏         | 800/34343 [09:58<25:10:51,  2.70s/it]

Closest words to the center word simple: ['mathfrak', 'deterministic', 'bilinear', 'countably', 'bijective', 'adjoint', 'boolean', 'functor', 'newnode', 'convolution']

Epoch 2/2:   3%|▎         | 899/34343 [11:07<2:34:24,  3.61it/s] 

Epoch 2 Batch 900 loss: 2.1261239051818848

Epoch 2/2:   3%|▎         | 900/34343 [11:07<3:34:47,  2.60it/s]

Closest words to the center word robert: ['leiserson', 'rivest', 'cormen', 'elke', 'physiologist', 'laureate', 'robert', 'geneticist', 'wadsworth', 'archibald']

Epoch 2/2:   3%|▎         | 999/34343 [12:15<2:07:05,  4.37it/s] 

Epoch 2 Batch 1000 loss: 2.14149808883667

Epoch 2/2:   3%|▎         | 1000/34343 [12:16<3:22:52,  2.74it/s]

Closest words to the center word ascribed: ['ascribed', 'attributed', 'epistle', 'ordained', 'referred', 'anh', 'gnostic', 'apocryphal', 'authorship', 'epistles']

Epoch 2/2:   3%|▎         | 1099/34343 [13:23<1:57:10,  4.73it/s] 

Epoch 2 Batch 1100 loss: 2.1283271312713623

Epoch 2/2:   3%|▎         | 1100/34343 [13:24<3:13:19,  2.87it/s]

Closest words to the center word alabama: ['township', 'abet', 'devry', 'chungcheong', 'alabama', 'ssn', 'busan', 'bangor', 'missouri', 'county']

Epoch 2/2:   3%|▎         | 1199/34343 [14:49<7:18:34,  1.26it/s] 

Epoch 2 Batch 1200 loss: 2.108813762664795

Epoch 2/2:   3%|▎         | 1200/34343 [14:50<6:59:21,  1.32it/s]

Closest words to the center word divers: ['divers', 'diving', 'competitions', 'curling', 'restaurants', 'grt', 'bowls', 'golf', 'handball', 'skiing']

Epoch 2/2:   4%|▍         | 1299/34343 [15:58<2:20:52,  3.91it/s] 

Epoch 2 Batch 1300 loss: 2.11685848236084

Epoch 2/2:   4%|▍         | 1300/34343 [15:59<3:27:35,  2.65it/s]

Closest words to the center word denominational: ['denominational', 'denominations', 'fundamentalist', 'evangelicalism', 'haredi', 'churches', 'reconstructionist', 'masorti', 'hutterites', 'unido']

Epoch 2/2:   4%|▍         | 1399/34343 [17:07<1:58:24,  4.64it/s] 

Epoch 2 Batch 1400 loss: 2.136810779571533

Epoch 2/2:   4%|▍         | 1400/34343 [17:08<4:46:10,  1.92it/s]

Closest words to the center word due: ['due', 'owing', 'reactivity', 'induce', 'desertification', 'inversely', 'cloudy', 'hyperinsulinism', 'mortality', 'contributes']

Epoch 2/2:   4%|▍         | 1499/34343 [18:31<46:13:55,  5.07s/it]

Epoch 2 Batch 1500 loss: 2.115029811859131

Epoch 2/2:   4%|▍         | 1500/34343 [18:32<34:14:56,  3.75s/it]

Closest words to the center word establish: ['unido', 'establish', 'wco', 'unctad', 'upu', 'ifrcs', 'wftu', 'enact', 'unmibh', 'wcl']

Epoch 2/2:   5%|▍         | 1599/34343 [19:39<5:19:40,  1.71it/s] 

Epoch 2 Batch 1600 loss: 2.08738374710083

Epoch 2/2:   5%|▍         | 1600/34343 [19:40<5:37:05,  1.62it/s]

Closest words to the center word alliance: ['alliance', 'ifrcs', 'unido', 'unctad', 'wco', 'opcw', 'wftu', 'ifc', 'iom', 'frud']

Epoch 2/2:   5%|▍         | 1699/34343 [20:48<2:19:00,  3.91it/s] 

Epoch 2 Batch 1700 loss: 2.10058856010437

Epoch 2/2:   5%|▍         | 1700/34343 [20:48<3:27:46,  2.62it/s]

Closest words to the center word second: ['second', 'third', 'fourth', 'fifth', 'sixth', 'seventh', 'ninth', 'rd', 'tenth', 'nd']

Epoch 2/2:   5%|▌         | 1799/34343 [21:59<1:54:44,  4.73it/s] 

Epoch 2 Batch 1800 loss: 2.1364266872406006

Epoch 2/2:   5%|▌         | 1800/34343 [21:59<3:02:41,  2.97it/s]

Closest words to the center word science: ['science', 'fiction', 'cognitivism', 'sciences', 'informatics', 'psychology', 'cognitive', 'engineering', 'epistemology', 'sociology']

Epoch 2/2:   6%|▌         | 1899/34343 [23:23<29:52:31,  3.31s/it]

Epoch 2 Batch 1900 loss: 2.1308605670928955

Epoch 2/2:   6%|▌         | 1900/34343 [23:24<22:43:45,  2.52s/it]

Closest words to the center word pragmatic: ['pragmatic', 'normative', 'egoism', 'falsifiable', 'cognitivism', 'libertarianism', 'agnosticism', 'rationality', 'consequentialism', 'relativism']

Epoch 2/2:   6%|▌         | 1999/34343 [24:35<4:27:08,  2.02it/s] 

Epoch 2 Batch 2000 loss: 2.123274564743042

Epoch 2/2:   6%|▌         | 2000/34343 [24:35<4:57:27,  1.81it/s]

Closest words to the center word also: ['also', 'wmo', 'disambiguation', 'wtoo', 'ifad', 'wftu', 'opcw', 'unido', 'ifrcs', 'wcl']

Epoch 2/2:   6%|▌         | 2099/34343 [25:43<2:01:03,  4.44it/s] 

Epoch 2 Batch 2100 loss: 2.0830719470977783

Epoch 2/2:   6%|▌         | 2100/34343 [25:44<3:11:09,  2.81it/s]

Closest words to the center word ingredients: ['sorghum', 'ingredients', 'spices', 'cassava', 'solvents', 'soybeans', 'beets', 'soy', 'alkaloids', 'dishes']

Epoch 2/2:   6%|▋         | 2199/34343 [26:52<1:53:15,  4.73it/s] 

Epoch 2 Batch 2200 loss: 2.109307289123535

Epoch 2/2:   6%|▋         | 2200/34343 [26:52<3:03:44,  2.92it/s]

Closest words to the center word york: ['york', 'giroux', 'schuster', 'knopf', 'ny', 'newark', 'scribner', 'berkley', 'schenectady', 'devry']

Epoch 2/2:   7%|▋         | 2299/34343 [28:16<22:40:05,  2.55s/it]

Epoch 2 Batch 2300 loss: 2.0878589153289795

Epoch 2/2:   7%|▋         | 2300/34343 [28:17<17:35:18,  1.98s/it]

Closest words to the center word france: ['france', 'belgium', 'portugal', 'spain', 'partement', 'luxembourg', 'netherlands', 'alsace', 'castile', 'nazaire']

Epoch 2/2:   7%|▋         | 2399/34343 [29:23<3:03:09,  2.91it/s] 

Epoch 2 Batch 2400 loss: 2.139266014099121

Epoch 2/2:   7%|▋         | 2400/34343 [29:24<3:58:10,  2.24it/s]

Closest words to the center word fire: ['fire', 'tank', 'projectiles', 'ammunition', 'bolt', 'dwt', 'combustion', 'projectile', 'torpedo', 'propelled']

Epoch 2/2:   7%|▋         | 2499/34343 [30:33<1:56:40,  4.55it/s] 

Epoch 2 Batch 2500 loss: 2.1110358238220215

Epoch 2/2:   7%|▋         | 2500/34343 [30:33<3:04:56,  2.87it/s]

Closest words to the center word a: ['a', 'pluriform', 'every', 'automorphism', 'any', 'another', 'monoid', 'agave', 'otimes', 'newnode']

Epoch 2/2:   8%|▊         | 2599/34343 [31:42<1:50:59,  4.77it/s] 

Epoch 2 Batch 2600 loss: 2.0861809253692627

Epoch 2/2:   8%|▊         | 2600/34343 [31:43<3:06:06,  2.84it/s]

Closest words to the center word or: ['or', 'ifrcs', 'grt', 'newnode', 'rajonas', 'chordata', 'insubstantial', 'conjugated', 'prev', 'ifad']

Epoch 2/2:   8%|▊         | 2699/34343 [33:04<11:38:27,  1.32s/it]

Epoch 2 Batch 2700 loss: 2.1319422721862793

Epoch 2/2:   8%|▊         | 2700/34343 [33:05<9:59:22,  1.14s/it] 

Closest words to the center word edited: ['edited', 'elke', 'hlich', 'rivest', 'leiserson', 'anh', 'routledge', 'pratchett', 'reprinted', 'cormen']

Epoch 2/2:   8%|▊         | 2799/34343 [34:12<2:25:22,  3.62it/s] 

Epoch 2 Batch 2800 loss: 2.1186113357543945

Epoch 2/2:   8%|▊         | 2800/34343 [34:13<3:27:22,  2.54it/s]

Closest words to the center word wealth: ['wealth', 'earnings', 'gdp', 'prosperity', 'glasnost', 'overgrazing', 'vitality', 'subsistence', 'income', 'urbanization']

Epoch 2/2:   8%|▊         | 2899/34343 [35:20<1:50:49,  4.73it/s] 

Epoch 2 Batch 2900 loss: 2.0913403034210205

Epoch 2/2:   8%|▊         | 2900/34343 [35:20<3:00:28,  2.90it/s]

Closest words to the center word has: ['has', 'had', 'have', 'ifrcs', 'hasn', 'pluriform', 'having', 'ifad', 'enjoys', 'possesses']

Epoch 2/2:   9%|▊         | 2999/34343 [36:29<1:49:42,  4.76it/s] 

Epoch 2 Batch 3000 loss: 2.0676462650299072

Epoch 2/2:   9%|▊         | 3000/34343 [36:30<2:59:12,  2.91it/s]

Closest words to the center word verb: ['verb', 'infinitive', 'verbs', 'perfective', 'participle', 'approximant', 'consonant', 'fricative', 'nouns', 'nominative']

Epoch 2/2:   9%|▉         | 3099/34343 [37:55<6:49:00,  1.27it/s] 

Epoch 2 Batch 3100 loss: 2.0898971557617188

Epoch 2/2:   9%|▉         | 3100/34343 [37:55<6:28:44,  1.34it/s]

Closest words to the center word into: ['into', 'agave', 'subdivided', 'onto', 'divided', 'diploid', 'across', 'northward', 'together', 'through']

Epoch 2/2:   9%|▉         | 3199/34343 [39:03<1:56:54,  4.44it/s] 

Epoch 2 Batch 3200 loss: 2.125943899154663

Epoch 2/2:   9%|▉         | 3200/34343 [39:04<3:09:01,  2.75it/s]

Closest words to the center word french: ['french', 'ois', 'fran', 'dutch', 'spanish', 'auguste', 'comte', 'belgian', 'portuguese', 'outre']

Epoch 2/2:  10%|▉         | 3299/34343 [40:10<1:50:56,  4.66it/s] 

Epoch 2 Batch 3300 loss: 2.0982842445373535

Epoch 2/2:  10%|▉         | 3300/34343 [40:11<3:12:07,  2.69it/s]

Closest words to the center word expected: ['expected', 'able', 'lifes', 'expectancy', 'estimated', 'grt', 'willing', 'calculate', 'homeomorphic', 'adjusted']

Epoch 2/2:  10%|▉         | 3399/34343 [41:36<44:11:25,  5.14s/it]

Epoch 2 Batch 3400 loss: 2.088789463043213

Epoch 2/2:  10%|▉         | 3400/34343 [41:36<32:38:19,  3.80s/it]

Closest words to the center word interventionist: ['interventionist', 'laissez', 'collectivism', 'pluriform', 'conservatism', 'totalitarian', 'faire', 'egoism', 'egalitarianism', 'authoritarianism']

Epoch 2/2:  10%|█         | 3499/34343 [42:44<2:39:00,  3.23it/s] 

Epoch 2 Batch 3500 loss: 2.083583354949951

Epoch 2/2:  10%|█         | 3500/34343 [42:45<3:42:12,  2.31it/s]

Closest words to the center word of: ['of', 'ifrcs', 'ifad', 'rajonas', 'icrm', 'ifc', 'wtoo', 'akan', 'wtro', 'wmo']

Epoch 2/2:  10%|█         | 3599/34343 [43:53<1:52:29,  4.55it/s] 

Epoch 2 Batch 3600 loss: 2.1149137020111084

Epoch 2/2:  10%|█         | 3600/34343 [43:54<3:05:02,  2.77it/s]

Closest words to the center word sugarcubes: ['allman', 'wagoner', 'sugarcubes', 'rockabilly', 'django', 'vocals', 'albums', 'beatles', 'singles', 'everly']

Epoch 2/2:  11%|█         | 3699/34343 [45:02<1:51:27,  4.58it/s] 

Epoch 2 Batch 3700 loss: 2.111382007598877

Epoch 2/2:  11%|█         | 3700/34343 [45:03<3:07:22,  2.73it/s]

Closest words to the center word mg: ['gwh', 'grt', 'twh', 'mg', 'cooh', 'kwh', 'pmid', 'kcal', 'dwt', 'pngimage']

Epoch 2/2:  11%|█         | 3799/34343 [46:27<8:47:37,  1.04s/it] 

Epoch 2 Batch 3800 loss: 2.096947431564331

Epoch 2/2:  11%|█         | 3800/34343 [46:27<7:52:46,  1.08it/s]

Closest words to the center word social: ['social', 'pluriform', 'socio', 'egalitarianism', 'marxian', 'conservatism', 'welfare', 'libertarian', 'libertarianism', 'collectivism']

Epoch 2/2:  11%|█▏        | 3899/34343 [47:35<2:21:06,  3.60it/s] 

Epoch 2 Batch 3900 loss: 2.1126112937927246

Epoch 2/2:  11%|█▏        | 3900/34343 [47:36<3:24:41,  2.48it/s]

Closest words to the center word massive: ['massive', 'pluriform', 'rapid', 'overgrazing', 'huge', 'shortage', 'sudden', 'catastrophic', 'recession', 'drought']

Epoch 2/2:  12%|█▏        | 3999/34343 [48:43<1:48:51,  4.65it/s] 

Epoch 2 Batch 4000 loss: 2.1012675762176514

Epoch 2/2:  12%|█▏        | 4000/34343 [48:44<3:03:07,  2.76it/s]

Closest words to the center word generally: ['generally', 'widely', 'commonly', 'universally', 'often', 'usually', 'hotly', 'broadly', 'inherently', 'humid']

Epoch 2/2:  12%|█▏        | 4099/34343 [49:53<1:45:54,  4.76it/s] 

Epoch 2 Batch 4100 loss: 2.126585006713867

Epoch 2/2:  12%|█▏        | 4100/34343 [50:09<42:11:39,  5.02s/it]

Closest words to the center word signed: ['signed', 'ratified', 'treaty', 'wetlands', 'runways', 'ratification', 'agreement', 'agreements', 'signing', 'accords']

Epoch 2/2:  12%|█▏        | 4199/34343 [51:18<6:24:49,  1.31it/s] 

Epoch 2 Batch 4200 loss: 2.1400294303894043

Epoch 2/2:  12%|█▏        | 4200/34343 [51:19<6:10:36,  1.36it/s]

Closest words to the center word by: ['by', 'ifrcs', 'icrm', 'ifc', 'agave', 'rajonas', 'unido', 'leiserson', 'ifad', 'iom']

Epoch 2/2:  13%|█▎        | 4299/34343 [52:27<2:07:40,  3.92it/s] 

Epoch 2 Batch 4300 loss: 2.1312642097473145

Epoch 2/2:  13%|█▎        | 4300/34343 [52:28<3:12:47,  2.60it/s]

Closest words to the center word most: ['most', 'lifes', 'poorest', 'earliest', 'highly', 'extremely', 'more', 'largest', 'greatest', 'less']

Epoch 2/2:  13%|█▎        | 4399/34343 [53:37<1:47:25,  4.65it/s] 

Epoch 2 Batch 4400 loss: 2.139698028564453

Epoch 2/2:  13%|█▎        | 4400/34343 [53:38<2:55:23,  2.85it/s]

Closest words to the center word is: ['is', 'satisfies', 'bijective', 'refers', 'consists', 'was', 'behaves', 'contains', 'corresponds', 'exists']

Epoch 2/2:  13%|█▎        | 4499/34343 [55:03<40:19:51,  4.87s/it]

Epoch 2 Batch 4500 loss: 2.091897964477539

Epoch 2/2:  13%|█▎        | 4500/34343 [55:03<29:56:05,  3.61s/it]

Closest words to the center word chosen: ['chosen', 'elected', 'appointed', 'ordained', 'vested', 'anointed', 'sworn', 'overridden', 'nominated', 'proportional']

Epoch 2/2:  13%|█▎        | 4599/34343 [56:12<5:04:27,  1.63it/s] 

Epoch 2 Batch 4600 loss: 2.1326465606689453

Epoch 2/2:  13%|█▎        | 4600/34343 [56:12<5:06:40,  1.62it/s]

Closest words to the center word praise: ['praise', 'everlasting', 'preach', 'forgiveness', 'almighty', 'nomination', 'myself', 'thee', 'amnon', 'sins']

Epoch 2/2:  14%|█▎        | 4699/34343 [57:22<1:56:12,  4.25it/s] 

Epoch 2 Batch 4700 loss: 2.07889461517334

Epoch 2/2:  14%|█▎        | 4700/34343 [57:23<2:54:00,  2.84it/s]

Closest words to the center word the: ['the', 'ifrcs', 'pluriform', 'icrm', 'nonsignatory', 'intelsat', 'nfc', 'ifc', 'ladoga', 'vernal']

Epoch 2/2:  14%|█▍        | 4799/34343 [58:31<1:44:50,  4.70it/s] 

Epoch 2 Batch 4800 loss: 2.122645616531372

Epoch 2/2:  14%|█▍        | 4800/34343 [58:32<2:45:25,  2.98it/s]

Closest words to the center word earliest: ['earliest', 'oldest', 'masoretic', 'finest', 'cretaceous', 'cambrian', 'hellenistic', 'antiquity', 'prehistoric', 'carboniferous']

Epoch 2/2:  14%|█▍        | 4899/34343 [59:55<28:35:38,  3.50s/it]

Epoch 2 Batch 4900 loss: 2.0987653732299805

Epoch 2/2:  14%|█▍        | 4900/34343 [59:55<21:32:21,  2.63s/it]

Closest words to the center word after: ['after', 'before', 'lasted', 'afterwards', 'shortly', 'thereafter', 'months', 'during', 'weeks', 'afterward']

Epoch 2/2:  15%|█▍        | 4999/34343 [1:01:05<3:23:53,  2.40it/s] 

Epoch 2 Batch 5000 loss: 2.0908498764038086

Epoch 2/2:  15%|█▍        | 5000/34343 [1:01:05<3:55:48,  2.07it/s]

Closest words to the center word australia: ['australia', 'tasmania', 'chungcheong', 'zealand', 'busan', 'canada', 'premiers', 'kitts', 'uruguay', 'queensland']

Epoch 2/2:  15%|█▍        | 5099/34343 [1:02:13<1:47:56,  4.52it/s] 

Epoch 2 Batch 5100 loss: 2.123389720916748

Epoch 2/2:  15%|█▍        | 5100/34343 [1:02:14<2:49:46,  2.87it/s]

Closest words to the center word early: ['early', 'late', 'mid', 'kwh', 'twh', 'gwh', 'availabilitymales', 'cretaceous', 'nineteenth', 'eighteenth']

Epoch 2/2:  15%|█▌        | 5199/34343 [1:03:22<1:44:56,  4.63it/s] 

Epoch 2 Batch 5200 loss: 2.087289333343506

Epoch 2/2:  15%|█▌        | 5200/34343 [1:03:23<2:47:42,  2.90it/s]

Closest words to the center word le: ['le', 'cegep', 'linguistique', 'estudios', 'serpento', 'quijote', 'dicis', 'histoire', 'esas', 'sur']

Epoch 2/2:  15%|█▌        | 5299/34343 [1:04:47<15:04:07,  1.87s/it]

Epoch 2 Batch 5300 loss: 2.09371018409729

Epoch 2/2:  15%|█▌        | 5300/34343 [1:04:48<12:06:57,  1.50s/it]

Closest words to the center word residing: ['residing', 'grt', 'ukrainians', 'populous', 'villages', 'wealthiest', 'sq', 'emigrated', 'bosniaks', 'poorest']

Epoch 2/2:  16%|█▌        | 5399/34343 [1:05:55<2:23:08,  3.37it/s] 

Epoch 2 Batch 5400 loss: 2.153334856033325

Epoch 2/2:  16%|█▌        | 5400/34343 [1:05:55<3:10:54,  2.53it/s]

Closest words to the center word thus: ['falsifiable', 'thus', 'integrable', 'countably', 'bijective', 'infty', 'adjoint', 'enthalpy', 'causal', 'cardinality']

Epoch 2/2:  16%|█▌        | 5499/34343 [1:07:05<1:44:24,  4.60it/s] 

Epoch 2 Batch 5500 loss: 2.08534574508667

Epoch 2/2:  16%|█▌        | 5500/34343 [1:07:06<2:40:43,  2.99it/s]

Closest words to the center word invention: ['invention', 'advent', 'inventor', 'microcomputer', 'ipf', 'synthesizers', 'combustion', 'luminiferous', 'synthesizer', 'introduction']

Epoch 2/2:  16%|█▋        | 5599/34343 [1:08:13<1:44:56,  4.57it/s] 

Epoch 2 Batch 5600 loss: 2.1243724822998047

Epoch 2/2:  16%|█▋        | 5600/34343 [1:08:14<2:48:03,  2.85it/s]

Closest words to the center word charles: ['charles', 'leiserson', 'cormen', 'earl', 'duchess', 'louis', 'ois', 'emmanuel', 'valois', 'henry']

Epoch 2/2:  17%|█▋        | 5699/34343 [1:09:39<8:10:31,  1.03s/it] 

Epoch 2 Batch 5700 loss: 2.0910654067993164

Epoch 2/2:  17%|█▋        | 5700/34343 [1:09:39<7:33:53,  1.05it/s]

Closest words to the center word terre: ['icrm', 'universidade', 'rajonas', 'ribeira', 'ouest', 'partement', 'noma', 'terre', 'classis', 'cegep']

Epoch 2/2:  17%|█▋        | 5799/34343 [1:10:48<1:48:30,  4.38it/s] 

Epoch 2 Batch 5800 loss: 2.112058162689209

Epoch 2/2:  17%|█▋        | 5800/34343 [1:10:48<2:52:03,  2.76it/s]

Closest words to the center word church: ['church', 'episcopal', 'churches', 'orthodox', 'communion', 'catholic', 'methodist', 'lutheran', 'anglican', 'anglicanism']

Epoch 2/2:  17%|█▋        | 5899/34343 [1:11:57<1:40:13,  4.73it/s] 

Epoch 2 Batch 5900 loss: 2.1148324012756348

Epoch 2/2:  17%|█▋        | 5900/34343 [1:11:58<2:45:46,  2.86it/s]

Closest words to the center word preferably: ['beets', 'cooked', 'stewed', 'waterborne', 'boiled', 'soy', 'grilled', 'dried', 'fried', 'cassava']

Epoch 2/2:  17%|█▋        | 5999/34343 [1:13:05<1:40:52,  4.68it/s] 

Epoch 2 Batch 6000 loss: 2.1314735412597656

Epoch 2/2:  17%|█▋        | 6000/34343 [1:13:20<38:13:06,  4.85s/it]

Closest words to the center word led: ['led', 'unido', 'contributed', 'ifrcs', 'resulted', 'helped', 'caused', 'unctad', 'attempted', 'prompted']

Epoch 2/2:  18%|█▊        | 6099/34343 [1:14:28<2:40:24,  2.93it/s] 

Epoch 2 Batch 6100 loss: 2.0925450325012207

Epoch 2/2:  18%|█▊        | 6100/34343 [1:14:29<3:28:39,  2.26it/s]

Closest words to the center word has: ['has', 'have', 'had', 'ifrcs', 'pluriform', 'hasn', 'having', 'hadn', 'enjoys', 'possesses']

Epoch 2/2:  18%|█▊        | 6199/34343 [1:15:39<1:44:02,  4.51it/s] 

Epoch 2 Batch 6200 loss: 2.115598440170288

Epoch 2/2:  18%|█▊        | 6200/34343 [1:15:40<2:46:35,  2.82it/s]

Closest words to the center word gravitational: ['gravitational', 'dipole', 'electromagnetic', 'mathbf', 'acceleration', 'velocity', 'rotational', 'inertial', 'angular', 'spacetime']

Epoch 2/2:  18%|█▊        | 6299/34343 [1:16:48<1:39:59,  4.67it/s] 

Epoch 2 Batch 6300 loss: 2.150808572769165

Epoch 2/2:  18%|█▊        | 6300/34343 [1:16:49<2:46:19,  2.81it/s]

Closest words to the center word rating: ['rating', 'gwh', 'grt', 'pluriform', 'kwh', 'twh', 'expenditures', 'mbit', 'dwt', 'mjs']

Epoch 2/2:  19%|█▊        | 6399/34343 [1:18:13<10:37:16,  1.37s/it]

Epoch 2 Batch 6400 loss: 2.1091442108154297

Epoch 2/2:  19%|█▊        | 6400/34343 [1:18:14<9:02:21,  1.16s/it] 

Closest words to the center word broadcast: ['broadcast', 'intelsat', 'shortwave', 'isps', 'intersputnik', 'stations', 'inmarsat', 'radio', 'aired', 'eutelsat']

Epoch 2/2:  19%|█▉        | 6499/34343 [1:19:21<2:18:38,  3.35it/s] 

Epoch 2 Batch 6500 loss: 2.0898923873901367

Epoch 2/2:  19%|█▉        | 6500/34343 [1:19:22<3:10:13,  2.44it/s]

Closest words to the center word forces: ['forces', 'troops', 'army', 'gendarmerie', 'marshal', 'armies', 'peacekeeping', 'armed', 'squadrons', 'force']

Epoch 2/2:  19%|█▉        | 6599/34343 [1:20:29<1:40:31,  4.60it/s] 

Epoch 2 Batch 6600 loss: 2.1494300365448

Epoch 2/2:  19%|█▉        | 6600/34343 [1:20:29<2:41:06,  2.87it/s]

Closest words to the center word done: ['done', 'insubstantial', 'import', 'duplicate', 'irrelevent', 'subnet', 'nonsignatory', 'redirect', 'jargon', 'iom']

Epoch 2/2:  20%|█▉        | 6699/34343 [1:21:40<1:38:13,  4.69it/s] 

Epoch 2 Batch 6700 loss: 2.068402051925659

Epoch 2/2:  20%|█▉        | 6700/34343 [1:21:40<2:43:11,  2.82it/s]

Closest words to the center word poverty: ['poverty', 'expectancy', 'est', 'income', 'unemployment', 'median', 'expenditures', 'mortality', 'population', 'wages']

Epoch 2/2:  20%|█▉        | 6799/34343 [1:23:04<7:46:08,  1.02s/it] 

Epoch 2 Batch 6800 loss: 2.1448311805725098

Epoch 2/2:  20%|█▉        | 6800/34343 [1:23:05<6:59:13,  1.10it/s]

Closest words to the center word ahmet: ['ebne', 'footballer', 'grt', 'kwh', 'abol', 'gwh', 'rovere', 'maaouya', 'violinist', 'finalist']

Epoch 2/2:  20%|██        | 6899/34343 [1:24:15<2:09:37,  3.53it/s] 

Epoch 2 Batch 6900 loss: 2.1025197505950928

Epoch 2/2:  20%|██        | 6900/34343 [1:24:15<2:58:44,  2.56it/s]

Closest words to the center word british: ['british', 'canadian', 'scottish', 'australian', 'american', 'naturalized', 'autodidacts', 'french', 'dutch', 'portuguese']

Epoch 2/2:  20%|██        | 6999/34343 [1:25:23<1:36:58,  4.70it/s] 

Epoch 2 Batch 7000 loss: 2.095928430557251

Epoch 2/2:  20%|██        | 7000/34343 [1:25:23<2:33:05,  2.98it/s]

Closest words to the center word america: ['america', 'chungcheong', 'mycological', 'busan', 'gyeonggi', 'nfc', 'gyeongsang', 'africa', 'saharan', 'caribbean']

Epoch 2/2:  21%|██        | 7099/34343 [1:26:31<1:36:07,  4.72it/s] 

Epoch 2 Batch 7100 loss: 2.082852602005005

Epoch 2/2:  21%|██        | 7100/34343 [1:26:47<36:38:03,  4.84s/it]

Closest words to the center word various: ['various', 'several', 'numerous', 'different', 'variety', 'differing', 'varying', 'multiple', 'many', 'ifad']

Epoch 2/2:  21%|██        | 7199/34343 [1:27:54<5:37:45,  1.34it/s] 

Epoch 2 Batch 7200 loss: 2.112398862838745

Epoch 2/2:  21%|██        | 7200/34343 [1:27:55<5:23:27,  1.40it/s]

Closest words to the center word liu: ['unido', 'liu', 'unctad', 'wco', 'wftu', 'unmibh', 'opcw', 'ifrcs', 'kapoor', 'shek']

Epoch 2/2:  21%|██▏       | 7299/34343 [1:29:04<1:53:12,  3.98it/s] 

Epoch 2 Batch 7300 loss: 2.092010021209717

Epoch 2/2:  21%|██▏       | 7300/34343 [1:29:04<2:51:32,  2.63it/s]

Closest words to the center word surgeon: ['surgeon', 'chemist', 'unido', 'physicist', 'philanthropist', 'physician', 'physiologist', 'zoologist', 'psychiatrist', 'mineralogist']

Epoch 2/2:  22%|██▏       | 7399/34343 [1:30:13<1:35:07,  4.72it/s] 

Epoch 2 Batch 7400 loss: 2.088667392730713

Epoch 2/2:  22%|██▏       | 7400/34343 [1:30:14<2:41:24,  2.78it/s]

Closest words to the center word one: ['gwh', 'kwh', 'twh', 'grt', 'hbk', 'pngimage', 'sfg', 'cyg', 'pmid', 'jul']

Epoch 2/2:  22%|██▏       | 7499/34343 [1:31:38<36:02:00,  4.83s/it]

Epoch 2 Batch 7500 loss: 2.1404001712799072

Epoch 2/2:  22%|██▏       | 7500/34343 [1:31:39<26:51:55,  3.60s/it]

Closest words to the center word wonderful: ['wonderful', 'stevie', 'austen', 'frankenstein', 'poirot', 'swaim', 'sherlock', 'marple', 'diddley', 'loves']

Epoch 2/2:  22%|██▏       | 7599/34343 [1:32:46<3:28:59,  2.13it/s] 

Epoch 2 Batch 7600 loss: 2.103727340698242

Epoch 2/2:  22%|██▏       | 7600/34343 [1:32:47<3:49:00,  1.95it/s]

Closest words to the center word them: ['them', 'findable', 'matres', 'him', 'prev', 'thy', 'confess', 'preach', 'oneself', 'baptize']

Epoch 2/2:  22%|██▏       | 7699/34343 [1:33:56<1:41:04,  4.39it/s] 

Epoch 2 Batch 7700 loss: 2.111868381500244

Epoch 2/2:  22%|██▏       | 7700/34343 [1:33:57<2:44:57,  2.69it/s]

Closest words to the center word which: ['which', 'cytosol', 'countably', 'ifrcs', 'endothermic', 'insoluble', 'enthalpy', 'symplectic', 'circularly', 'gangetic']

Epoch 2/2:  23%|██▎       | 7799/34343 [1:35:05<1:35:31,  4.63it/s] 

Epoch 2 Batch 7800 loss: 2.120771884918213

Epoch 2/2:  23%|██▎       | 7800/34343 [1:35:06<2:30:37,  2.94it/s]

Closest words to the center word latex: ['latex', 'findable', 'iom', 'cassava', 'soy', 'lossy', 'codecs', 'ifad', 'scsi', 'ipf']

Epoch 2/2:  23%|██▎       | 7899/34343 [1:36:31<19:07:55,  2.60s/it]

Epoch 2 Batch 7900 loss: 2.0719943046569824

Epoch 2/2:  23%|██▎       | 7900/34343 [1:36:31<15:06:32,  2.06s/it]

Closest words to the center word mongoloids: ['mongoloids', 'elevations', 'lifes', 'incomes', 'humid', 'reactivity', 'rainfall', 'humidity', 'differentiable', 'estimators']

Epoch 2/2:  23%|██▎       | 7999/34343 [1:37:39<2:32:21,  2.88it/s] 

Epoch 2 Batch 8000 loss: 2.151247024536133

Epoch 2/2:  23%|██▎       | 8000/34343 [1:37:40<3:17:14,  2.23it/s]

Closest words to the center word even: ['even', 'lifes', 'aesthetically', 'cheaper', 'weren', 'quicker', 'forgiving', 'falsifiable', 'worse', 'computationally']

Epoch 2/2:  24%|██▎       | 8099/34343 [1:38:47<1:33:29,  4.68it/s] 

Epoch 2 Batch 8100 loss: 2.1032638549804688

Epoch 2/2:  24%|██▎       | 8100/34343 [1:38:48<2:33:56,  2.84it/s]

Closest words to the center word standard: ['standard', 'iec', 'cccc', 'bbbb', 'ansi', 'tiberian', 'iso', 'utf', 'phonetic', 'standards']

Epoch 2/2:  24%|██▍       | 8199/34343 [1:39:56<1:31:29,  4.76it/s] 

Epoch 2 Batch 8200 loss: 2.1376395225524902

Epoch 2/2:  24%|██▍       | 8200/34343 [1:39:57<2:33:47,  2.83it/s]

Closest words to the center word il: ['il', 'bwv', 'anh', 'ebne', 'linguistique', 'stosunku', 'cegep', 'landsmannschaft', 'leiserson', 'studi']

Epoch 2/2:  24%|██▍       | 8299/34343 [1:41:20<9:20:42,  1.29s/it] 

Epoch 2 Batch 8300 loss: 2.1120400428771973

Epoch 2/2:  24%|██▍       | 8300/34343 [1:41:21<8:02:19,  1.11s/it]

Closest words to the center word headache: ['headache', 'nausea', 'dizziness', 'diarrhea', 'vectorborne', 'vomiting', 'congenital', 'inhibitors', 'symptoms', 'hypothyroidism']

Epoch 2/2:  24%|██▍       | 8399/34343 [1:42:42<3:56:36,  1.83it/s] 

Epoch 2 Batch 8400 loss: 2.1168150901794434

Epoch 2/2:  24%|██▍       | 8400/34343 [1:42:42<4:34:22,  1.58it/s]

Closest words to the center word aforementioned: ['aforementioned', 'privy', 'ecumenical', 'mormon', 'epistle', 'immaculate', 'curia', 'nevi', 'cluetrain', 'justices']

Epoch 2/2:  25%|██▍       | 8499/34343 [1:44:04<1:45:12,  4.09it/s] 

Epoch 2 Batch 8500 loss: 2.105393409729004

Epoch 2/2:  25%|██▍       | 8500/34343 [1:44:05<2:48:06,  2.56it/s]

Closest words to the center word coal: ['coal', 'bauxite', 'hydropower', 'potash', 'fuels', 'kwh', 'petroleum', 'zinc', 'beets', 'gwh']

Epoch 2/2:  25%|██▌       | 8599/34343 [1:45:23<1:33:03,  4.61it/s] 

Epoch 2 Batch 8600 loss: 2.0791122913360596

Epoch 2/2:  25%|██▌       | 8600/34343 [1:45:24<2:43:14,  2.63it/s]

Closest words to the center word graphics: ['graphics', 'raster', 'ipf', 'opengl', 'lossy', 'bitmap', 'usb', 'lossless', 'codecs', 'programmable']

Epoch 2/2:  25%|██▌       | 8699/34343 [1:46:51<2:55:29,  2.44it/s] 

Epoch 2 Batch 8700 loss: 2.095363140106201

Epoch 2/2:  25%|██▌       | 8700/34343 [1:46:51<3:23:12,  2.10it/s]

Closest words to the center word if: ['if', 'prev', 'bijective', 'newnode', 'forall', 'refutable', 'otimes', 'bijection', 'countably', 'ldots']

Epoch 2/2:  26%|██▌       | 8799/34343 [1:48:03<1:39:47,  4.27it/s] 

Epoch 2 Batch 8800 loss: 2.0790231227874756

Epoch 2/2:  26%|██▌       | 8800/34343 [1:48:03<2:29:34,  2.85it/s]

Closest words to the center word j: ['leiserson', 'elke', 'rivest', 'polskiej', 'cormen', 'stosunku', 'otimes', 'leq', 'landsmannschaft', 'hlich']

Epoch 2/2:  26%|██▌       | 8899/34343 [1:49:12<1:29:55,  4.72it/s] 

Epoch 2 Batch 8900 loss: 2.159534215927124

Epoch 2/2:  26%|██▌       | 8900/34343 [1:49:13<2:21:50,  2.99it/s]

Closest words to the center word it: ['it', 'findable', 'prev', 'bijective', 'testable', 'falsifiable', 'empirically', 'metrizable', 'refutable', 'unwise']

Epoch 2/2:  26%|██▌       | 8999/34343 [1:50:38<12:44:57,  1.81s/it]

Epoch 2 Batch 9000 loss: 2.1130452156066895

Epoch 2/2:  26%|██▌       | 9000/34343 [1:50:39<10:16:01,  1.46s/it]

Closest words to the center word see: ['see', 'disambiguation', 'yird', 'redirects', 'list', 'newnode', 'topics', 'terrier', 'wtoo', 'wtro']

Epoch 2/2:  26%|██▋       | 9099/34343 [1:51:48<2:24:48,  2.91it/s] 

Epoch 2 Batch 9100 loss: 2.0795629024505615

Epoch 2/2:  26%|██▋       | 9100/34343 [1:51:49<3:02:15,  2.31it/s]

Closest words to the center word final: ['final', 'finals', 'nfc', 'playoff', 'championship', 'divisional', 'fantasy', 'afc', 'champions', 'redskins']

Epoch 2/2:  27%|██▋       | 9199/34343 [1:53:00<1:34:52,  4.42it/s] 

Epoch 2 Batch 9200 loss: 2.129692554473877

Epoch 2/2:  27%|██▋       | 9200/34343 [1:53:00<2:23:57,  2.91it/s]

Closest words to the center word a: ['a', 'pluriform', 'groupoid', 'prev', 'automorphism', 'bijective', 'newnode', 'descriptor', 'agave', 'lastnode']

Epoch 2/2:  27%|██▋       | 9299/34343 [1:54:11<1:27:59,  4.74it/s] 

Epoch 2 Batch 9300 loss: 2.0988025665283203

Epoch 2/2:  27%|██▋       | 9300/34343 [1:54:11<2:33:43,  2.72it/s]

Closest words to the center word late: ['late', 'mid', 'kwh', 'gwh', 'nfc', 'early', 'nineteenth', 'twh', 'seventeenth', 'afc']

Epoch 2/2:  27%|██▋       | 9399/34343 [1:55:35<8:56:45,  1.29s/it] 

Epoch 2 Batch 9400 loss: 2.0639731884002686

Epoch 2/2:  27%|██▋       | 9400/34343 [1:55:36<7:35:19,  1.10s/it]

Closest words to the center word placed: ['placed', 'worn', 'interred', 'inscribed', 'buried', 'inserted', 'touches', 'mounted', 'located', 'stacked']

Epoch 2/2:  28%|██▊       | 9499/34343 [1:56:43<1:55:07,  3.60it/s] 

Epoch 2 Batch 9500 loss: 2.1140642166137695

Epoch 2/2:  28%|██▊       | 9500/34343 [1:56:44<2:42:05,  2.55it/s]

Closest words to the center word success: ['success', 'acclaim', 'popularity', 'airplay', 'downturn', 'successes', 'garnered', 'notoriety', 'comeback', 'postseason']

Epoch 2/2:  28%|██▊       | 9599/34343 [1:57:53<1:28:22,  4.67it/s] 

Epoch 2 Batch 9600 loss: 2.0933616161346436

Epoch 2/2:  28%|██▊       | 9600/34343 [1:57:54<2:18:16,  2.98it/s]

Closest words to the center word divorced: ['divorced', 'remarried', 'married', 'caesaris', 'wed', 'householder', 'fathered', 'raped', 'agrippina', 'antonia']

Epoch 2/2:  28%|██▊       | 9699/34343 [1:59:02<1:30:31,  4.54it/s] 

Epoch 2 Batch 9700 loss: 2.0646674633026123

Epoch 2/2:  28%|██▊       | 9700/34343 [1:59:02<2:24:05,  2.85it/s]

Closest words to the center word opponents: ['opponents', 'masculists', 'ideologies', 'collectivism', 'oppose', 'capitalists', 'supporters', 'detractors', 'unwillingness', 'egalitarianism']

Epoch 2/2:  29%|██▊       | 9799/34343 [2:00:25<5:09:20,  1.32it/s] 

Epoch 2 Batch 9800 loss: 2.098212242126465

Epoch 2/2:  29%|██▊       | 9800/34343 [2:00:26<4:54:13,  1.39it/s]

Closest words to the center word erasmus: ['yoannis', 'erasmus', 'stwertka', 'palaeologus', 'mamre', 'judaica', 'dramatists', 'nevi', 'duchess', 'friedrich']

Epoch 2/2:  29%|██▉       | 9899/34343 [2:01:36<1:40:16,  4.06it/s] 

Epoch 2 Batch 9900 loss: 2.0927066802978516

Epoch 2/2:  29%|██▉       | 9900/34343 [2:01:36<2:26:44,  2.78it/s]

Closest words to the center word replicating: ['replicating', 'eukaryotic', 'oxidizing', 'polymers', 'adjoint', 'deterministic', 'probabilistic', 'glycogen', 'multicellular', 'positron']

Epoch 2/2:  29%|██▉       | 9999/34343 [2:02:45<1:26:33,  4.69it/s] 

Epoch 2 Batch 10000 loss: 2.116197347640991

Epoch 2/2:  29%|██▉       | 10000/34343 [2:02:46<2:17:33,  2.95it/s]

Closest words to the center word they: ['they', 'we', 'you', 'matres', 'gesserit', 'liars', 'lifes', 'microtubules', 'themselves', 'findable']

Epoch 2/2:  29%|██▉       | 10099/34343 [2:04:10<32:24:59,  4.81s/it]

Epoch 2 Batch 10100 loss: 2.0954537391662598

Epoch 2/2:  29%|██▉       | 10100/34343 [2:04:11<23:56:45,  3.56s/it]

Closest words to the center word international: ['ifrcs', 'ifc', 'iom', 'icrm', 'ifad', 'international', 'opcw', 'icftu', 'acct', 'ibrd']

Epoch 2/2:  30%|██▉       | 10199/34343 [2:05:18<3:08:10,  2.14it/s] 

Epoch 2 Batch 10200 loss: 2.101100206375122

Epoch 2/2:  30%|██▉       | 10200/34343 [2:05:19<3:37:36,  1.85it/s]

Closest words to the center word led: ['led', 'unido', 'contributed', 'helped', 'ifrcs', 'caused', 'unctad', 'resulted', 'icrm', 'maaouya']

Epoch 2/2:  30%|██▉       | 10299/34343 [2:06:28<1:27:50,  4.56it/s] 

Epoch 2 Batch 10300 loss: 2.0999958515167236

Epoch 2/2:  30%|██▉       | 10300/34343 [2:06:28<2:17:49,  2.91it/s]

Closest words to the center word parker: ['parker', 'clooney', 'bandleader', 'giroux', 'stevie', 'holliday', 'astaire', 'saxophonist', 'gillespie', 'zant']

Epoch 2/2:  30%|███       | 10399/34343 [2:07:39<1:27:27,  4.56it/s] 

Epoch 2 Batch 10400 loss: 2.1005687713623047

Epoch 2/2:  30%|███       | 10400/34343 [2:07:40<2:29:13,  2.67it/s]

Closest words to the center word addresses: ['findable', 'addresses', 'rfc', 'dhcp', 'ipv', 'dns', 'address', 'servers', 'ip', 'iec']

Epoch 2/2:  31%|███       | 10499/34343 [2:09:04<16:37:46,  2.51s/it]

Epoch 2 Batch 10500 loss: 2.1011013984680176

Epoch 2/2:  31%|███       | 10500/34343 [2:09:05<13:04:12,  1.97s/it]

Closest words to the center word thought: ['thought', 'monistic', 'falsifiable', 'rationalism', 'materialism', 'panentheism', 'pantheism', 'leninism', 'empirically', 'testable']

Epoch 2/2:  31%|███       | 10599/34343 [2:10:14<1:58:40,  3.33it/s] 

Epoch 2 Batch 10600 loss: 2.0845556259155273

Epoch 2/2:  31%|███       | 10600/34343 [2:10:15<2:47:37,  2.36it/s]

Closest words to the center word while: ['while', 'gangetic', 'although', 'inflicted', 'haliotis', 'though', 'hand', 'whilst', 'tropics', 'callithrix']

Epoch 2/2:  31%|███       | 10699/34343 [2:11:21<1:24:27,  4.67it/s] 

Epoch 2 Batch 10700 loss: 2.097136974334717

Epoch 2/2:  31%|███       | 10700/34343 [2:11:22<2:16:52,  2.88it/s]

Closest words to the center word one: ['gwh', 'hbk', 'pngimage', 'kwh', 'grt', 'twh', 'cyg', 'sfg', 'jul', 'mjs']

Epoch 2/2:  31%|███▏      | 10799/34343 [2:12:30<1:22:57,  4.73it/s] 

Epoch 2 Batch 10800 loss: 2.103605270385742

Epoch 2/2:  31%|███▏      | 10800/34343 [2:12:31<2:19:16,  2.82it/s]

Closest words to the center word security: ['opcw', 'unido', 'unctad', 'security', 'iom', 'ifad', 'icrm', 'ifrcs', 'ifc', 'pluriform']

Epoch 2/2:  32%|███▏      | 10899/34343 [2:13:57<4:02:28,  1.61it/s] 

Epoch 2 Batch 10900 loss: 2.095186710357666

Epoch 2/2:  32%|███▏      | 10900/34343 [2:13:57<4:09:25,  1.57it/s]

Closest words to the center word character: ['character', 'villain', 'characters', 'protagonist', 'scrooge', 'superhero', 'animated', 'toriyama', 'elric', 'hitchhiker']

Epoch 2/2:  32%|███▏      | 10999/34343 [2:15:06<1:36:48,  4.02it/s] 

Epoch 2 Batch 11000 loss: 2.1248416900634766

Epoch 2/2:  32%|███▏      | 11000/34343 [2:15:07<2:32:50,  2.55it/s]

Closest words to the center word mobil: ['kwh', 'mobil', 'gwh', 'twh', 'dwt', 'pluriform', 'intelsat', 'exxon', 'nasdaq', 'motors']

Epoch 2/2:  32%|███▏      | 11099/34343 [2:16:47<2:00:34,  3.21it/s] 

Epoch 2 Batch 11100 loss: 2.1157004833221436

Epoch 2/2:  32%|███▏      | 11100/34343 [2:16:48<2:50:25,  2.27it/s]

Closest words to the center word logical: ['logical', 'adjoint', 'axiom', 'propositional', 'mathfrak', 'ponens', 'axiomatic', 'cognitivism', 'deterministic', 'countable']

Epoch 2/2:  33%|███▎      | 11199/34343 [2:18:38<29:11:27,  4.54s/it]

Epoch 2 Batch 11200 loss: 2.076874256134033

Epoch 2/2:  33%|███▎      | 11200/34343 [2:18:40<22:29:43,  3.50s/it]

Closest words to the center word players: ['players', 'playoffs', 'linemen', 'orioles', 'browns', 'astros', 'nhl', 'bengals', 'broncos', 'teams']

Epoch 2/2:  33%|███▎      | 11299/34343 [2:20:07<4:00:56,  1.59it/s] 

Epoch 2 Batch 11300 loss: 2.1086976528167725

Epoch 2/2:  33%|███▎      | 11300/34343 [2:20:08<4:26:09,  1.44it/s]

Closest words to the center word facilitate: ['facilitate', 'nonsignatory', 'ifrcs', 'iom', 'ifad', 'opcw', 'stimulate', 'motivate', 'induce', 'ifc']

Epoch 2/2:  33%|███▎      | 11399/34343 [2:21:36<1:44:33,  3.66it/s] 

Epoch 2 Batch 11400 loss: 2.0916242599487305

Epoch 2/2:  33%|███▎      | 11400/34343 [2:21:37<2:46:34,  2.30it/s]

Closest words to the center word minister: ['minister', 'prime', 'csu', 'ministers', 'cdu', 'succeeds', 'fdp', 'boutros', 'chancellor', 'strau']

Epoch 2/2:  33%|███▎      | 11499/34343 [2:23:01<1:36:37,  3.94it/s] 

Epoch 2 Batch 11500 loss: 2.126551866531372

Epoch 2/2:  33%|███▎      | 11500/34343 [2:23:02<2:31:42,  2.51it/s]

Closest words to the center word mind: ['mind', 'phenomenology', 'cognition', 'egoism', 'cognitivism', 'objectivism', 'consciousness', 'metaphysics', 'intellect', 'brahman']

Epoch 2/2:  34%|███▍      | 11599/34343 [2:24:55<22:51:01,  3.62s/it]

Epoch 2 Batch 11600 loss: 2.094184637069702

Epoch 2/2:  34%|███▍      | 11600/34343 [2:24:56<17:26:12,  2.76s/it]

Closest words to the center word performance: ['performance', 'throughput', 'lossy', 'compression', 'efficiency', 'macroeconomic', 'paced', 'superscalar', 'airplay', 'lossless']

Epoch 2/2:  34%|███▍      | 11699/34343 [2:26:50<5:50:58,  1.08it/s] 

Epoch 2 Batch 11700 loss: 2.1219887733459473

Epoch 2/2:  34%|███▍      | 11700/34343 [2:26:52<7:47:46,  1.24s/it]

Closest words to the center word has: ['has', 'hasn', 'had', 'have', 'ifrcs', 'pluriform', 'having', 'possesses', 'hadn', 'ifad']

Epoch 2/2:  34%|███▍      | 11799/34343 [2:28:21<1:34:59,  3.96it/s] 

Epoch 2 Batch 11800 loss: 2.106555938720703

Epoch 2/2:  34%|███▍      | 11800/34343 [2:28:22<2:15:25,  2.77it/s]

Closest words to the center word which: ['which', 'enthalpy', 'cytosol', 'insoluble', 'countably', 'piezoelectric', 'centripetal', 'halide', 'symplectic', 'endothermic']

Epoch 2/2:  35%|███▍      | 11899/34343 [2:30:03<1:39:25,  3.76it/s] 

Epoch 2 Batch 11900 loss: 2.116100549697876

Epoch 2/2:  35%|███▍      | 11900/34343 [2:30:03<2:22:48,  2.62it/s]

Closest words to the center word south: ['south', 'chungcheong', 'busan', 'irian', 'gyeonggi', 'sulawesi', 'north', 'tenggara', 'savannas', 'southwest']

Epoch 2/2:  35%|███▍      | 11999/34343 [2:31:54<13:58:33,  2.25s/it]

Epoch 2 Batch 12000 loss: 2.086256504058838

Epoch 2/2:  35%|███▍      | 12000/34343 [2:31:54<10:59:20,  1.77s/it]

Closest words to the center word possessed: ['possessed', 'transcendent', 'misled', 'lifes', 'immanent', 'harkonnen', 'peleus', 'knew', 'exercised', 'deified']

Epoch 2/2:  35%|███▌      | 12099/34343 [2:33:26<2:26:01,  2.54it/s] 

Epoch 2 Batch 12100 loss: 2.1063146591186523

Epoch 2/2:  35%|███▌      | 12100/34343 [2:33:27<2:58:50,  2.07it/s]

Closest words to the center word collaboration: ['collaboration', 'wagoner', 'collaborated', 'interviews', 'friendship', 'collaborator', 'finalist', 'diffie', 'interview', 'emmylou']

Epoch 2/2:  36%|███▌      | 12199/34343 [2:35:50<2:06:20,  2.92it/s] 

Epoch 2 Batch 12200 loss: 2.123460531234741

Epoch 2/2:  36%|███▌      | 12200/34343 [2:35:51<3:11:04,  1.93it/s]

Closest words to the center word any: ['any', 'anything', 'newnode', 'lastnode', 'prev', 'every', 'baryonic', 'whatsoever', 'differentiable', 'antiderivatives']

Epoch 2/2:  36%|███▌      | 12299/34343 [2:38:04<1:54:01,  3.22it/s] 

Epoch 2 Batch 12300 loss: 2.0970840454101562

Epoch 2/2:  36%|███▌      | 12300/34343 [2:38:05<3:06:39,  1.97it/s]

Closest words to the center word ad: ['ad', 'argumentum', 'hominem', 'bc', 'bce', 'reductio', 'ce', 'hoc', 'yoannis', 'propter']

Epoch 2/2:  36%|███▌      | 12399/34343 [2:40:50<9:16:06,  1.52s/it] 

Epoch 2 Batch 12400 loss: 2.1157491207122803

Epoch 2/2:  36%|███▌      | 12400/34343 [2:40:51<7:46:29,  1.28s/it]

Closest words to the center word twice: ['twice', 'innings', 'lifes', 'playoffs', 'strikeouts', 'batted', 'grt', 'intercalary', 'inning', 'centimeters']

Epoch 2/2:  36%|███▋      | 12499/34343 [2:42:33<2:11:32,  2.77it/s] 

Epoch 2 Batch 12500 loss: 2.1198954582214355

Epoch 2/2:  36%|███▋      | 12500/34343 [2:42:34<2:57:29,  2.05it/s]

Closest words to the center word five: ['gwh', 'grt', 'kwh', 'hbk', 'twh', 'cyg', 'sfg', 'pngimage', 'mjs', 'jul']

Epoch 2/2:  37%|███▋      | 12599/34343 [2:45:08<1:47:23,  3.37it/s] 

Epoch 2 Batch 12600 loss: 2.142111301422119

Epoch 2/2:  37%|███▋      | 12600/34343 [2:45:09<2:43:01,  2.22it/s]

Closest words to the center word compilers: ['compilers', 'lisp', 'macros', 'opengl', 'bytecode', 'applets', 'implementations', 'brainfuck', 'debugger', 'compiler']

Epoch 2/2:  37%|███▋      | 12699/34343 [2:47:06<1:36:24,  3.74it/s] 

Epoch 2 Batch 12700 loss: 2.0801544189453125

Epoch 2/2:  37%|███▋      | 12700/34343 [2:47:33<48:57:58,  8.14s/it]

Closest words to the center word approved: ['approved', 'ratified', 'iom', 'amended', 'fda', 'unicameral', 'kwh', 'impeached', 'ifad', 'ratification']

Epoch 2/2:  37%|███▋      | 12799/34343 [2:49:07<4:52:12,  1.23it/s] 

Epoch 2 Batch 12800 loss: 2.1307029724121094

Epoch 2/2:  37%|███▋      | 12800/34343 [2:49:07<4:51:26,  1.23it/s]

Closest words to the center word auger: ['auger', 'icrm', 'lorentz', 'physicist', 'coulomb', 'ibrd', 'leiserson', 'krebs', 'nsted', 'ment']

Epoch 2/2:  38%|███▊      | 12899/34343 [2:50:40<1:33:24,  3.83it/s] 

Epoch 2 Batch 12900 loss: 2.14245343208313

Epoch 2/2:  38%|███▊      | 12900/34343 [2:50:41<2:32:01,  2.35it/s]

Closest words to the center word stadium: ['stadium', 'chargers', 'playoffs', 'texans', 'afc', 'nfc', 'oakland', 'playoff', 'broncos', 'steelers']

Epoch 2/2:  38%|███▊      | 12999/34343 [2:52:15<1:43:25,  3.44it/s] 

Epoch 2 Batch 13000 loss: 2.105532646179199

Epoch 2/2:  38%|███▊      | 13000/34343 [2:52:16<3:24:16,  1.74it/s]

Closest words to the center word protected: ['protected', 'regulated', 'unido', 'wastes', 'dwt', 'administered', 'vesicles', 'ifrcs', 'dumping', 'pluriform']

Epoch 2/2:  38%|███▊      | 13099/34343 [2:54:17<34:21:46,  5.82s/it]

Epoch 2 Batch 13100 loss: 2.138997793197632

Epoch 2/2:  38%|███▊      | 13100/34343 [2:54:18<25:29:28,  4.32s/it]

Closest words to the center word subsequent: ['subsequent', 'culminated', 'perestroika', 'amanullah', 'abbasid', 'glasnost', 'ensuing', 'hastened', 'mauryan', 'resulted']

Epoch 2/2:  38%|███▊      | 13199/34343 [2:55:59<2:08:13,  2.75it/s] 

Epoch 2 Batch 13200 loss: 2.1364693641662598

Epoch 2/2:  38%|███▊      | 13200/34343 [2:56:00<2:47:41,  2.10it/s]

Closest words to the center word competitively: ['competitively', 'kart', 'soybeans', 'aroma', 'firmware', 'skiing', 'bikes', 'gba', 'ductile', 'harmonicas']

Epoch 2/2:  39%|███▊      | 13299/34343 [2:57:41<1:52:28,  3.12it/s] 

Epoch 2 Batch 13300 loss: 2.0943915843963623

Epoch 2/2:  39%|███▊      | 13300/34343 [2:57:42<3:12:14,  1.82it/s]

Closest words to the center word clairvoyance: ['clairvoyance', 'cognitivism', 'testable', 'falsifiable', 'therapies', 'memetics', 'deconstructive', 'paranormal', 'anecdotal', 'anthropic']

Epoch 2/2:  39%|███▉      | 13399/34343 [2:59:12<1:37:28,  3.58it/s] 

Epoch 2 Batch 13400 loss: 2.108501434326172

Epoch 2/2:  39%|███▉      | 13400/34343 [2:59:13<2:34:05,  2.27it/s]

Closest words to the center word and: ['ifrcs', 'ifad', 'ifc', 'icrm', 'unmibh', 'opcw', 'unctad', 'rajonas', 'classis', 'icftu']

Epoch 2/2:  39%|███▉      | 13499/34343 [3:01:10<5:41:02,  1.02it/s] 

Epoch 2 Batch 13500 loss: 2.14046573638916

Epoch 2/2:  39%|███▉      | 13500/34343 [3:01:11<5:51:50,  1.01s/it]

Closest words to the center word premier: ['premier', 'premiers', 'finalist', 'famers', 'chungcheong', 'coach', 'diamondbacks', 'nfc', 'football', 'footballers']

Epoch 2/2:  40%|███▉      | 13599/34343 [3:02:46<1:55:13,  3.00it/s] 

Epoch 2 Batch 13600 loss: 2.1126320362091064

Epoch 2/2:  40%|███▉      | 13600/34343 [3:02:47<2:40:46,  2.15it/s]

Closest words to the center word sperm: ['sperm', 'diploid', 'eukaryotic', 'haploid', 'secrete', 'homologous', 'hormones', 'gamete', 'chromosomes', 'mitochondria']

Epoch 2/2:  40%|███▉      | 13699/34343 [3:04:21<1:34:50,  3.63it/s] 

Epoch 2 Batch 13700 loss: 2.0808300971984863

Epoch 2/2:  40%|███▉      | 13700/34343 [3:04:22<2:29:12,  2.31it/s]

Closest words to the center word narrow: ['narrow', 'steep', 'sloping', 'humid', 'mountainous', 'gauge', 'kilometer', 'km', 'elevations', 'westerly']

Epoch 2/2:  40%|████      | 13799/34343 [3:06:09<33:13:50,  5.82s/it]

Epoch 2 Batch 13800 loss: 2.062546730041504

Epoch 2/2:  40%|████      | 13800/34343 [3:06:10<24:34:07,  4.31s/it]

Closest words to the center word were: ['were', 'are', 'persecuted', 'massacred', 'outnumbered', 'raids', 'aediles', 'serbs', 'ukrainians', 'byzantines']

Epoch 2/2:  40%|████      | 13899/34343 [3:07:41<4:24:37,  1.29it/s] 

Epoch 2 Batch 13900 loss: 2.090404987335205

Epoch 2/2:  40%|████      | 13900/34343 [3:07:42<4:24:32,  1.29it/s]

Closest words to the center word chicago: ['chicago', 'devry', 'abet', 'culver', 'kansas', 'illinois', 'urbana', 'detroit', 'moines', 'tacoma']

Epoch 2/2:  41%|████      | 13999/34343 [3:09:10<1:45:37,  3.21it/s] 

Epoch 2 Batch 14000 loss: 2.1093571186065674

Epoch 2/2:  41%|████      | 14000/34343 [3:09:11<2:32:37,  2.22it/s]

Closest words to the center word breeding: ['breeding', 'ferus', 'shrubs', 'crocodiles', 'habitat', 'cassava', 'prey', 'tusks', 'larvae', 'vectorborne']

Epoch 2/2:  41%|████      | 14099/34343 [3:10:53<1:39:30,  3.39it/s] 

Epoch 2 Batch 14100 loss: 2.117936134338379

Epoch 2/2:  41%|████      | 14100/34343 [3:10:54<2:35:55,  2.16it/s]

Closest words to the center word revolution: ['revolution', 'bolsheviks', 'revolutionary', 'uprising', 'socialist', 'rsdlp', 'kuomintang', 'anarchism', 'sino', 'maoist']

Epoch 2/2:  41%|████▏     | 14199/34343 [3:12:49<26:48:52,  4.79s/it]

Epoch 2 Batch 14200 loss: 2.1263201236724854

Epoch 2/2:  41%|████▏     | 14200/34343 [3:12:50<19:57:05,  3.57s/it]

Closest words to the center word jeong: ['jeong', 'tiberian', 'hunmin', 'polskiej', 'cccc', 'cooh', 'eum', 'stosunku', 'rzeczypospolitej', 'ttt']

Epoch 2/2:  42%|████▏     | 14299/34343 [3:14:17<3:23:05,  1.64it/s] 

Epoch 2 Batch 14300 loss: 2.089411497116089

Epoch 2/2:  42%|████▏     | 14300/34343 [3:14:18<4:14:42,  1.31it/s]

Closest words to the center word morning: ['morning', 'evening', 'friday', 'saturday', 'afternoon', 'selamat', 'night', 'tonight', 'thursday', 'solstice']

Epoch 2/2:  42%|████▏     | 14399/34343 [3:15:56<1:38:47,  3.36it/s] 

Epoch 2 Batch 14400 loss: 2.0882961750030518

Epoch 2/2:  42%|████▏     | 14400/34343 [3:15:57<2:24:45,  2.30it/s]

Closest words to the center word does: ['does', 'findable', 'did', 'doesn', 'didn', 'factum', 'wouldn', 'opcw', 'do', 'iom']

Epoch 2/2:  42%|████▏     | 14499/34343 [3:17:39<1:27:26,  3.78it/s] 

Epoch 2 Batch 14500 loss: 2.1101858615875244

Epoch 2/2:  42%|████▏     | 14500/34343 [3:17:39<2:32:22,  2.17it/s]

Closest words to the center word m: ['elke', 'm', 'cdots', 'ldots', 'stosunku', 'otimes', 'cdot', 'nchen', 'ttt', 'leq']

Epoch 2/2:  43%|████▎     | 14599/34343 [3:19:45<25:34:12,  4.66s/it]

Epoch 2 Batch 14600 loss: 2.106454849243164

Epoch 2/2:  43%|████▎     | 14600/34343 [3:19:45<19:05:12,  3.48s/it]

Closest words to the center word ouen: ['yoannis', 'basilica', 'churchyard', 'methodius', 'grenadines', 'sistine', 'kitts', 'chapel', 'hagia', 'benedictine']

Epoch 2/2:  43%|████▎     | 14699/34343 [3:21:29<2:48:26,  1.94it/s] 

Epoch 2 Batch 14700 loss: 2.0788698196411133

Epoch 2/2:  43%|████▎     | 14700/34343 [3:21:30<3:19:19,  1.64it/s]

Closest words to the center word terrorist: ['terrorist', 'qaeda', 'opcw', 'ifrcs', 'hezbollah', 'unido', 'terrorism', 'paramilitary', 'islamist', 'peacekeeping']

Epoch 2/2:  43%|████▎     | 14799/34343 [3:23:04<1:33:37,  3.48it/s] 

Epoch 2 Batch 14800 loss: 2.0820975303649902

Epoch 2/2:  43%|████▎     | 14800/34343 [3:23:05<2:20:17,  2.32it/s]

Closest words to the center word republicans: ['republicans', 'bolsheviks', 'conservatives', 'wingers', 'democrats', 'voters', 'whigs', 'liberals', 'militias', 'communists']

Epoch 2/2:  43%|████▎     | 14899/34343 [3:24:44<1:40:07,  3.24it/s] 

Epoch 2 Batch 14900 loss: 2.08479642868042

Epoch 2/2:  43%|████▎     | 14900/34343 [3:24:44<2:29:13,  2.17it/s]

Closest words to the center word replacing: ['pluriform', 'replacing', 'imac', 'dwt', 'replaced', 'ipf', 'powerbook', 'otimes', 'cccc', 'bbbb']

Epoch 2/2:  44%|████▎     | 14999/34343 [3:26:59<11:40:42,  2.17s/it]

Epoch 2 Batch 15000 loss: 2.081838607788086

Epoch 2/2:  44%|████▎     | 15000/34343 [3:26:59<9:28:21,  1.76s/it] 

Closest words to the center word this: ['this', 'enthalpy', 'epimenides', 'empirically', 'findable', 'falsifiability', 'minimax', 'strictest', 'casuistry', 'irreducibly']

Epoch 2/2:  44%|████▍     | 15099/34343 [3:28:29<1:48:28,  2.96it/s] 

Epoch 2 Batch 15100 loss: 2.1021761894226074

Epoch 2/2:  44%|████▍     | 15100/34343 [3:28:30<2:20:39,  2.28it/s]

Closest words to the center word but: ['but', 'though', 'unless', 'liars', 'countably', 'lifes', 'nor', 'preregular', 'because', 'consummated']

Epoch 2/2:  44%|████▍     | 15199/34343 [3:30:04<1:24:06,  3.79it/s] 

Epoch 2 Batch 15200 loss: 2.1122500896453857

Epoch 2/2:  44%|████▍     | 15200/34343 [3:30:04<2:07:30,  2.50it/s]

Closest words to the center word called: ['called', 'agave', 'termed', 'referred', 'labiodental', 'leontopithecus', 'endomorphism', 'denoted', 'disambiguation', 'paracompact']

Epoch 2/2:  45%|████▍     | 15299/34343 [3:31:48<1:58:21,  2.68it/s] 

Epoch 2 Batch 15300 loss: 2.0835440158843994

Epoch 2/2:  45%|████▍     | 15300/34343 [3:31:49<2:46:14,  1.91it/s]

Closest words to the center word or: ['or', 'alkoxide', 'clonic', 'ifrcs', 'waterborne', 'ketone', 'tachycardia', 'unsaturated', 'newnode', 'sibilant']

Epoch 2/2:  45%|████▍     | 15399/34343 [3:34:04<5:51:17,  1.11s/it] 

Epoch 2 Batch 15400 loss: 2.1030282974243164

Epoch 2/2:  45%|████▍     | 15400/34343 [3:34:05<5:24:45,  1.03s/it]

Closest words to the center word refer: ['refer', 'describe', 'refers', 'referred', 'relate', 'adhere', 'refered', 'belong', 'referring', 'classify']

Epoch 2/2:  45%|████▌     | 15499/34343 [3:35:51<1:42:49,  3.05it/s] 

Epoch 2 Batch 15500 loss: 2.0975685119628906

Epoch 2/2:  45%|████▌     | 15500/34343 [3:35:52<2:29:14,  2.10it/s]

Closest words to the center word civil: ['civil', 'liberties', 'pluriform', 'criminal', 'secedes', 'judicial', 'suffrage', 'icao', 'williamite', 'confederate']

Epoch 2/2:  45%|████▌     | 15599/34343 [3:37:24<1:17:05,  4.05it/s] 

Epoch 2 Batch 15600 loss: 2.0705533027648926

Epoch 2/2:  45%|████▌     | 15600/34343 [3:37:24<1:57:43,  2.65it/s]

Closest words to the center word not: ['not', 'liars', 'unable', 'refutable', 'repent', 'empirically', 'disprove', 'necessarily', 'wouldn', 'nor']

Epoch 2/2:  46%|████▌     | 15699/34343 [3:39:30<37:02:00,  7.15s/it]

Epoch 2 Batch 15700 loss: 2.1161015033721924

Epoch 2/2:  46%|████▌     | 15700/34343 [3:39:31<26:55:43,  5.20s/it]

Closest words to the center word less: ['less', 'lifes', 'more', 'cheaper', 'thicker', 'softer', 'viscous', 'than', 'hotter', 'denser']

Epoch 2/2:  46%|████▌     | 15799/34343 [3:41:09<2:42:23,  1.90it/s] 

Epoch 2 Batch 15800 loss: 2.139280080795288

Epoch 2/2:  46%|████▌     | 15800/34343 [3:41:09<2:57:06,  1.74it/s]

Closest words to the center word to: ['to', 'findable', 'inability', 'able', 'unable', 'obliged', 'prev', 'willing', 'stimulate', 'attempt']

Epoch 2/2:  46%|████▋     | 15899/34343 [3:43:03<1:25:23,  3.60it/s] 

Epoch 2 Batch 15900 loss: 2.098975658416748

Epoch 2/2:  46%|████▋     | 15900/34343 [3:43:04<2:06:48,  2.42it/s]

Closest words to the center word screen: ['screen', 'bitmap', 'ipf', 'backlit', 'scrolling', 'playstation', 'gba', 'raster', 'clicking', 'rgb']

Epoch 2/2:  47%|████▋     | 15999/34343 [3:44:43<1:49:17,  2.80it/s] 

Epoch 2 Batch 16000 loss: 2.151726722717285

Epoch 2/2:  47%|████▋     | 16000/34343 [3:44:44<2:29:53,  2.04it/s]

Closest words to the center word speed: ['speed', 'speeds', 'voltage', 'velocity', 'throughput', 'torque', 'subsonic', 'mbit', 'mhz', 'voltages']

Epoch 2/2:  47%|████▋     | 16099/34343 [3:46:49<7:16:01,  1.43s/it] 

Epoch 2 Batch 16100 loss: 2.0663247108459473

Epoch 2/2:  47%|████▋     | 16100/34343 [3:46:50<6:08:33,  1.21s/it]

Closest words to the center word libraries: ['libraries', 'cygwin', 'compilers', 'netbsd', 'findable', 'gpled', 'browsers', 'directories', 'recompilation', 'gnu']

Epoch 2/2:  47%|████▋     | 16199/34343 [3:48:33<1:56:08,  2.60it/s] 

Epoch 2 Batch 16200 loss: 2.0803465843200684

Epoch 2/2:  47%|████▋     | 16200/34343 [3:48:34<2:36:48,  1.93it/s]

Closest words to the center word its: ['its', 'ifrcs', 'ifad', 'their', 'icrm', 'ifc', 'iom', 'findable', 'nonsignatory', 'iho']

Epoch 2/2:  47%|████▋     | 16299/34343 [3:50:22<1:54:51,  2.62it/s] 

Epoch 2 Batch 16300 loss: 2.09684681892395

Epoch 2/2:  47%|████▋     | 16300/34343 [3:50:24<4:55:49,  1.02it/s]

Closest words to the center word decided: ['decided', 'refused', 'vowed', 'persuaded', 'wanted', 'opted', 'agreed', 'didn', 'announced', 'obliged']

Epoch 2/2:  48%|████▊     | 16399/34343 [3:52:12<1:27:55,  3.40it/s] 

Epoch 2 Batch 16400 loss: 2.0894036293029785

Epoch 2/2:  48%|████▊     | 16400/34343 [3:52:40<42:56:31,  8.62s/it]

Closest words to the center word cathedral: ['cathedral', 'basilica', 'churchyard', 'sistine', 'yoannis', 'chapel', 'abbey', 'sepulchre', 'convent', 'hagia']

Epoch 2/2:  48%|████▊     | 16499/34343 [3:54:18<5:18:15,  1.07s/it] 

Epoch 2 Batch 16500 loss: 2.095620632171631

Epoch 2/2:  48%|████▊     | 16500/34343 [3:54:19<5:20:31,  1.08s/it]

Closest words to the center word atoms: ['atoms', 'protons', 'covalent', 'electrons', 'orbitals', 'ions', 'neutrons', 'particles', 'hydrogen', 'molecules']

Epoch 2/2:  48%|████▊     | 16599/34343 [3:56:04<1:55:28,  2.56it/s] 

Epoch 2 Batch 16600 loss: 2.1025757789611816

Epoch 2/2:  48%|████▊     | 16600/34343 [3:56:06<3:16:47,  1.50it/s]

Closest words to the center word an: ['an', 'iom', 'ifrcs', 'ifc', 'nonsignatory', 'unido', 'argumentum', 'laia', 'wmo', 'wtro']

Epoch 2/2:  49%|████▊     | 16699/34343 [3:57:51<1:32:37,  3.17it/s] 

Epoch 2 Batch 16700 loss: 2.0925910472869873

Epoch 2/2:  49%|████▊     | 16700/34343 [3:57:51<2:09:48,  2.27it/s]

Closest words to the center word being: ['being', 'having', 'acutely', 'orally', 'divinely', 'universally', 'heretical', 'oxidizing', 'insoluble', 'amplify']

Epoch 2/2:  49%|████▉     | 16799/34343 [3:59:55<32:53:01,  6.75s/it]

Epoch 2 Batch 16800 loss: 2.0983142852783203

Epoch 2/2:  49%|████▉     | 16800/34343 [3:59:56<24:09:14,  4.96s/it]

Closest words to the center word a: ['a', 'pluriform', 'agave', 'newnode', 'insubstantial', 'keying', 'unital', 'prev', 'globicephala', 'enthalpy']

Epoch 2/2:  49%|████▉     | 16899/34343 [4:01:36<4:09:39,  1.16it/s] 

Epoch 2 Batch 16900 loss: 2.0604052543640137

Epoch 2/2:  49%|████▉     | 16900/34343 [4:01:37<4:08:10,  1.17it/s]

Closest words to the center word of: ['of', 'ifrcs', 'ifad', 'rajonas', 'wtoo', 'unido', 'ifc', 'icrm', 'nazarene', 'wftu']

Epoch 2/2:  49%|████▉     | 16999/34343 [4:03:11<1:22:44,  3.49it/s] 

Epoch 2 Batch 17000 loss: 2.0964388847351074

Epoch 2/2:  50%|████▉     | 17000/34343 [4:03:12<2:02:30,  2.36it/s]

Closest words to the center word area: ['area', 'sq', 'unpaved', 'runways', 'ecoregion', 'irrigated', 'subtropical', 'isthmus', 'km', 'kilometers']

Epoch 2/2:  50%|████▉     | 17099/34343 [4:04:46<1:10:31,  4.08it/s] 

Epoch 2 Batch 17100 loss: 2.1100945472717285

Epoch 2/2:  50%|████▉     | 17100/34343 [4:04:47<1:53:03,  2.54it/s]

Closest words to the center word days: ['days', 'months', 'grt', 'weeks', 'hours', 'minutes', 'lifes', 'lasts', 'gregorian', 'friday']

Epoch 2/2:  50%|█████     | 17199/34343 [4:06:41<23:06:24,  4.85s/it]

Epoch 2 Batch 17200 loss: 2.0566067695617676

Epoch 2/2:  50%|█████     | 17200/34343 [4:06:42<17:16:30,  3.63s/it]

Closest words to the center word years: ['years', 'grt', 'decades', 'months', 'weeks', 'servicemales', 'availabilitymales', 'lifes', 'gwh', 'males']

Epoch 2/2:  50%|█████     | 17299/34343 [4:08:10<2:30:49,  1.88it/s] 

Epoch 2 Batch 17300 loss: 2.074434280395508

Epoch 2/2:  50%|█████     | 17300/34343 [4:08:11<2:42:48,  1.74it/s]

Closest words to the center word core: ['core', 'cisc', 'zseries', 'xt', 'graphical', 'desktop', 'sparc', 'athlon', 'risc', 'microprocessor']

Epoch 2/2:  51%|█████     | 17399/34343 [4:09:55<5:27:25,  1.16s/it] 

Epoch 2 Batch 17400 loss: 2.108968734741211

Epoch 2/2:  51%|█████     | 17400/34343 [4:09:55<4:51:27,  1.03s/it]

Closest words to the center word both: ['both', 'bilabial', 'mutually', 'mizrahi', 'respective', 'maronites', 'trigonometric', 'fricatives', 'subtraction', 'laminal']

Epoch 2/2:  51%|█████     | 17499/34343 [4:11:48<2:47:20,  1.68it/s] 

Epoch 2 Batch 17500 loss: 2.0946757793426514

Epoch 2/2:  51%|█████     | 17500/34343 [4:11:49<2:59:41,  1.56it/s]

Closest words to the center word of: ['of', 'ifrcs', 'ifad', 'rajonas', 'wtoo', 'icrm', 'ifc', 'unido', 'akan', 'nazarene']

Epoch 2/2:  51%|█████     | 17599/34343 [4:13:39<12:51:04,  2.76s/it]

Epoch 2 Batch 17600 loss: 2.121558666229248

Epoch 2/2:  51%|█████     | 17600/34343 [4:13:40<10:11:19,  2.19s/it]

Closest words to the center word tag: ['tag', 'pngimage', 'mjs', 'mjd', 'cccc', 'bbbb', 'komm', 'findable', 'meine', 'jesu']

Epoch 2/2:  52%|█████▏    | 17699/34343 [4:15:14<1:47:47,  2.57it/s] 

Epoch 2 Batch 17700 loss: 2.097524642944336

Epoch 2/2:  52%|█████▏    | 17700/34343 [4:15:15<2:19:00,  2.00it/s]

Closest words to the center word club: ['club', 'bruins', 'cyclopedia', 'texans', 'broncos', 'diamondbacks', 'stadium', 'vsl', 'sox', 'coppa']

Epoch 2/2:  52%|█████▏    | 17799/34343 [4:16:47<59:40,  4.62it/s]   

Epoch 2 Batch 17800 loss: 2.089198589324951

Epoch 2/2:  52%|█████▏    | 17800/34343 [4:16:48<1:35:35,  2.88it/s]

Closest words to the center word those: ['those', 'lifes', 'felonies', 'consenting', 'neopagans', 'disapprove', 'upu', 'unido', 'nontrinitarian', 'denominations']

Epoch 2/2:  52%|█████▏    | 17899/34343 [4:18:00<59:40,  4.59it/s]   

Epoch 2 Batch 17900 loss: 2.1561672687530518

Epoch 2/2:  52%|█████▏    | 17900/34343 [4:18:01<1:56:20,  2.36it/s]

Closest words to the center word moment: ['moment', 'dipole', 'newnode', 'tangent', 'eccentricity', 'velocity', 'prev', 'penumbra', 'rotational', 'hyperfocal']

Epoch 2/2:  52%|█████▏    | 17999/34343 [4:19:30<5:07:31,  1.13s/it] 

Epoch 2 Batch 18000 loss: 2.089468002319336

Epoch 2/2:  52%|█████▏    | 18000/34343 [4:19:31<4:30:23,  1.01it/s]

Closest words to the center word mode: ['mode', 'bitmap', 'debugger', 'ipf', 'usb', 'cpu', 'processor', 'xt', 'scsi', 'asynchronous']

Epoch 2/2:  53%|█████▎    | 18099/34343 [4:20:44<2:01:01,  2.24it/s] 

Epoch 2 Batch 18100 loss: 2.1045188903808594

Epoch 2/2:  53%|█████▎    | 18100/34343 [4:20:45<2:18:03,  1.96it/s]

Closest words to the center word created: ['created', 'pluriform', 'coined', 'unido', 'nonsignatory', 'popularized', 'invented', 'supplanted', 'founded', 'formed']

Epoch 2/2:  53%|█████▎    | 18199/34343 [4:21:56<59:45,  4.50it/s]   

Epoch 2 Batch 18200 loss: 2.0475542545318604

Epoch 2/2:  53%|█████▎    | 18200/34343 [4:21:57<1:34:06,  2.86it/s]

Closest words to the center word is: ['is', 'refers', 'differentiable', 'satisfies', 'diffeomorphism', 'bijective', 'metrizable', 'converges', 'endomorphism', 'subgroup']

Epoch 2/2:  53%|█████▎    | 18299/34343 [4:23:07<56:31,  4.73it/s]   

Epoch 2 Batch 18300 loss: 2.0971145629882812

Epoch 2/2:  53%|█████▎    | 18300/34343 [4:23:23<22:11:16,  4.98s/it]

Closest words to the center word furigana: ['furigana', 'hiragana', 'kanji', 'katakana', 'diacritic', 'perfective', 'digraphs', 'approximant', 'phonemes', 'devanagari']

Epoch 2/2:  54%|█████▎    | 18399/34343 [4:24:36<1:44:15,  2.55it/s] 

Epoch 2 Batch 18400 loss: 2.1299948692321777

Epoch 2/2:  54%|█████▎    | 18400/34343 [4:24:37<2:34:48,  1.72it/s]

Closest words to the center word tel: ['tel', 'allafrica', 'icrm', 'rajons', 'coquitlam', 'universidade', 'ifad', 'unmibh', 'opcw', 'stadtbahn']

Epoch 2/2:  54%|█████▍    | 18499/34343 [4:25:56<1:21:16,  3.25it/s] 

Epoch 2 Batch 18500 loss: 2.0718226432800293

Epoch 2/2:  54%|█████▍    | 18500/34343 [4:25:57<1:49:22,  2.41it/s]

Closest words to the center word against: ['against', 'ifrcs', 'unido', 'unmibh', 'prosecute', 'mughals', 'slobodan', 'libel', 'wco', 'invading']

Epoch 2/2:  54%|█████▍    | 18599/34343 [4:27:23<59:59,  4.37it/s]   

Epoch 2 Batch 18600 loss: 2.0688459873199463

Epoch 2/2:  54%|█████▍    | 18600/34343 [4:27:24<1:37:17,  2.70it/s]

Closest words to the center word after: ['after', 'before', 'shortly', 'lasted', 'afterwards', 'afterward', 'thereafter', 'rafik', 'aegisthus', 'abruptly']

Epoch 2/2:  54%|█████▍    | 18699/34343 [4:29:09<6:09:38,  1.42s/it] 

Epoch 2 Batch 18700 loss: 2.076733112335205

Epoch 2/2:  54%|█████▍    | 18700/34343 [4:29:10<5:07:04,  1.18s/it]

Closest words to the center word than: ['than', 'lifes', 'less', 'heavier', 'cheaper', 'sweeter', 'denser', 'thicker', 'considerably', 'faster']

Epoch 2/2:  55%|█████▍    | 18799/34343 [4:30:21<1:20:14,  3.23it/s] 

Epoch 2 Batch 18800 loss: 2.106260061264038

Epoch 2/2:  55%|█████▍    | 18800/34343 [4:30:22<1:43:31,  2.50it/s]

Closest words to the center word has: ['has', 'hasn', 'ifrcs', 'have', 'had', 'pluriform', 'hadn', 'ifad', 'ifc', 'enjoys']

Epoch 2/2:  55%|█████▌    | 18899/34343 [4:31:32<57:20,  4.49it/s]   

Epoch 2 Batch 18900 loss: 2.090738296508789

Epoch 2/2:  55%|█████▌    | 18900/34343 [4:31:32<1:28:23,  2.91it/s]

Closest words to the center word actor: ['actor', 'actress', 'footballer', 'comedian', 'cricketer', 'pngimage', 'comedienne', 'singer', 'dramatist', 'laureate']

Epoch 2/2:  55%|█████▌    | 18999/34343 [4:32:42<54:13,  4.72it/s]   

Epoch 2 Batch 19000 loss: 2.079223394393921

Epoch 2/2:  55%|█████▌    | 19000/34343 [4:32:43<1:27:56,  2.91it/s]

Closest words to the center word employed: ['employed', 'dwt', 'used', 'trained', 'practiced', 'conscripted', 'invented', 'appointed', 'oxidizing', 'exploited']

Epoch 2/2:  56%|█████▌    | 19099/34343 [4:34:08<4:29:40,  1.06s/it] 

Epoch 2 Batch 19100 loss: 2.087118625640869

Epoch 2/2:  56%|█████▌    | 19100/34343 [4:34:09<4:03:16,  1.04it/s]

Closest words to the center word primer: ['primer', 'ifrcs', 'findable', 'ifad', 'nonsignatory', 'ifc', 'erowid', 'rajonas', 'tutorial', 'radiology']

Epoch 2/2:  56%|█████▌    | 19199/34343 [4:35:18<1:11:42,  3.52it/s] 

Epoch 2 Batch 19200 loss: 2.094693183898926

Epoch 2/2:  56%|█████▌    | 19200/34343 [4:35:19<1:39:26,  2.54it/s]

Closest words to the center word also: ['also', 'disambiguation', 'wmo', 'wftu', 'wtoo', 'yird', 'ifrcs', 'wcl', 'unido', 'ifad']

Epoch 2/2:  56%|█████▌    | 19299/34343 [4:36:27<54:34,  4.59it/s]   

Epoch 2 Batch 19300 loss: 2.088292121887207

Epoch 2/2:  56%|█████▌    | 19300/34343 [4:36:28<1:26:57,  2.88it/s]

Closest words to the center word words: ['words', 'pronouns', 'nouns', 'verbs', 'cognates', 'adjectives', 'morphemes', 'digraphs', 'phonemes', 'consonants']

Epoch 2/2:  56%|█████▋    | 19399/34343 [4:37:40<53:26,  4.66it/s]   

Epoch 2 Batch 19400 loss: 2.1207566261291504

Epoch 2/2:  56%|█████▋    | 19400/34343 [4:37:56<20:54:01,  5.04s/it]

Closest words to the center word usually: ['usually', 'typically', 'calcite', 'intravenous', 'circularly', 'hydride', 'often', 'unicellular', 'normally', 'waterborne']

Epoch 2/2:  57%|█████▋    | 19499/34343 [4:39:05<3:13:23,  1.28it/s] 

Epoch 2 Batch 19500 loss: 2.0922622680664062

Epoch 2/2:  57%|█████▋    | 19500/34343 [4:39:05<3:06:51,  1.32it/s]

Closest words to the center word considerable: ['considerable', 'substantial', 'tremendous', 'significant', 'pluriform', 'decreased', 'huge', 'enormous', 'disproportionate', 'greater']

Epoch 2/2:  57%|█████▋    | 19599/34343 [4:40:16<1:02:09,  3.95it/s] 

Epoch 2 Batch 19600 loss: 2.075504779815674

Epoch 2/2:  57%|█████▋    | 19600/34343 [4:40:17<1:48:26,  2.27it/s]

Closest words to the center word groups: ['groups', 'ethnic', 'abelian', 'homomorphisms', 'bamar', 'denominations', 'mestizo', 'amerindian', 'malayo', 'group']

Epoch 2/2:  57%|█████▋    | 19699/34343 [4:41:26<52:01,  4.69it/s]   

Epoch 2 Batch 19700 loss: 2.104104518890381

Epoch 2/2:  57%|█████▋    | 19700/34343 [4:41:27<1:27:50,  2.78it/s]

Closest words to the center word trademark: ['trademark', 'license', 'copyleft', 'infringement', 'copyright', 'abandonware', 'gpl', 'kazaa', 'netbsd', 'fsf']

Epoch 2/2:  58%|█████▊    | 19799/34343 [4:42:54<19:40:54,  4.87s/it]

Epoch 2 Batch 19800 loss: 2.1101269721984863

Epoch 2/2:  58%|█████▊    | 19800/34343 [4:42:55<14:32:43,  3.60s/it]

Closest words to the center word sword: ['sword', 'tamarin', 'spear', 'excalibur', 'diomedes', 'dresses', 'lyre', 'hilt', 'wakizashi', 'uther']

Epoch 2/2:  58%|█████▊    | 19899/34343 [4:44:04<2:03:31,  1.95it/s] 

Epoch 2 Batch 19900 loss: 2.1287410259246826

Epoch 2/2:  58%|█████▊    | 19900/34343 [4:44:04<2:16:28,  1.76it/s]

Closest words to the center word three: ['gwh', 'grt', 'pngimage', 'twh', 'kwh', 'cyg', 'mjs', 'hbk', 'sfg', 'lup']

Epoch 2/2:  58%|█████▊    | 19999/34343 [4:45:15<54:44,  4.37it/s]   

Epoch 2 Batch 20000 loss: 2.0664329528808594

Epoch 2/2:  58%|█████▊    | 20000/34343 [4:45:16<1:26:05,  2.78it/s]

Closest words to the center word ironic: ['ironic', 'epistolary', 'antithesis', 'sarcastic', 'deprecating', 'falsifiability', 'monistic', 'bhagavad', 'arguable', 'retelling']

Epoch 2/2:  59%|█████▊    | 20099/34343 [4:46:28<50:25,  4.71it/s]   

Epoch 2 Batch 20100 loss: 2.0766873359680176

Epoch 2/2:  59%|█████▊    | 20100/34343 [4:46:29<1:19:26,  2.99it/s]

Closest words to the center word following: ['following', 'gregorian', 'leap', 'rafik', 'fao', 'bloodiest', 'williamite', 'listing', 'preceding', 'pseudocode']

Epoch 2/2:  59%|█████▉    | 20199/34343 [4:47:57<10:08:52,  2.58s/it]

Epoch 2 Batch 20200 loss: 2.061518669128418

Epoch 2/2:  59%|█████▉    | 20200/34343 [4:47:57<7:51:39,  2.00s/it] 

Closest words to the center word links: ['links', 'webelements', 'wmo', 'icrm', 'external', 'allafrica', 'nonsignatory', 'ifrcs', 'wtoo', 'ifc']

Epoch 2/2:  59%|█████▉    | 20299/34343 [4:49:07<1:24:28,  2.77it/s] 

Epoch 2 Batch 20300 loss: 2.0836377143859863

Epoch 2/2:  59%|█████▉    | 20300/34343 [4:49:08<1:45:04,  2.23it/s]

Closest words to the center word malaysia: ['malaysia', 'tuvalu', 'allafrica', 'rupee', 'nepal', 'dinar', 'bissau', 'swaziland', 'tajikistan', 'laos']

Epoch 2/2:  59%|█████▉    | 20399/34343 [4:50:18<49:53,  4.66it/s]   

Epoch 2 Batch 20400 loss: 2.1162683963775635

Epoch 2/2:  59%|█████▉    | 20400/34343 [4:50:19<1:24:02,  2.77it/s]

Closest words to the center word ten: ['ten', 'grt', 'sixty', 'forty', 'fifteen', 'twelve', 'thirty', 'totaling', 'hundred', 'eleven']

Epoch 2/2:  60%|█████▉    | 20499/34343 [4:51:28<48:52,  4.72it/s]   

Epoch 2 Batch 20500 loss: 2.0981311798095703

Epoch 2/2:  60%|█████▉    | 20500/34343 [4:51:29<1:22:32,  2.79it/s]

Closest words to the center word external: ['webelements', 'icrm', 'external', 'links', 'ifc', 'ifrcs', 'nonsignatory', 'ibrd', 'wmo', 'iom']

Epoch 2/2:  60%|█████▉    | 20599/34343 [4:52:57<5:29:22,  1.44s/it] 

Epoch 2 Batch 20600 loss: 2.108396053314209

Epoch 2/2:  60%|█████▉    | 20600/34343 [4:52:58<4:41:32,  1.23s/it]

Closest words to the center word u: ['u', 'poz', 'polskiej', 'codepoint', 'rzeczypospolitej', 'rightarrow', 'forall', 'textrm', 'dz', 'ustawa']

Epoch 2/2:  60%|██████    | 20699/34343 [4:54:06<53:51,  4.22it/s]   

Epoch 2 Batch 20700 loss: 2.1024155616760254

Epoch 2/2:  60%|██████    | 20700/34343 [4:54:07<1:31:43,  2.48it/s]

Closest words to the center word version: ['version', 'ipf', 'versions', 'bwv', 'edition', 'imac', 'remastered', 'athlon', 'cccc', 'wikisource']

Epoch 2/2:  61%|██████    | 20799/34343 [4:55:16<50:02,  4.51it/s]   

Epoch 2 Batch 20800 loss: 2.1139543056488037

Epoch 2/2:  61%|██████    | 20800/34343 [4:55:17<1:18:53,  2.86it/s]

Closest words to the center word new: ['new', 'york', 'schuster', 'ny', 'giroux', 'farrar', 'ticker', 'bangor', 'dunedin', 'straus']

Epoch 2/2:  61%|██████    | 20899/34343 [4:56:26<47:12,  4.75it/s]   

Epoch 2 Batch 20900 loss: 2.1296777725219727

Epoch 2/2:  61%|██████    | 20900/34343 [4:56:26<1:18:38,  2.85it/s]

Closest words to the center word allegedly: ['unido', 'wco', 'ifrcs', 'unmibh', 'allegedly', 'hazmi', 'unctad', 'wftu', 'zarqawi', 'aegisthus']

Epoch 2/2:  61%|██████    | 20999/34343 [4:57:52<1:30:38,  2.45it/s] 

Epoch 2 Batch 21000 loss: 2.129845142364502

Epoch 2/2:  61%|██████    | 21000/34343 [4:57:52<1:47:01,  2.08it/s]

Closest words to the center word black: ['callithrix', 'black', 'leontopithecus', 'tamarin', 'marmoset', 'saguinus', 'eulemur', 'dasyprocta', 'mico', 'capuchin']

Epoch 2/2:  61%|██████▏   | 21099/34343 [4:59:03<51:58,  4.25it/s]   

Epoch 2 Batch 21100 loss: 2.1261391639709473

Epoch 2/2:  61%|██████▏   | 21100/34343 [4:59:03<1:19:53,  2.76it/s]

Closest words to the center word to: ['to', 'findable', 'inability', 'triadic', 'prev', 'unable', 'able', 'suspend', 'willing', 'unwilling']

Epoch 2/2:  62%|██████▏   | 21199/34343 [5:00:12<47:33,  4.61it/s]   

Epoch 2 Batch 21200 loss: 2.128490686416626

Epoch 2/2:  62%|██████▏   | 21200/34343 [5:00:13<1:19:14,  2.76it/s]

Closest words to the center word of: ['of', 'ifrcs', 'wtoo', 'nazarene', 'rajonas', 'ifad', 'akan', 'bentheim', 'mycological', 'icrm']

Epoch 2/2:  62%|██████▏   | 21299/34343 [5:01:38<6:39:27,  1.84s/it] 

Epoch 2 Batch 21300 loss: 2.0650970935821533

Epoch 2/2:  62%|██████▏   | 21300/34343 [5:01:39<5:25:22,  1.50s/it]

Closest words to the center word participated: ['participated', 'starred', 'competed', 'culminated', 'fought', 'embroiled', 'resulted', 'engaged', 'campaigned', 'wco']

Epoch 2/2:  62%|██████▏   | 21399/34343 [5:02:47<1:16:45,  2.81it/s] 

Epoch 2 Batch 21400 loss: 2.1159958839416504

Epoch 2/2:  62%|██████▏   | 21400/34343 [5:02:48<1:37:51,  2.20it/s]

Closest words to the center word lively: ['lively', 'tikka', 'nightlife', 'natured', 'flavoured', 'morbid', 'thriving', 'sentimental', 'vibrant', 'twinned']

Epoch 2/2:  63%|██████▎   | 21499/34343 [5:03:57<47:25,  4.51it/s]   

Epoch 2 Batch 21500 loss: 2.096832513809204

Epoch 2/2:  63%|██████▎   | 21500/34343 [5:03:58<1:17:58,  2.75it/s]

Closest words to the center word work: ['work', 'rediscovery', 'doctorate', 'goethe', 'phenomenology', 'virtuosity', 'insights', 'husserl', 'contribution', 'dissertation']

Epoch 2/2:  63%|██████▎   | 21599/34343 [5:05:09<45:46,  4.64it/s]   

Epoch 2 Batch 21600 loss: 2.1141245365142822

Epoch 2/2:  63%|██████▎   | 21600/34343 [5:05:10<1:14:38,  2.85it/s]

Closest words to the center word celebrated: ['celebrated', 'feast', 'commemorated', 'solstice', 'canonized', 'beltane', 'liturgics', 'imbolc', 'tishri', 'midsummer']

Epoch 2/2:  63%|██████▎   | 21699/34343 [5:06:36<5:01:08,  1.43s/it] 

Epoch 2 Batch 21700 loss: 2.095954418182373

Epoch 2/2:  63%|██████▎   | 21700/34343 [5:06:37<4:11:05,  1.19s/it]

Closest words to the center word magnificent: ['magnificent', 'borghese', 'yoannis', 'sistine', 'piazza', 'basilica', 'piccadilly', 'sarcophagus', 'erected', 'cathedral']

Epoch 2/2:  63%|██████▎   | 21799/34343 [5:07:47<58:40,  3.56it/s]   

Epoch 2 Batch 21800 loss: 2.1025989055633545

Epoch 2/2:  63%|██████▎   | 21800/34343 [5:07:48<1:19:41,  2.62it/s]

Closest words to the center word tied: ['tied', 'dealt', 'homeomorphic', 'aligned', 'connected', 'isomorphic', 'awarded', 'reconcile', 'replayed', 'rewarded']

Epoch 2/2:  64%|██████▍   | 21899/34343 [5:08:57<45:32,  4.55it/s]   

Epoch 2 Batch 21900 loss: 2.114682912826538

Epoch 2/2:  64%|██████▍   | 21900/34343 [5:08:58<1:12:55,  2.84it/s]

Closest words to the center word narrowly: ['narrowly', 'landslide', 'bingu', 'bolsheviks', 'kamenev', 'mwai', 'karzai', 'reelected', 'votes', 'fianna']

Epoch 2/2:  64%|██████▍   | 21999/34343 [5:10:11<44:19,  4.64it/s]   

Epoch 2 Batch 22000 loss: 2.0741190910339355

Epoch 2/2:  64%|██████▍   | 22000/34343 [5:10:12<1:10:52,  2.90it/s]

Closest words to the center word cost: ['cost', 'kwh', 'gwh', 'throughput', 'expenditures', 'costs', 'bandwidth', 'speeds', 'twh', 'latency']

Epoch 2/2:  64%|██████▍   | 22099/34343 [5:11:39<2:40:51,  1.27it/s] 

Epoch 2 Batch 22100 loss: 2.089301586151123

Epoch 2/2:  64%|██████▍   | 22100/34343 [5:11:39<2:34:29,  1.32it/s]

Closest words to the center word received: ['received', 'honorary', 'graduated', 'earned', 'doctorate', 'garnered', 'awarded', 'won', 'gained', 'attended']

Epoch 2/2:  65%|██████▍   | 22199/34343 [5:12:50<49:29,  4.09it/s]   

Epoch 2 Batch 22200 loss: 2.1115972995758057

Epoch 2/2:  65%|██████▍   | 22200/34343 [5:12:50<1:17:49,  2.60it/s]

Closest words to the center word feller: ['leiserson', 'cormen', 'cricketer', 'elke', 'bckgr', 'peckinpah', 'astaire', 'footballer', 'cullen', 'feller']

Epoch 2/2:  65%|██████▍   | 22299/34343 [5:14:05<44:22,  4.52it/s]   

Epoch 2 Batch 22300 loss: 2.0672473907470703

Epoch 2/2:  65%|██████▍   | 22300/34343 [5:14:05<1:12:57,  2.75it/s]

Closest words to the center word love: ['love', 'totoro', 'lust', 'nnhilde', 'passionate', 'loves', 'vanity', 'thy', 'unrequited', 'wagoner']

Epoch 2/2:  65%|██████▌   | 22399/34343 [5:15:32<17:31:29,  5.28s/it]

Epoch 2 Batch 22400 loss: 2.09804368019104

Epoch 2/2:  65%|██████▌   | 22400/34343 [5:15:33<13:00:50,  3.92s/it]

Closest words to the center word one: ['gwh', 'kwh', 'pngimage', 'hbk', 'grt', 'mjw', 'twh', 'mjs', 'cyg', 'sfg']

Epoch 2/2:  66%|██████▌   | 22499/34343 [5:16:43<1:35:35,  2.07it/s] 

Epoch 2 Batch 22500 loss: 2.1084938049316406

Epoch 2/2:  66%|██████▌   | 22500/34343 [5:16:44<1:54:48,  1.72it/s]

Closest words to the center word suspicious: ['suspicious', 'wary', 'intellectually', 'underestimated', 'consummated', 'exacerbated', 'harshly', 'overdose', 'aware', 'incapable']

Epoch 2/2:  66%|██████▌   | 22599/34343 [5:18:21<59:02,  3.32it/s]   

Epoch 2 Batch 22600 loss: 2.0844850540161133

Epoch 2/2:  66%|██████▌   | 22600/34343 [5:18:21<1:25:20,  2.29it/s]

Closest words to the center word change: ['change', 'desertification', 'enthalpy', 'changes', 'hazardous', 'variability', 'biodiversity', 'findable', 'pollutants', 'drift']

Epoch 2/2:  66%|██████▌   | 22699/34343 [5:19:58<51:54,  3.74it/s]   

Epoch 2 Batch 22700 loss: 2.1036622524261475

Epoch 2/2:  66%|██████▌   | 22700/34343 [5:19:58<1:17:43,  2.50it/s]

Closest words to the center word said: ['said', 'replied', 'rumoured', 'moneo', 'quipped', 'glad', 'hafsa', 'told', 'remarried', 'believed']

Epoch 2/2:  66%|██████▋   | 22799/34343 [5:22:12<10:56:27,  3.41s/it]

Epoch 2 Batch 22800 loss: 2.102323293685913

Epoch 2/2:  66%|██████▋   | 22800/34343 [5:22:13<8:22:27,  2.61s/it] 

Closest words to the center word heretic: ['heretic', 'divinely', 'patriarch', 'habakkuk', 'forgiveness', 'prophet', 'absolution', 'marcion', 'arius', 'pius']

Epoch 2/2:  67%|██████▋   | 22899/34343 [5:23:48<57:46,  3.30it/s]   

Epoch 2 Batch 22900 loss: 2.1062870025634766

Epoch 2/2:  67%|██████▋   | 22900/34343 [5:23:49<1:28:42,  2.15it/s]

Closest words to the center word by: ['by', 'ifrcs', 'leiserson', 'maaouya', 'icrm', 'unido', 'ifc', 'ifad', 'rajonas', 'nonsignatory']

Epoch 2/2:  67%|██████▋   | 22999/34343 [5:25:26<56:11,  3.37it/s]   

Epoch 2 Batch 23000 loss: 2.136038303375244

Epoch 2/2:  67%|██████▋   | 23000/34343 [5:25:27<1:20:00,  2.36it/s]

Closest words to the center word computers: ['computers', 'cpus', 'microcomputer', 'consoles', 'xt', 'macs', 'peripherals', 'scsi', 'microprocessors', 'powerpc']

Epoch 2/2:  67%|██████▋   | 23099/34343 [5:27:11<55:20,  3.39it/s]   

Epoch 2 Batch 23100 loss: 2.081343173980713

Epoch 2/2:  67%|██████▋   | 23100/34343 [5:27:11<1:20:10,  2.34it/s]

Closest words to the center word accuses: ['accuses', 'wco', 'unido', 'wftu', 'unmibh', 'wmo', 'unctad', 'weu', 'wtoo', 'wcl']

Epoch 2/2:  68%|██████▊   | 23199/34343 [5:29:22<2:39:13,  1.17it/s] 

Epoch 2 Batch 23200 loss: 2.132946491241455

Epoch 2/2:  68%|██████▊   | 23200/34343 [5:29:23<2:33:21,  1.21it/s]

Closest words to the center word in: ['in', 'kwh', 'gwh', 'annexes', 'nfc', 'rajonas', 'sfg', 'wct', 'births', 'anh']

Epoch 2/2:  68%|██████▊   | 23299/34343 [5:31:01<1:22:42,  2.23it/s] 

Epoch 2 Batch 23300 loss: 2.139177083969116

Epoch 2/2:  68%|██████▊   | 23300/34343 [5:31:02<1:44:21,  1.76it/s]

Closest words to the center word graded: ['dolomite', 'beets', 'cassava', 'montane', 'graded', 'lifes', 'broadleaf', 'humid', 'oxides', 'soluble']

Epoch 2/2:  68%|██████▊   | 23399/34343 [5:32:48<44:29,  4.10it/s]   

Epoch 2 Batch 23400 loss: 2.080181121826172

Epoch 2/2:  68%|██████▊   | 23400/34343 [5:32:49<1:17:34,  2.35it/s]

Closest words to the center word god: ['god', 'yahweh', 'yhwh', 'brahman', 'allah', 'omnipotent', 'vishnu', 'almighty', 'forgiveness', 'incarnate']

Epoch 2/2:  68%|██████▊   | 23499/34343 [5:34:53<15:37:54,  5.19s/it]

Epoch 2 Batch 23500 loss: 2.0523595809936523

Epoch 2/2:  68%|██████▊   | 23500/34343 [5:34:54<11:53:46,  3.95s/it]

Closest words to the center word spots: ['spots', 'leontopithecus', 'isosceles', 'mantled', 'saguinus', 'tamarin', 'tailed', 'callithrix', 'grt', 'hexagonal']

Epoch 2/2:  69%|██████▊   | 23599/34343 [5:36:42<2:13:40,  1.34it/s] 

Epoch 2 Batch 23600 loss: 2.092559814453125

Epoch 2/2:  69%|██████▊   | 23600/34343 [5:36:43<2:14:33,  1.33it/s]

Closest words to the center word need: ['need', 'intend', 'pluperfect', 'newnode', 'incentive', 'posteriori', 'want', 'factum', 'require', 'disprove']

Epoch 2/2:  69%|██████▉   | 23699/34343 [5:38:25<1:01:12,  2.90it/s] 

Epoch 2 Batch 23700 loss: 2.08243465423584

Epoch 2/2:  69%|██████▉   | 23700/34343 [5:38:26<1:51:52,  1.59it/s]

Closest words to the center word electrolysis: ['electrolysis', 'halides', 'hydride', 'alkoxide', 'anode', 'nitric', 'phosphorylation', 'cathode', 'pyruvate', 'carbonyl']

Epoch 2/2:  69%|██████▉   | 23799/34343 [5:40:05<52:40,  3.34it/s]   

Epoch 2 Batch 23800 loss: 2.127439498901367

Epoch 2/2:  69%|██████▉   | 23800/34343 [5:40:05<1:14:04,  2.37it/s]

Closest words to the center word benz: ['benz', 'daimler', 'audi', 'kwh', 'mercedes', 'gwh', 'bugatti', 'bmw', 'aston', 'twh']

Epoch 2/2:  70%|██████▉   | 23899/34343 [5:42:18<9:17:21,  3.20s/it] 

Epoch 2 Batch 23900 loss: 2.15342116355896

Epoch 2/2:  70%|██████▉   | 23900/34343 [5:42:18<7:08:55,  2.46s/it]

Closest words to the center word bhutan: ['bhutan', 'allafrica', 'rupee', 'kitts', 'tuvalu', 'swaziland', 'dinar', 'escudo', 'tajikistan', 'chungcheong']

Epoch 2/2:  70%|██████▉   | 23999/34343 [5:44:02<1:31:50,  1.88it/s] 

Epoch 2 Batch 24000 loss: 2.0733962059020996

Epoch 2/2:  70%|██████▉   | 24000/34343 [5:44:03<1:42:34,  1.68it/s]

Closest words to the center word flutes: ['flutes', 'tremolo', 'basses', 'cellos', 'harmonicas', 'woodwind', 'clarinets', 'clarinet', 'diatonic', 'contrabass']

Epoch 2/2:  70%|███████   | 24099/34343 [5:45:36<46:28,  3.67it/s]   

Epoch 2 Batch 24100 loss: 2.1080574989318848

Epoch 2/2:  70%|███████   | 24100/34343 [5:45:36<1:17:58,  2.19it/s]

Closest words to the center word weaknesses: ['weaknesses', 'irritability', 'strengths', 'eyesight', 'melee', 'flaws', 'assumptions', 'tardive', 'inhibitors', 'myocardial']

Epoch 2/2:  70%|███████   | 24199/34343 [5:47:17<42:58,  3.93it/s]   

Epoch 2 Batch 24200 loss: 2.142861843109131

Epoch 2/2:  70%|███████   | 24200/34343 [5:47:18<1:38:53,  1.71it/s]

Closest words to the center word scroll: ['scroll', 'polycarbonate', 'fretboard', 'platters', 'socket', 'rotor', 'cylindrical', 'keypad', 'fret', 'tubing']

Epoch 2/2:  71%|███████   | 24299/34343 [5:49:17<6:47:51,  2.44s/it] 

Epoch 2 Batch 24300 loss: 2.1393821239471436

Epoch 2/2:  71%|███████   | 24300/34343 [5:49:18<5:29:12,  1.97s/it]

Closest words to the center word states: ['states', 'federated', 'emirates', 'micronesia', 'united', 'tuvalu', 'nations', 'seceded', 'oecs', 'eapc']

Epoch 2/2:  71%|███████   | 24399/34343 [5:50:51<1:09:01,  2.40it/s] 

Epoch 2 Batch 24400 loss: 2.1135880947113037

Epoch 2/2:  71%|███████   | 24400/34343 [5:50:52<1:25:38,  1.94it/s]

Closest words to the center word under: ['under', 'auspices', 'gpl', 'mamluk', 'durrani', 'mughals', 'fatimid', 'salih', 'maaouya', 'tokugawa']

Epoch 2/2:  71%|███████▏  | 24499/34343 [5:52:26<49:12,  3.33it/s]   

Epoch 2 Batch 24500 loss: 2.0778377056121826

Epoch 2/2:  71%|███████▏  | 24500/34343 [5:52:27<1:10:20,  2.33it/s]

Closest words to the center word changed: ['changed', 'shifted', 'reverted', 'findable', 'faded', 'altered', 'ratified', 'traced', 'renamed', 'recovered']

Epoch 2/2:  72%|███████▏  | 24599/34343 [5:54:01<48:12,  3.37it/s]   

Epoch 2 Batch 24600 loss: 2.0834269523620605

Epoch 2/2:  72%|███████▏  | 24600/34343 [5:54:02<1:11:41,  2.27it/s]

Closest words to the center word brown: ['callithrix', 'eulemur', 'brown', 'leontopithecus', 'tamias', 'tamarin', 'aquilegia', 'mesoplodon', 'haliotis', 'beaked']

Epoch 2/2:  72%|███████▏  | 24699/34343 [5:56:09<4:14:31,  1.58s/it] 

Epoch 2 Batch 24700 loss: 2.1234207153320312

Epoch 2/2:  72%|███████▏  | 24700/34343 [5:56:09<3:39:53,  1.37s/it]

Closest words to the center word had: ['had', 'hadn', 'fathered', 'aegisthus', 'thyestes', 'hath', 'hasn', 'clytemnestra', 'has', 'betrothed']

Epoch 2/2:  72%|███████▏  | 24799/34343 [5:57:42<55:55,  2.84it/s]   

Epoch 2 Batch 24800 loss: 2.1185073852539062

Epoch 2/2:  72%|███████▏  | 24800/34343 [5:57:42<1:10:05,  2.27it/s]

Closest words to the center word card: ['card', 'cards', 'nfc', 'debit', 'afc', 'betting', 'playoffs', 'bobble', 'newnode', 'gba']

Epoch 2/2:  73%|███████▎  | 24899/34343 [5:59:12<42:06,  3.74it/s]   

Epoch 2 Batch 24900 loss: 2.102382183074951

Epoch 2/2:  73%|███████▎  | 24900/34343 [5:59:13<1:05:01,  2.42it/s]

Closest words to the center word in: ['in', 'kwh', 'gwh', 'anh', 'annexes', 'births', 'throughout', 'anterselva', 'twh', 'availabilitymales']

Epoch 2/2:  73%|███████▎  | 24999/34343 [6:00:43<1:32:59,  1.67it/s] 

Epoch 2 Batch 25000 loss: 2.1095993518829346

Epoch 2/2:  73%|███████▎  | 25000/34343 [6:01:05<17:56:32,  6.91s/it]

Closest words to the center word years: ['years', 'grt', 'decades', 'months', 'servicemales', 'weeks', 'totaling', 'availabilitymales', 'injures', 'lifes']

Epoch 2/2:  73%|███████▎  | 25099/34343 [6:02:33<1:52:50,  1.37it/s] 

Epoch 2 Batch 25100 loss: 2.1300365924835205

Epoch 2/2:  73%|███████▎  | 25100/34343 [6:02:34<1:52:17,  1.37it/s]

Closest words to the center word hscsd: ['bbbb', 'nonsignatory', 'iom', 'wavelet', 'cccc', 'asynchronous', 'ifrcs', 'opcw', 'bilinear', 'sys']

Epoch 2/2:  73%|███████▎  | 25199/34343 [6:04:00<43:30,  3.50it/s]   

Epoch 2 Batch 25200 loss: 2.0810372829437256

Epoch 2/2:  73%|███████▎  | 25200/34343 [6:04:02<1:36:25,  1.58it/s]

Closest words to the center word are: ['are', 'were', 'aren', 'unicellular', 'protists', 'celled', 'differ', 'rectangles', 'consist', 'solutes']

Epoch 2/2:  74%|███████▎  | 25299/34343 [6:05:35<43:32,  3.46it/s]   

Epoch 2 Batch 25300 loss: 2.08467960357666

Epoch 2/2:  74%|███████▎  | 25300/34343 [6:05:37<1:24:27,  1.78it/s]

Closest words to the center word checking: ['checking', 'chaining', 'newnode', 'prev', 'asynchronous', 'firstnode', 'findable', 'avl', 'pointers', 'macros']

Epoch 2/2:  74%|███████▍  | 25399/34343 [6:07:31<13:00:05,  5.23s/it]

Epoch 2 Batch 25400 loss: 2.134082794189453

Epoch 2/2:  74%|███████▍  | 25400/34343 [6:07:32<9:37:47,  3.88s/it] 

Closest words to the center word constitution: ['constitution', 'bicameral', 'ratified', 'unicameral', 'legislature', 'referendum', 'constitutional', 'amendment', 'monarchy', 'eldr']

Epoch 2/2:  74%|███████▍  | 25499/34343 [6:09:05<1:06:59,  2.20it/s] 

Epoch 2 Batch 25500 loss: 2.1233327388763428

Epoch 2/2:  74%|███████▍  | 25500/34343 [6:09:06<1:57:29,  1.25it/s]

Closest words to the center word project: ['nonsignatory', 'ifrcs', 'ifad', 'project', 'ifc', 'iom', 'sourceforge', 'idb', 'looksmart', 'gnat']

Epoch 2/2:  75%|███████▍  | 25599/34343 [6:10:40<44:31,  3.27it/s]   

Epoch 2 Batch 25600 loss: 2.1135940551757812

Epoch 2/2:  75%|███████▍  | 25600/34343 [6:10:40<1:05:59,  2.21it/s]

Closest words to the center word valid: ['valid', 'bijective', 'provable', 'metrizable', 'ponens', 'refutable', 'unital', 'differentiable', 'surjective', 'enumerable']

Epoch 2/2:  75%|███████▍  | 25699/34343 [6:12:17<39:27,  3.65it/s]   

Epoch 2 Batch 25700 loss: 2.1290810108184814

Epoch 2/2:  75%|███████▍  | 25700/34343 [6:12:18<1:14:39,  1.93it/s]

Closest words to the center word bobby: ['finalist', 'bobby', 'kreutzmann', 'satchel', 'bruins', 'footballer', 'bourque', 'lesh', 'foxx', 'darrell']

Epoch 2/2:  75%|███████▌  | 25799/34343 [6:14:11<2:23:53,  1.01s/it] 

Epoch 2 Batch 25800 loss: 2.092529535293579

Epoch 2/2:  75%|███████▌  | 25800/34343 [6:14:11<2:12:11,  1.08it/s]

Closest words to the center word a: ['a', 'agave', 'pluriform', 'globicephala', 'lemur', 'keying', 'insubstantial', 'groupoid', 'newnode', 'callithrix']

Epoch 2/2:  75%|███████▌  | 25899/34343 [6:15:40<43:13,  3.26it/s]   

Epoch 2 Batch 25900 loss: 2.114318370819092

Epoch 2/2:  75%|███████▌  | 25900/34343 [6:15:41<1:00:17,  2.33it/s]

Closest words to the center word they: ['they', 'capybaras', 'we', 'you', 'liars', 'aediles', 'gesserit', 'feeders', 'matres', 'findable']

Epoch 2/2:  76%|███████▌  | 25999/34343 [6:17:15<35:20,  3.94it/s]   

Epoch 2 Batch 26000 loss: 2.0930535793304443

Epoch 2/2:  76%|███████▌  | 26000/34343 [6:17:16<52:30,  2.65it/s]

Closest words to the center word ethics: ['ethics', 'cognitivism', 'egoism', 'intuitionism', 'ethical', 'jurisprudence', 'epistemology', 'reductionism', 'objectivism', 'relativism']

Epoch 2/2:  76%|███████▌  | 26099/34343 [6:19:10<15:07:11,  6.60s/it]

Epoch 2 Batch 26100 loss: 2.076605796813965

Epoch 2/2:  76%|███████▌  | 26100/34343 [6:19:11<11:09:00,  4.87s/it]

Closest words to the center word june: ['june', 'gwh', 'january', 'april', 'july', 'february', 'august', 'november', 'kwh', 'december']

Epoch 2/2:  76%|███████▋  | 26199/34343 [6:20:58<2:05:28,  1.08it/s] 

Epoch 2 Batch 26200 loss: 2.129329204559326

Epoch 2/2:  76%|███████▋  | 26200/34343 [6:20:59<1:57:37,  1.15it/s]

Closest words to the center word studies: ['studies', 'ethologists', 'therapies', 'maxillofacial', 'subfields', 'anthropology', 'psychologists', 'informatics', 'critiques', 'physiology']

Epoch 2/2:  77%|███████▋  | 26299/34343 [6:22:31<47:39,  2.81it/s]   

Epoch 2 Batch 26300 loss: 2.129953384399414

Epoch 2/2:  77%|███████▋  | 26300/34343 [6:22:32<1:04:33,  2.08it/s]

Closest words to the center word the: ['ifrcs', 'pluriform', 'nonsignatory', 'tamarin', 'leontopithecus', 'ifad', 'the', 'ifc', 'iom', 'saguinus']

Epoch 2/2:  77%|███████▋  | 26399/34343 [6:23:59<33:27,  3.96it/s]   

Epoch 2 Batch 26400 loss: 2.112628936767578

Epoch 2/2:  77%|███████▋  | 26400/34343 [6:23:59<52:50,  2.51it/s]

Closest words to the center word told: ['told', 'swaim', 'tells', 'moneo', 'replied', 'complains', 'distraught', 'joked', 'afraid', 'hafsa']

Epoch 2/2:  77%|███████▋  | 26499/34343 [6:26:01<11:06:50,  5.10s/it]

Epoch 2 Batch 26500 loss: 2.110123872756958

Epoch 2/2:  77%|███████▋  | 26500/34343 [6:26:02<8:16:45,  3.80s/it] 

Closest words to the center word brigades: ['brigades', 'battalions', 'battalion', 'brigade', 'ifrcs', 'einsatzgruppen', 'waffen', 'regiment', 'peacekeeping', 'iom']

Epoch 2/2:  77%|███████▋  | 26599/34343 [6:27:37<1:26:41,  1.49it/s] 

Epoch 2 Batch 26600 loss: 2.069441795349121

Epoch 2/2:  77%|███████▋  | 26600/34343 [6:27:38<1:34:16,  1.37it/s]

Closest words to the center word mathematician: ['mathematician', 'physicist', 'astronomer', 'mineralogist', 'physiologist', 'zoologist', 'philosopher', 'xaver', 'neurologist', 'philologist']

Epoch 2/2:  78%|███████▊  | 26699/34343 [6:29:27<42:04,  3.03it/s]   

Epoch 2 Batch 26700 loss: 2.099580764770508

Epoch 2/2:  78%|███████▊  | 26700/34343 [6:29:27<57:48,  2.20it/s]

Closest words to the center word probability: ['probability', 'cardinality', 'polynomial', 'mathfrak', 'hamiltonian', 'polynomials', 'entropy', 'lebesgue', 'integrable', 'widehat']

Epoch 2/2:  78%|███████▊  | 26799/34343 [6:30:58<36:36,  3.44it/s]   

Epoch 2 Batch 26800 loss: 2.114670515060425

Epoch 2/2:  78%|███████▊  | 26800/34343 [6:30:59<54:35,  2.30it/s]

Closest words to the center word fact: ['fact', 'provable', 'epimenides', 'unsurprising', 'rooted', 'unital', 'testable', 'eukaryotes', 'believe', 'misconception']

Epoch 2/2:  78%|███████▊  | 26899/34343 [6:33:00<6:26:43,  3.12s/it] 

Epoch 2 Batch 26900 loss: 2.118748188018799

Epoch 2/2:  78%|███████▊  | 26900/34343 [6:33:01<5:10:20,  2.50s/it]

Closest words to the center word the: ['pluriform', 'ifrcs', 'leontopithecus', 'tamarin', 'the', 'nonsignatory', 'gangetic', 'saguinus', 'ifad', 'callithrix']

Epoch 2/2:  79%|███████▊  | 26999/34343 [6:34:20<44:27,  2.75it/s]   

Epoch 2 Batch 27000 loss: 2.0952489376068115

Epoch 2/2:  79%|███████▊  | 27000/34343 [6:34:21<54:40,  2.24it/s]

Closest words to the center word own: ['own', 'findable', 'respective', 'dealings', 'abilities', 'humility', 'egoism', 'countrymen', 'tireless', 'teg']

Epoch 2/2:  79%|███████▉  | 27099/34343 [6:35:32<27:25,  4.40it/s]   

Epoch 2 Batch 27100 loss: 2.113372802734375

Epoch 2/2:  79%|███████▉  | 27100/34343 [6:35:33<42:06,  2.87it/s]

Closest words to the center word left: ['left', 'otimes', 'arctan', 'right', 'cdot', 'vec', 'rangle', 'kx', 'cdots', 'operatorname']

Epoch 2/2:  79%|███████▉  | 27199/34343 [6:36:44<25:21,  4.69it/s]   

Epoch 2 Batch 27200 loss: 2.0929782390594482

Epoch 2/2:  79%|███████▉  | 27200/34343 [6:36:45<43:00,  2.77it/s]

Closest words to the center word bit: ['bit', 'kbit', 'pngimage', 'xt', 'mbit', 'megabyte', 'kb', 'bytes', 'kilobytes', 'mjs']

Epoch 2/2:  79%|███████▉  | 27299/34343 [6:38:12<2:47:55,  1.43s/it] 

Epoch 2 Batch 27300 loss: 2.0733766555786133

Epoch 2/2:  79%|███████▉  | 27300/34343 [6:38:13<2:30:21,  1.28s/it]

Closest words to the center word suitable: ['suitable', 'optimized', 'feedstock', 'computationally', 'unsuitable', 'transshipment', 'consuming', 'suited', 'glycogen', 'metrizable']

Epoch 2/2:  80%|███████▉  | 27399/34343 [6:39:52<50:22,  2.30it/s]   

Epoch 2 Batch 27400 loss: 2.10170841217041

Epoch 2/2:  80%|███████▉  | 27400/34343 [6:39:53<59:28,  1.95it/s]

Closest words to the center word cuttlefish: ['cuttlefish', 'beets', 'cassava', 'infraorder', 'lepidoptera', 'waterborne', 'secrete', 'lemurs', 'suborder', 'soybeans']

Epoch 2/2:  80%|████████  | 27499/34343 [6:41:49<30:21,  3.76it/s]   

Epoch 2 Batch 27500 loss: 2.0607285499572754

Epoch 2/2:  80%|████████  | 27500/34343 [6:41:50<46:29,  2.45it/s]

Closest words to the center word equatorial: ['equatorial', 'bissau', 'allafrica', 'verde', 'faso', 'gambia', 'landlocked', 'ivoire', 'escudo', 'brazzaville']

Epoch 2/2:  80%|████████  | 27599/34343 [6:43:28<29:33,  3.80it/s]   

Epoch 2 Batch 27600 loss: 2.1012046337127686

Epoch 2/2:  80%|████████  | 27600/34343 [6:43:29<45:18,  2.48it/s]

Closest words to the center word reliability: ['reliability', 'throughput', 'flexibility', 'usability', 'latency', 'efficiency', 'recompilation', 'compressive', 'bandwidth', 'tensile']

Epoch 2/2:  81%|████████  | 27699/34343 [6:45:40<2:37:38,  1.42s/it] 

Epoch 2 Batch 27700 loss: 2.1068646907806396

Epoch 2/2:  81%|████████  | 27700/34343 [6:45:40<2:13:49,  1.21s/it]

Closest words to the center word scientific: ['scientific', 'cognitivism', 'anthropological', 'empirical', 'anthropic', 'nativist', 'epistemological', 'falsifiable', 'psychoanalytic', 'pseudoscience']

Epoch 2/2:  81%|████████  | 27799/34343 [6:47:19<26:54,  4.05it/s]   

Epoch 2 Batch 27800 loss: 2.085338592529297

Epoch 2/2:  81%|████████  | 27800/34343 [6:47:20<42:33,  2.56it/s]

Closest words to the center word gospel: ['gospel', 'gospels', 'epistle', 'barnabas', 'epistles', 'ephesians', 'apostles', 'hebrews', 'jude', 'philippians']

Epoch 2/2:  81%|████████  | 27899/34343 [6:48:51<41:51,  2.57it/s]   

Epoch 2 Batch 27900 loss: 2.1243529319763184

Epoch 2/2:  81%|████████  | 27900/34343 [6:48:52<59:54,  1.79it/s]

Closest words to the center word centres: ['centres', 'iom', 'rajonas', 'centers', 'destinations', 'outskirts', 'tourist', 'shopping', 'wangfujing', 'ifrcs']

Epoch 2/2:  82%|████████▏ | 27999/34343 [6:50:48<14:27:43,  8.21s/it]

Epoch 2 Batch 28000 loss: 2.1093854904174805

Epoch 2/2:  82%|████████▏ | 28000/34343 [6:50:50<11:08:05,  6.32s/it]

Closest words to the center word sense: ['sense', 'cognitivism', 'physicalism', 'prescriptive', 'priori', 'soundness', 'reductionism', 'formalization', 'propositional', 'posteriori']

Epoch 2/2:  82%|████████▏ | 28099/34343 [6:52:21<38:21,  2.71it/s]   

Epoch 2 Batch 28100 loss: 2.108693838119507

Epoch 2/2:  82%|████████▏ | 28100/34343 [6:52:22<49:26,  2.10it/s]

Closest words to the center word rather: ['rather', 'thicker', 'fairer', 'semivowel', 'nonexistence', 'less', 'aesthetically', 'overly', 'sweeter', 'relying']

Epoch 2/2:  82%|████████▏ | 28199/34343 [6:53:54<27:50,  3.68it/s]   

Epoch 2 Batch 28200 loss: 2.105180501937866

Epoch 2/2:  82%|████████▏ | 28200/34343 [6:53:55<42:41,  2.40it/s]

Closest words to the center word two: ['gwh', 'grt', 'twh', 'kwh', 'hbk', 'cyg', 'sfg', 'pngimage', 'pmid', 'mjs']

Epoch 2/2:  82%|████████▏ | 28299/34343 [6:55:27<22:54,  4.40it/s]   

Epoch 2 Batch 28300 loss: 2.079437255859375

Epoch 2/2:  82%|████████▏ | 28300/34343 [6:55:28<37:24,  2.69it/s]

Closest words to the center word rejected: ['rejected', 'denounced', 'contradicted', 'persecuted', 'affirmed', 'ratified', 'criticized', 'dismissed', 'rejects', 'accepted']

Epoch 2/2:  83%|████████▎ | 28399/34343 [6:57:26<2:08:37,  1.30s/it] 

Epoch 2 Batch 28400 loss: 2.10756516456604

Epoch 2/2:  83%|████████▎ | 28400/34343 [6:57:27<2:09:23,  1.31s/it]

Closest words to the center word changed: ['changed', 'reverted', 'shifted', 'findable', 'altered', 'renamed', 'ported', 'abolished', 'faded', 'ratified']

Epoch 2/2:  83%|████████▎ | 28499/34343 [6:59:01<34:51,  2.79it/s]   

Epoch 2 Batch 28500 loss: 2.1471188068389893

Epoch 2/2:  83%|████████▎ | 28500/34343 [6:59:01<46:10,  2.11it/s]

Closest words to the center word republican: ['republican', 'pluriform', 'sinn', 'presidential', 'bingu', 'democrat', 'eldr', 'reelected', 'senator', 'sandinista']

Epoch 2/2:  83%|████████▎ | 28599/34343 [7:00:23<27:12,  3.52it/s]  

Epoch 2 Batch 28600 loss: 2.1108620166778564

Epoch 2/2:  83%|████████▎ | 28600/34343 [7:00:24<41:03,  2.33it/s]

Closest words to the center word george: ['leiserson', 'george', 'earl', 'cormen', 'marquess', 'eldridge', 'custis', 'rivest', 'frideric', 'hervey']

Epoch 2/2:  83%|████████▎ | 28671/34343 [7:01:28<26:28,  3.57it/s]   Exception ignored in: <bound method IPythonKernel._clean_thread_parent_frames of <ipykernel.ipkernel.IPythonKernel object at 0x104ea8110>>
Traceback (most recent call last):
  File "/Users/ravi.mandliya/.pyenv/versions/3.11.7/envs/blog/lib/python3.11/site-packages/ipykernel/ipkernel.py", line 775, in _clean_thread_parent_frames
    def _clean_thread_parent_frames(

KeyboardInterrupt: 
Epoch 2/2:  84%|████████▎ | 28691/34343 [7:01:56<1:23:07,  1.13it/s] 

---------------------------------------------------------------------------

KeyboardInterrupt                         Traceback (most recent call last)

Cell In[69], line 1
----> 1 word2vec.train(text, batch_size=2048)

Cell In[67], line 62, in Word2Vec.train(self, text, batch_size)
     60 # Create tqdm wrapper inside the loop for each epoch
     61 tqdm_dataloader = tqdm(dataloader, total=len(dataloader), desc=f"Epoch {epoch+1}/{self.num_epochs}")
---> 62 for idx, (target, context, negs) in enumerate(tqdm_dataloader):
     63   self.optimizer.zero_grad()
     64   pos_score, neg_score = self.model(target, context, negs)

File ~/.pyenv/versions/3.11.7/envs/blog/lib/python3.11/site-packages/tqdm/std.py:1181, in tqdm.__iter__(self)
   1178 time = self._time
   1180 try:
-> 1181     for obj in iterable:
   1182         yield obj
   1183         # Update and possibly print the progressbar.
   1184         # Note: does not call self.update(1) for speed optimisation.

File ~/.pyenv/versions/3.11.7/envs/blog/lib/python3.11/site-packages/torch/utils/data/dataloader.py:631, in _BaseDataLoaderIter.__next__(self)
    628 if self._sampler_iter is None:
    629     # TODO(https://github.com/pytorch/pytorch/issues/76750)
    630     self._reset()  # type: ignore[call-arg]
--> 631 data = self._next_data()
    632 self._num_yielded += 1
    633 if self._dataset_kind == _DatasetKind.Iterable and \
    634         self._IterableDataset_len_called is not None and \
    635         self._num_yielded > self._IterableDataset_len_called:

File ~/.pyenv/versions/3.11.7/envs/blog/lib/python3.11/site-packages/torch/utils/data/dataloader.py:675, in _SingleProcessDataLoaderIter._next_data(self)
    673 def _next_data(self):
    674     index = self._next_index()  # may raise StopIteration
--> 675     data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    676     if self._pin_memory:
    677         data = _utils.pin_memory.pin_memory(data, self._pin_memory_device)

File ~/.pyenv/versions/3.11.7/envs/blog/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py:51, in _MapDatasetFetcher.fetch(self, possibly_batched_index)
     49         data = self.dataset.__getitems__(possibly_batched_index)
     50     else:
---> 51         data = [self.dataset[idx] for idx in possibly_batched_index]
     52 else:
     53     data = self.dataset[possibly_batched_index]

File ~/.pyenv/versions/3.11.7/envs/blog/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py:51, in <listcomp>(.0)
     49         data = self.dataset.__getitems__(possibly_batched_index)
     50     else:
---> 51         data = [self.dataset[idx] for idx in possibly_batched_index]
     52 else:
     53     data = self.dataset[possibly_batched_index]

Cell In[63], line 91, in Word2VecDataset.__getitem__(self, idx)
     88 target, context = self.pairs[idx]
     90 # Generate negative samples on-the-fly
---> 91 negs = torch.multinomial(
     92   self.negative_sample_weights,
     93   self.negative_sample_counts,
     94   replacement=True
     95 )
     97 return torch.tensor(target, dtype=torch.long), torch.tensor(context, dtype=torch.long), negs

KeyboardInterrupt:

# print random words in the vocabulary
random.sample(words, 10)

['after',
 'took',
 'asserts',
 'russian',
 'in',
 'performed',
 'day',
 'two',
 'serving',
 'conference']

word2vec.most_similar("happiness")

['happiness',
 'goodness',
 'repent',
 'compassion',
 'righteousness',
 'atonement',
 'forgiveness',
 'sinful',
 'immanent',
 'rationality']