{ "cells": [ { "cell_type": "markdown", "id": "b39e2edb", "metadata": {}, "source": [ "# Transformer: Language Modeling\n", "\n", "Train a three-block, width-128 causal transformer on Tiny Shakespeare, a 1,115,394-character corpus distributed with Karpathy’s char-rnn examples. Contiguous 80/10/10 text ranges define train/validation/test. Windows do not cross split boundaries.\n", "\n", "Source: [Tiny Shakespeare](https://github.com/karpathy/char-rnn/tree/master/data/tinyshakespeare). The underlying Shakespeare texts are public domain. This run uses the file SHA-256 recorded in the experiment metadata.\n", "\n", "The outputs below are saved from local GPU training. Documentation builds render these saved outputs without running cells." ] }, { "cell_type": "code", "execution_count": 1, "id": "9d1878f3", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.126781Z", "iopub.status.busy": "2026-10-14T13:51:33.126619Z", "iopub.status.idle": "2026-10-14T13:51:33.832739Z", "shell.execute_reply": "2026-10-14T13:51:33.832114Z" }, "tags": [ "hide-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "JAXsaw 0.1.0 · JAX 0.5.3\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[CudaDevice(id=0)]\n" ] } ], "source": [ "from pathlib import Path\n", "import sys, json, inspect\n", "ROOT = Path.cwd() if (Path.cwd() / 'examples').exists() else Path.cwd().parent\n", "sys.path.insert(0,str(ROOT))\n", "import jax\n", "import numpy as np\n", "import pandas as pd\n", "import jaxsaw as js\n", "from jaxsaw import models, training\n", "from IPython.display import display, Image, Code\n", "RUN = ROOT / 'runs' / 'gallery/transformer'\n", "print(f\"JAXsaw {js.__version__} · JAX {jax.__version__}\")\n", "print(jax.devices())" ] }, { "cell_type": "markdown", "id": "d769c167", "metadata": {}, "source": [ "## Architecture\n", "\n", "Context length 128; fixed 128-symbol ASCII vocabulary; output `[B,T,128]`. Targets are shifted by one character. Attention uses boolean allowed-position masks. Training uses cross-entropy and a learning rate of 0.0003.\n", "\n", "The definition below is the actual package implementation used by this run." ] }, { "cell_type": "code", "execution_count": 2, "id": "ec2d8bbf", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.834255Z", "iopub.status.busy": "2026-10-14T13:51:33.833969Z", "iopub.status.idle": "2026-10-14T13:51:33.862777Z", "shell.execute_reply": "2026-10-14T13:51:33.862355Z" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
class TinyLanguageModel(nn.Module):\n",
       "    def __init__(self, vocab_size, max_length, width=32, depth=2, heads=4, *, rngs):\n",
       "        super().__init__()\n",
       "        self.max_length = max_length\n",
       "        self.token = nn.Embedding(vocab_size, width, rngs=rngs)\n",
       "        self.position = nn.Embedding(max_length, width, rngs=rngs)\n",
       "        self.blocks = nn.ModuleList(\n",
       "            [TransformerBlock(width, heads, rngs=rngs) for _ in range(depth)]\n",
       "        )\n",
       "        self.norm = nn.LayerNorm(width)\n",
       "        self.head = nn.Linear(width, vocab_size, rngs=rngs)\n",
       "\n",
       "    def forward(self, tokens, *, valid=None):\n",
       "        if tokens.shape[1] > self.max_length:\n",
       "            raise ValueError(\"Token sequence exceeds max_length\")\n",
       "        length = tokens.shape[1]\n",
       "        x = self.token(tokens) + self.position(jnp.arange(length))[None]\n",
       "        mask = nn.attention.causal_mask(length)\n",
       "        if valid is not None:\n",
       "            mask = mask & nn.attention.padding_mask(valid)\n",
       "        for block in self.blocks:\n",
       "            x = block(x, mask=mask)\n",
       "        return self.head(self.norm(x))\n",
       "
\n" ], "text/latex": [ "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", "\\PY{k}{class}\\PY{+w}{ }\\PY{n+nc}{TinyLanguageModel}\\PY{p}{(}\\PY{n}{nn}\\PY{o}{.}\\PY{n}{Module}\\PY{p}{)}\\PY{p}{:}\n", " \\PY{k}{def}\\PY{+w}{ }\\PY{n+nf+fm}{\\PYZus{}\\PYZus{}init\\PYZus{}\\PYZus{}}\\PY{p}{(}\\PY{n+nb+bp}{self}\\PY{p}{,} \\PY{n}{vocab\\PYZus{}size}\\PY{p}{,} \\PY{n}{max\\PYZus{}length}\\PY{p}{,} \\PY{n}{width}\\PY{o}{=}\\PY{l+m+mi}{32}\\PY{p}{,} \\PY{n}{depth}\\PY{o}{=}\\PY{l+m+mi}{2}\\PY{p}{,} \\PY{n}{heads}\\PY{o}{=}\\PY{l+m+mi}{4}\\PY{p}{,} \\PY{o}{*}\\PY{p}{,} \\PY{n}{rngs}\\PY{p}{)}\\PY{p}{:}\n", " \\PY{n+nb}{super}\\PY{p}{(}\\PY{p}{)}\\PY{o}{.}\\PY{n+nf+fm}{\\PYZus{}\\PYZus{}init\\PYZus{}\\PYZus{}}\\PY{p}{(}\\PY{p}{)}\n", " \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{max\\PYZus{}length} \\PY{o}{=} \\PY{n}{max\\PYZus{}length}\n", " \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{token} \\PY{o}{=} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{Embedding}\\PY{p}{(}\\PY{n}{vocab\\PYZus{}size}\\PY{p}{,} \\PY{n}{width}\\PY{p}{,} \\PY{n}{rngs}\\PY{o}{=}\\PY{n}{rngs}\\PY{p}{)}\n", " \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{position} \\PY{o}{=} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{Embedding}\\PY{p}{(}\\PY{n}{max\\PYZus{}length}\\PY{p}{,} \\PY{n}{width}\\PY{p}{,} \\PY{n}{rngs}\\PY{o}{=}\\PY{n}{rngs}\\PY{p}{)}\n", " \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{blocks} \\PY{o}{=} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{ModuleList}\\PY{p}{(}\n", " \\PY{p}{[}\\PY{n}{TransformerBlock}\\PY{p}{(}\\PY{n}{width}\\PY{p}{,} \\PY{n}{heads}\\PY{p}{,} \\PY{n}{rngs}\\PY{o}{=}\\PY{n}{rngs}\\PY{p}{)} \\PY{k}{for} \\PY{n}{\\PYZus{}} \\PY{o+ow}{in} \\PY{n+nb}{range}\\PY{p}{(}\\PY{n}{depth}\\PY{p}{)}\\PY{p}{]}\n", " \\PY{p}{)}\n", " \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{norm} \\PY{o}{=} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{LayerNorm}\\PY{p}{(}\\PY{n}{width}\\PY{p}{)}\n", " \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{head} \\PY{o}{=} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{Linear}\\PY{p}{(}\\PY{n}{width}\\PY{p}{,} \\PY{n}{vocab\\PYZus{}size}\\PY{p}{,} \\PY{n}{rngs}\\PY{o}{=}\\PY{n}{rngs}\\PY{p}{)}\n", "\n", " \\PY{k}{def}\\PY{+w}{ }\\PY{n+nf}{forward}\\PY{p}{(}\\PY{n+nb+bp}{self}\\PY{p}{,} \\PY{n}{tokens}\\PY{p}{,} \\PY{o}{*}\\PY{p}{,} \\PY{n}{valid}\\PY{o}{=}\\PY{k+kc}{None}\\PY{p}{)}\\PY{p}{:}\n", " \\PY{k}{if} \\PY{n}{tokens}\\PY{o}{.}\\PY{n}{shape}\\PY{p}{[}\\PY{l+m+mi}{1}\\PY{p}{]} \\PY{o}{\\PYZgt{}} \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{max\\PYZus{}length}\\PY{p}{:}\n", " \\PY{k}{raise} \\PY{n+ne}{ValueError}\\PY{p}{(}\\PY{l+s+s2}{\\PYZdq{}}\\PY{l+s+s2}{Token sequence exceeds max\\PYZus{}length}\\PY{l+s+s2}{\\PYZdq{}}\\PY{p}{)}\n", " \\PY{n}{length} \\PY{o}{=} \\PY{n}{tokens}\\PY{o}{.}\\PY{n}{shape}\\PY{p}{[}\\PY{l+m+mi}{1}\\PY{p}{]}\n", " \\PY{n}{x} \\PY{o}{=} \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{token}\\PY{p}{(}\\PY{n}{tokens}\\PY{p}{)} \\PY{o}{+} \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{position}\\PY{p}{(}\\PY{n}{jnp}\\PY{o}{.}\\PY{n}{arange}\\PY{p}{(}\\PY{n}{length}\\PY{p}{)}\\PY{p}{)}\\PY{p}{[}\\PY{k+kc}{None}\\PY{p}{]}\n", " \\PY{n}{mask} \\PY{o}{=} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{attention}\\PY{o}{.}\\PY{n}{causal\\PYZus{}mask}\\PY{p}{(}\\PY{n}{length}\\PY{p}{)}\n", " \\PY{k}{if} \\PY{n}{valid} \\PY{o+ow}{is} \\PY{o+ow}{not} \\PY{k+kc}{None}\\PY{p}{:}\n", " \\PY{n}{mask} \\PY{o}{=} \\PY{n}{mask} \\PY{o}{\\PYZam{}} \\PY{n}{nn}\\PY{o}{.}\\PY{n}{attention}\\PY{o}{.}\\PY{n}{padding\\PYZus{}mask}\\PY{p}{(}\\PY{n}{valid}\\PY{p}{)}\n", " \\PY{k}{for} \\PY{n}{block} \\PY{o+ow}{in} \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{blocks}\\PY{p}{:}\n", " \\PY{n}{x} \\PY{o}{=} \\PY{n}{block}\\PY{p}{(}\\PY{n}{x}\\PY{p}{,} \\PY{n}{mask}\\PY{o}{=}\\PY{n}{mask}\\PY{p}{)}\n", " \\PY{k}{return} \\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{head}\\PY{p}{(}\\PY{n+nb+bp}{self}\\PY{o}{.}\\PY{n}{norm}\\PY{p}{(}\\PY{n}{x}\\PY{p}{)}\\PY{p}{)}\n", "\\end{Verbatim}\n" ], "text/plain": [ "class TinyLanguageModel(nn.Module):\n", " def __init__(self, vocab_size, max_length, width=32, depth=2, heads=4, *, rngs):\n", " super().__init__()\n", " self.max_length = max_length\n", " self.token = nn.Embedding(vocab_size, width, rngs=rngs)\n", " self.position = nn.Embedding(max_length, width, rngs=rngs)\n", " self.blocks = nn.ModuleList(\n", " [TransformerBlock(width, heads, rngs=rngs) for _ in range(depth)]\n", " )\n", " self.norm = nn.LayerNorm(width)\n", " self.head = nn.Linear(width, vocab_size, rngs=rngs)\n", "\n", " def forward(self, tokens, *, valid=None):\n", " if tokens.shape[1] > self.max_length:\n", " raise ValueError(\"Token sequence exceeds max_length\")\n", " length = tokens.shape[1]\n", " x = self.token(tokens) + self.position(jnp.arange(length))[None]\n", " mask = nn.attention.causal_mask(length)\n", " if valid is not None:\n", " mask = mask & nn.attention.padding_mask(valid)\n", " for block in self.blocks:\n", " x = block(x, mask=mask)\n", " return self.head(self.norm(x))" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "display(Code(inspect.getsource(models.TinyLanguageModel), language=\"python\"))" ] }, { "cell_type": "markdown", "id": "ee75da9b", "metadata": {}, "source": [ "## Objective and optimization\n", "\n", "Training uses an explicit step. Data iteration, evaluation, and artifact writing remain outside compiled computation." ] }, { "cell_type": "code", "execution_count": 3, "id": "709e2ff5", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.863709Z", "iopub.status.busy": "2026-10-14T13:51:33.863613Z", "iopub.status.idle": "2026-10-14T13:51:33.868682Z", "shell.execute_reply": "2026-10-14T13:51:33.868220Z" }, "tags": [ "hide-input" ] }, "outputs": [ { "data": { "text/html": [ "
@jit\n",
       "def classifier_step(model, optimizer, batch):\n",
       "    x, y = batch\n",
       "\n",
       "    def objective(candidate):\n",
       "        logits = candidate(x)\n",
       "        return F.cross_entropy(logits, y), (logits.argmax(-1) == y).sum()\n",
       "\n",
       "    (loss, correct), grads = value_and_grad(objective, has_aux=True)(model)\n",
       "    optimizer.step(grads)\n",
       "    return {\"loss\": loss, \"correct\": correct, \"count\": y.size}\n",
       "
\n" ], "text/latex": [ "\\begin{Verbatim}[commandchars=\\\\\\{\\}]\n", "\\PY{n+nd}{@jit}\n", "\\PY{k}{def}\\PY{+w}{ }\\PY{n+nf}{classifier\\PYZus{}step}\\PY{p}{(}\\PY{n}{model}\\PY{p}{,} \\PY{n}{optimizer}\\PY{p}{,} \\PY{n}{batch}\\PY{p}{)}\\PY{p}{:}\n", " \\PY{n}{x}\\PY{p}{,} \\PY{n}{y} \\PY{o}{=} \\PY{n}{batch}\n", "\n", " \\PY{k}{def}\\PY{+w}{ }\\PY{n+nf}{objective}\\PY{p}{(}\\PY{n}{candidate}\\PY{p}{)}\\PY{p}{:}\n", " \\PY{n}{logits} \\PY{o}{=} \\PY{n}{candidate}\\PY{p}{(}\\PY{n}{x}\\PY{p}{)}\n", " \\PY{k}{return} \\PY{n}{F}\\PY{o}{.}\\PY{n}{cross\\PYZus{}entropy}\\PY{p}{(}\\PY{n}{logits}\\PY{p}{,} \\PY{n}{y}\\PY{p}{)}\\PY{p}{,} \\PY{p}{(}\\PY{n}{logits}\\PY{o}{.}\\PY{n}{argmax}\\PY{p}{(}\\PY{o}{\\PYZhy{}}\\PY{l+m+mi}{1}\\PY{p}{)} \\PY{o}{==} \\PY{n}{y}\\PY{p}{)}\\PY{o}{.}\\PY{n}{sum}\\PY{p}{(}\\PY{p}{)}\n", "\n", " \\PY{p}{(}\\PY{n}{loss}\\PY{p}{,} \\PY{n}{correct}\\PY{p}{)}\\PY{p}{,} \\PY{n}{grads} \\PY{o}{=} \\PY{n}{value\\PYZus{}and\\PYZus{}grad}\\PY{p}{(}\\PY{n}{objective}\\PY{p}{,} \\PY{n}{has\\PYZus{}aux}\\PY{o}{=}\\PY{k+kc}{True}\\PY{p}{)}\\PY{p}{(}\\PY{n}{model}\\PY{p}{)}\n", " \\PY{n}{optimizer}\\PY{o}{.}\\PY{n}{step}\\PY{p}{(}\\PY{n}{grads}\\PY{p}{)}\n", " \\PY{k}{return} \\PY{p}{\\PYZob{}}\\PY{l+s+s2}{\\PYZdq{}}\\PY{l+s+s2}{loss}\\PY{l+s+s2}{\\PYZdq{}}\\PY{p}{:} \\PY{n}{loss}\\PY{p}{,} \\PY{l+s+s2}{\\PYZdq{}}\\PY{l+s+s2}{correct}\\PY{l+s+s2}{\\PYZdq{}}\\PY{p}{:} \\PY{n}{correct}\\PY{p}{,} \\PY{l+s+s2}{\\PYZdq{}}\\PY{l+s+s2}{count}\\PY{l+s+s2}{\\PYZdq{}}\\PY{p}{:} \\PY{n}{y}\\PY{o}{.}\\PY{n}{size}\\PY{p}{\\PYZcb{}}\n", "\\end{Verbatim}\n" ], "text/plain": [ "@jit\n", "def classifier_step(model, optimizer, batch):\n", " x, y = batch\n", "\n", " def objective(candidate):\n", " logits = candidate(x)\n", " return F.cross_entropy(logits, y), (logits.argmax(-1) == y).sum()\n", "\n", " (loss, correct), grads = value_and_grad(objective, has_aux=True)(model)\n", " optimizer.step(grads)\n", " return {\"loss\": loss, \"correct\": correct, \"count\": y.size}" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "display(Code(inspect.getsource(training.classifier_step), language=\"python\"))" ] }, { "cell_type": "markdown", "id": "ba8ab0ba", "metadata": {}, "source": [ "## Reproduce the run\n", "\n", "Select one GPU before launching Python or Jupyter: `CUDA_VISIBLE_DEVICES=0`. The recipe below creates a new run directory and does not overwrite completed results. Leave `TRAIN=False` to inspect an existing local run. Checkpoints are gitignored; this notebook retains the figures and metrics for publication." ] }, { "cell_type": "code", "execution_count": 4, "id": "31c28e59", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.869419Z", "iopub.status.busy": "2026-10-14T13:51:33.869328Z", "iopub.status.idle": "2026-10-14T13:51:33.876413Z", "shell.execute_reply": "2026-10-14T13:51:33.875961Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "python -m examples.run transformer --download --output runs/gallery/transformer --epochs 20 --limit 30000 --eval-limit 2048 --batch-size 64 --context-length 128 --model-width 128 --model-depth 3 --text-file data/tinyshakespeare.txt\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochsbatch_sizeseedtraining_examplesvalidation_examplestest_examples
0206402788120482048
\n", "
" ], "text/plain": [ " epochs batch_size seed training_examples validation_examples \\\n", "0 20 64 0 27881 2048 \n", "\n", " test_examples \n", "0 2048 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Training device: cuda:0 NVIDIA GeForce RTX 5090\n" ] } ], "source": [ "from examples.train_gallery import RECIPES\n", "import subprocess\n", "TRAIN = False\n", "command = [sys.executable, '-m', 'examples.run', 'transformer', '--download',\n", " '--output', str(RUN), *RECIPES['transformer']]\n", "print(' '.join(part.replace(str(ROOT) + '/', '')\n", " for part in ['python', *command[1:]]))\n", "if TRAIN:\n", " subprocess.run(command,cwd=ROOT,check=True)\n", "if not (RUN / 'report.json').exists():\n", " raise FileNotFoundError('Run the recipe locally before refreshing notebook outputs.')\n", "report = json.loads((RUN / 'report.json').read_text())\n", "configuration = report['config']\n", "display(pd.DataFrame([{'epochs':configuration['epochs'], 'batch_size':configuration['batch_size'],\n", " 'seed':configuration['seed'], 'training_examples':report['train_samples'],\n", " 'validation_examples':report['validation_samples'],\n", " 'test_examples':report['test_samples']}]))\n", "print('Training device:',', '.join(report['devices']))" ] }, { "cell_type": "markdown", "id": "a486af64", "metadata": {}, "source": [ "## Training history\n", "\n", "The figure reports actual epoch metrics. For supervised tasks, the checkpoint with the lowest validation loss is selected before test evaluation." ] }, { "cell_type": "code", "execution_count": 5, "id": "4ca2605b", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.877173Z", "iopub.status.busy": "2026-10-14T13:51:33.877081Z", "iopub.status.idle": "2026-10-14T13:51:33.880016Z", "shell.execute_reply": "2026-10-14T13:51:33.879606Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjksIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvJkbTWQAAAAlwSFlzAAAPYQAAD2EBqD+naQAAYjxJREFUeJzt3Xd0VHX+PvBnWqakTHrv1EiJgICBVRGQ5iKIoiJHYHXXFlxdl/0p6yqgu2JfO5YVkBVEQUC+4KKABJBeld4SkkB6mySTZCaZub8/bmaSIT1MS+Z5nTNn2r133jOZhIdPuxJBEAQQERERkceQuroAIiIiInIuBkAiIiIiD8MASERERORhGACJiIiIPAwDIBEREZGHYQAkIiIi8jAMgEREREQehgGQiIiIyMMwABIRERF5GAZAIiIiIg/DAEhERETkYRgAiYiIiDwMAyARERGRh5G7ugB3YDabkZOTA19fX0gkEleXQ0RERNQqQRBQUVGByMhISKUdb89jAASQk5ODmJgYV5dBRERE1CHZ2dmIjo7u8H4MgAB8fX0BiB+in5+fi6shIiIial15eTliYmKsGaaj3C4ALl68GOvWrcPZs2ehVqsxYsQIvP766+jTp0+r+5WVleGFF17AunXrUFJSgri4OLz77ruYNGlSm69p6fb18/NjACQiIqIuo7ND19wuAO7cuROpqakYOnQo6urq8Pe//x3jxo3D6dOn4e3t3ew+RqMRd9xxB0JDQ7F27VpERUUhMzMT/v7+zi2eiIiIqAtwuwC4ZcsWm/vLly9HaGgojhw5gltvvbXZfZYuXYqSkhLs3bsXCoUCABAfH+/oUomIiIi6JLdfBkan0wEAAgMDW9xm48aNSElJQWpqKsLCwtC/f3+8+uqrMJlMziqTiIiIqMtwuxbAxsxmM5555hmMHDkS/fv3b3G79PR0/Pzzz5g5cyZ++OEHXLx4EU8++SRqa2uxYMGCJtsbDAYYDAbr/fLycofUT0RE5Comkwm1tbWuLoM6SaFQQCaTOez4bh0AU1NTcfLkSfzyyy+tbmc2mxEaGorPPvsMMpkMQ4YMwdWrV/Hmm282GwAXL16MRYsWOapsIiIilxEEAXl5eSgrK3N1KXSd/P39ER4e7pA1it02AM6dOxebNm3Crl272lzfJiIioklSTkpKQl5eHoxGI7y8vGy2nz9/Pp599lnrfctUaiIioq7OEv5CQ0Oh0Wh4goMuSBAEVFVVoaCgAICYc+zN7QKgIAh46qmnsH79eqSlpSEhIaHNfUaOHIlVq1bBbDZbV8M+f/48IiIimoQ/AFAqlVAqlXavnYiIyJVMJpM1/AUFBbm6HLoOarUaAFBQUIDQ0FC7dwe73SSQ1NRUfPXVV1i1ahV8fX2Rl5eHvLw8VFdXW7eZNWsW5s+fb73/xBNPoKSkBE8//TTOnz+PzZs349VXX0Vqaqor3gIREZFLWMb8aTQaF1dC9mD5OTpiLKfbtQAuWbIEADBq1Cibx5ctW4Y5c+YAALKysmzOexcTE4Mff/wRf/nLXzBw4EBERUXh6aefxnPPPeessomIiNwGu327B0f+HN0uAAqC0OY2aWlpTR5LSUnB/v37HVARERERUffidl3ARERERNcjPj4e7777rl2OlZaWBolE0u1mVbtdCyARERF5nlGjRuHGG2+0S3A7dOhQi6ePJREDIBEREbk9QRBgMpkgl7cdXUJCQpxQUdfGLmAiIiJyqTlz5mDnzp147733IJFIIJFIsHz5ckgkEvzvf//DkCFDoFQq8csvv+DSpUuYMmUKwsLC4OPjg6FDh2Lbtm02x7u2C1gikeA///kP7r77bmg0GvTq1QsbN27sdL3fffcd+vXrB6VSifj4eLz99ts2z3/88cfo1asXVCoVwsLCcO+991qfW7t2LQYMGAC1Wo2goCCMHTsWer2+07V0FlsAneRiQSW+O3oFPko5Um/v6epyiIjIAwiCgOpak0teW62QtXsW63vvvYfz58+jf//+ePnllwEAp06dAgA8//zzeOutt5CYmIiAgABkZ2dj0qRJ+Ne//gWlUokVK1Zg8uTJOHfuHGJjY1t8jUWLFuGNN97Am2++iQ8++AAzZ85EZmYmAgMDO/S+jhw5gvvuuw8LFy7E/fffj7179+LJJ59EUFAQ5syZg8OHD+PPf/4z/vvf/2LEiBEoKSnB7t27AQC5ubmYMWMG3njjDdx9992oqKjA7t272zUB1t4YAJ0kv7wGS9IuITHYmwGQiIicorrWhBte+tElr3365fHQeLUvZmi1Wnh5eUGj0SA8PBwAcPbsWQDAyy+/jDvuuMO6bWBgIJKTk633X3nlFaxfvx4bN27E3LlzW3yNOXPmYMaMGQCAV199Fe+//z4OHjyICRMmdOh9vfPOOxgzZgxefPFFAEDv3r1x+vRpvPnmm5gzZw6ysrLg7e2N3//+9/D19UVcXBwGDRoEQAyAdXV1mDZtGuLi4gAAAwYM6NDr2wu7gJ0kMUQcjJpVUoVak9nF1RAREXUNN910k839yspKzJs3D0lJSfD394ePjw/OnDmDrKysVo8zcOBA621vb2/4+flZT7XWEWfOnMHIkSNtHhs5ciQuXLgAk8mEO+64A3FxcUhMTMRDDz2ElStXoqqqCgCQnJyMMWPGYMCAAZg+fTo+//xzlJaWdrgGe2ALoJOE+6mgVshQXWtCdkkVEkN8XF0SERF1c2qFDKdfHu+y17aHa2fzzps3D1u3bsVbb72Fnj17Qq1W495774XRaGz1OAqFwua+RCKB2Wz/BhlfX18cPXoUaWlp+Omnn/DSSy9h4cKFOHToEPz9/bF161bs3bsXP/30Ez744AO88MILOHDgQLtOfWtPbAF0EolEgoRg8UucXuj8wZ5EROR5JBIJNF5yl1w6ehYLLy8vmExtj1fcs2cP5syZg7vvvhsDBgxAeHg4Ll++3MlPqOOSkpKwZ8+eJjX17t3ber5euVyOsWPH4o033sBvv/2Gy5cv4+effwYg/kxGjhyJRYsW4dixY/Dy8sL69eudVr8FWwCdKDHEG6dzy5FeVAkgzNXlEBERuY34+HgcOHAAly9fho+PT4utc7169cK6deswefJkSCQSvPjiiw5pyWvJX//6VwwdOhSvvPIK7r//fuzbtw8ffvghPv74YwDApk2bkJ6ejltvvRUBAQH44YcfYDab0adPHxw4cADbt2/HuHHjEBoaigMHDqCwsBBJSUlOq9+CLYBOZOn2zShiCyAREVFj8+bNg0wmww033ICQkJAWx/S98847CAgIwIgRIzB58mSMHz8egwcPdlqdgwcPxrfffovVq1ejf//+eOmll/Dyyy9jzpw5AAB/f3+sW7cOo0ePRlJSEj755BN8/fXX6NevH/z8/LBr1y5MmjQJvXv3xj/+8Q+8/fbbmDhxotPqt5AIrph77GbKy8uh1Wqh0+ng5+fnsNfZcOwqnvnmOIYlBOLbx1Ic9jpEROSZampqkJGRgYSEBKhUKleXQ9eptZ/n9WYXtgA6kWUmMMcAEhERkSsxADqRZRJIUaUB5TW1Lq6GiIiIHn/8cfj4+DR7efzxx11dnsNwEogT+aoUCPFVorDCgIxCPZJj/F1dEhERkUd7+eWXMW/evGafc+SwMFdjAHSyxGBvFFYYkF5UyQBIRETkYqGhoQgNDXV1GU7HLmAns4wDzOA4QCIiInIRBkAnSwwWl4K5xKVgiIiIyEUYAJ2MLYBERETkagyATmaZCZxRpIfZ7PFLMBIREZELMAA6WUygBnKpBNW1JuSV17i6HCIiIvJADIBOppBJERukAcBTwhEREdlLfHw83n333XZtK5FIsGHDBofW4+4YAF0gMdhyRpBKF1dCREREnogB0AUSQ+pnAnMiCBEREbkAA6ALNJ4IQkRE5Ok+++wzREZGwmw22zw+ZcoUPPzww7h06RKmTJmCsLAw+Pj4YOjQodi2bZvdXv/EiRMYPXo01Go1goKC8Oijj6KysqGXLi0tDcOGDYO3tzf8/f0xcuRIZGZmAgB+/fVX3H777fD19YWfnx+GDBmCw4cP2602R2EAdAFrF3ARu4CJiMiBBAEw6l1zEdq/0sX06dNRXFyMHTt2WB8rKSnBli1bMHPmTFRWVmLSpEnYvn07jh07hgkTJmDy5MnIysq67o9Ir9dj/PjxCAgIwKFDh7BmzRps27YNc+fOBQDU1dVh6tSpuO222/Dbb79h3759ePTRRyGRSAAAM2fORHR0NA4dOoQjR47g+eefh0KhuO66HI2ngnMBSxfwldJqGOpMUMplLq6IiIi6pdoq4NVI17z233MAL+92bRoQEICJEydi1apVGDNmDABg7dq1CA4Oxu233w6pVIrk5GTr9q+88grWr1+PjRs3WoNaZ61atQo1NTVYsWIFvL3Fej/88ENMnjwZr7/+OhQKBXQ6HX7/+9+jR48eAICkpCTr/llZWfjb3/6Gvn37AgB69ep1XfU4C1sAXSDYxwu+SjkEAcgsrnJ1OURERC43c+ZMfPfddzAYDACAlStX4oEHHoBUKkVlZSXmzZuHpKQk+Pv7w8fHB2fOnLFLC+CZM2eQnJxsDX8AMHLkSJjNZpw7dw6BgYGYM2cOxo8fj8mTJ+O9995Dbm6uddtnn30Wf/zjHzF27Fi89tpruHTp0nXX5AxsAXQBiUSCxBBv/HpFh/TCSvQO83V1SURE1B0pNGJLnKteuwMmT54MQRCwefNmDB06FLt378a///1vAMC8efOwdetWvPXWW+jZsyfUajXuvfdeGI1GR1TexLJly/DnP/8ZW7ZswTfffIN//OMf2Lp1K26++WYsXLgQDz74IDZv3oz//e9/WLBgAVavXo27777bKbV1FgOgiySG+IgBkBNBiIjIUSSSdnfDuppKpcK0adOwcuVKXLx4EX369MHgwYMBAHv27MGcOXOsoaqyshKXL1+2y+smJSVh+fLl0Ov11lbAPXv2QCqVok+fPtbtBg0ahEGDBmH+/PlISUnBqlWrcPPNNwMAevfujd69e+Mvf/kLZsyYgWXLlrl9AGQXsIskWNcCZAAkIiICxG7gzZs3Y+nSpZg5c6b18V69emHdunU4fvw4fv31Vzz44INNZgxfz2uqVCrMnj0bJ0+exI4dO/DUU0/hoYceQlhYGDIyMjB//nzs27cPmZmZ+Omnn3DhwgUkJSWhuroac+fORVpaGjIzM7Fnzx4cOnTIZoygu2ILoIskhnAxaCIiosZGjx6NwMBAnDt3Dg8++KD18XfeeQcPP/wwRowYgeDgYDz33HMoLy+3y2tqNBr8+OOPePrppzF06FBoNBrcc889eOedd6zPnz17Fl9++SWKi4sRERGB1NRUPPbYY6irq0NxcTFmzZqF/Px8BAcHY9q0aVi0aJFdanMkiSB0YJ52N1VeXg6tVgudTgc/Pz+nvObpnHJMen83AjQKHHtpnFNek4iIureamhpkZGQgISEBKpXK1eXQdWrt53m92YVdwC4SHywOji2tqkWp3jmDWImIiIgABkCX0XjJEakV0zwXhCYiIrKPlStXwsfHp9lLv379XF2e2+AYQBdKCPFGjq4G6YV6DIkLdHU5REREXd5dd92F4cOHN/tcVzhDh7MwALpQYrAP9lws5lIwREREduLr6wtfX66v2xZ2AbuQZSZwBpeCISIiIidiAHQh61qAHANIRER2ZK818si1HPlzZBewC/UI8QEAXC6ugsksQCaVuLgiIiLqyry8vCCVSpGTk4OQkBB4eXlBIuG/LV2NIAgwGo0oLCyEVCqFl5eX3V+DAdCFIv3V8JJLYawzI6esGjGBHTtvIhERUWNSqRQJCQnIzc1FTo6LzgFMdqPRaBAbGwup1P4dtm4XABcvXox169bh7NmzUKvVGDFiBF5//XWb8/G1ZvXq1ZgxYwamTJmCDRs2OLbY6ySTShAfpMH5/EpcKqxkACQiouvm5eWF2NhY1NXVwWQyuboc6iSZTAa5XO6wFly3C4A7d+5Eamoqhg4dirq6Ovz973/HuHHjcPr0aetJmlty+fJlzJs3D7fccouTqr1+icE+OJ9fifRCPUa1L+MSERG1SiKRQKFQcNkTapHbBcAtW7bY3F++fDlCQ0Nx5MgR3HrrrS3uZzKZMHPmTCxatAi7d+9GWVmZgyu1D+tMYC4FQ0RERE7i9rOAdTodACAwsPWFkl9++WWEhobikUceafOYBoMB5eXlNhdX4UxgIiIicja3DoBmsxnPPPMMRo4cif79+7e43S+//IIvvvgCn3/+ebuOu3jxYmi1WuslJibGXiV3WGL9TOB0rgVIRERETuLWATA1NRUnT57E6tWrW9ymoqICDz30ED7//HMEBwe367jz58+HTqezXrKzs+1Vcocl1rcA5upqUGWsc1kdRERE5Dncbgygxdy5c7Fp0ybs2rUL0dHRLW536dIlXL58GZMnT7Y+Zlk4US6X49y5c+jRo4fNPkqlEkql0jGFd1CAtxcCNAqUVtUio0iPfpFaV5dERERE3ZzbBUBBEPDUU09h/fr1SEtLQ0JCQqvb9+3bFydOnLB57B//+AcqKirw3nvvubR7t70SQ3xwJLOUAZCIiIicwu0CYGpqKlatWoXvv/8evr6+yMvLAwBotVqo1WoAwKxZsxAVFYXFixdDpVI1GR/o7+8PAK2OG3QnCcHeOJJZynGARERE5BRuFwCXLFkCABg1apTN48uWLcOcOXMAAFlZWQ5ZFdtVLEvBpBdyJjARERE5ntsFQEEQ2twmLS2t1eeXL19un2KcJDFYnAnMtQCJiIjIGbpPM1oX1tACqG9XACYiIiK6HgyAbiAuSAOpBKgw1KGw0uDqcoiIiKibYwB0A0q5DNEBGgBABieCEBERkYMxALqJhlPCMQASERGRYzEAugnOBCYiIiJnYQB0E5ZTwnEmMBERETkaA6CbSAwRl4LhYtBERETkaAyAbsLSBZxVUoVak9nF1RAREVF3xgDoJsJ8VVArZKgzC8guqXJ1OURERNSNMQC6CalU0jATmN3ARERE5EAMgG7E0g3MiSBERETkSAyAbiTRuhYgl4IhIiIix2EAdCOWmcCX2AVMREREDsQA6EYSuBYgEREROQEDoBuxjAEsrDCgoqbWxdUQERFRd8UA6EZ8VQqE+CoBcCYwEREROQ4DoJthNzARERE5GgOgm+kRYlkLkDOBiYiIyDEYAN1MYnD9OYHZAkhEREQOwgDoZng2ECIiInI0BkA30/hsIGaz4OJqiIiIqDtiAHQzMYEayKUSVNeakF9R4+pyiIiIqBtiAHQzCpkUsYEaAOwGJiIiIsdgAHRDiZwJTERERA7EAOiGrBNBOBOYiIiIHIAB0A0lhtQvBcMuYCIiInIABkA3lGhtAWQXMBEREdkfA6AbSqgfA3iltBqGOpOLqyEiIqLuhgHQDYX4KOGrlEMQgMziKleXQ0RERN0MA6AbkkgkjWYCcxwgERER2RcDoJtK4DhAIiIichAGQDfFmcBERETkKAyAbqrxOYGJiIiI7IkB0E1Zu4B5NhAiIiKyMwZAN2UJgKVVtSjVG11cDREREXUnDIBuSuMlR4RWBYCnhCMiIiL7YgB0Yw1LwbAbmIiIiOyHAdCNJQbXzwRmCyARERHZEQOgG7OMA8zgUjBERERkRwyAbszaBczFoImIiMiOGADdWI/6xaAvF1fBZBZcXA0RERF1F24XABcvXoyhQ4fC19cXoaGhmDp1Ks6dO9fqPp9//jluueUWBAQEICAgAGPHjsXBgwedVLHjRPqr4SWXwlhnRk5ZtavLISIiom7C7QLgzp07kZqaiv3792Pr1q2ora3FuHHjoNe3PA4uLS0NM2bMwI4dO7Bv3z7ExMRg3LhxuHr1qhMrtz+ZVIL4IA0A4BJnAhMREZGdSARBcOu+xcLCQoSGhmLnzp249dZb27WPyWRCQEAAPvzwQ8yaNavN7cvLy6HVaqHT6eDn53e9JdvV4/89gi2n8rBg8g34w8gEV5dDREREbuB6s4vcATXZlU6nAwAEBga2e5+qqirU1ta2uI/BYIDBYLDeLy8vv74iHSjBuhYgZwITERGRfbhdF3BjZrMZzzzzDEaOHIn+/fu3e7/nnnsOkZGRGDt2bLPPL168GFqt1nqJiYmxV8l2lxjMmcBERERkX24dAFNTU3Hy5EmsXr263fu89tprWL16NdavXw+VStXsNvPnz4dOp7NesrOz7VWy3VmWguFagERERGQvbtsFPHfuXGzatAm7du1CdHR0u/Z566238Nprr2Hbtm0YOHBgi9splUoolUp7lepQlrOB5OhqUGWsg8bLbX9kRERE1EW4XQugIAiYO3cu1q9fj59//hkJCe2b+PDGG2/glVdewZYtW3DTTTc5uErnCfD2QoBGAQDI4CnhiIiIyA7cLgCmpqbiq6++wqpVq+Dr64u8vDzk5eWhurphHbxZs2Zh/vz51vuvv/46XnzxRSxduhTx8fHWfSoru8e4Oesp4RgAiYiIyA7cLgAuWbIEOp0Oo0aNQkREhPXyzTffWLfJyspCbm6uzT5GoxH33nuvzT5vvfWWK96C3SXWnxGEM4GJiIjIHtxuQFl7liVMS0uzuX/58mXHFGNPJenAb2sALw0w4qkO7WqdCMIWQCIiIrIDt2sB7LaKLwFprwKH/tPhXa1LwfBsIERERGQHDIDOEnszIJEBpZeBso4tO9O4C9jNT9xCREREXQADoLMofYHIQeLty790aNfYQA0kEqDCUIeiSqMDiiMiIiJPwgDoTPG/E687GABVChmiA9QA2A1MRERE148B0JkSbhGvL+/q8K6WBaHTORGEiIiIrhMDoDPF1I8DLMsCSjM7tCvXAiQiIiJ7YQB0JqUPEDVYvJ25p0O79gjhTGAiIiKyDwZAZ4uv7wbO2N2h3awzgdkCSERERNeJAdDZOjkRxNIFnFVchVqT2d5VERERkQdhAHS2mOGAVA7oOjYOMNxPBbVChjqzgOySKgcWSERERN0dA6CzKX2AqCHi7Q60AkqlEk4EISIiIrtgAHQFazdwx8YBJlgngjAAEhERUecxALpC43GAHTi1Ww/LOYGLOBOYiIiIOo8B0BVihgNSBaDLBsraPw6QLYBERERkDwyAruDl3TAOsAPLwfBsIERERGQPDICu0onlYCwtgIUVBlTU1DqiKiIiIvIADICu0olxgH4qBYJ9lAA4E5iIiIg6jwHQVSzjAMuvAKUZ7d4tkeMAiYiI6DoxALqKlwaIvkm83YFuYOs5gdkCSERERJ3EAOhKnRkHaFkKppBLwRAREVHnMAC6Uvwt4nXG7naPA7TOBGYXMBEREXUSA6ArRQ8FZF5ARQ5Qkt6uXSxjADOK9BA6sIg0ERERkQUDoCt5aYCojo0DjAnUQC6VoLrWhLzyGgcWR0RERN0VA6CrJdR3A7fzvMAKmRSxgRoA7AYmIiKizmEAdLVOrAdonQjCmcBERETUCQyArmYdB5jb4XGAnAlMREREncEA6GoKNRA9TLydsatduySGcCYwERERdR4DoDvo4HqAli5gng6OiIiIOoMB0B10cBygpQv4SmkVDHUmR1ZGRERE3RADoDuIHgrIlEBlHlB8sc3NQ3yU8FXKYRaArOIqJxRIRERE3QkDoDtQqICY+nGA7VgORiKRIKG+FfASxwESERFRBzEAuosOjgNMtC4Fw5nARERE1DEMgO7Ccl7gdo8DFGcCZ7AFkIiIiDqIAdBdRA0B5CqgMh8outDm5lwMmoiIiDqLAdBdKFTiZBCgXeMAuRg0ERERdRYDoDtJuFW8bsc4QEsLYGlVLUr1RkdWRURERN0MA6A76cB6gBovOSK0KgDsBiYiIqKOYQB0J5ZxgPoCoOh8m5uzG5iIiIg6gwHQnciVQMxw8XY7xgHylHBERETUGQyA7sayHExGOyaCBItLwaRzKRgiIiLqALcLgIsXL8bQoUPh6+uL0NBQTJ06FefOnWtzvzVr1qBv375QqVQYMGAAfvjhBydU6wAdGAdo6QJmCyARERF1hNsFwJ07dyI1NRX79+/H1q1bUVtbi3HjxkGvbznk7N27FzNmzMAjjzyCY8eOYerUqZg6dSpOnjzpxMrtJGoIIFcDVUVAYevB19ICmFGsh8nc9uLRRERERAAgEYR2nHbChQoLCxEaGoqdO3fi1ltvbXab+++/H3q9Hps2bbI+dvPNN+PGG2/EJ5980uZrlJeXQ6vVQqfTwc/Pz261d9qKKUB6GjDpLWDYn1rczGQWkPTiFhhNZuz+f7cjJlDjvBqJiIjIZa43u7hdC+C1dDodACAwMLDFbfbt24exY8faPDZ+/Hjs27fPobU5jLUbuPVxgDKpBHFBYujjUjBERETUXm4dAM1mM5555hmMHDkS/fv3b3G7vLw8hIWF2TwWFhaGvLy8Zrc3GAwoLy+3ubiVDpwXmEvBEBERUUe5dQBMTU3FyZMnsXr1arsed/HixdBqtdZLTEyMXY9/3SIHAwoNUFUMFJxpddPEEM4EJiIioo5x2wA4d+5cbNq0CTt27EB0dHSr24aHhyM/P9/msfz8fISHhze7/fz586HT6ayX7Oxsu9VtF3KvRusBtn5aOK4FSERERB3ldgFQEATMnTsX69evx88//4yEhIQ290lJScH27dttHtu6dStSUlKa3V6pVMLPz8/m4nbaOQ6wB7uAiYiIqIPkri7gWqmpqVi1ahW+//57+Pr6WsfxabVaqNVqAMCsWbMQFRWFxYsXAwCefvpp3HbbbXj77bdx5513YvXq1Th8+DA+++wzl72P65ZQP+P58i+A2QxIm8/qlqVgcnQ1qDLWQePldj9SIiIicjNu1wK4ZMkS6HQ6jBo1ChEREdbLN998Y90mKysLubm51vsjRozAqlWr8NlnnyE5ORlr167Fhg0bWp044vYiB4njAKtLgMKWxwEGeHvBX6MAAFwuqnJWdURERNSFuV1zUXuWJUxLS2vy2PTp0zF9+nQHVOQiMgUQezNw6WexFTCsX4ubJgZ742hWGdKLKnFDpBt2ZxMREZFbcbsWQGrEuhxM6+MALTOBMzgTmIiIiNqBAdCdNV4P0GxucTPLTGAuBk1ERETtwQDoziJvBBTeQHUpUHC6xc04E5iIiIg6ggHQnckUQFz9UjatrAeYUD8TOL1I364xlEREROTZGADdXTvWA4wL0kAiASpq6lBUaXRSYURERNRVMQC6u3aMA1QpZIgOENdIZDcwERERtYUB0N1F3Ah4+QA1ZUDBqRY3s3QD85RwRERE1BYGQHcnkwOx9eMAM1ruBk7kTGAiIiJqJwbArsA6DrDliSCcCUxERETtxQDYFSTUjwPM3NPiOMDGM4GJiIiIWsMA2BWEJwNevuI4wPwTzW6SWN8CmFVchVpTy4tGExERETEAdgUyeZvrAYb7qaBWyFBnFnCltNqJxREREVFXwwDYVTReDqYZUqkE8cEcB0hERERtYwDsKqwTQfYAZlOzmyRaJ4JwHCARERG1jAGwqwgfCCj9AIMOyGthHCCXgiEiIqJ2YADsKmRyIG6EeLuFbuBELgVDRERE7cAA2JW0cV7gRC4FQ0RERO3AANiVWAJg5t5mxwEm1LcAFlYYUFFT68zKiIiIqAthAOxKwgcCSi1gKAfyfmvytJ9KgWAfJQCeE5iIiIhaxgDYlUhlDeMAWzgvMGcCExERUVsYALuaNs4LzJnARERE1BYGwK7GEgCz9gGmuiZPcyYwERERtYUBsKsJHwCoWh4HaJkJzDGARERE1BK7BcDs7GxcuXLFev/gwYN45pln8Nlnn9nrJQioHwc4UrzdzHIwlpnAGUV6CILgzMqIiIioi7BbAHzwwQexY8cOAEBeXh7uuOMOHDx4EC+88AJefvlle70MAa2OA4wN1EAmlaDKaEJeeY2TCyMiIqKuwG4B8OTJkxg2bBgA4Ntvv0X//v2xd+9erFy5EsuXL7fXyxAAxN8iXmc2HQeokEkRG6gBAGRwJjARERE1w24BsLa2FkqluAbdtm3bcNdddwEA+vbti9zcXHu9DAFAWH9A5Q8YK4DcX5s8bZkJfInjAImIiKgZdguA/fr1wyeffILdu3dj69atmDBhAgAgJycHQUFB9noZAgCptNVxgD1CxYkgJ66UObEoIiIi6irsFgBff/11fPrppxg1ahRmzJiB5ORkAMDGjRutXcNkRwn13cDNjAMc1ScEAPDDiTxUG5ueMo6IiIg8m9xeBxo1ahSKiopQXl6OgIAA6+OPPvooNBqNvV6GLGzWA6wFZArrUzcnBCEmUI3skmr872Qupg2OdlGRRERE5I7s1gJYXV0Ng8FgDX+ZmZl49913ce7cOYSGhtrrZcgitB+gDgCMlU3GAUqlEkwfEgMA+PZwtiuqIyIiIjdmtwA4ZcoUrFixAgBQVlaG4cOH4+2338bUqVOxZMkSe70MWbQxDvCeIdGQSID96SXIKq5ycnFERETkzuwWAI8ePYpbbhHHpa1duxZhYWHIzMzEihUr8P7779vrZagxy3IwGU0DYJS/Gr/rGQwAWHuErYBERETUwG4BsKqqCr6+vgCAn376CdOmTYNUKsXNN9+MzMxMe70MNWYdB7hfHAd4jXuHiGP/1h65ApOZZwUhIiIikd0CYM+ePbFhwwZkZ2fjxx9/xLhx4wAABQUF8PPzs9fLUGOhNwDqQKBWD+Qcb/L0+H7h8FPJkaOrwd5LRc6vj4iIiNyS3QLgSy+9hHnz5iE+Ph7Dhg1DSkoKALE1cNCgQfZ6GWpMKgXiLeMAdzV5WqWQ4a4bIwEAaw5fafI8EREReSa7BcB7770XWVlZOHz4MH788Ufr42PGjMG///1ve70MXSu+5fUAAeC+m8TZwFtO5UFX1bSbmIiIiDyP3QIgAISHh2PQoEHIycnBlStii9OwYcPQt29fe74MNWYJgC2MAxwQpUXfcF8Y68zY+OtVJxdHRERE7shuAdBsNuPll1+GVqtFXFwc4uLi4O/vj1deeQVms9leL0PXCukLaIKA2irg6tEmT0skEutkkDVH2A1MREREdgyAL7zwAj788EO89tprOHbsGI4dO4ZXX30VH3zwAV588UV7vQxdq431AAHg7kFRkEsl+O2KDmfzyp1YHBEREbkjuwXAL7/8Ev/5z3/wxBNPYODAgRg4cCCefPJJfP7551i+fLm9Xoaak3CreN3COMAgHyXGJIlnY+FkECIiIrJbACwpKWl2rF/fvn1RUlLS7uPs2rULkydPRmRkJCQSCTZs2NDmPitXrkRycjI0Gg0iIiLw8MMPo7i4uCPld22W9QCzDwB1xmY3sUwGWX/sKox17JInIiLyZHYLgMnJyfjwww+bPP7hhx9i4MCB7T6OXq9HcnIyPvroo3Ztv2fPHsyaNQuPPPIITp06hTVr1uDgwYP405/+1O7X7PJC+gKaYHEcYE7TcYAAcFvvEIT4KlGiN+LnswVOLpCIiIjcidxeB3rjjTdw5513Ytu2bdY1APft24fs7Gz88MMP7T7OxIkTMXHixHZvv2/fPsTHx+PPf/4zACAhIQGPPfYYXn/99Y69ga5MIhFbAU9vEMcBxt7cZBO5TIppg6Pw6c50rDmcjQn9w51fJxEREbkFu7UA3nbbbTh//jzuvvtulJWVoaysDNOmTcOpU6fw3//+114v00RKSoo1ZAqCgPz8fKxduxaTJk1qcR+DwYDy8nKbS5dn6QZuYRwgAEwfInYDp50vREF5jTOqIiIiIjdk13UAIyMj8a9//QvfffcdvvvuO/zzn/9EaWkpvvjiC3u+jI2RI0di5cqVuP/+++Hl5YXw8HBotdpWu5AXL14MrVZrvcTExDisPqexrgd4AKgzNLtJz1AfDI71h8ksYN0xrglIRETkqewaAF3h9OnTePrpp/HSSy/hyJEj2LJlCy5fvozHH3+8xX3mz58PnU5nvWRnZzuxYgcJ6QN4hwB11c2uB2hhmQzy7eFsCILgrOqIiIjIjXT5ALh48WKMHDkSf/vb3zBw4ECMHz8eH3/8MZYuXYrc3Nxm91EqlfDz87O5dHmWcYBAq93Adw6MgEohRXqhHkezypxTGxEREbmVLh8Aq6qqIJXavg2ZTAYAntfCZQ2Au1rcxFelwKQBEQCANYe7QcsnERERddh1zwKeNm1aq8+XlZV16HiVlZW4ePGi9X5GRgaOHz+OwMBAxMbGYv78+bh69SpWrFgBAJg8eTL+9Kc/YcmSJRg/fjxyc3PxzDPPYNiwYYiMjOzw++nSLOMAsw+K4wDlymY3mz4kBuuOXsWm33Lx0uQboPGy22RwIiIi6gKu+19+rVbb5vOzZs1q9/EOHz6M22+/3Xr/2WefBQDMnj0by5cvR25uLrKysqzPz5kzBxUVFfjwww/x17/+Ff7+/hg9erRnLQNjEdwb8A4F9AXA1SNA3IhmNxueEIjYQA2ySqrwvxN5uKf+XMFERETkGSSCx/WTNlVeXg6tVgudTtf1xwOu+QNwah0w6u/AqOda3OyD7Rfw9tbzuDkxEKsfTXFigURERHS9rje7dPkxgHQN6zjA3a1uds+QaEgkwP70EmQW651QGBEREbkLBsDuJuFW8frKIaC25cWeI/3V+F3PYADA2iNXnFEZERERuQkGwO4mqCfgEwbU1QBXD7e66fT6NQG/O3IFJrPHjwQgIiLyGAyA3U3j9QDPtn4O5nE3hMFPJUeOrgZ7LhY5oTgiIiJyBwyA3dGA6eL1wU+B/FMtbqZSyDDlxigAwBp2AxMREXkMBsDuqPcEoO/vAXMdsPHPgNnU4qaWU8P9eCoPuqpaZ1VIRERELsQA2B1JJMCkNwGlnzgO8NB/Wty0f5Qf+ob7wlhnxsZfrzqxSCIiInIVBsDuyi8SGLtQvL1tEVDW/GnfJBKJdTLIt4fZDUxEROQJGAC7syF/AGJTgFo9sPlZoIU1v6feGAmFTIITV3U4k1vu5CKJiIjI2RgAuzOpFJj8PiDzAi78BJz8rtnNgnyUGNM3DACwhq2ARERE3R4DYHcX0hu49W/i7f89B1SVNLvZfUPF8wFvOH4Vxjqzs6ojIiIiF2AA9AQjnwFCkoCqIuDHF5rd5NZeIQj1VaJEb8TPZ/OdWx8RERE5FQOgJ5B7AXd9AEAC/LoKuLSj6SYyKaYNFlsBORmEiIioe2MA9BQxQ4Fhj4q3/+9pwFjVZJPpN4kBMO1cAfLLWz6PMBEREXVtDICeZMyLgF80UJYJpL3a5OkeIT4YEhcAswCsO8o1AYmIiLorBkBPovQFfv+OeHvfR0DO8Sab3FffCrjmcDaEFpaNISIioq6NAdDT9B4P9L8HEMzAxqcAU53N03cOjIRaIUN6kR5Hs0pdVCQRERE5EgOgJ5rwOqAOAPJ+A/Z/ZPOUj1KOSQMiAADfHuJkECIiou6IAdAT+YQA4/4l3t7xKlB8yeZpy2SQTb/loMpYd+3eRERE1MUxAHqqGx8EEm4D6mqATX+xOU3c8IRAxAVpoDea8MOJPBcWSURERI7AAOipJBJg8ruAXA1k7ASOr2r0lATThzRMBiEiIqLuhQHQkwUmArfPF2//+HegssD61LTB0ZBIgAMZJcgs1ruoQCIiInIEBkBPd3MqED4QqCkTzxVcL9JfjVt6hQAA1h7hZBAiIqLuhAHQ08nk4mniJDLg1Drg3BbrU5Zu4LVHrsBk5pqARERE3QUDIAGRNwIpqeLtzc8ChgoAwB03hEGrViBXV4NfLha5rj4iIiKyKwZAEo2aDwTEA+VXge0vAwBUChmm3BgJgJNBiIiIuhMGQBJ5aYDJ74m3D34OZB8EANx3UwwA4KdT+SirMrqqOiIiIrIjBkBqkDgKuHEmAEE8TVydEf0i/dA33BdGkxkbf81xdYVERERkBwyAZGvcPwHvEKDwLPDLvyGRSKytgN+yG5iIiKhbYAAkW5pAYOLr4u3dbwGF5zB1UBQUMglOXi3H6Zxy19ZHRERE140BkJrqNw3oNR4wGYGNf0agWo6xSWEAgDVH2ApIRETU1TEAUlMSCXDn24CXD5C9Hziy1NoNvOHYVRjrzC4ukIiIiK4HAyA1zz8GGPOSeHvrQtwSZkSorxKlVbXYfibftbURERHRdWEApJYN/SMQPRQwVkC+5f/hnsFRADgZhIiIqKtjAKSWSWXiaeKkCuDcZswJ+A0AsPN8IfLLa1xcHBEREXUWAyC1LjQJ+N1fAABhv7yI22LkMAvAd0evuLgwIiIi6iwGQGrbrfOA4N5AZT4WqL4BAKw9fAWCILi4MCIiIuoMBkBqm1wJTH4fAJCY/R1uVZxFepEeRzJLXVwYERERdQYDILVPXApw08MAgLfVS6GEkZNBiIiIuigGQGq/sQsB3wiEGK/gz/J12PxbLvSGOldXRURERB3kdgFw165dmDx5MiIjIyGRSLBhw4Y29zEYDHjhhRcQFxcHpVKJ+Ph4LF261PHFehqVVlwgGsBj8s2IrU3HDydyXVwUERERdZTbBUC9Xo/k5GR89NFH7d7nvvvuw/bt2/HFF1/g3Llz+Prrr9GnTx8HVunB+t4JJN0FOUx4TfE51h7OdHVFRERE1EFyVxdwrYkTJ2LixInt3n7Lli3YuXMn0tPTERgYCACIj493UHUEAJj0JszpaUg2pKN/9te4XDQI8cHerq6KiIiI2sntWgA7auPGjbjpppvwxhtvICoqCr1798a8efNQXV3t6tK6L99wSMf9EwDwV/ka/LTnoIsLIiIioo5wuxbAjkpPT8cvv/wClUqF9evXo6ioCE8++SSKi4uxbNmyZvcxGAwwGAzW++Xl5c4qt/sYPAvF+/6LoKJDGHB8IUy/vxUymczVVREREVE7dPkWQLPZDIlEgpUrV2LYsGGYNGkS3nnnHXz55ZcttgIuXrwYWq3WeomJiXFy1d2ARALf6R/BAAVShOMofvd3QOY+V1dFRERE7dDlA2BERASioqKg1WqtjyUlJUEQBFy50vzpyubPnw+dTme9ZGdzPbvO8ArrgwP9F6JCUCO04jSwbALw7SygJMPVpREREVErunwAHDlyJHJyclBZWWl97Pz585BKpYiOjm52H6VSCT8/P5sLdc6t987F0sFrsapuNEyCBDj9PfDRMOCnF4EanavLIyIioma4XQCsrKzE8ePHcfz4cQBARkYGjh8/jqysLABi692sWbOs2z/44IMICgrCH/7wB5w+fRq7du3C3/72Nzz88MNQq9WueAse56nJI3F4wAJMMi7GHmEAYDICe98H3h8EHPwcMHGxaCIiInfidgHw8OHDGDRoEAYNGgQAePbZZzFo0CC89NJLAIDc3FxrGAQAHx8fbN26FWVlZbjpppswc+ZMTJ48Ge+//75L6vdEUqkEr987EBG9h2Cm4XnMlcyH0b8nUFUM/DAPWDICuLDV1WUSERFRPYkgCIKri3C18vJyaLVa6HQ6dgdfhypjHR78/ACOZ5ch2k+OTSMvwn//W0B1ibhBjzHAuH8CYTe4tlAiIqIu7nqzi9u1AFLXpfGSY9mcoegR4o0r5XWYfqQ/dH86CKTMBaQK4NJ24JORwP89A1QWurpcIiIij8UASHYV4O2FFY8MR7ifChcKKvHwNxdQffvLwNyDQNJdgGAGjiwTxwf+8m+gtsbVJRMREXkcBkCyuyh/Nb58eBj8VHIcySxF6qqjqNXGA/f/F5jzAxBxI2CsALYtBD4aCpxcB3AkAhERkdMwAJJD9An3xdI5Q6GUS/Hz2QLMX3cCgiAA8SOBP+0A7v4U8I0EyrKAtX8Alo4HrhxxddlEREQegQGQHOam+EB89OBgyKQSrD1yBW/8eE58QioFkh8AnjoMjPo7oNAA2QeA/4wGvvsjUMaFuYmIiByJAZAcauwNYVh89wAAwJK0S/jil0ZnCfHyBkY9Bzx1FLhxJgAJcGIN8OFNwPZXAEOFa4omIiLq5hgAyeHuGxqDv43vAwB4ZdNpbDh21XYDvwhg6sfAo2lA3O+Auhpg91vAB0OAoysAs8n5RRMREXVjDIDkFE+O6oE/jIwHAMxb8yt2nW9mGZjIG4E5m4D7VwKBiUBlPrDxKeDTW4H0NGeWS0RE1K0xAJJTSCQSvHjnDbgrORJ1ZgGPf3UEv2aXNbchkPR74MkDwPjFgEoL5J8EVkwBVj0A5J92eu1ERETdDc8EAp4JxJmMdWY8vPwQfrlYhEBvL6x5PAU9Qnxa3qGqBNj5OnDoP4C5/pzCETcCA6YD/e8Ru4+JiIg8zPVmFwZAMAA6W6WhDg9+vh+/XdEhyl+NdU+OQJifqvWdii4A218Gzv3QEAQhARJuAQbcByRNBtT+ji6diIjILTAA2gEDoPMVVRow/ZN9yCjSo2+4L755LAVataLtHfVFwOkNwG9rgOz9DY/LlEDvcWLLYK/xgKKNQElERNSFMQDaAQOga2SXVGHakr0orDBgWHwgVjwyDCqFrP0HKM0ETq4Vw2DhmYbHlVrghsliGIy/BZB24JhERERdAAOgHTAAus7pnHLc/+k+VBjqMO6GMHw8czDksg7OTRIEIP8UcOJb4MR3QPmVhud8wsWxggOni2MHJRK71k9EROQKDIB2wADoWvvTizFr6UEY68yYMSwGr949AJLOBjWzGcjaJ4bBUxuAmrKG54J6ia2CA+4FgnrYo3QiIiKXYAC0AwZA19tyMhdPrjwKswD8eXRPPDuuz/UftM4IXNwmnl3k3A/iAtMWUUPEySP97gZ8w67/tYiIiJyIAdAOGADdw8oDmXhh/UkAwKK7+mH2iHj7HdxQAZzZJIbB9B2AYBYfl0iBxFFiy2Df3wMq/vyJiMj9MQDaAQOg+3hv2wX8e9t5SCTABzMG4fcDI+3/IpUFwKn1wG/fAlcPNzwuVwG9JwAD7wN63gHIvez/2kRE1LWZzUCtHjBaLpXitaGy4bZRL57Rqvc4h5XBAGgHDIDuQxAEvPT9Kfx3fyYUMgmW/2EYRvYMdtwLFl8CTn4nhsHiCw2Pq/yBHrcDsSlA7M1AWH/OJiYi6qpMtUB5jjguvLmwZmzuvr75bWv17XvNAdOBe/7jsLfEAGgHDIDuxWQW8NTXR/HDiTz4KOVY/ejN6B+ldeyLCgKQ+6vYRXxiLVCZZ/u8ly8QM6whEEYNAbw0jq2JiIjaZjYDVUWALhvQXQXKrwK6Kw2X8qtARR4AO8cdiRTw8gG8vBtdfBoei70ZGP6YfV+zEQZAO2AAdD+GOhPmLD2EfenFCPbxwtrHRyA+2Ns5L242AdkHgMw9QNZ+IPsgYCi33UYqF5eVib25PhSmAN5BzqmPiMiT1JQ3H+oa3zYZ2z6OzAvQBDUf1qz3vQHltY97i40A126nULt0aTEGQDtgAHRPFTW1uP/T/TidW47YQA3WPpGCUF8XnOHDbAIKTgOZ+8QlZrL2ARW5TbcL7t0oEN4MBCRw3UEiotbUGcSuWWuoq2/Faxzyrv0PeLMkgG844BcFaKMbLo3va4IBaQfXmXVjDIB2wADovgoqanDvkn3IKqnCDRF++Oaxm+Grascp4xxJEICyLLF10BIIC8823c4n3DYQhvUHZHLn10tE5GyCAFSXiv9ZLs8FKnLE6/Krto9VFbfveCr/5kOd5b5vhMdN3GMAtAMGQPd2uUiPez/Zi6JKIwbH+uPd+wchNsjNxt9VlYjdxln7xGB49ShgrrXdxssHiB4qBsK4lPpxhE7q1iYishdTLVCZ3yjY1V+uDXt11e07nlzVesudX5TYLUs2GADtgAHQ/Z28qsMDn+1HpaEOKoUUz4ztjUd+lwBFR08b5yy11UDOMSBzb/04wgMtjCNMBmKGAyF9geBe4tlKvIPZdUxE7Wc2A4JJHK5ic92Zx+ufq6sRJ040CXY54lJa7Z1QoQ4E/CLFFjq/CMA3UrxvfSwSUAfwb14nMADaAQNg13C5SI+/rz+BvZfELoOkCD+8Nm0AkmP8XVtYe5hNQMGZhhbCrH1iV0hzVFoxCAb3AoJ6NgTDwERA4YIxkETkWIIgLk9SkS+2rFUWiCsRVOY3eqz+8TpDQ0izBDdXkMrFANdisKt/TqF2TX0egAHQDhgAuw5BEPDd0av45+bTKKuqhVQCzB4Rj7+O6wMfZRcbX2cZR3j1CFB0QVyHsCwbLf/PWgL4xzQfDv0i+T9oIndTZwT0hfVhrkBsUbOGu8b38wGTwXF1SGTiOqY211JxGZMmj9Xfl3kBPqGNgp0l5EWIXbLdbEJFV8QAaAcMgF1PUaUB/9x0GhuO5wAAIrUq/PPu/hjdt4uf17e2GihJbwiERRcbrg26lvdTeANBPRoCoSUgBvXk2Bmi9jKbxSBWV38xGcQQZzKIXaLW20bb7WrKmg931SUde32VVpw85hMqzmj1CWu4+IYB3qFiL0CLga6Fx6lbYgC0AwbArmvn+UL8Y8MJZJeIg43vHBiBBZNvcM1yMY4kCGJLgjUYXhDPYlJ8ASjJaL0byDeiobXQP078R0blBygt134N117ebEkk92aqE8fT1pQB1WVAjU68XaMTL5bHDBXXhDljo1B3zWOWx8119q9XKrcNctZwF1of9q4Jd0TtxABoBwyAXVuVsQ7vbbuA//ySAZNZgK9Kjr9PSsL9N8VAKvWAMGOqBUovXxMOL4rXVUUdO5ZEBih9Ww6ITa61TR/38mGIpJYJgng6rcbBrbUgd+02xgonFSoB5EpAphSXF5GrxG5RubL+WiU+LlOK33ublrtG4U4dwFY4cggGQDtgAOweTl7VYf66EzhxVewqHRYfiFen9UfPUF8XV+ZC1aWNupEviLP5asrrW1B09df19+3V+iGR1odIbf3F/5rrZi7qRs8xQHZtltbqknSxdbokXbyUZgClmWKgs8d3TeHd6Hvj3/S7pPS1DW2Nw5xMaRvgrAGv0fMyBb+H5NYYAO2AAbD7qDOZ8eW+TLz90zlUGU3wkknx5O098MSoHlDKZa4uz30Jgjj+0BIIa3TimENrWGzp2gEhUiJtISj6Nx8k1f71sw6juNC2s5jN4ix2S7CzhLySy+J1rb7tY0jlDT/La4Nc4/8Q2DzW6HGZixeEJ3IxBkA7YADsfq6UVuHFDSex41whAKBHiDcWTxuIYQmBLq6sG2suRF7brdfspayhi+/axbM7QiKtXzw2Rpwt7R/b6HacuKCsXGmnN2tnRn2j5T8sy33UNHO+0mZOPO+osx+YasWZ6o1b8ixhr/RyG+delYiffWC8uHxRYKJ4asSAeHGdS5UWUGjYwkZ0HRgA7YABsHsSBAGbT+Ri4cbTKKoUl1iYMSwWz0/sC62arQduRxDE0NNcSKwubTlAVpeIi9O252TwPmGNgmGsGA61sQ237XlmFlNd/RIgjYPdNWu6Wa6NlZ1/HamihZPXX3uS+2ZOem+5VBY0asmrv9Zli+vNtfa6AXFisLOEvMD62/6x7hu2iboJBkA7YADs3nRVtXhtyxl8fTAbABDiq8TCyf0waUA4JGyB6B7MZkBfIK6jWJYphpeybLEFS1d/XVvV9nHUgc0HQ0tgVGnFFktLeLt2kd7G11XFaPfZEgCxRazxTFG5SqzZWCm2EBr14m1D/X1HrhtnIVc3CnYJtmFPGy0uNUJELsEAaAcMgJ7hQHox5q8/gfRCcXzSmL6heGVqf0T6c6X6bk8QxPM167LEMFiW3RAMy7LFx2taWWfRQiLr2JkXJLL6GaGhDcGuccizzBz1Cev4eo2m2kbBsD4cXhsWLbcNFc0/brmvDmjaiheQIM5o5X+SiNwSA6AdMAB6jppaEz5Ou4QlaRdRaxLg7SXDX8f1wewR8ZB5wpIx1LIa3TXBMKtRC2K27ZI6Ku01a7uFNR/yNEFcAoSIHIIB0A4YAD3PhfwKzF93AoczSwEAydFaLJ42EDdE8udPLTDqxYkqmiAu2EtELscAaAcMgJ7JbBbw9aEsvPbDWVQY6iCTSvCnWxLx9JheUHtxbBMREbmv680u7JsgjyWVSjBzeBy2/fU2TBoQDpNZwCc7L2H8u7uw+0Khq8sjIiJyGLYAgi2AJNp6Oh8vfX8SuboaAOIkkYd/l4ARPYI4W5iIiNxKt2sB3LVrFyZPnozIyEhIJBJs2LCh3fvu2bMHcrkcN954o8Pqo+7rjhvCsPXZ2zBnRDwkEmD72QLM/M8BTHh3N74+mIWa2g7M/iQiInJjbhcA9Xo9kpOT8dFHH3Vov7KyMsyaNQtjxoxxUGXkCXyUciy8qx+2P3sbZqXEQeMlw7n6CSM3L96O17ecRa6u2tVlEhERXRe37gKWSCRYv349pk6d2ua2DzzwAHr16gWZTIYNGzbg+PHj7X4ddgFTS3TVtVhzOBvL917GlVIx+MmkEkzsH44/jEzA4Fh/dg8TEZHTdbsu4M5YtmwZ0tPTsWDBgnZtbzAYUF5ebnMhao5WrcAfb0nEzr/djk8fGoLhCYEwmQVs+i0X9yzZi6kf7cH3x6/CWNfKKbOIiIjcTJcPgBcuXMDzzz+Pr776CnK5vF37LF68GFqt1nqJiYlxcJXU1cmkEozvF45vHkvB5j//DtOHRMNLLsWvV3R4evVx/O71n/HB9gsornTC6bmIiIiuU5cOgCaTCQ8++CAWLVqE3r17t3u/+fPnQ6fTWS/Z2dkOrJK6m36RWrw5PRn7nh+Nv97RG6G+ShRUGPD21vNIee1n/G3Nrzidw1ZlIiJyX116DGBZWRkCAgIgkzUs2ms2myEIAmQyGX766SeMHj26zdfhGEC6HsY6M344kYtlezLw65WG88nenBiIP4xMwNikMJ5mjoiI7Op6s0v7+kzdlJ+fH06cOGHz2Mcff4yff/4Za9euRUJCgosqI0/iJZdi6qAoTLkxEkezyrBsTwb+dzIP+9NLsD+9BDGBasxOicd9Q2Pgp1K4ulwiIiL3C4CVlZW4ePGi9X5GRgaOHz+OwMBAxMbGYv78+bh69SpWrFgBqVSK/v372+wfGhoKlUrV5HEiR5NIJBgSF4AhcQHI1VXjv/sysepgFrJLqvHPzWfwztbzuHdINOaMiEdiiI+ryyUiIg/mdgHw8OHDuP322633n332WQDA7NmzsXz5cuTm5iIrK8tV5RG1S4RWjf83oS+eGt0LG45fxbI9GTifX4kV+zKxYl8mbu8Tgj+MTMAtvYK5jAwRETmdW48BdBaOASRHEwQBey8VY9meDGw/WwDLb13PUB/MGRGP3w+MgL/Gy7VFEhFRl3G92YUBEAyA5FyXi/T4ct9lrDl8BZWGOgDiMjMpiUEY3z8c428IQ6ifysVVEhGRO2MAtAMGQHKFipparDl8Bd8ezsbZvAqb5wbH+mNC/3CM7xeOuCBvF1VIRETuigHQDhgAydUuF+nx46k8bDmVh2NZZTbPJUX4YXy/MEzoH44+Yb4cM0hERAyA9sAASO4kT1eDrafFMLg/vQQmc8OvaHyQRuwm7heOG6P9IeX6gkREHokB0A4YAMldleqN2H62AFtO5mHXhUKbcw6H+Skxvl84JvQLx7CEQMhlXfrEPkRE1AEMgHbAAEhdgd5Qh7RzhdhyKg87zhZYJ5AAgL9GgbFJYZjQLxy/6xUMlULWypGIiKirYwC0AwZA6moMdSbsvViMLSfzsPVMPkr0Rutz3l4yjOobivH9wnF7nxD48uwjRETdDgOgHTAAUldWZzLjcGYptpzMw4+n8pCrq7E+5yWT4ne9gjGhXzjG3hCGQG+uNUhE1B0wANoBAyB1F4Ig4LcrOnFG8ck8pBfprc9JJcDQ+EDc2jsEt/QKRv9ILSeREBF1UQyAdsAASN2RIAi4WFCJLSfFGcWncsptng/QKDCyZzBu6RWMW3qFINJf7aJKiYiooxgA7YABkDxBdkkV0s4XYvf5Quy7VIyKRpNIAKBHiDdu6SW2Dt6cGARvpdudKpyIiOoxANoBAyB5mjqTGb9eKcOu80XYfaEQx7PL0Gi5QShkEgyODbC2DvaP0kLG7mIiIrfBAGgHDIDk6XTVtdh3qRi7LxRi94UiZJVU2Tzvb+ku7hmMW3qHIIrdxURELsUAaAcMgES2Mov12H1BbB3ce6kYFTW23cWJId5iGOwVgpt7BMGH3cVERE7FAGgHDIBELRO7i3XW1sHj2WU2p6eTSyUYHBdgbR0cwO5iIiKHYwC0AwZAovYrr2noLv7lQhEuF9t2F2vVCozsGYSUxCAMjgtAnzBfnqaOiMjOGADtgAGQqPOyiquw+6IYBvdcLEL5Nd3F3l4yJMf4Y0hcAAbHBmBQrD/8NVyQmojoejAA2gEDIJF91JnM+O2qDr9cKMKhyyU4nlXWZLkZQFxyxhIIh8QFoEeIDxelJiLqAAZAO2AAJHIMk1lcjPpIZimOZpXiaGapzdlJLPxUcgyqD4ODYwOQHKPlOYyJiFrBAGgHDIBEzlOiN+JYlhgIj2SW4tdsHaprTTbbSCVA7zBfm1bCuCANJBK2EhIRAQyAdsEASOQ6dSYzzuZVWFsJj2SW4kppdZPtgry9MCg2AIPj/DEkNgADo/2h9pK5oGIiItdjALQDBkAi91JQXmMNg0ezynDiig5Gk9lmG7lUghsi/axdxgOi/JEY7M2xhETkERgA7YABkMi9GepMOJVTjqOZYig8klmKggpDk+28vWToF6XFgCgtBkZr0T9Ki4QghkIi6n4YAO2AAZCoaxEEAVfLqnE0qwxHM0tx4qoOp3J0qKk1N9nWRylHv0g/ayAcGO2PuEANQyERdWkMgHbAAEjU9dWZzLhUqMeJqzqcuFJWHwrLYahrGgp9lXL0j9JiQLTYWjggSstJJkTUpTAA2gEDIFH3VGcy42JhJX67osPJqzr8dkWH07nlMDYTCv1UDaFwYJQ/BkRpEROoZigkIrfEAGgHDIBEnqPWZMaF/EoxEF4tw4mr5TjTQijUqhViC2GjlsLoAIZCInI9BkA7YAAk8my1JjPO51fgxBWd2IV8VYezuRVNZh4DYvdxn3Bf9I3wRVKEH/qG+6FPuC98lHIXVE5EnooB0A4YAInoWsY6MRT+Zg2FZTiXV4FaU/N/MmMDNegb7ou+EX5Iqr/mZBMichQGQDtgACSi9qg1mZFeqMfZvHKcya2ovy5HfnnTJWkAQK2QoU+4L5IifNE33E8MiOF+0Gp4mjsiuj4MgHbAAEhE16NEb8TZvHKcrQ+FZ/MqcC6votkZyAAQqVWhb4QYCJMi/JAU4Yv4IG/IZVInV05EXRUDoB0wABKRvZnMAi4X63E2twJncsutrYZXy5qe5g4AvORS9A7zsWkp7BXmg1BfJSedEFETDIB2wABIRM5SXlOLc3kVOJtbjjP112fzKlBlNDW7vZ9Kjl5hvugV6oOeoT7oHeaLXmE+CPdTMRgSeTAGQDtgACQiVzKbBVwprcZpa0thOS7kV+JysR7mFv5C+yjl6Bnqg16hPugV5mMNiZFaNSeeEHkABkA7YAAkIndkqDMho0iPC/mVuFBQiQv5FbhQUInLRXrUtZAMNV4y9GzcWhjqg16hvogOYDAk6k4YAO2AAZCIuhJjnRmZxXqcz6/EhQIxFF7Mr0R6UWWLy9SoFFL0CLG0GPpar2MDNZAxGBJ1OQyAdsAASETdQa3JjMziKlwsqGhoNSyoxKXCymbPdAKIk08Sg73RI8QHPUK80SPUB4nBPkgM8YY3F7cmclsMgHbAAEhE3VmdyYzs0mprF7Ll+mJBZYtL1QBAhFbVJBj2CPXmBBQiN8AAaAcMgETkiUxmAVdKq5BeqMelQrGl8FKBeLtYb2xxP28vGRLrg6F4LQbD+CBvqBQyJ74DIs/FAGgHDIBERLbKqoy41CgYWkJiZnEVTC1MQJFIgJgAjW0wrG89DPL2YqshkR11uwC4a9cuvPnmmzhy5Ahyc3Oxfv16TJ06tcXt161bhyVLluD48eMwGAzo168fFi5ciPHjx7f7NRkAiYjax1hnRlZJVZNgeLGgEhU1dS3up1UrkBjibR1fmBgshsS4IA1bDYk64Xqzi9uN8NXr9UhOTsbDDz+MadOmtbn9rl27cMcdd+DVV1+Fv78/li1bhsmTJ+PAgQMYNGiQEyomIvIcXnKpdZmZxgRBQFGl0SYUWi5XSquhq67FsawyHMsqs9lPIgGi/NVIqJ+IkhDsLQbEEB9E+Km4dA2Rg7hdC2BjEomkzRbA5vTr1w/3338/XnrppXZtzxZAIiLHqak14XKxHpcK9EgvrERGkR6XisTbrbUaqhRSxAd5W1sOreEw2AdajcKJ74DI/XS7FsDrZTabUVFRgcDAQFeXQkREAFQKWf05jm3/kRIEAcV6I9IL9cgoElsO0+uDYVZJFWpqzTibV4GzeRVNjhnk7YXEEO/6UOhT34LojZhADZRydikTtaXbBcC33noLlZWVuO+++1rcxmAwwGAwWO+Xl5c7ozQiImpEIpEg2EeJYB8lhiXY/qe9zmTGldJqsbWwsBLpRXpkFOqRXlSJ/HIDivVGFOuNOHS51GY/qQSICdQgIViclRwXpLFeRwdo4CWXOvMtErmtbhUAV61ahUWLFuH7779HaGhoi9stXrwYixYtcmJlRETUEXKZFPHB3ogP9sbtfW3/nusNdcgoamgtFFsQxdt6owmZxVXILK4CUGizn1QCRAWorYEwLrA+IAZ7IzaQk1HIs3SbMYCrV6/Gww8/jDVr1uDOO+9sddvmWgBjYmI4BpCIqAsTBAGFFQZcqg+EmSV6ZBZV4XKxHpnFVaiuNbW6f4RW1ajF0BvxQRrEBmkQF+QNH54VhdwMxwAC+Prrr/Hwww9j9erVbYY/AFAqlVAqlU6ojIiInEUikSDUT4VQPxVSegTZPGcJh5klVbhcJAZCSzC8XKRHhaEOuboa5OpqsD+9pMmxg32UiK8Pg/FBGsQF118HenNCCnVJbhcAKysrcfHiRev9jIwMHD9+HIGBgYiNjcX8+fNx9epVrFixAoDY7Tt79my89957GD58OPLy8gAAarUaWq3WJe+BiIjcS+NwODTedryhIAgoraqtD4R6XC6qQmaxHpklYldyid6IokoDiioNOJxZ2uTYWrUCUf5qRAeoER2gQXSAGlEBDfe1agZEcj9u1wWclpaG22+/vcnjs2fPxvLlyzFnzhxcvnwZaWlpAIBRo0Zh586dLW7fHlwGhoiIWqKrrkWWtcVQj8vFVdbrwgpDm/v7KuX1gVBTHwobwmGUvxr+GgXPkkId1u3OBOIKDIBERNQZekMdrpRW42pZFa6UVou3S6txpVS839o5lS28vWRiGGwUDqP8G8JiIE+jR83gGEAiIiIX8VbK0SfcF33CfZt9vspYh5yyamQ3Ew6vllWjsMIAvdGEc/kVOJffdL1DAFArZDbhMCZAg5hATf21Glo1WxCp4xgAiYiIHETjJUfPUF/0DG0+INbUmnC1zBIMxXB4tazhdkGFAdW1JlwsEM+33BxfpRzRgRrEBKjrg6EasUFiQIwO0EDtxeVtqCkGQCIiIhdRKWToEeKDHiE+zT5vqDMht6zGGgizS6uQXVJtvS6qNKDCUIczueU4k9v8SQ2CfZSICVRbWwwbtyBG+KugkHFxbE/EAEhEROSmlHKZdUHs5lQbTbbBsMQ2JFbU1FlnMB/LKmuyv1QCRGjVtsGw0e0QHyWkUnYvd0cMgERERF2U2kuGXmG+6BXWfBezrqq2PhBe23pYhezSahjrzGIXdFk19qPp+odeMqnN+EPbmcwMiF0ZAyAREVE3pdUooNVo0T+q6bq4ZrOAwkqDbThsdDuvvAZGkxkZReKZVZpjGxBtw2FMgBrBDIhuiwGQiIjIA0mlEoT5qRDmp8JN1yyODQB1JjPyysXxh9klDcvcWGYx5+qq2w6Icimi/dXNrIPIgOhqXAcQXAeQiIioo2pNZuTpamxCYbZliZv6gGhuI2Eo5VJE+asR6a+2Xkf6qxAVIN6P0KrhJeckleZwHUAiIiJyOoVMWj9pRAMgqMnzloBoCYWNg6IlIBrqzEgv0iO9hRZEiQQI8VGKAbE+FEZqVYgK0IhB0Z/rIHYWAyARERHZnW1AbMpYZ2lBrEKOrgZXS6uRU1aNHJ0YEK+WiQGxoMKAggoDjmeXNXscby9Zfctho5Dor0KUvxgSw/1UkHOpmyYYAImIiMjpvORSxAZpEBvUfEAUBAEleiOulonB8GpZ05BYrDdCbzThQkElLrSwULZUAoT7qRDpr0aEvxoRWjEURmhVCK+/hPgoPS4kMgASERGR25FIJAjyUSLIR4mB0f7NblNTa6oPh82HxJyyatSaBOToapCjqwEyS5s9jlQChPqKYTBCK06MsQTECK0YGkP9lFDKu89ZVRgAiYiIqEtSKWRIDPFBYgtnUjGbBRRVGqxrHebpapCrq0GergZ55eJ1fnkN6syCeL+8BsezW369IG8va0i0hEPbsKiCxqtrRKuuUSURERFRB0mlEoT6qRDqp8Kg2IBmtzGZBRRXGpBbHw7zyy0hsVq8rg+KhjozivVGFOuNOJXT/Gn3AMBPJUe4VoUxSWF4bkJfR72168YASERERB5L1igkJsc0v40gCCirqq0PhGIwzLe0JpY3tCpWGupQXlOH8ppKDIjyd+r76CgGQCIiIqJWSCQSBHh7IcDbCzdEtrzmXkVNrbUFMUDj5cQKO44BkIiIiMgOfFUK+KoU6Bna/LmZ3YlnzXkmIiIiIgZAIiIiIk/DAEhERETkYRgAiYiIiDwMAyARERGRh2EAJCIiIvIwDIBEREREHoYBkIiIiMjDMAASEREReRgGQCIiIiIPwwBIRERE5GEYAImIiIg8DAMgERERkYdhACQiIiLyMAyARERERB5G7uoC3IEgCACA8vJyF1dCRERE1DZLZrFkmI5iAARQUVEBAIiJiXFxJURERETtV1FRAa1W2+H9JEJno2M3YjabkZOTA19fX0gkEleX4zLl5eWIiYlBdnY2/Pz8XF2OS/GzaMDPQsTPoQE/iwb8LBrws2jgjM9CEARUVFQgMjISUmnHR/SxBRCAVCpFdHS0q8twG35+fh7/y2vBz6IBPwsRP4cG/Cwa8LNowM+igaM/i860/FlwEggRERGRh2EAJCIiIvIwDIBkpVQqsWDBAiiVSleX4nL8LBrwsxDxc2jAz6IBP4sG/CwadIXPgpNAiIiIiDwMWwCJiIiIPAwDIBEREZGHYQAkIiIi8jAMgB5i8eLFGDp0KHx9fREaGoqpU6fi3Llzre6zfPlySCQSm4tKpXJSxY6zcOHCJu+rb9++re6zZs0a9O3bFyqVCgMGDMAPP/zgpGodKz4+vslnIZFIkJqa2uz23ek7sWvXLkyePBmRkZGQSCTYsGGDzfOCIOCll15CREQE1Go1xo4diwsXLrR53I8++gjx8fFQqVQYPnw4Dh486KB3YB+tfQ61tbV47rnnMGDAAHh7eyMyMhKzZs1CTk5Oq8fszO+YO2jrOzFnzpwm72vChAltHrerfSeAtj+L5v5uSCQSvPnmmy0esyt+L9rzb2dNTQ1SU1MRFBQEHx8f3HPPPcjPz2/1uJ39+2JPDIAeYufOnUhNTcX+/fuxdetW1NbWYty4cdDr9a3u5+fnh9zcXOslMzPTSRU7Vr9+/Wze1y+//NLitnv37sWMGTPwyCOP4NixY5g6dSqmTp2KkydPOrFixzh06JDN57B161YAwPTp01vcp7t8J/R6PZKTk/HRRx81+/wbb7yB999/H5988gkOHDgAb29vjB8/HjU1NS0e85tvvsGzzz6LBQsW4OjRo0hOTsb48eNRUFDgqLdx3Vr7HKqqqnD06FG8+OKLOHr0KNatW4dz587hrrvuavO4HfkdcxdtfScAYMKECTbv6+uvv271mF3xOwG0/Vk0/gxyc3OxdOlSSCQS3HPPPa0et6t9L9rzb+df/vIX/N///R/WrFmDnTt3IicnB9OmTWv1uJ35+2J3AnmkgoICAYCwc+fOFrdZtmyZoNVqnVeUkyxYsEBITk5u9/b33XefcOedd9o8Nnz4cOGxxx6zc2Wu9/TTTws9evQQzGZzs8931+8EAGH9+vXW+2azWQgPDxfefPNN62NlZWWCUqkUvv766xaPM2zYMCE1NdV632QyCZGRkcLixYsdUre9Xfs5NOfgwYMCACEzM7PFbTr6O+aOmvssZs+eLUyZMqVDx+nq3wlBaN/3YsqUKcLo0aNb3aY7fC+u/bezrKxMUCgUwpo1a6zbnDlzRgAg7Nu3r9ljdPbvi72xBdBD6XQ6AEBgYGCr21VWViIuLg4xMTGYMmUKTp065YzyHO7ChQuIjIxEYmIiZs6ciaysrBa33bdvH8aOHWvz2Pjx47Fv3z5Hl+lURqMRX331FR5++OFWz4ndXb8TjWVkZCAvL8/m567VajF8+PAWf+5GoxFHjhyx2UcqlWLs2LHd6rui0+kgkUjg7+/f6nYd+R3rStLS0hAaGoo+ffrgiSeeQHFxcYvbesp3Ij8/H5s3b8YjjzzS5rZd/Xtx7b+dR44cQW1trc3PuG/fvoiNjW3xZ9yZvy+OwADogcxmM5555hmMHDkS/fv3b3G7Pn36YOnSpfj+++/x1VdfwWw2Y8SIEbhy5YoTq7W/4cOHY/ny5diyZQuWLFmCjIwM3HLLLaioqGh2+7y8PISFhdk8FhYWhry8PGeU6zQbNmxAWVkZ5syZ0+I23fU7cS3Lz7YjP/eioiKYTKZu/V2pqanBc889hxkzZrR6ftOO/o51FRMmTMCKFSuwfft2vP7669i5cycmTpwIk8nU7Pae8J0AgC+//BK+vr5tdnt29e9Fc/925uXlwcvLq8l/iFr7GXfm74sjyJ32SuQ2UlNTcfLkyTbHXqSkpCAlJcV6f8SIEUhKSsKnn36KV155xdFlOszEiROttwcOHIjhw4cjLi4O3377bbv+B9tdffHFF5g4cSIiIyNb3Ka7fieobbW1tbjvvvsgCAKWLFnS6rbd9XfsgQcesN4eMGAABg4ciB49eiAtLQ1jxoxxYWWutXTpUsycObPNCWFd/XvR3n87uwq2AHqYuXPnYtOmTdixYweio6M7tK9CocCgQYNw8eJFB1XnGv7+/ujdu3eL7ys8PLzJjK78/HyEh4c7ozynyMzMxLZt2/DHP/6xQ/t11++E5WfbkZ97cHAwZDJZt/yuWMJfZmYmtm7d2mrrX3Pa+h3rqhITExEcHNzi++rO3wmL3bt349y5cx3+2wF0re9FS/92hoeHw2g0oqyszGb71n7Gnfn74ggMgB5CEATMnTsX69evx88//4yEhIQOH8NkMuHEiROIiIhwQIWuU1lZiUuXLrX4vlJSUrB9+3abx7Zu3WrTEtbVLVu2DKGhobjzzjs7tF93/U4kJCQgPDzc5udeXl6OAwcOtPhz9/LywpAhQ2z2MZvN2L59e5f+rljC34ULF7Bt2zYEBQV1+Bht/Y51VVeuXEFxcXGL76u7fica++KLLzBkyBAkJyd3eN+u8L1o69/OIUOGQKFQ2PyMz507h6ysrBZ/xp35++IQTptuQi71xBNPCFqtVkhLSxNyc3Otl6qqKus2Dz30kPD8889b7y9atEj48ccfhUuXLglHjhwRHnjgAUGlUgmnTp1yxVuwm7/+9a9CWlqakJGRIezZs0cYO3asEBwcLBQUFAiC0PRz2LNnjyCXy4W33npLOHPmjLBgwQJBoVAIJ06ccNVbsCuTySTExsYKzz33XJPnuvN3oqKiQjh27Jhw7NgxAYDwzjvvCMeOHbPObn3ttdcEf39/4fvvvxd+++03YcqUKUJCQoJQXV1tPcbo0aOFDz74wHp/9erVglKpFJYvXy6cPn1aePTRRwV/f38hLy/P6e+vvVr7HIxGo3DXXXcJ0dHRwvHjx23+dhgMBusxrv0c2vodc1etfRYVFRXCvHnzhH379gkZGRnCtm3bhMGDBwu9evUSampqrMfoDt8JQWj790MQBEGn0wkajUZYsmRJs8foDt+L9vzb+fjjjwuxsbHCzz//LBw+fFhISUkRUlJSbI7Tp08fYd26ddb77fn74mgMgB4CQLOXZcuWWbe57bbbhNmzZ1vvP/PMM0JsbKzg5eUlhIWFCZMmTRKOHj3q/OLt7P777xciIiIELy8vISoqSrj//vuFixcvWp+/9nMQBEH49ttvhd69ewteXl5Cv379hM2bNzu5asf58ccfBQDCuXPnmjzXnb8TO3bsaPZ3wvJ+zWaz8OKLLwphYWGCUqkUxowZ0+QziouLExYsWGDz2AcffGD9jIYNGybs37/fSe+oc1r7HDIyMlr827Fjxw7rMa79HNr6HXNXrX0WVVVVwrhx44SQkBBBoVAIcXFxwp/+9KcmQa47fCcEoe3fD0EQhE8//VRQq9VCWVlZs8foDt+L9vzbWV1dLTz55JNCQECAoNFohLvvvlvIzc1tcpzG+7Tn74ujSeoLIyIiIiIPwTGARERERB6GAZCIiIjIwzAAEhEREXkYBkAiIiIiD8MASERERORhGACJiIiIPAwDIBEREZGHYQAkIiIi8jAMgEREbkgikWDDhg2uLoOIuikGQCKia8yZMwcSiaTJZcKECa4ujYjILuSuLoCIyB1NmDABy5Yts3lMqVS6qBoiIvtiCyARUTOUSiXCw8NtLgEBAQDE7tklS5Zg4sSJUKvVSExMxNq1a232P3HiBEaPHg21Wo2goCA8+uijqKystNlm6dKl6NevH5RKJSIiIjB37lyb54uKinD33XdDo9GgV69e2Lhxo2PfNBF5DAZAIqJOePHFF3HPPffg119/xcyZM/HAAw/gzJkzAAC9Xo/x48cjICAAhw4dwpo1a7Bt2zabgLdkyRKkpqbi0UcfxYkTJ7Bx40b07NnT5jUWLVqE++67D7/99hsmTZqEmTNnoqSkxKnvk4i6KYGIiGzMnj1bkMlkgre3t83lX//6lyAIggBAePzxx232GT58uPDEE08IgiAIn332mRAQECBUVlZan9+8ebMglUqFvLw8QRAEITIyUnjhhRdarAGA8I9//MN6v7KyUgAg/O9//7Pb+yQiz8UxgEREzbj99tuxZMkSm8cCAwOtt1NSUmyeS0lJwfHjxwEAZ86cQXJyMry9va3Pjxw5EmazGefOnYNEIkFOTg7GjBnTag0DBw603vb29oafnx8KCgo6+5aIiKwYAImImuHt7d2kS9Ze1Gp1u7ZTKBQ29yUSCcxmsyNKIiIPwzGARESdsH///ib3k5KSAABJSUn49ddfodfrrc/v2bMHUqkUffr0ga+vL+Lj47F9+3an1kxEZMEWQCKiZhgMBuTl5dk8JpfLERwcDABYs2YNbrrpJvzud7/DypUrcfDgQXzxxRcAgJkzZ2LBggWYPXs2Fi5ciMLCQjz11FN46KGHEBYWBgBYuHAhHn/8cYSGhmLixImoqKjAnj178NRTTzn3jRKRR2IAJCJqxpYtWxAREWHzWJ8+fXD27FkA4gzd1atX48knn0RERAS+/vpr3HDDDQAAjUaDH3/8EU8//TSGDh0KjUaDe+65B++88471WLNnz0ZNTQ3+/e9/Y968eQgODsa9997rvDdIRB5NIgiC4OoiiIi6EolEgvXr12Pq1KmuLoWIqFM4BpCIiIjIwzAAEhEREXkYjgEkIuogjpwhoq6OLYBEREREHoYBkIiIiMjDMAASEREReRgGQCIiIiIPwwBIRERE5GEYAImIiIg8DAMgERERkYdhACQiIiLyMAyARERERB6GAZCIiIjIwzAAEhEREXkYBkAiIiIiD8MASERERORhGACJiIiIPMz/B/JMJevr2VQuAAAAAElFTkSuQmCC", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Selected epoch: 10\n" ] } ], "source": [ "display(Image(filename=str(RUN / 'learning.png')))\n", "if 'val_loss' in report['history'][0]:\n", " best_epoch = 1 + int(np.argmin([row['val_loss'] for row in report['history']]))\n", " print('Selected epoch:',best_epoch)" ] }, { "cell_type": "markdown", "id": "2f2fac1b", "metadata": {}, "source": [ "## Evaluation\n", "\n", "Validation loss eventually rises while training loss continues falling, so the best validation checkpoint is restored. The sampled passage shows learned character and dialogue structure but contains grammatical errors and invented words. This is a small character model, not a general-purpose language model." ] }, { "cell_type": "code", "execution_count": 6, "id": "5890b908", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.880779Z", "iopub.status.busy": "2026-10-14T13:51:33.880689Z", "iopub.status.idle": "2026-10-14T13:51:33.884961Z", "shell.execute_reply": "2026-10-14T13:51:33.884566Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MetricValue
0test_loss1.787328
1perplexity5.973470
\n", "
" ], "text/plain": [ " Metric Value\n", "0 test_loss 1.787328\n", "1 perplexity 5.973470" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Timed training loop (including validation/checkpoint work): 49.61 seconds\n" ] } ], "source": [ "metrics = {name:value for name,value in report.items()\n", " if name in ('test_loss','test_accuracy','majority_baseline_accuracy',\n", " 'persistence_baseline_mse','perplexity','test_auc','test_brier')}\n", "if metrics:\n", " display(pd.DataFrame(list(metrics.items()),columns=['Metric','Value']))\n", "print('Timed training loop (including validation/checkpoint work):',round(report['training_seconds'],2),'seconds')" ] }, { "cell_type": "code", "execution_count": 7, "id": "39b35a38", "metadata": { "execution": { "iopub.execute_input": "2026-10-14T13:51:33.885710Z", "iopub.status.busy": "2026-10-14T13:51:33.885621Z", "iopub.status.idle": "2026-10-14T13:51:33.887587Z", "shell.execute_reply": "2026-10-14T13:51:33.887231Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "First Citizen:\n", "Then, I must be not be so; if you had been reason\n", "Do at one this last before thou shalt the mind\n", "deservice of your gorder parties\n", "His treasure I will to the bear that is young women to\n", "His hand, the wind of place in himself,\n", "So monthy this stood I will dire the battle fierce\n", "And harbbler: therefore holjesty and Rome;\n", "But if all half lone\n", "The whop then of ceremony words, and Angelo\n", "As a man's head \n" ] } ], "source": [ "print((RUN / 'generation.txt').read_text())" ] }, { "cell_type": "markdown", "id": "fcb2c53e", "metadata": {}, "source": [ "## Artifacts and further work\n", "\n", "`report.json` records the configuration and measured outcomes. `data.json` records split/preprocessing information; `best.npz` (supervised tasks) and `last.npz` contain model/training state. Figures and predictions are saved separately. Use the checkpoint tutorial to restore an existing architecture.\n", "\n", "Change one factor at a time, retain validation-based selection, and avoid choosing configurations or samples to improve the apparent test result." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.20" }, "jaxsaw": { "executed_utc": "2026-10-14T13:51:34Z", "gpu_selection": "0", "kind": "model", "requires_figure": true, "run_directory": "gallery/transformer", "source_sha256": "e0bcadee26012c4bed06af35ad708cd0ebffe043c3750fa2c7b0784ca7bf3758", "task": "transformer", "source_rename": { "package_from": "neuraljaxlib", "package_to": "jaxsaw", "alias_from": "nj", "alias_to": "js" } } }, "nbformat": 4, "nbformat_minor": 5 }