{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a04bf7a2",
   "metadata": {},
   "source": [
    "# Handtrained sample · one HoloAssist episode\n",
    "Watch a 38-second GoPro battery change, load it with Hugging Face Datasets, and turn real frames into a PyTorch tensor. Original video and 17 temporal annotations are from [HoloAssist](https://holoassist.github.io/), under CDLA-Permissive-2.0. Handtrained prepared the excerpt, metadata and import checks.\n",
    "\n",
    "**Run all cells in order.** Locally, open this notebook from the unzipped sample folder with Python 3.12. In Colab, upload this notebook; the next cell will ask for the sample ZIP. A Hugging Face account or GPU is not needed. The notebook has been executed locally; Colab is a provided path, not a certified runtime.\n",
    "\n",
    "This proves a small delivery can be inspected and loaded. It does not establish annotation accuracy, model performance, robot-policy compatibility or a commercial acceptance rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e51736ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import sys, zipfile\n",
    "\n",
    "ROOT = Path.cwd()\n",
    "if not (ROOT / \"manifest.json\").exists():\n",
    "    if \"google.colab\" in sys.modules:\n",
    "        from google.colab import files\n",
    "        print(\"Upload handtrained-holoassist-v1.0.1.zip downloaded from the sample page\")\n",
    "        uploaded = files.upload()\n",
    "        archives = [Path(name) for name in uploaded if name.endswith(\".zip\")]\n",
    "    else:\n",
    "        archives = list(ROOT.glob(\"handtrained-holoassist-v1.0.1.zip\"))\n",
    "    if not archives:\n",
    "        raise FileNotFoundError(\"Unzip the sample and run this notebook inside its folder, or place the sample ZIP beside it.\")\n",
    "    target = ROOT / \"handtrained-sample-unpacked\"\n",
    "    target.mkdir(exist_ok=True)\n",
    "    with zipfile.ZipFile(archives[0]) as z:\n",
    "        for member in z.infolist():\n",
    "            destination = (target / member.filename).resolve()\n",
    "            if not destination.is_relative_to(target.resolve()):\n",
    "                raise ValueError(\"Unsafe archive path\")\n",
    "            if member.file_size > 100_000_000:\n",
    "                raise ValueError(\"Unexpectedly large archive member\")\n",
    "        z.extractall(target)\n",
    "    ROOT = target / \"handtrained-holoassist-v1.0.1\"\n",
    "assert (ROOT / \"manifest.json\").exists(), ROOT\n",
    "sys.path.insert(0, str(ROOT))\n",
    "print(\"Sample found:\", ROOT.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7637be4",
   "metadata": {},
   "source": [
    "## Install the tested notebook packages\n",
    "This step needs internet access and can take several minutes on a fresh machine. PyAV includes the video decoder in its wheel; this notebook does not require a separate FFmpeg command-line installation. FiftyOne has a separate quickstart in the README."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5fe3fbb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import subprocess, time\n",
    "t0 = time.perf_counter()\n",
    "subprocess.check_call([sys.executable, \"-m\", \"pip\", \"install\", \"--quiet\", \"-r\", str(ROOT / \"requirements-notebook.txt\")])\n",
    "print(f\"Dependency check/install: {time.perf_counter() - t0:.2f}s\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "458dfe9b",
   "metadata": {},
   "source": [
    "## Check the delivery before loading\n",
    "SHA-256 checksums detect changed or incomplete files. They do not certify label quality. Validation decodes all 1,140 frames, checks presentation times, verifies the source-to-clip offset, and tests action boundaries and frame supports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6ae7c06",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sample_utils import validate, read_json, load_huggingface, frame_batch, active_labels\n",
    "t0 = time.perf_counter()\n",
    "report = validate(ROOT)\n",
    "print(report)\n",
    "print(f\"Validation: {time.perf_counter() - t0:.2f}s\")\n",
    "manifest = read_json(ROOT / \"manifest.json\")\n",
    "labels = read_json(ROOT / \"labels.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c76a8e1",
   "metadata": {},
   "source": [
    "## Load one row with Hugging Face Datasets\n",
    "The package is a standard VideoFolder: MP4 plus metadata.jsonl. We set Video(decode=False), then use PyAV for decoding. This makes the decoder explicit and avoids a system TorchCodec/FFmpeg version dependency. No full HoloAssist download is performed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "716ee956",
   "metadata": {},
   "outputs": [],
   "source": [
    "t0 = time.perf_counter()\n",
    "dataset = load_huggingface(ROOT)\n",
    "row = dataset[0]\n",
    "assert len(dataset) == 1\n",
    "assert row[\"episode_id\"] == manifest[\"episode_id\"]\n",
    "assert len(row[\"actions\"]) == 17\n",
    "print(dataset)\n",
    "print(\"Episode:\", row[\"episode_id\"])\n",
    "print(\"Video:\", Path(row[\"video\"][\"path\"]).name)\n",
    "print(f\"Hugging Face load: {time.perf_counter() - t0:.2f}s\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddc80231",
   "metadata": {},
   "source": [
    "## Watch the delivered video\n",
    "Audio was removed. The source is a variable-frame-rate HoloLens recording; the supplied excerpt is normalized to 30 fps. That may duplicate or drop frames and does not create new captured information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "797377f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Video, display\n",
    "display(Video(filename=row[\"video\"][\"path\"], embed=True, width=720))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c500ba3a",
   "metadata": {},
   "source": [
    "## Produce a frame batch with synchronized labels\n",
    "The tensor uses [time, channel, height, width] with RGB bytes. Frames are resized only for this small example batch. Canonical labels use seconds [start, end); overlapping labels remain separate and an empty list means unannotated time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4132fa3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from PIL import Image\n",
    "t0 = time.perf_counter()\n",
    "batch, timestamps = frame_batch(row[\"video\"][\"path\"])\n",
    "assert batch.shape == (5, 3, 126, 224)\n",
    "assert batch.dtype == torch.uint8\n",
    "print(\"Batch:\", tuple(batch.shape), batch.dtype)\n",
    "for frame, timestamp in zip(batch, timestamps):\n",
    "    print(f\"{timestamp:5.2f}s\", [a[\"label\"] for a in active_labels(labels, timestamp)])\n",
    "    display(Image.fromarray(frame.permute(1, 2, 0).numpy()))\n",
    "print(f\"Batch decode: {time.perf_counter() - t0:.2f}s\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0331fc5e",
   "metadata": {},
   "source": [
    "## Inspect the taxonomy and source records\n",
    "Verb, noun, correctness and source label IDs come from HoloAssist. A source label saying Correct Action is an inherited annotation, not a measured acceptance rate. Handtrained adds stable sample IDs, clip-relative times and file structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ead3e35",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "display(pd.DataFrame([{k: a[k] for k in [\"source_label_id\", \"level\", \"label\", \"start\", \"end\", \"source_start\", \"source_end\"]} for a in labels]))\n",
    "print(read_json(ROOT / \"taxonomy.json\"))\n",
    "assert len(active_labels(labels, 6.0)) == 2  # preserved overlap\n",
    "assert active_labels(labels, 28.0) == []     # preserved gap\n",
    "print(\"Overlap and gap checks passed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbae8a63",
   "metadata": {},
   "source": [
    "## What to do next\n",
    "For visual review, follow the FiftyOne instructions in README.md. It imports the same video with temporal actions, source IDs and frame-level active labels. A reviewer can filter task and review status, inspect action clips and record issues.\n",
    "\n",
    "For model work, choose a task such as action recognition or video representation learning, define splits by original recording/person, then build the task-specific preprocessing. A single excerpt cannot provide meaningful train/test evaluation. This sample contains no robot commands or measured robot state.\n",
    "\n",
    "To host on Hugging Face later, upload this folder into an authorized dataset repository with the included dataset card. The repository has not been created or published by this notebook; README.md shows the explicit publishing command."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}