--- license: cdla-permissive-2.0 pretty_name: Handtrained HoloAssist single-episode sample language: - en size_categories: - n<1K task_categories: - video-classification tags: - egocentric-video - temporal-action-labels - sample configs: - config_name: default data_files: - split: train path: data/* --- # Handtrained HoloAssist sample v1.0.1 A real 38-second battery-change excerpt from HoloAssist, packaged for inspection and loading. Original video and semantic annotations: HoloAssist. Handtrained adds excerpt selection, video normalization, timestamp mapping, metadata, import scripts and technical checks. This is not footage collected by Handtrained or in Indonesia. ## Start here Unzip handtrained-holoassist-v1.0.1.zip. Open a terminal inside the resulting folder. Use Python 3.12; the tested platform is macOS arm64. Installation requires internet access. No account or GPU is required to inspect the local sample. ### Hugging Face Datasets and PyTorch ```sh python3.12 -m venv .venv source .venv/bin/activate python -m pip install -r requirements-notebook.txt python -m pip install jupyterlab jupyter lab quickstart.ipynb ``` Run all notebook cells. It checks every checksum, decodes all 1,140 frames, loads one VideoFolder row with Hugging Face Datasets, plays the video, and creates a [5, 3, 126, 224] RGB uint8 PyTorch batch with timestamp-aligned labels. The notebook uses Video(decode=False) and PyAV explicitly; it does not require TorchCodec or a separate FFmpeg installation for this path. Colab alternative: open https://colab.research.google.com/, upload quickstart.ipynb, run all cells, and upload the sample ZIP when prompted. Colab execution has not been verified in this release. The notebook does not point to a nonexistent Hugging Face repository or silently download the full HoloAssist dataset. ### FiftyOne Install the FFmpeg command-line tools first (on macOS with Homebrew: `brew install ffmpeg`; Linux distributions provide an ffmpeg package). In the same virtual environment: ```sh python -m pip install -r requirements-fiftyone.txt python import_fiftyone.py --launch ``` Open http://127.0.0.1:5151. Click the video thumbnail; enable frames.active_actions to see all active labels during playback. `actions` contains the 16 temporal detections. `coarse_action` contains the task. Exact seconds, source IDs, verbs, nouns and inherited correctness remain available as label attributes. Filter task/source/review_status in the sidebar. To view each action as a separate clip in Python: `dataset.to_clips("actions")`. The importer creates a uniquely named review dataset and never overwrites an existing one. With --launch it persists in the local FiftyOne database; without --launch the test dataset is removed after import. The default database lives in .fiftyone-db beside the script. You can supply a new --name or --port. Stop the server with Ctrl-C. Remove this particular dataset through FiftyOne when finished; do not delete other datasets. ### Validate without opening a notebook ```sh python sample_utils.py python import_fiftyone.py ``` A successful result means file and import checks passed. It does not certify semantic label accuracy or model fitness. ## Contents and schema - data/episode.mp4: H.264/yuv420p, 896 x 504, no audio, 38 seconds, 30 fps, 1,140 frames. - data/metadata.jsonl: one VideoFolder record linking file_name to the MP4, plus episode/task/source and action metadata. - manifest.json: release identifier, dimensions, source interval, transformations and limitations. - labels.json: 16 fine actions and one coarse action; all inherited. `id` is stable, `source_label_id` is the original integer. `start` and `end` are clip seconds. `source_start` and `source_end` are original session seconds. `attributes` preserves the source object. - original-annotations.json: only the selected original annotation records, unchanged. - action-labels.csv and taxonomy.json: flat inspection view and documented verb/noun mapping. - fiftyone-labels.json and fiftyone-coarse.json: native temporal detections and coarse task support. - provenance.json: source URLs, original file hash, archive member/byte range and exact FFmpeg transformation. - checksums.json: SHA-256 of the immutable release files, excluding checksums.json itself. Changing an output notebook in place will change its checksum; retain the original release for revalidation. - validation-report.json: measured test environment, results and timing limits. - asset-validation.json: video and source-label structural checks; sampled visual review. - tested-versions.json and requirements-*.txt: tested direct dependency versions. - LICENSE-CDLA-Permissive-2.0.txt and NOTICE.txt: data license and attribution. LICENSE-CODE.txt covers the packaging scripts and notebook under MIT. ## Time and frame conventions Source session: R0027-12-GoPro. Source range: [29.0, 67.0) seconds. `source_seconds = clip_seconds + 29.0`. Seconds are half-open intervals: start included, end excluded. Multiple actions may be active. No active fine label means unannotated time; it does not mean the person is idle. We preserve 5.028 seconds of overlapping fine annotations and 5.9 seconds with no fine annotation. These are durations, not timestamps or quality scores. FiftyOne uses 1-based inclusive frame supports. Frame k has presentation time (k-1)/30. Support begins at ceil(start*30)+1 and ends at ceil(end*30), with numerical tolerance at exact boundaries. Exact seconds stay in attributes because frame conversion quantizes boundaries by less than one frame. All 1,140 frame memberships are tested against the seconds convention. The source is approximately 24.46 fps and variable-rate; 30 fps normalization repeats/drops frames. It does not add captured temporal detail. Original millisecond label values are precision of storage, not independently measured millisecond accuracy. ## Rights and limitations Data is distributed under CDLA-Permissive-2.0 with the full license. Source recording and annotation credit belongs to HoloAssist. The source license is not an additional participant release or endorsement of Handtrained. Packaging code has a separate MIT license. This is one short excerpt, not representative diversity or a benchmark. Audio, depth, hand pose, gaze, force, robot state and robot commands are absent. Source Correct Action values are inherited annotator judgments. Natural blur, occlusion and annotation gaps remain. Only sampled visual review was performed; no independent annotation-accuracy study, exhaustive privacy audit, commercial acceptance rate or model-performance test is claimed. The train split is the default VideoFolder container name, not an approved training split. Do not invent train/test evidence from one recording. A real project requires episode/person/site-aware split rules and leakage checks. ## Publish to Hugging Face later This package is ready for a dataset repository but has not been uploaded to the Hub. The website remains a private preview. To publish, first choose an authorized Hugging Face namespace and intended audience. From this folder, authenticate with your own account and upload an explicit private repository: ```sh hf auth login hf repo create YOUR_ORG/handtrained-holoassist-sample --repo-type dataset --private hf upload YOUR_ORG/handtrained-holoassist-sample . . --repo-type dataset --exclude '.venv/**' '.fiftyone-db/**' '__pycache__/**' '.ipynb_checkpoints/**' ``` Use your actual authorized repository ID. Do not commit tokens, local database files or participant identity records. Verify the Hub video viewer after upload, then test a pinned snapshot from a second machine before advertising hosted compatibility. Hugging Face hosting and access are separate from the Datasets library used by this notebook. ## Troubleshooting - Missing manifest: change directory into the unzipped sample folder. The MP4 alone is not the complete delivery. - Checksum failure: extract an untouched ZIP again. If the notebook was executed and saved, its output changed; retain the original notebook for checksum validation. - Missing ffmpeg/ffprobe in FiftyOne: install FFmpeg and reopen the terminal so it is on PATH. - Database cannot start: ensure the folder is writable and your OS is supported by the installed FiftyOne release. A security policy may prevent its local MongoDB process; inspect the error rather than disabling system protections. - Port 5151 busy: add --port 5152. Existing datasets are never deleted automatically. - No video: verify data/episode.mp4 exists, try the MP4 in a browser, and inspect the validation output. - Installation issues on another OS: consult the pinned versions and official tool docs. Windows/Linux/Colab were not verified in this macOS test. ## Sources HoloAssist project: https://holoassist.github.io/ Source release and schema: https://holoassist.github.io/data_links/README.html Data license: https://cdla.dev/permissive-2-0/ Hugging Face video loading: https://huggingface.co/docs/datasets/video_load Hugging Face video datasets on the Hub: https://huggingface.co/docs/hub/en/datasets-video FiftyOne temporal imports: https://docs.voxel51.com/user_guide/import_datasets.html#fiftyone-temporal-detection ## Release 1.0.1 Branding and package names now use Handtrained. Video, source annotations, episode IDs and loader behavior are unchanged from 1.0.0. The taxonomy identifier uses the new brand; its classes and mappings are unchanged. Checksums cover this release.