Robot training data from hospitality work: what labs need, and what a clean pilot looks like.
Humanoid and manipulation models learn from first-person video of people doing real tasks. Housekeeping, kitchen and laundry work is repeated, step-by-step handling of objects, which is what those models need. This guide sets out what a lab-grade capture programme has to get right. It is based on the pilot Hospara is preparing for Q4 2026.
Short answer. It is first-person (egocentric) video of housekeeping, kitchen, laundry, pool and garden tasks. It is captured to a specification agreed with a robotics lab. A consent chain covers the property, each staff participant and any accidental capture. Labs use it to train manipulation and humanoid models. Hospara is preparing a Q4 2026 pilot in Bali.
What egocentric video is
Video recorded from the worker’s point of view, usually with a head-mounted phone or camera. It shows hands, tools and objects during a task. Labs use it to pre-train and fine-tune models that map perception to action.
It is video only, unless extra sensors are added. By itself it contains no robot actions, joint states or depth.
Why hospitality tasks
Bed making, bathroom resets, mopping, dishwashing, folding, pool skimming and garden work are repeated hundreds of times a week. They follow written procedures. They involve soft and deformable objects, liquids, tools and changing light.
That combination is scarce in existing datasets, which skew to kitchens in a few countries.
What a specification contains
- The task family and the number of repetitions
- The camera device and mount, angle, resolution and frame rate
- The pace of work
- Left- and right-handed participants
- Failure and recovery cases
- Lighting conditions and variety of environments
- The acceptance criteria the buyer’s quality checks will apply
Capture without a specification produces footage nobody can use.
The consent chain
- A separate, specific authorisation from each property owner
- A signed participation agreement for each staff participant, in their own language
- Recording only in vacant units
- A policy that prevents capture of guests, documents and screens
- Blurring before export
- A consent log for each clip
- A written procedure to quarantine and delete anything captured by accident
Under Indonesia’s Personal Data Protection Law (PDP Law, UU 27/2022), the lawful basis and the safeguards for cross-border transfer are reviewed with counsel before anything leaves the country.
Delivery
Clips are split by task, with timestamps and JSON manifests. Blur is applied, and each clip has its consent log.
On request, we package clips in RLDS or LeRobot directory layouts, carrying video and manifests only. Annotation of hands or objects is available through partners, also on request.
How Hospara is approaching it
A Q4 2026 pilot in participating Bali properties. It starts with a paid calibration stage in one or two authorised units. Volume is then agreed for each specification.
Nothing has been recorded or delivered yet. Details for robotics labs.
Common questions
Does egocentric video include robot actions or depth?
No. It is video plus task manifests, unless extra sensors are agreed. Packaging in RLDS or LeRobot layouts does not add actions, state or depth.
How is staff consent handled?
Each staff participant signs a participation agreement in their own language. They are paid for each accepted hour, on top of salary. They can stop future recording at any time, with no effect on their employment. Training and recording time count as working hours.
Can a lab ship its own capture hardware?
Yes. The default is a head-mounted phone. A GoPro, smart glasses or a lab rig can be used, with the lab’s calibration procedure.
Related
Planning a capture programme?
Tell us the tasks, the volume and the acceptance criteria you need. We will say plainly what the pilot can deliver.