Quantifying the Frontal-to-Ceiling Domain Gap for YOLO-Based Hand Gesture Recognition in Smart Homes

Creative Commons License

Beşenk U., Naim S. E., Öcal Ö., Öztemel M., Özkurt A., Selver M. A.

SENSORS, cilt.26, sa.18, ss.1-27, 2026 (SCI-Expanded, Scopus)

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 26 Sayı: 18
  • Basım Tarihi: 2026
  • Doi Numarası: 10.3390/s26185735
  • Dergi Adı: SENSORS
  • Derginin Tarandığı İndeksler: Academic Search Ultimate (EBSCO), Biomedical Reference Collection: Corporate Edition (EBSCO), Health Research Premium Collection (ProQuest), Scopus, Science Citation Index Expanded (SCI-EXPANDED), Compendex, EMBASE, INSPEC, MEDLINE, Directory of Open Access Journals
  • Sayfa Sayıları: ss.1-27
  • Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
  • Dokuz Eylül Üniversitesi Adresli: Evet

Özet

Vision-based hand gesture recognition (HGR) systems are predominantly developed for frontal camera viewpoints, whereas smart-home cameras are often ceiling-mounted, creating a viewpoint-induced domain gap. To investigate this issue, we collected and manually annotated CeilGest, an 18-class ceiling-view hand gesture dataset comprising 156,282 annotated frames from 68 participants recorded in distinct domestic environments. We then systematically evaluated frontal-to-ceiling transfer using YOLO-based detectors trained on HaGRID and compared their performance with an in-domain CeilGest-trained model. On identical ceiling-view footage, the frontal-trained YOLOv8n produced approximately 24× more class-to-class misclassified frames than the in-domain model (486 vs. 20 across 27,000 frames); this large paired difference remained evident when temporal dependence within gesture holds was taken into account. The effect was strongly class-dependent, with AP decreasing by up to 5.5 percentage points for the worst-affected gesture, while the aggregate same-architecture mAP50 difference was 0.6 percentage points. Across five YOLOv8 variants evaluated on frontal HaGRID, mAP50 remained at 0.995, supporting selection of the lightweight YOLOv8n for edge deployment. The complete ceiling-view HGR pipeline was implemented on Raspberry Pi 5 using NCNN and Jetson Orin Nano using TensorRT. Mean inference latency was 74.69 ± 4.72 ms and 12.72 ± 0.12 ms, respectively. During a 10 min continuous Raspberry Pi 5 test, mean inference latency increased by 17.6% and junction temperature reached 90.8 °C with thermal throttling. Power consumption and INT8 inference were not evaluated. Overall, the results show that training–deployment viewpoint consistency is a major consideration for ceiling-mounted HGR and establish an in-domain supervised baseline relative to frontal-only training without domain adaptation.