Conditional Weighted Ensemble of Transferred Models for Camera Based Onboard Pedestrian Detection in Railway Driver Support Systems

Toprak T., Belenlioglu B., Aydin B., Guzelis C., SELVER M. A.

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, vol.69, no.5, pp.5041-5054, 2020 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 69 Issue: 5
  • Publication Date: 2020
  • Doi Number: 10.1109/tvt.2020.2983825
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, Aerospace Database, Applied Science & Technology Source, Business Source Elite, Business Source Premier, Communication Abstracts, Compendex, Computer & Applied Sciences, Environment Index, INSPEC, Metadex, Civil Engineering Abstracts
  • Page Numbers: pp.5041-5054
  • Keywords: Pedestrian detection, railway transportation, transfer learning, classifier ensembles, CLASSIFICATION, PERFORMANCE, EXTRACTION, FEATURES
  • Dokuz Eylül University Affiliated: Yes


Pedestrian Detection (PD) is one of the most studied issues of driver assistance systems. Although a tremendous effort is already given to create datasets and to develop classifiers for cars, studies about railway systems remain very limited. This article shows that direct application of neither existing advanced object detectors (such as AlexNet, VGG, YOLOetc.), nor specifically created systems for PD(such as Caltech/INRIA trained classifiers), can provide enough performance to overcome railway specific challenges. Fortunately, it is also shown that without waiting the collection of a mature dataset for railways as comprehensively diverse and annotated as the existing ones for cars, a Transfer Learning (TL) approach to fine-tune various successful deep models (pre-trained using both extensive image and pedestrian datasets) to railway PD tasks provides an effective solution. To achieve TL, a new RAil-Way PEdestrian Dataset (RAWPED) is collected and annotated. Then, a novel three-stage system is designed. At its first stage, a feature-classifier fusion is created to overcome the localization and adaptation limitations of deep models. At the second stage, the complementarity of the transferred models and diversity of their results are exploited by conducted measurements and analyses. Based on the findings, at the third stage, a novel learning strategy is developed to create an ensemble, which conditionally weights the outputs of individual models and performs consistently better than its components. The proposed system is shown to achieve a log average miss rate of 0.34 and average precision of 0.93, which are significantly better than the performance of compared well-established models.