Recent Publications

A Data-driven Approach for Adding Facade Details to Textured LoD2 CityGML Models

Xingzi Zhang,∗, Franziska Lippoldt, Kan Chen, Henry Johan and Marius Erdt

LoD3 CityGML models (with facade elements, e.g., windows and doors) have many applications, however, they are not easy to acquire, while LoD2 models (only roofs and walls) are currently largely available. In this paper, we propose to generate LoD3 models by adding facade details to textured LoD2 models using a data-driven approach.

A Novel Bayesian Model Incorporating Deep Neural Network and Statistical Shape Model for Pancreas Segmentation

Jingting Ma, Feng Lin, Stefan Wesarg, and Marius Erdt

Deep neural networks have achieved significant success in medical image segmentation in recent years. However, poor contrast to surrounding tissues and high flexibility of anatomical structure of the interest object are still challenges.

A Novel Robust Kernel Principal Component Analysis for Nonlinear Statistical Shape Modeling from Erroneous Data

Jingting Ma, Anqi Wang, Feng Lin, Stefan Wesarg, Marius Erdt

Statistical Shape Models (SSMs) have achieved considerable success in medical image segmentation. A high quality SSM is able to approximate the main plausible variances of a given anatomical structure to guide segmentation.

A Two-point Map-based Interface for Architectural Walkthrough

Kan Chen, Eugene Lee

Architectural walkthrough of 3D buildings usually requires the user to manipulate the position and orientation of a camera for explo- ration, measurement, and annotation. In this paper, we propose a novel two-point interaction method by interacting with the corre- sponding 2D floor plans to navigate in 3D buildings.

A Two-stage Outlier Filtering Framework for City-Scale Localization using 3D SfM Point Clouds

Wentao Cheng, Kan Chen, Weisi Lin, Michael Goesele, Xinfeng Zhang, Yabin Zhang

3D Structure-based localization aims to estimate the 6-DOF camera pose of a query image by means of feature matches against a 3D Structure-from-Motion (SfM) point cloud. For city-scale SfM point clouds with tens of millions of points, it becomes more and more difficult to disambiguate matches.

An Appearance-Driven Method for Converting Polygon Soup Building Models for 3D Geospatial Applications

Kan Chen, Henry Johan, Marius Erdt

Polygon soup building models are fine for visual- ization purposes such as in games and movies. They, however, are not suitable for 3D geospatial applications which require geometrical analysis, since they lack connectivity information and may contain intersections internally between their parts.

Appearance-Driven Conversion of Polygon Soup Building Models with Level of Detail

Kan Chen, Henry Johan and Marius Erdt

In many 3D applications, building models in polygon-soup representation are commonly used for the purposes of visualization, for example, in movies and games. Their appearances are fine, however geometry-wise, they may have limited information of connectivity and may have internal intersections between their parts. Therefore, they are not well-suited to be directly used in 3D geospatial applications, which usually require geometric analysis.

ArchGANs: Stylized Colorization Prototyping for Architectural Line Drawing

Wenyuan Tao, Han Jiang, Qian Sun, Mu Zhang, Kan Chen, Marius Erdt

Architectural illustration using line drawing with colorization is an important tool and art format. In this paper, in order to generate a natural-looking and high quality watercolor-like colorization for architectural line drawing, we propose a novel Generative Ad- versarial Network (GAN) approach, namely ArchGANs.

Assessing Haptic Video Interaction with Neurocognitive Tools

Shahzad Rasool, Xiyuan Hou, Yisi Liu, Alexei Sourin, Olga Sourina

Haptics interaction is a form of a user-cmputer interaction where physical forces are delivered to the user via vibrations, displacements and rotations of special haptic devices. When quality of the experience of the haptic intereaction is asessed, mostly subjective tests using various questionnaires are performed.

Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization

Wentao Cheng, Weisi Lin, Kan Chen, Xingfeng Zhang

Image-based localization (IBL) aims to estimate the 6DOF camera pose for a given query image. The camera pose can be computed from 2D-3D matches between a query image and Structure-from-Motion (SfM) models. Despite recent advances in IBL, it remains difficult to simultaneously resolve the memory consumption and match ambiguity problems of large SfM models.

Cross Dataset Workload Classification Using Encoded Wavelet Decomposition Features

Wei Lun Lim, Olga Sourina, Lipo Wang

For practical application, it is desirable for a trained classified system to be independent of task and/or subject. In this study, we show one-way transfer between two independent EEG workload datasets: from a large multitasking dataset with 48 subjects to a second Stroop test dataset with 18 subjects.

Detection of Humanoid Robot Design Preferences Using EEG and Eye Tracker

Yisi Liu, Fan Li, Lin Hei Tang, Zirui Lan, Jian Cui, Olga Sourina

Currently, many modern humanoid robots have little appeal due to their simple designs and bland appearances. To provide reccomendations for designers and improve the designs of humanoid robots, a study of human's perception on humanoid robot design is conducted using Electroencephalography (EEG), eye tracking information and questionnaires.

Domain Adaptation Techniques for EEG-based Emotion Recognition: A Comparative Study on Two Public Datasets

Zirui Lan, Olga Sourina, Lipo Wang, Reinhold Scherer

Affective brain-computer interface (aBCI) introduces personal affective factors to human-computer interaction. The state-of-the-art aBCI tailors its classifier to each individual user to achieve accurate emotion classification. A subject-independent classifier that is trained on pooled data from multiple subjects generally leads to inferior accuracy, due to the fact that encephalogram (EEG) patterns vary from subject to subject.

EEG-based Cadets Training and Performance Assessment System in Maritime Virtual Simulator

Yisi Liu, Zirui Lan, Olga Sourina, Hui Ping Liew, Gopala Krishnan

Deep investment in the maritime industries has led to many cutting edge technological advances in shipping navigation and operational safety to ensure safe and efficient logistical transportations. However, even with the best technology equipped onboard, maritime accidents are still occurring with at least three quarters of them attributed to human errors.

EEG-based Cross-subject Mental Fatigue Recognition

Yisi Liu, Zirui Lan, Jian Cui, Olga Sourina, Wolfgang Müller-Wittig

Mental fatigue is common at work places, and it can lead to decreased attention, vigilance and cognitive performance, which is dangerous in the situations such as driving, vessel maneuvering, etc. By directly measuring the neurophysiological activities happening in the brain, electroencephalography (EEG) signal can be used as a good indicator of mental fatigue.

EEG-based Evaluation of Mental Fatigue Using Machine Learning Algorithms

Yisi Liu, Zirui Lan, Han Hua Glenn Khoo, King Ho Holden Li, Olga Sourina, Wolfgang Mueller- Wittig

When people are exhausted both physically and mentally from overexertion, they experience fatigue. Fatigue can lead to a decrease in motivation and vigilance which may result in certain accidents or injuries. It is crucial to monitor fatigue in workplace for safety reasons and well-being of the workers.

EEG-based Human Factors Evaluation of Air Traffic Control Operators (ATCOs) for Optimal Training

Yisi Liu, Zirui Lan, Fitri Traspsilawati, Olga Sourina, Chun-Hsien Chen, Wolfgang Mueller-Wittig

To deal with the increasing demands in Air Traffic Control (ATC), new working place designs are proposed and developed that need novel human factors evaluation tools. In this paper, we propose a novel application of Electroencephalogram (EEG)-based emotion, workload, and stress recognition algorithms to investigate the optimal length of training for Air Traffic Control Officers (ATCOs) to learn working with three-dimensional (3D) display as a supplementary to the existing 2D display.

EEG-based Mental Workload and Stress Recognition of Crew Members in Maritime Virtual Simulator: A Case Study

Yisi Liu, Salem Chandrasekaran Harihara Subramaniam, Olga Sourina, Serene Hui Ping Liew, Gopala Krishnan

Many studies have shown that the majority of maritime accidents/incidents are attributed to human errors as the initiating cause. Efforts have been made to study human factors that can result in a safer maritime transportation.

EEG-based Recognition of Driver State Related to Situation Awareness Using Graph Convolutional Networks

Ruilin Li, Zirui Lan, Jian Cui, Olga Sourina, Lipo Wang

Extracting intra- and inter- subject parameters from Electroencephalogram (EEG) representing different Situation Awareness (SA) status is a critical challenge for objective SA recognition. Most of the existing work focuses on the subject-dependent classification that applies power spectrum density (PSD) features. In this paper, we propose a novel spectral-spatial (S-S) model for cross-subject fatigue-related SA recognition.

Evaluation of Humanoid Robot Design based on Global Eye-tracking Metrics

Fan Li, Danni Chang, Yisi Liu, Jian Cui, Shanshan Feng, Ning Huang, Chun-Hsien Chen, Olga Sourina

The first impression of robot appearance normally affects the interaction with physical robots. Hence, it is critically important to evaluate the humanoid robot appearance design. This study towards evaluating humanoid robot design based on global eye-tracking metrics.

Exploration of Natural Free-Hand Interaction for Shape Modeling using Leap Motion Controller

Jian Cui, Arjan Kuijper, Alexei Sourin

In this paper, we propose a web-enabled shape modeling system with natural free-hand interaction, which can be easily learned by users while imposing least mental load on them. The deformation interface allows for performing various deformations, including stretching, compressing, squeezing, enlarging, twisting and tapering, on shapes interactively mimicking how they are done in real life.

Generation of 3D Building Models from City Area Maps

Roman Martel, Chaoqun Dong, Kan Chen, Henry Johan, Marius Erdt

In this paper, we propose a pipeline that converts buildings described in city area maps to 3D models in the CityGML LOD1 standard. The input documents are scanned city area maps provided by a city authority. The city area maps were recorded and stored over a long time period.

Generation of 3D Building Models from City Area Maps by Fusing Information from Multiple Sources

Chaoqun Dong, Roman Martel, Kan Chen, Henry Johan, Marius Erdt

This paper proposes a pipeline for the automatic generation of 3D building models for city areas based on information from archived city area maps. These maps are typically created and archived by city authorities for several decades. As such, they exhibit several challenging prop- erties like a mixture of handwritings and typewriter font styles, varying layouts and notation standards, low contrast and physical damages. To tackle these challenges, we pro- pose to extract and fuse information from multiple sources.

Hierarchical Eye-Tracking Data Analytics for Human Fatigue Detection at a Traffic Control Center

Fan Li, Chun-Hsien Chen, Gangyan Xu, Li-Pheng Khoo

Eye-tracking-based human fatigue detection at traffic control centers suffers from an unavoidable problem of low-quality eye-tracking data caused by noisy and missing gaze points. In this study, the authors conducted pioneering work by investigating the effects of data quality on eye- tracking-based fatigue indicators and by proposing a hierarchical-based interpolation approach to extract the eye- tracking-based fatigue indicators from low-quality eye- tracking data.

High Performance City Rendering in Vulkan

Alex Zhang, Kan Chen, Henry Johan, Marius Erdt

City scale scenes often contain large amounts of geometry and texture that cannot altogether fit on GPU memory. Our ongoing work seek to minimise texture memory usage by streaming only view-relevant textures and to improve rendering performance using parallel opportunities offered by Vulkan, the latest generation of graphics API.

High Performance Texture Streaming and Rendering of Large Textured 3D Cities

Alex Zhang, Kan Chen, Henry Johan, Marius Erdt

We introduce a novel, high performing, bandwidth-aware texture streaming system for progressive texturing of buildings in large 3D cities, with optional texture pre-processing. We seek to maintain high and consistent texture streaming performance across different city datasets, and to address the high memory binding latency in hardware virtual textures.

Human Factors Assessment in VR-based Firefighting Training in Maritime: A Pilot Study

Yisi Liu, Zirui Lan, Benedikt Tschoerner, Satinder Singh Virdi, Jian Cui, Fan Li, Olga Sourina, Daniel Zhang, David Chai, Wolfgang Mueller-Wittig

Virtual Reality (VR) has been used for training aircraft pilots, maritime seafarers, operators, etc as it provides an immersive environment with realistic lifelike quality. We developed and implemented a VR-based Liquefied Natural Gas (LNG) firefighting simulation system with head-mounted displays (HMD) and novel human factors evaluation that could train and assess both technical and non-technical skills in the firefighting scenarios.

Individual Alpha Peak Frequency Based Features For Subject Dependent EEG Workload Classification

Wei Lun Lim, Olga Sourina, Lipo Wang, Yisi Liu

The individual alpha peak frequency (IAPF) is an important biological indicator in Electroencephalogram (EEG) studies, with many research publications linking it to various cognitive functions

Inter-Subject Transfer Learning for EEG-based Mental Fatigue Recognition

Yisi Liu, Zirui Lan, Jian Cui, Olga Sourina, Wolfgang Mueller-Wittig

Mental fatigue is one of the major factors leading to human errors. To avoid failures caused by mental fatigue, researchers are working on ways to detect/monitor fatigue using different types of signals. Electroencephalography (EEG) signal is one of the most popular methods to recognize mental fatigue since it directly measures the neurophysiological activities in the brain. Current EEG-based fatigue recognition algorithms are usually subject-specific, which means a classifier needs to be trained per subject. However, as fatigue may need a relatively long period to induce, collecting training data from each new user could be time-consuming and troublesome.

Interactive Authoring of Bending and Twisting Motions of Short Plants Using Hand Gestures

Kan Chen, Henry Johan

In this paper, we propose an approach to interactively author the bending and twist- ing motions of short plants using hand gestures, especially suitable for grass, flowers and leaves. Our method is based on the observations that hand motions can represent the bending and twisting motions of short plants and using a hand to describe motions is natural and proficient for human.

Interactive Rendering of Translucent Materials under Area Lights using Voxels and Poisson Disk Samples

Ming Di Koa, Henry Johan, Alexei Sourin

Interactive rendering of translucent materials in virtual worlds has always proved to be challenging. Rendering their indirect illumination produces further challenges. In our work, we develop a voxel illumination framework for translucent materials illuminated by area lights.

Interactive Screenspace Fragment Rendering for Direct Illumination from Area Lights Using Gradient Aware Subdivision and Radial Basis Function Interpolation

Ming Di Koa, Henry Johan, Alexei Sourin

Interactive rendering of direct illumination from area lights in virtual worlds has always proven to be challenging. In this paper, we propose a deferred multi resolution approach for rendering direct illumination from area lights.

Interactive Shape Modeling Using Leap Motion Controller

Jian Cui, Alexei Sourin

Compared to commonly used 2D tracking devices, 3D hand tracking by low-cost optical cameras can provide more degrees of freedom, as well as natural gestures for interaction done in virtual spaces. However, it is not considered to be precise enough for 3D modeling due to the problems of hand jitter, jump release, and occlusion.

Large-Scale 3D Shape Retrieval from ShapeNet Core55

Manolis Savva, Fisher Yu, Hao Su, Asako Kanezaki, Takahiko Furuya, Ryutarou Ohbuchi, Zhichao Zhou, Rui Yu, Song Bai, Xiang Bai, Masaki Aono, Atsushi Tatsuma, S. Thermos, A. Axenopoulos, G. Th. Papadopoulos, P. Daras, Xiao Deng, Zhouhui Lian, Bo Li, Henry Johan, Yijuan Lu, Sanjeev Mk

With the advent of commodity 3D capturing devices and better 3D modeling tools, 3D shape content is becoming increasingly prevalent. Therefore, the need for shape retrieval algorithms to handle large-scale shape repositories is more and more important.

Mid-air Interaction with Optical Tracking for 3D Modeling

Compared to common 2D interaction done with mouse and other 2D tracking devices, 3D hand tracking with low-cost optical cameras can provide more degrees of freedom, as well as natural gestures, when shape modeling is done in virtual spaces.

Mobile EEG-based Situation Awareness Recognition for Air Traffic Controllers

Lee Guan Yeo, Haoqi Sun, Yisi Liu, Fitri Trapsilawati, Olga Sourina, Chun-Hsien Chen, Wolfgang Mueller-Wittig, Wei Tech Ang

With the growing volume and complexity of air traffic, air traffic controllers (ATCOs) encounter heavier burden nowadays. Therefore, human factors study in air traffic control (ATC) is increasingly essential, paving the way to a safer air transportation system.

Neurofeedback Training for Rifle Shooters to Improve Cognitive Ability

Yisi Liu, Salem Chandrasekaran Harihara Subramaniam, Olga Sourina, Eesha Shah, Joshua Chua, Kirill Ivanov

Neurofeedback training is one type of the biofeedback training that allows the subject do self-regulation during the training according to his/her real-time brain activities recognized from Electroencephalogram (EEG) and given to him/her through visual, audio or haptic feedback.

Neuroscience Based Design: Fundamentals and Applications

Olga Sourina, Yisi Liu, Xiyuan Hou, Wei Lun Lim, Wolfgang Mueller-Wittig, Lipo Wang, Dimitrios Konovessis, Chun-Hsien Chen, Wei Tech Ang

Neuroscience-based or neuroscience-informed design is a new application area of Brain-Computer Interaction (BCI). It takes its roots in study of human well-being in architecture, human factors study in engineering and manufacturing including neuroergonomics.

Nonlinear Statistical Shape Modeling for Ankle Bone Segmentation Using A Novel Kernelized Robust PCA

Jingting Ma, Anqi Wang, Feng Lin, Stefan Wesarg, and Marius Erdt

Statistical shape models (SSMs) are widely employed in medical image segmentation. However, an inferior SSM will degenerate the quality of segmentations. It is challenging to derive an efficient model because: (1) often the training datasets are corrupted by noise and/or artifacts; (2) conventional SSM is not capable to capture nonlinear vari- abilities of a population of shape.

Process Data-Based Knowledge Discovery in Additive Manufacturing of Ceramic Materials by Multi-Material Jetting (CerAM MMJ)

Valentin Lang, Steven Weingarten, Hajo Wiemer, Uwe Scheithauer, Felix Glausch, Robert Johne, Alexander Michaelis and Steffen Ihlenfeldt

Multi-material jetting (CerAM MMJ, previously T3DP) enables the additive manufacturing of ceramics, metals, glass and hardmetals, demonstrating comparatively high solid contents of the processed materials. The material is applied drop by drop onto a substrate. The droplets can be adapted to the component to be produced by a large degree of freedom in parameterization. Thus, large volumes can be processed quickly and fine structures can be displayed in detail, based on the droplet size. Data-driven methods are applied to build process knowledge and to contribute to the optimization of CerAM MMJ manufacturing processes.

Psychophysiological Evaluation of Seafarers to Improve Training in Maritime Virtual Simulator

Yisi Liu, Zirui Lan, Jian Cui, Gopala Krishnan, Olga Sourina, Dimitrios Konovessis, Hock Eng Ang, Wolfgang Mueller-Wittig

Over years, safety in maritime industries has been reinforced by many state-of- the-art technologies. However, the accident rate hasn’t dropped significantly with the advanced technology onboard. The main cause of this phenomenon is human errors which drive researchers to study human factors in maritime domain. One of the key factors that contribute to human performance is their mental states such as cognitive workload and stress

RGB-D to CAD Retrieval with ObjectNN Dataset

Binh-Son Hua, Quang-Trung Truong, Minh-Khoi Tran, Quang-Hieu Pham, Asako Kanezaki, Tang Lee, HungYueh Chiang, Winston Hsu, Bo Li, Yijuan Lu, Henry Johan, Shoki Tashiro, Masaki Aono, Minh-Triet Tran, Viet-Khoi Pham, Hai-Dang Nguyen, Vinh-Tiep Nguyen, Quang-Thang Tran, Thuyen V. Phan, Bao Truong, Minh N. Do, Anh-Duc Duong, Lap-Fai Yu, Duc Thanh Nguyen, Sai-Kit Yeung

The goal of this track is to study and evaluate the performance of 3D object retrieval algorithms using RGB-D data. This is inspired from the practical need to pair an object acquired from a consumer-grade depth camera to CAD models available in public datasets on the Internet.

SAFE: An EEG Dataset for Stable Affective Feature Selection

Zirui Lan, Yisi Liu, Olga Sourina, Lipo Wang, Reinhold Scherer, Gernot Müller-Putz

An affective brain-computer interface (aBCI) is a direct communication pathway between human brain and computer, via which the computer tries to recognize the affective states of its user and respond accordingly. As aBCI introduces personal affective factors into human- computer interaction, it could potentially enrich the user’s experience during the interaction. Successful emotion recognition plays a key role in such a system.  

Shape Retrieval of Non-rigid 3D Human Models

D.Pickup, X. Sun, P. L. Rosin, R. R. Martin, Z. Cheng, Z. Lian, M. Aono, A. Ben Hamza, A. Bronstein, M. Bronstein, S. Bu, U. Castellani, S. Cheng, V. Garro, A. Giachetti, A. Godil, L. Isaia, J. Han, H. Johan, L. Lai, B. Li, C. Li, H. Li, R. Litman, X. Liu, Z. Liu, Y. Lu, L. Sun, G. Tam, A. Tatsuma & J. Ye

3D models of humans are commonly used within computer graphics and vision, and so the ability to distinguish between body shapes is an important shape retrieval problem. We extend our recent paper which provided a benchmark for testing non-rigid 3D shape retrieval algorithms on 3D human models.

Stable Feature Selection for EEG-based Emotion Recognition

Zirui Lan, Olga Sourina, Lipo Wang, Yisi Liu

Affective brain-computer interface (aBCI) introduces personal affective factors into human-computer interactions, which could potentially enrich the user’s experience during the interaction with a computer. However, affective neural patterns are volatile even within the same subject.

STEW: Simultaneous Task EEG Workload Dataset

Wei Lun Lim, Olga Sourina, Lipo Wang

This paper describes an open access electroencephalography (EEG) dataset for multitasking mental workload activity induced by a single-session simultaneous capacity (SIMKAP) experiment with 48 subjects.

Sketch-based 3D Model Retrieval Utilizing Adaptive View Clustering and Semantic Information

Bo Li, Yijuan Lu, Henry Johan, Ribel Fares

Searching for relevant 3D models based on hand-drawn sketches is both intuitive and important for many applications, such as sketch-based 3D modeling and recognition, human computer interaction, 3D animation, game design, and etc.

Tangible Images of Real Life Scenes

Xingzi Zhang, Michael Goesele, Alexei Sourin

Haptic technologies allow for adding a new “touching” modality into virtual scenes. However, 3D reconstruction of real life scene often results in millions of polygons which cannot be simultaneously visualized and haptically rendered.

Unsupervised Feature Learning for EEG-based Emotion Recognition

Zirui Lan, Olga Sourina, Lipo Wang, Reinhold Scherer, Gernot Müller-Putz

Spectral band power features are one of the most widely used features in the studies of electroencephalogram (EEG)-based emotion recognition. The power spectral density of EEG signals is partitioned into different bands such as delta, theta, alpha and beta band etc.

Using Support Vector Regression to Estimate Valence Level from EEG

Zirui Lan, Gernot R. Müller-Putz, Lipo Wang, Yisi Liu, Olga Sourina, Reinhold Scherer

Emotion recognition is an integral part of affective computing. An affective brain-computer-interface (BCI) can benefit the user in a number of applications. In most existing studies, EEG (electroencephalograph)-based emotion recognition is explored in a classificatory manner.

Voxel-Based Interactive Rendering of Translucent Materials under Area Lights using Sparse Samples

Ming Di Koa, Henry Johan, Alexei Sourin

Interactive rendering of translucent materials in virtual worlds has always proved to be challenging. In our work, we develop a voxel illumination framework for translucent materials illuminated by area lights. Our voxel illumination framework consists of two voxel structures.