Overview

Artificial Intelligence

Deep Learning (or Machine Learning with Neural Networks) is a very popular approach for designing data-driven artificial intelligence (AI). It is used in many of our applications for object detection, image enhancement, data synthesis etc.

As deep learning is compute intensive, our recent research efforts is concerned by rethinking convolutional filters with DCTs in CNNs for encouraging sparsity and compression of the network, to help in their training and portability to embedded devices.


Road Scene Analysis

Many applications such as autonomous navigation, urban planning and asset monitoring, rely on the availability of accurate information about objects (e.g road signs, poles, trees) and their geolocations.

Using a combination of fully convolutional neural networks for processing geolocated street view images, we recently propose to geolocate assets of interest using a Markov Random field for performing multi-sensor information fusion.


Climate Change

Climate change is affecting people and their way of life including transportation of goods and people. Our recent work proposes to analyse satellite imagery in combination with street view imagery posted on social media during flooding events, for assessing the passability of roads.

Automatic detection of passable roads after floods in remote sensed and social media data
K. Ahmad, K. Pogorelov, M. Riegler, O. Ostroukhova, P. Halvorsen, N. Conci, R. Dahyot, Signal Processing: Image Communication, Volume 74, May 2019, Pages 110-118
DOI:10.1016/j.image.2019.02.002
ArXiv:1901.03298

Colouring

Colour transfer aims at changing the colour feel of an image (target image) using the colour palette of another image (palette image). Colour transfer can also be applied to videos, and for creating dynamic effects on static images. Colour transfer is used for pre-processing of image data in many image based applications, as well as in the post processing film industry.

For the past decade, we have proposed several approaches to solve this problem based in Optimal Transport and information theory as cost functions to minimize between colour distributions.


Digital Social Twins

We create a geotagged virtual world to visualise people's visual interests and their sentiment as captured from their social network activities. The 3D map is created automatically using information from OpenStreetMap and Google Street View. The Unreal game engine is used to animate the 3D world where lights represent flash photography of images posted on Twitter. Areas accumulating the most lights are associated with high popularity scores.

Social Media based 3D Visual Popularity
A. Bulbul and R. Dahyot, Computer & Graphics, volume 63, pages 28-36, April 2017. [PDF]
DOI:10.1016/j.cag.2017.01.005
Image by Enrique Meseguer from Pixabay

Image Enhancement

Heterogeneous asynchroneous raw data streams often required pre-processing to be used as part of intelligent systems. Several of our recent work provides solutions to image denoising (e.g. JPG artifacts), super-resolution, and registration.

Image by FunkyFocus from Pixabay

Game Engine and GIS

Game engines play a crucial roles in applications such as urban planning, architecture, autonomous driving, filmmaking, etc. Unreal Engine was used in our work on digital twins, and recently Unity was used to create an interactive visit of Trinity (using our drone TCD dataset and information from openstreet map). See below some of our early WebGL visualisation results.

Spherical & reflective surfaces

Several software solutions exist for creating a 3d mesh from multiple RGB images. Artefacts occurs however when dealing with reflective surfaces. We have proposed a solution for spherical reflective surfaces.


3D Reconstruction of Reflective Spherical Surfaces from Multiple Images
A. Bulbul, M. Grogan and R. Dahyot, Irish Machine Vision and Image Processing conference, pages 19-26, (Permanent link to full book: http://hdl.handle.net/2262/74714) ISBN 978-0-9934207-0-2, August 2015, demo.

Robust AI

Intelligent systems need to be resilient to unexpected subsets of (outlier) data occurring in the training and/or test sets. An example of application is robust recognition of objects in images (e.g. traffic signs) when these are partially occluded (e.g. by tree branches). My PhD thesis explored robust cost function able to indicate which pixels are used or discarded to infer the recognised object (explainable AI).

Robust Visual Recognition of Colour Images
R. Dahyot, P. Charbonnier and F. Heitz, IEEE conference on Computer Vision and Pattern Recognition (CVPR'00), volume 1, pages 685-690, Juin 2000, Hilton Head Island USA, (poster).
DOI:10.1109/CVPR.2000.855886
A Bayesian approach to object detection using probabilistic appearance-based models
R. Dahyot, P. Charbonnier and F. Heitz, Pattern Analysis and Applications, Vol. 7, No 3, pp. 317-332, December 2004.
DOI:10.1007/s10044-004-0230-5
[PDF]