Overview

Object Detection & Geotagging

Many applications such as autonomous navigation, urban planning and asset monitoring, rely on the availability of accurate information about objects and their geolocations. Using a combination of fully convolutional neural networks for processing geolocated street view images, we propose to geolocate assets of interest using a Markov Random field for performing multi-sensor information fusion.


Colour transfer

Colour transfer aims at changing the colour feel of an image (target image) using the colour palette of another image (palette image). Colour transfer can also be applied to videos, and for creating dynamic effects on static images.


Climate Change

Automatic detection of passable roads after floods in remote sensed and social media data
K. Ahmad, K. Pogorelov, M. Riegler, O. Ostroukhova, P. Halvorsen, N. Conci, R. Dahyot, Signal Processing: Image Communication, Volume 74, May 2019, Pages 110-118
DOI:10.1016/j.image.2019.02.002
ArXiv:1901.03298

Deep Learning in Artificial Intelligence

Deep Learning (or Machine Learning with Neural Networks) is a very popular approach for designing data-driven artificial intelligence (AI). It is used in many of our applications for object detection, image superresolution and denoising, data synthesis etc.


Denoising and SuperResolution

Super-Resolution on Degraded Low-Resolution Images Using Convolutional Neural Networks
F. Albluwi, V. Krylov and R. Dahyot, European Signal Processing Conference (Eusipco), A Coruna Spain, September 2019.
[DOI:10.23919/EUSIPCO.2019.8903000] [IEEE Xplore]
[PDF] [Github]
More publications...
Denoising RENOIR Image Dataset with DBSR
Fatma Albluwi, Vladimir A. Krylov and R. Dahyot, Irish Machine Vision and Image Processing, Technological University Dublin, 28-30 August 2019, pages 76-79, ISBN 978-0-9934207-4-0
[DOI:10.21427/g34k-8r27]
[PDF]
Image Deblurring And Super-Resolution Using Deep Convolutional Neural Networks
F. Albluwi, V. Krylov and R. Dahyot, IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2018), September 2018, Aalborg, Danemark. [PDF] [Github]
DOI:10.1109/MLSP.2018.8516983
Artifacts reduction in JPEG-Compressed Images using CNNs
F. Albluwi, V. Krylov and R. Dahyot Irish Machine Vision and Image Processing conference (IMVIP 2018), 2018. [PDF]
Published in IMVIP e-book of proceedings with ISBN 978-0-9934207-3-3. http://hdl.handle.net/2262/89508
[Github]

Social Media based 3D Visual Popularity

We create a geotagged virtual world to visualise people's visual interests and their sentiment as captured from their social network activities. The 3D map is created automatically using information from OpenStreetMap and Google Street View. The Unreal game engine is used to animate the 3D world where lights represent flash photography of images posted on Twitter. Areas accumulating the most lights are associated with high popularity scores.

Populating virtual cities using social media

We propose to automatically populate geo-located virtual cities by harvesting and analyzing online contents shared on social networks and websites.

Object Detection & Geotagging

Many applications such as autonomous navigation, urban planning and asset monitoring, rely on the availability of accurate information about objects and their geolocations. My PhD thesis dealt with the detection and the recognition of static objects (poles, traffic signs,...) on the road side in road scene video databases. More recently we have proposed a pipeline combining fully convolutional neural networks for processing geolocated street view images, and a Markov Random field for performing multi-sensor information fusion. Our approach AIMapIt allows to create a list of GPS coordinates of the detected objects with high accuracy.

Illustration

Demos

AIMapIt: Automatic Detection and Geotagging of Stationary Objects from Street Level Imagery

Publications

Automatic Discovery and Geotagging of Objects from Street View Imagery
V. A. Krylov, E. Kenny and R. Dahyot, Remote Sensing, vol. 10, number 5, 2018,
DOI:10.3390/rs10050661
ArXiv:1708.08417
[Bibtex] [Github]
Object Geolocation from Crowdsourced Street Level Imagery
V. Krylov and R. Dahyot, International Workshop on Urban Reasoning (UrbReas 2018) in conjunction with the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), Dublin, Ireland, September 2018. [PDF]
DOI:10.1007/978-3-030-13453-2_7
Object Geolocation using MRF based Multi-Sensor Fusion
V. Krylov and R. Dahyot, IEEE International Conference on Image Processing (ICIP 2018), October 2018, Greece. [PDF]
DOI:10.1109/ICIP.2018.8451458
Modeles probabilistes d'apparence et estimation robuste
P. Charbonnier, R. Dahyot, T. Vik and F. Heitz, chapter in the book Detection et reconnaissance de la signalisation verticale par analyse d'images edited by P. Foucher, Etudes et Recherches des laboratoires des Ponts et Chaussees, CR53 July 2010, ISBN 978-2-7208-2578-1
[PDF]
Analyse d'images sequentielles de scenes routieres par modeles d'apparence pour la gestion du reseau routier
Appearance based road scene video analysis for the management of the road network
R. Dahyot, PhD Thesis University of Strasbourg France, November 2001. Published ISBN10 (2-7208-2028-1) in 2003.
http://theses.fr/2001STR13130
[PDF]
Unsupervised Statistical Detection of Changing Objects in Camera-in-Motion Video
R. Dahyot, P. Charbonnier and F. Heitz, IEEE International Conference on Image Processing (ICIP'01), October 2001, Greece.
DOI:10.1109/ICIP.2001.959126
Matlab/Octave code on Github
Robust Visual Recognition of Colour Images
R. Dahyot, P. Charbonnier and F. Heitz, IEEE conference on Computer Vision and Pattern Recognition (CVPR'00), volume 1, pages 685-690, Juin 2000, Hilton Head Island USA, (poster).
DOI:10.1109/CVPR.2000.855886
Non-Supervised Robust Visual Recognition of Colour images using Half-Quadratic Theory
R. Dahyot, P. Charbonnier and F.Heitz, European Workshop on Content-Based Multimedia Indexing (CBMI'99), october 1999, Toulouse FRANCE (poster).

Deep Learning and Harmonic networks

Many applications in computer vision rely on the use of machine learning techniques for creating data-driven artificial intelligence (AI) as opposed to model-driven AI. In particular, several of our targetted applications (e.g. object geolocation, image denoising and super-resolution ) use deep learning based on Neural Networks, and some efforts in the team aim at both understanding and reducing their complexity.

Publications

Harmonic Networks for Image Classification
M. Ulicny, V. Krylov and R. Dahyot, British Machine Vision Conference (BMVC), Cardiff UK, 9-12 September 2019.
[PDF]
[Github]
Harmonic Networks with Limited Training Samples
M. Ulicny, V. Krylov and R. Dahyot, European Signal Processing Conference (Eusipco), A Coruna Spain, September 2019.
[DOI:10.23919/EUSIPCO.2019.8902831] [IEEE Xplore]
ArXiv:1905.00135
[Github]
AI Pipeline - bringing AI to you. End-to-end integration of data, algorithms and deployment tools
M. de Prado, J. Su, R. Dahyot, R. Saeed, L. Keller and N. Vallez, HiPEAC 2019 workshop Emerging Deep Learning Accelerator January 2019.
[PDF]
ArXiv:1901.05049
Performance-Oriented Neural Architecture Search
A. Anderson, J. Su, R. Dahyot and D. Gregg, International Conference on High Performance Computing & Simulation, July 2019.
[PDF]
ArXiv:2001.02976
Harmonic Networks: Integrating Spectral Information into CNNs
Matej Ulicny, Vladimir A. Krylov, Rozenn Dahyot, December 2018
ArXiv:1812.03205
[Github]
On Using CNN with Compressed (DCT Based) Image Data
M Ulicny, and R. Dahyot, Irish Machine Vision and Image Processing conference (IMVIP 2017), Maynooth University, 30/08-1/09/2017
pages 44-51, ISBN 978-0-9934207-2-6 [PDF] [Bibtex]
Permanent link to the ebook: http://eprints.maynoothuniversity.ie/8841/

Colour Transfer

Colour transfer aims at changing the colour feel of an image (target image) using the colour palette of another image (palette image). Colour transfer can also be applied to videos, and creating dynamic effects on static images. Several techniques have been proposed in my team: all looks at how to register efficiently colour distributions with (supervised, semi-supervised) or without (unsupervised) input correspondences. Our works consider Optimal Transport and L2 divergence as cost functions to minimize between colour distributions.

Online code and demos

Alghamdi et al Grogan et al Pitie et al
2019- 2014-2019 2004-2007

Publications

Patch-Based Colour Transfer with Optimal Transport
H. Alghamdi, M. Grogan and R. Dahyot, European Signal Processing Conference (Eusipco), A Coruna Spain, September 2019.
[DOI:10.23919/EUSIPCO.2019.8902611 ] [IEEE Xplore]
[PDF]
[GitHub]
Entropic Regularisation of Robust Optimal Transport
Rozenn Dahyot, Hana Alghamdi and Mairead Grogan, Irish Machine Vision and Image Processing, Technological University Dublin, 28-30 August 2019
ArXiv:1905.12678
[DOI:10.21427/w611-mb37]
[PDF]
L2 Divergence for Robust Colour Transfer
M. Grogan and R. Dahyot, Computer Vision and Image Understanding, Volume 181, April 2019, Pages 39-49
DOI:10.1016/j.cviu.2019.02.002
[PDF]
[Github]
L2 Divergence for Robust Colour Transfer
M. Grogan and R. Dahyot, Computer Vision and Image Understanding, Volume 181, April 2019, Pages 39-49
DOI:10.1016/j.cviu.2019.02.002
[PDF]
[Github]
User Interaction for Image Recolouring using L2
M. Grogan, R. Dahyot and A. Smolic, in Conference on Visual Media Production (CVMP), London, December 2017. [PDF]
Online DEMO - Awarded Best Paper at CVMP
DOI:10.1145/3150165.3150171
Robust Registration of Gaussian Mixtures for Colour Transfer
M. Grogan and R. Dahyot, May 2017
ArXiv:1705.06091
Online Demo and webpage
L2 Registration for Colour Transfer
M. Grogan, M. Prasad and R. Dahyot, European Signal Processing Conference (Eusipco), ISBN 978-0-9928626-4-0, Nice France, September 2015. Code and demo
DOI:10.1109/EUSIPCO.2015.7362799
L2 registration for Colour Transfer in Videos
M. Grogan and R. Dahyot, in Conference on Visual Media Production (CVMP), London, November 2015
Code and demo - awarded Best Student Poster at CVMP
DOI:10.1145/2824840.2824862
Automated Colour Grading using Colour Distribution Transfer
F. Pitie, A. Kokaram and R. Dahyot, in Computer Vision and Image Understanding, vol. 107, July-August, Elsevier, pp.123-137, 2007. DOI:10.1016/j.cviu.2006.11.011
[PDF] [GitHub]
N-Dimensional Probability Density Function Transfer and its Application to Colour Transfer
F. Pitie, A. Kokaram and R. Dahyot, in proceedings of the IEEE International Conference on Computer Vision (ICCV'05), Beijing, China, Oct. 2005.
DOI:10.1109/ICCV.2005.166
Code
Towards Automated Colour Grading
F. Pitie, A. C. Kokaram and R. Dahyot, 2nd European Conference on Visual Media Production (IEE CVMP 2005), London November 2005. Code

Social Media based 3D Visual Popularity

We create a geotagged virtual world to visualise people s visual interests and their sentiment as captured from their social network activities. Using game engine technologies, lights in the virtual environment are used to highlight areas with high popularity score.

Publications

Social Media based 3D Visual Popularity
A. Bulbul and R. Dahyot, Computer & Graphics, volume 63, pages 28-36, April 2017. [PDF]
DOI:10.1016/j.cag.2017.01.005

Populating virtual cities using social media

We propose to automatically populate geo-located virtual cities by harvesting and analyzing online contents shared on social networks and websites. We show how pose and motion paths of agents can be realistically rendered using information gathered from social media. 3d cities are automatically generated using open-source information available online. Our final rendering of both static and dynamic urban scenes is generated using Unreal game engine.

Publications

Populating Virtual Cities using Social Media
A. Bulbul and R. Dahyot, Computer Animation and Virtual Worlds journal 2017 [PDF]
DOI:10.1002/cav.1742

Spherical & reflective surfaces

Several software solutions exist for creating a 3d mesh from multiple RGB images. Artefacts occurs however when dealing with reflective surfaces. We propose a solution for spherical reflective surfaces.

Walton statue

walton sculpture without spherical refinement

walton sculpture with spherical refinement

Berkeley sphere

berkeley sphere without spherical refinement

berkeley sphere with spherical refinement

Publications

3D Reconstruction of Reflective Spherical Surfaces from Multiple Images
A. Bulbul, M. Grogan and R. Dahyot, Irish Machine Vision and Image Processing conference, pages 19-26, (Permanent link to full book: http://hdl.handle.net/2262/74714) ISBN 978-0-9934207-0-2, August 2015, demo.
-->