Colm G.

Data Engineer

460 dollar
Freelancer
9 ans
Londres, ROYAUME-UNI

Mon expérience

Voir plus

Laser Focus DataMarch 2018 - Présent

We are a team of consultant data engineers and data scientists. We specialise in helping small to medium sized organisations to harness their data in order to make superior decisions.
Voir plus

Guardian News & MediaMarch 2019 - May 2019

The Guardian have multiple different systems (> 10) from which they must extract or delete data in order to comply with GDPR. Before I joined, the process of extracting and deleting data was either manual, or performed by multiple python scripts. Unfortunately the scripts were not written by professional developers and were brittle and difficult to maintain. I implemented a single application which interacted with each of the individual systems to extract or delete data, so as to be less cumbersome and more easily maintainable than the previous system. The application was written in Scala using the Play framework, and interacted with web UIs and the Guardian’s datalake on Amazon Athena.
Voir plus

dunnhumbyMarch 2018 - August 2018

The goal of this project was moving an existing app written in R and Scala onto Spark, in order to reduce the running time of the app from several days to several hours. As it turned out, the work also involved Dockerising the project and implementing a continuous integration pipeline using GitLab. I managed to gain the expected performance benefits using Spark.

Voir plus

Betsson GroupNovember 2016 - January 2018

I build Big Data products for external and internal consumption, mostly using open source software on the JVM.

I help to maintain the existing ETL systems built on  Kafka, Nifi, Hive and HBase.

I work hand-in-hand with our data scientists to productionise their machine-learning models, mostly using Scala on Spark.
Voir plus

CrownpeakJuly 2014 - November 2016

Maintaining the existing Java codebase (bug fixes, product enhancements).
Implementing new functionality in Scala.
Migrating existing functionality to a new Big Data architecture written in Scala, on top of a Spark, Kafka, Cassandra infrastructure.
Re-skinning the product using JSF and CSS.
Rapid prototyping of potential new product features using a wide range of technologies (back-end and front-end).
Voir plus

Allied Irish BanksOctober 2012 - September 2013

•    Business plan analysis
•    Financial projection analysis
•    Maintaining relationships with clients
•    Liaising with upper management
•    Market research
Voir plus

Spikeys Ice GripsAugust 2011 - January 2012

•    Market research
•    Due diligence
•    Liaising with Chinese client representative
•    Sales
•    After sales service
•    Marketing
•    Supply chain design and management

Mes compétences

Big Data

Hive, Apache Kafka, Spark

Databases

MySQL

Analysis methods and tools

Agile Methodology

Languages

JAVA SE, Scala, CSS3, JavaScript, HTML, HTML5

IT Infrastructure

Docker, Cloud Computing

Others

Problem Solving, Teamwork, Leadership, Database Design, Commercial

Frameworks

AJAX

Mes études et formations

Functional Programming Principles in Scala - Coursera2014 - 2014

Graduate Diploma in Information Technology - Dublin City University2013 - 2014

MSc. - Trinity College, Dublin2011 - 2012

Bachelor's Degree - Trinity College Dublin2006 - 2010