AFAYNOU M.

Data Engineer

415 dollar
Freelancer
2 ans
Paris, FRANCE

Mon expérience

Voir plus

MobiliseJanuary 2020 - Présent

 1- Gathering Requirement from customer
2- Using AWS DMS to import data from various data source and store it in a perticular data format in AWS S3
3- Using AWS DMS to do an incremental load

4- Using AWS lambda to create table in glue catalog using
AWS athena by triggering a AWS lambda function
5- Dynamically creating the table using properties file in glue catalog
6- Using AWS CloudWatch to check the logs for data
7- Partition the table in AWS glue to reduce the cost while querying from Atena
8- Following best practices to reduce the cost from client

prospective
9- Using Pyspark to process data using AWS glue
10- Using AWS Redshift to store the final processed data
Voir plus

Mobilise Cloud · Contract Part-timeOctober 2020 - November 2021

Mission: RATP Group

AWS stacks( DMS, S3, Glue, Althena, Lambda, Redshift,...)

Voir plus

DeezerApril 2020 - August 2020

 Building and automatization of the Batch Data Pipeline
(Data Warehouse) using AWS cloud services (S3, EC2, Redshift, etc.) and language Python ( pandas,

pyspark, psycopg2, boto3, etc)

PROFESSIONAL ACADEMIC

M2 Data Science 'D2SN': parcours Big Data

Mes compétences

Environment of Development

GitLab

Databases

PostGreSQL, Cassandra, NoSQL

Analysis methods and tools

DevOps

Technologies

AWS

IT Infrastructure

Git, Azure, Amazon Redshift, Cloud Computing, Docker

Big Data

Spark, Big Data, Hadoop

Languages

Python, SQL

Other

Apache Cassandra, Amazon Web Services, Data Warehousing

Mes études et formations

M2 Data Science 'D2SN': parcours Big Data - UGE2020 - 2021

M1 Applied Mathematics: parcours Mathematics of Modelling - UPMC2019 - 2020