Frisma R.

DATA ENGINEER

561 dollar
Freelancer
4 ans
London, UNITED KINGDOM

Mon expérience

Voir plus

AccentureJanuary 2019 - Présent

Project: Develop end to end DataOps solution with ETL framework using PySpark/Scala and build integrated ETL pipelines for data processing orchestrated through DevOps Containerized platform built on Kubernetes
 Roles & Responsibilities:
• Developed data pre-processing module for batch and streaming data across multiple data sources using Pyspark
• Implemented test and simulation framework for automated testing of data transformation module and wrote unit test cases of ETL application
• Developed parameterized Jenkins pipeline using groovy script.
• Implemented Dask multiprocessing with dynamic spark clusters to invoke Dask executors.
• Developed web based applications using python
• Played key role in implementing EDA (Explo- ratory data analysis), data quality, data cleaning and data validation using Python scripting.
• Developed data logging and lineage monitoring programs using Python, Java, shell scripting to push logs to data analytics solution.
• Implemented API approach to package application with Flask API
• Created standard operating procedure documentation to assist SDLC process and user documentation for easy implementation of solution.
Voir plus

BTC techno labs cochinSeptember 2017 - April 2018

Project: Develop fingerprint valuation algorithm and implement voice-based system for blinds using google voice modules. 
 Roles & Responsibilities:

• Developed fingerprint valuation algorithm using Python
• Developed python packages to handle user level exceptions 
• Developed stored procedures and functions on Oracle database
• Web based development experience utilizing python.

Mes compétences

Test Cases, SQL, SDLC, Scala, Python, PySpark, Pandas, Oracle, Microsoft SQL Server, Kubernetes, Jenkins, JavaScript, Java, HTML, HDFS, ETL, DevOps, Data processing, Big Data, Azure DevOps, AWS, API, Analytics, Amazon Identity and Access Management (IAM), Amazon EC2