Marius B.

Data Engineer

390 dollar
Freelancer
2 ans
Cluj-Napoca, ROUMANIE

Mon expérience

Voir plus

YardiAugust 2019 - Présent

- Aggregate large datasets from multiple sources.
- Identify patterns, analyze and interpret complex datasets in order to be able to extract relevant information.
- Research missing/incomplete/faulty data.
- Assess, ensure and maintain data quality.
- Create /Use /Maintain Perl & SQL scripts responsible for ETL transactions, data parsing, and tasks automatization.
- Track errors and provide ongoing appropriate inter-departmental communication and monthly or daily data reports.
Stack: Perl, Oracle SQL, Git, SVN, Linux, RegEx, Jenkins.
Voir plus

DXC LuxoftMay 2021 - Présent

- Implement and develop auto-orchestrated Data Ingestion Framework (internal framework).

- Leverage Azure Cloud to create and maintain ETL/ELT environments.

- Develop new features within the framework or as part of the native data-flow leading to it.

- SME in initial deploying and start of the framework in new clients environment.


• Stack: Python, PySpark, DataBricks, Azure DevOps, Azure Data Factory, SQL, Azure SQL, Delta Lakes.

Voir plus

SteelcaseJanuary 2021 - Présent

• Work in an Agile team to support multiple stakeholders and fulfill the data need in:

-- ML /AI projects for Data Science

-- Visualization projects for Data Analytics.

• Leverage Azure Data Factory, Azure DataBricks, Azure DevOps, Azure SQL, Snowflake... to create and maintain ETL/ELT data pipelines having multiple ingestion points / transformations of variable data formats.

• Lead new features and high impact projects which coordinate resources from

multiple sources and teams in a decentralized department.

• Leverage and improve DataBricks spark environment for efficient data manipulation / transformation.

• Drive the improvement of current project architecture as follows:

 -- Make use of the new Repos feature in DataBricks to improve upon current CI/CD.

 -- Apply different layers to project structure to drive away from heavy notebooks scripting and towards modularization of code.

 -- Migrate towards more functional & OOP structured code for better QA.


• Stack: Python, PySpark, DataBricks, Azure DevOps, Azure Data Factory, SQL, Java, Snowflake, Oracle SQL, Azure SQL, Delta Lakes.

Mes compétences

Big Data

PySpark

Technologies

Azure Data Factory

Languages

Perl, Regex, OOP, JavaScript, Bash, MVC, Python, HTML, Java, Oracle PL/SQL, SQL

Analysis methods and tools

JIRA, Apache Maven, Subversion (svn), Kanban, DevOps, Confluence, Design Patterns

Business Intelligence

ETL, Business Intelligence, Tableau Software

Databases

PostGreSQL, Oracle, MySQL, Microsoft SQL Server

Computer Tools

Microsoft Excel, MS Office

Environment of Development

GitLab, Maven

IT Infrastructure

Azure DevOps, Azure Cloud, Docker, Linux, Git

Middleware

Jenkins

Frameworks

Spring, REST, JPA

Others

GitHub, Data analysis, Communication, Leadership, Project Management, Artificial Intelligence, GitFlow, Analytics

Other

Oracle SQL, Cascading Style Sheets, Scrum Methodology, Python Programming, Error handling, Regular Expressions, Data Analyst, Research Analyst, BigData Development, Snowflake, Senior Data Engineer, Windows Azure Platform, Azure SQL, develop auto, Real Estate Market Analyst, Apache Subversion, Bachelors Degree, Data Collection, LinkedIn, Data Engineer, tableau, FOCUS, Microsoft Office, RDBMS, Object Oriented Analysis/Design, Perl Programming, Line Coordinator, Spring Framework, industry~it