Juan Carlos M.

Data Engineer

Externalisation nearshore
12 ans
Guadalajara, MEXICO
Contactez-nous à propos de ce profil

Mon expérience

Voir plus

FreelanceJuin 2020 - Présent

Data Engineer/ETL Developer Freelancer.

As Data Engineer/ETL Developer:
  • Developed ETL scripts to extract data from source to target schemas (MySQL to SQL Azure). Developed several SQL scripts to extract and aggregate data into SQL Azure.
  • Developed Dimensional model for sales reporting service and visualize the data in Power BI.
  • Managed all Power BI users and config data sync between Azure SQL and Power BI.
  • Daily meeting with Data Stakeholders and Management to review progress along with data cleansing rules.
Functional Experience: Scrum methodology, and Data Engineer/ETL Developer.
Tools: Azure Data Factory, SQL scripting, SQL developer, Azure.
Voir plus

Tiempo Development MexicoAvril 2020 - Présent

Sr. Data Engineer within a company that provides design, development and deployment services to software companies and enterprises.
  • DWH development and design of new web analytic solution.
  • Designed and developed several ETL packages under Snowflake platform, Query Performance Optimization.
Voir plus

FreelanceMai 2020 - Présent

Data Engineer/ETL Developer Freelancer.

As Data Engineer/ETL Developer:
  • Developed and designed tasks within Snowflake to consume data from S3 to Snowflake.
  • Set up permissions in AWS to allow Snowflake connection.
  • Set up and designed data model in Snowflake (SQL Data Modeler).
  • Developed ETL scripts and scheduled them in Snowflake, followed a type 1 SCD.
  • Created complex queries for data app.
  • Daily meetings with Data Stakeholders and Management to review progress along with data cleansing rules.
Functional Experience:    Scrum methodology, and Data Engineer/ETL Developer.
Tools:    Snowflake, SQL scripting, SQL developer, AWS.
Voir plus

Data Analyst/ETL Developer (HP). - SofttekJanvier 2019 - Avril 2020

Data Analyst/ETL Developer (HP).

As Data Analyst/ETL Developer:
  • Developed ETL scripts to extract data from source to target schemas (SAP to SAP -HANA). Developed several SQL scripts to extract data into csv files.
  • Helped the team with data Analysis and Data cleansing activities.
  • Daily meeting with Data Stakeholders and Management to review progress along with data cleansing rules.
Functional Experience:    Scrum methodology, and Data Analyst/ETL Developer.
Tools:    SQL scripting, SQL developer, AWS.
Voir plus

Data Engineer, Bloksports - SofttekSeptembre 2019 - Janvier 2020

Data Engineer, Bloksports.

As Data Engineer:
  • Designed a new transactional database.
  • Designed data flow diagrams for possible dimensional model and for analytical purposes.
  • Designed a new database in PostgreSQL and for ETL processes I proposed to used Glue.
  • Reviewed slow running queries in current database and updated the scripts accordingly to improve performance.
Functional Experience:    Scrum methodology, and Data Engineer/ETL Developer.
Tools:    PostgreSQL, SQL scripting, Database design, SQL developer, AWS.
Voir plus

FreelanceFévrier 2018 - Octobre 2019

Data Warehouse and BI Consultant, Freelancer. 
Bombas Mejorada S.A de C.V.(http://www.mejorada.mx/).

As Data Warehouse Consultant:
  • Designed Finances Data Mart.
  • Created all mapping Documents for the ETL process.
  • Created Physical and Logical design of Data Mart.
  • Created along with General Manager Business rules of all ETL.
  • Created Data Dictionary of ERP and New Data Mart.
  • Designed ETL using SSIS.
  • Designed ETL Code for incremental Data Loads.
  • Designed Reporting SQL statement.

As BI Consultant:
  • Implemented Power BI for all reporting within the Company.
  • Implemented Power BI for new Data Mart.
  • Created all reports for Data Mart.
  • Coached Management and Users on how to use Power BI.
  • Created KPIs for Finances Department.
Functional Experience: Scrum methodology, and Data Analyst/BI Consultant.
Tools: SQL scripting, SSIS, SQL Server 2014, Power BI, Python scripting.
Voir plus

Data Engineer, Consumer Track - SofttekMars 2019 - Août 2019

Data Engineer, Consumer Track.

As Data Engineer:
  • Created new dataflows using Python and Pentaho Kettle.
  • Troubleshooted data issues in DWH.
  • Created new extracts for new Data mart.
  • Modified theexisting database to accommodate new reports.
  • Created several S3 buckets to store new extracts.
  • Created several scripts in Snowflake for QA purposes and for new data extracts.
Functional Experience:    Scrum methodology, and Data Engineer/ETL Developer.
Tools:    Snowflake, AWS S3, Pentaho, Python, SQL scripting.
Voir plus

Data Analyst/ Hive Consultant - SofttekAvril 2017 - Avril 2019

Data Analyst/ Hive Consultant, Legal (DXC-HPE).

As Data Engineer/Analyst/Hive Consultant:
  • Developed ETL scripts to extract data from source to target schemas (Hive DWH). We used Sqoop, Python, SSDT and other tools to extract data into csv files to later upload them into HDFS and Vertica database.
  • Helped the team with data load and defining partitions by assets to maximize the ETL process.
  • In charge of peer reviewing to ensure business rules.
  • Helped with CR and any logic changes. 
Functional Experience: Scrum methodology, and Data Analyst/Hive Consultant.
Tools: SQL scripting, Hadoop – Hive, HP-ALM, Python scripting, AWS.
Voir plus

Hadoop Consultant (HPI) - SofttekAvril 2017 - Août 2017

Hadoop Consultant (HPI).

As Hadoop Consultant:
  • Coached and trained Softtek team on the new project. The project architecture was on AWS and HIVE as Data Warehouse database. 
  • Helped the team to execute and create Python scripts to ensure data integrity.
  • Helped the team with one to one training on writing and optimizing Hive Queries.
  • Proposed alternatives for Visualization tool and created POC with Power BI.
Functional Experience:    Scrum methodology, and Data Analyst/Hive Consultant.
Tools:    AWS-Hortonworks(Hive), Power BI, SQL scripting, Python scripting.
Voir plus

FreelanceJanvier 2017 - Août 2017

ETL Developer - Freelancer, Easy Expunctions (Remote).

As ETL Developer:
  • Cleansed documents to import into a Staging table and from staging to dimensional model using Python.
  • Wrote several Python scripts to help format and cleanse files before loading them into staging.
  • Created several Pentaho Jobs to load data.
Functional Experience:    Scrum methodology, and ETL developing.
Tools:    Python, SQL scripting, Pentaho Kettle, PostgreSQL developing, SQL server developing, Scrum methodology.
Voir plus

Informatica ETL Developer, Finance(HP Enterprise-SSIT) - SofttekFévrier 2016 - Janvier 2017

Informatica ETL Developer, Finance (HP Enterprise-SSIT).

As ETL Developer:
  • Reviewed, analyzed and made changes to existing ETL scripts for either new wave of ETL’s or issues with the workload in Production or UAT environments.
  • New Code for new extracts to run daily or on demand in HP Enterprise Data Warehouse.
  • Tech lead therefore I was in charge of peer reviewing and I was responsible for the ETL and BI process.
  • In charge of translating the requirements into Informatica mappings, and their respective sessions and assigning sessions to existing new workflow.
Functional Experience: Scrum methodology, and BI/ETL Developer.
Tools: SQL scripting, Informatica IPC, Excel reporting, Vertica.
Voir plus

Data Architect, Legal (HPE Global ITSS) - SofttekFévrier 2016 - Janvier 2017

Data Architect, Legal (HPE Global ITSS).

As Data Architect:
  • In charge of Metadata creation in Vertica Environment, to meet Business recommendations.
  • Analysis of new scripts and creation of them to copy data from Hadoop over to Vertica DB.
  • Copying data from Hadoop over to Vertica DB.
Functional Experience: Scrum methodology, and Data Architect.
Tools: SQL scripting, Excel reporting, Hadoop – Hive, Vertica Database design, and Development.
Voir plus

Data Analyst and ETL Developer - SofttekFévrier 2016 - Janvier 2017

Data Analyst and ETL Developer, Legal (HPE Global ITSS).

As Data Analyst:
  • Troubleshoot CR, and data errors due to ETL scripts or due to incorrect mapping logic.
  • Worked with BA to analyze issues on data quality.
  • Created new scripts in Python or Unix to analyze data or to analyze constraints.
As ETL Developer:
  • Reviewed existing ETL scripts, analyzed and made necessary changes for either new wave of ETL’s or issues with the workload in Production or UAT environments.
  • New Code for new extracts to run daily or on demand in HP Enterprise Data Warehouse.
  • Reviewed Code of the team because I acted as a Tech lead and responsible for the ETL and BI process.
  • In charge of translating the requirements into Hive to Informatica mappings, and their respective sessions and assigning sessions to existing new workflow.
Functional Experience: Scrum methodology, and ETL Developer.
Tools: SQL scripting, IPC, Excel reporting and HPSM tool, Hadoop -Hive.
Voir plus

FreelanceFévrier 2014 - Mars 2016

Freelance Microsoft Developer.

As SSIS Developer:
Company: Garanti Autoparts (http://www.garantimobil.com.mx/)

- The project consists of, extracting data from the Operational database (Visual Fox Pro database) and moving it to SQL Server 2014 (Sometimes using SQL server 2008 R2 Express as an instance to move the data from 32 bit to 64 bit) for further analysis (Excel Reports using VBA).
  • Milestone, I designed the workflow from VFP and excel sheets, so the data was cleansed in a Loading Zone inside SQL server database, following a star schema so that the company’s data can grow into a data warehouse later on. This was a big hit because leadership in the company can take into considerations historical data to purchase and to sell auto parts depending on the region and model of the car.
  • Developed an in-house system to track down orders so the entire cycle of inventory is tracked and reported for analysis and adjustments if necessary (.net, Jquery, and C# along with my own custom class to move data from VFP (free tables) into SQL Server Express 2008 R2).
  • Created SQL queries for several departments and Company’s CEO.
  • Implemented Power BI so each department can customize their own dashboards and see data graphically.
  • Developed Several in-house reports using VBA (Macros), ranging from inventory management to data extracts.
Functional Experience: ETL Developer, .Net Developer, Solution Architect.
Voir plus

BI Developer, ETL Developer, PM - SofttekFévrier 2015 - Janvier 2016

BI Developer, ETL Developer, PM for Year end-Closing and Rollover, Finance (HP Enterprise SSIT).

As BI Developer:
  • Modified reports for Internal BI tools.
  • Analyzed existing code and reports to make sure they follow Finance department requirements.
  • In charge of modifying existing code and reports so they will meet new Finance changes.

As ETL Developer:
  • Reviewed existing ETL scripts and analyzed the logic with BA.
  • New Code for new extracts to run daily or on demand in HP Enterprise Data Warehouse.
  • Peer reviewing other’s team member codes.

As PM:
  • Daily meeting with Domain to set up Requirements, Change requests and workload of the team.
  • Managed a team of 3.
  • Followed the Scrum methodology to have daily 30 minutes calls with the team for any questions, doubts, the team might have.
  • Weekly reports and meetings with the Project Owner to coordinate releases and implementation plan and schedules of deliverables.
- I acted, most of the time, as a Tech lead in this project, and I was in charge of several SQL scripts used in the Financial fiscal year closing cycle.

Functional Experience: Scrum methodology, and BI/ETL Developer.
Tools: SQL scripting, Excel reporting and HPSM tool, HP Internal BI Tools(YMS, YOTTA).
Voir plus

Bi Developer, Data Modeler - SofttekFévrier 2015 - Janvier 2016

Bi Developer, Data Modeler, Tech Lead for Sales Compensation Reporting (HP- Enterprise SSIT).

As ETL/BI Developer:
  • Created ETL script for new extract.
  • Created XML for several new extracts.
  • In charge of all the extracts from the Data warehouse to the downstream system (Sales Compensation)
  • Modified existing code for existing Extracts and developed new extracts according to requirements.
  • Fixed production issues of Extracts.

As Data Modeler:
  • Modified several tables to handle separation logic of the company.
  • Created Dimensional Model in Erwin and E/R Studio.
  • Daily meetings with Data warehouse Architects to define new table structure and implementation plan.
  • In charge of L4 support on Productions issues on HP data warehouse.
As Tech Lead:
  • Tech lead for a group of 3 people, trained on internal HP BI tools and software.
  • Helped with new resources with code and internal procedures.
Functional Experience: BI Developer, Data Modeler for Relational and Dimensional models.
Tools: SQL scripting, HP Internal BI tools (Yotta and YMS), Excel reporting.
Voir plus

Developer on Call - SofttekJanvier 2014 - Février 2015

Developer on Call (HP-DS) / Tech Lead.
  • L4 support on Productions issues on HP data warehouse.
  • In charge of submitting fixes to Production issues when SQL jobs failed.
  • Analysis of scripts, and implementation ofbest practices when troubleshooting.
  • Implemented with the team a bug tracker to speed up the resolution process, since we were able to track similar issues or Jobs recurring over time.
  • In charge of scheduling of 4 resources and coordinating with the client.
Functional Experience: SQL scripts troubleshooting, Tidal, Dial, and internal HP data extract tools used.
Tools: SQL scripting, Tidal and Dial.
Voir plus

Data Modeler - SofttekAvril 2014 - Novembre 2014

Data Modeler (Sales and Comp IT, HP-DS).
  • Acted as a data modeler for a Data warehouse project.
  • Designed new extracts and Tables to be used for new extract, along with Business analyst and Tech lead requirements.
  • Developed SQL scripts to extract data and create reports (Entire ETL Process).
  • Helped on Production issues, by troubleshooting SQL Scripts or Jobs.
  • Vast usage of HP internal Business intelligence tools.
Functional Experience: SQL scripts troubleshooting, and internal HP data extract tools used.
Voir plus

Garanti MobilOctobre 2013 - Février 2014

Systems Manager within a company in Mexico.
  • Implemented a helpdesk system based on .Net and SQL Express 2008 database, which helped with internal communication from headquarters to branches.
  • Implemented AD DS on headquarters to help control permissions to sensitive data.
  • Implemented a speed counter system to allow fast shipping on local orders and trained staff on the use of company’s ERP.
  • Maintained ERP´s written in VFP.
  • Managed more than 50 concurrent users and established rules and procedures for the entire company, established tier levels of escalation when troubleshooting arose.
  • Team Leader for the new ERP the company purchased, establishing daily task for the entire staff, as well as a workload to migrate data to the new System.
  • Refactored and created procedures with VBA, C#, and T-SQL to extract information stored in VFP database to update the handheld system so the ERP and the handheld software were synced especially with inventory and bin locations.
  • Created procedures and naming convention for the entire company’s email addresses.
  • Created Linked servers infrastructure to ETL data from VFP tables to SQL Server.
  • Coordinated a team of 2 people to give support, development, tests.
Functional Experience: Documenting, VBA and Excel Macros, Access and Access SQL, Design company procedures and Company escalation tiers, Proposed and implemented new ERP using and more up-to-date technology.
Voir plus

Premier Products (Remote)Avril 2009 - Juillet 2013

Database Administrator for a distributor of aftermarket auto parts.

As SQL Developer:
  • Created complex queries and reports using VBA and excel maintain datasets and a production database of more than 350000 items.
  • Created development databases and tables to test updates/inserts for integrity as well as jobs in production, worked closely with the development team to maintain company´s software.
  • Created reports using VBA and Macros for upper management and to maintain data.
  • Worked closely with vendors to keep records up-to-date.
  • Participated and was in charge of migrating VB6 database to SQL Server 2008 database and new ERP (Sage Mas 200 to Sage Mas 500).
  • Created an app to create new UPCs for any new item coming in into any of the Company’s warehouses.

As Database administrator clerk:
  • In charge of maintaining the database and updating/inserting and deleting records.
  • Daily review of datasets and update accordingly, as well as update UPC and implemented company’s UPC for daily usage in shipping and delivery.
  • Well versed and hands-on experience on SQL Server 2008R2.

Mes études et formations

Bachelor's Degree in Computer Engineering - Technological Institute of Aguascalientes.2012 - 2015

Computer Information System - Penn Foster College.2007 - 2009