A seasoned Senior Data Engineer played a key role in migrating a traditional bank to a Cloud-Based NeoBank. Proficient in AWS, PySpark, Python, and data quality frameworks, they transformed financial reporting and designed comprehensive data quality solutions. With prior experience as a Data Analyst, they excelled in data pipeline design, integration, and dashboard creation, showcasing a proven track record of optimizing data processes for organizational success.
AWS
Python
• Develop ETL processes to migrate an On-Premises Bank, to a Cloud-Based NeoBank using tools such as the AWS Cloud, Pyspark, Python, SQL, Pytest, JUnit, JFrog, SonarQube, FluidAttacks, Terraform, Azure Pipelines, and Data Quality with Great Expectations.
• Improved the average time for Financial Reporting from 1 report per month to 12 reports per month.
• Designed the Data Quality framework for the entire organization leveraging proprietary Python developments using frameworks such as Great Expectations, DAMA-UK for Governance and Spark.
• Developed Azure Pipelines and provisioned infrastructure using Terraform, to develop AWS Glue Jobs, and other required infrastructure to perform the ETL processes..
Skills: Engineering
• Design strategies to build data pipelines aimed at the ingestion of data necessary to carry out all the processes managed by the units of the Vice Dean.
• Establish the mechanisms for building and feeding databases used in the units of the Vice Dean, allowing easy access to obtain information and assuring data quality.
• Perform data integration jobs using Python, Pandas and Numpy libraries, in order to clean academic data coming from public repositories and websites.
• Access and download reports from a centralized repository in Oracle in order to build Dahsboards on PowerBI and present reports to the Board of Directors.
• Define the schemas of academic databases, in order to create dashboards with demographic information and academical KPI’s (Current Students, Graduates, GPA, courses taken, etc).
• Support the Bi-Annual Courses Scheduling team to load information regarding courses per program that the students will be taking for their respective bachelor, master and PhD degrees.
• Clean and transform academical and student-related information, complying with national regulations, to build periodical reports and Dashboards that are presented to the National Ministry of Education.
Tools: Jupyter Lab, Python, Pandas, PowerBI, Oracle
• Built frameworks to capture data from CSV files using Python, and organized the captured data using Microsoft Excel (VBA) tools, in order to help the periodical reports needed at the dependencies of the Vice Dean.
• Performed ETL jobs from reports sent from other departments of the National University of Colombia, in order to build dashboards to be used at the Vice Dean meetings.
• Automated processes to support queries of academic information by the Units of the Vice Dean.
• Supported the analysis of the budgetary projection of the postgraduate curricular programs proposed to be offered through cooperation agreements, and to monitor and control those that are implemented, among other administrative tasks.