Cavendish Professionals
This listing was posted on Careers4a.
Senior Data Engineer
Description:
Senior Data Engineer with DatabricksProject Overview: Our client is embarking on a significant transformation project aimed at replatforming their financial system. This ambitious initiative includes two parallel efforts: one focusing on all financial data and the other on non-financial data. We are in the early stages of consolidating data from various sources into a single Delta Lake data warehouse, and we are looking for a Senior Data Engineer to lead this critical endeavor.Senior Data Engineer with DatabricksProject Overview: Our client is embarking on a significant transformation project aimed at replatforming their financial system. This ambitious initiative includes two parallel efforts: one focusing on all financial data and the other on non-financial data. We are in the early stages of consolidating data from various sources into a single Delta Lake data warehouse, and we are looking for a Senior Data Engineer to lead this critical endeavor.Responsibilities: Lead the integration and ingestion of data from diverse sources into the Delta Lake platform, ensuring data consistency and availability.Collaborate with various teams to identify all data sources, including SFTP/flat files, and APIs.Design and implement systematic processes for data ingestion and integration.Conduct extensive data analysis to support the consolidation of organizational data into a single data warehouse.Utilize Databricks for data ingestion and reporting, transitioning data from multiple data warehouses to a unified Delta Lake platform.Work closely with the development team to understand project scope, gather full requirements, and plan subsequent phases.Participate in daily standups, sprint planning, backlog refinement, and retrospectives within a scaled scrum framework.Exhibit flexibility and adaptability in navigating unknowns and evolving project requirements. Requirements: Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.Solid understanding of the software development life cycle.Proven experience with Python, Spark, and SQL for data engineering and analysis.Strong data analysis skills, with the ability to interpret and manipulate large datasets.Experience with Databricks for data ingestion and transformation.Familiarity with Delta Lake and data warehousing concepts.Excellent problem-solving skills and the ability to see the big picture and understand the broader context of assigned tasks.Strong teamwork and collaboration skills, with a demonstrated ability to work effectively in a team-oriented environment.Knowledge of reporting tools like Power BI.Knowledge of Data Science concepts is a plus.Strong communication, interpersonal, and presentation skills. Start Date: ASAPLocation: RemoteUS Hours Needed: Minimum until 6pm CETType of Employment: B2B
Company:
Cavendish Professionals
Posted:
July 5 on Careers4a
Visit Our Partner Website
This listing was posted on another website. Click here to open: Go to Careers4a
Important Safety Tips
- Always meet the employer in person.
- Avoid sharing sensitive personal and financial information.
- Avoid employment offers that require a deposit or investment.
To learn more, visit the Safety Center or click here to report this listing.