Team Lead Data Engineer
Primary Duties And Responsibilities
We are seeking an experienced Senior Data Engineer Team Lead to join our team and drive innovation through data. This role combines advanced technical expertise in data engineering with leadership skills to guide and mentor a team in delivering impactful data solutions. The ideal candidate has a knack for data analysis, modelling, and transformation, and thrives in translating business needs into actionable technical strategies. As a Team Lead, this role starts with pre-execution phases to provide critical technical input. You will oversee a project team, ensuring successful execution, and deliver technical guidance to meet and exceed client expectations.
Core Data Engineering Responsibilities
- Identify valuable data sources and automate collection processes.
- Perform pre-processing of structured and unstructured data.
- Analyse large datasets to uncover trends, patterns, and actionable insights.
- Design, build, and develop relational and star-schema data models.
- Architect, develop, and optimize data warehouses and database solutions.
- Implement ETL solutions using tools like SSIS, Azure Data Factory, and SQL Server procedures.
- Develop and maintain real-time analytics and batch processing frameworks.
- Utilize Big Data technologies like Hadoop, Spark, and Kafka for distributed storage and processing.
- Implement cloud-based analytical applications in environments such as AWS and Microsoft Azure.
- Collaborate with engineering and product development teams to deliver data-driven solutions.
- Provide technical guidance and day-to-day mentoring to team members.
- Foster effective communication and collaboration to meet team objectives.
- Monitor project execution, ensuring adherence to quality standards and timelines.
- Identify and address technical and project risks proactively.
- Cultivate a culture of accountability, trust, and continuous improvement.
- Manage client expectations through consistent communication and effective issue resolution.
- Engage in pre-execution phases of projects, providing valuable technical input and strategic guidance.
- Translate business requirements into technical strategies, ensuring alignment with client goals.
- Propose solutions and strategies to address business challenges using data insights.
- Perform database performance tuning and optimization.
- Present information through data visualization tools like Tableau or Power BI.
- Update database structures and maintain data quality processes.
- Research and implement emerging data engineering tools and methodologies.
- Tertiary degree, diploma, or certificate in a related field (e.g., BSc Computer Science, B.IT, Informatics).
- DP-203: Data Engineering on Microsoft Azure Certification.
- 5-7 years of experience as a Data Engineer or Database Developer.
- Expertise in data mining, large-scale data modelling, and business requirements analysis.
- Proficiency in designing and implementing data integration and ETL solutions.
- Experience with Dimensional Modelling and/or Data Vault methodologies.
- Working knowledge of data quality processes and master data management.
- Familiarity with Big Data technologies (Hadoop, Spark) and event processing tools (Kafka, Event Hub).
- Proficiency in database languages (T-SQL, ANSI SQL, PL/SQL).
- Experience with cloud-based tools like Azure SQL, Synapse, Data Lake, and Databricks.
- Additional skills in Tableau, Power BI, Python, R, Scala, and advanced mathematics are advantageous.
Technical and Analytical Skills
- Critical Thinking: Evaluate solutions logically to identify strengths and weaknesses.
- Systems Analysis: Understand how systems operate and how changes impact outcomes.
- Complex Problem Solving: Develop and implement solutions for intricate challenges.
- Technical Literacy: Assess how software solutions fit into organizational structures.
- Data Expertise: Strong understanding of database architecture, data integration, and analytics tools.
- Team Leadership: Guide and inspire teams to achieve goals while fostering a collaborative culture.
- Schedule Management: Ensure development milestones are met on time for self and team.
- Client Engagement: Communicate effectively with C-level stakeholders to align on project vision and deliverables.
- Strategic Execution: Translate organizational strategies into measurable, executable projects.
- Database Development: MS SQL, SSIS, SSAS, SSRS.
- Big Data Tools: Hadoop, Spark, Kafka, Event Hub.
- Programming Languages: Python, Java, C#, R, Scala.
- Cloud Solutions: Azure Synapse, Azure Data Factory, Data Lake, Databricks.
- Visualization Tools: Tableau, Power BI.
- Analytical mindset and business acumen.
- Ability to meet tight deadlines and manage complex tasks independently.
- Valid drivers license and own transport.
- Flexibility and adaptability to dynamic environments.
We seek understanding.
We make a difference.
Weve got your back.
We are leaders.
We are reliable.
We are brave.