Role Profile
Role title
Data Engineer
Location
Johannesburg
Contract type
Permanent, Full Time
Reporting to
Global Head of Data Operations
Role purpose
As a Data Engineer, you will be a driving force towards data engineering excellence.
Working with other data engineers, analysts, and the architecture function, youll be involved in the building out of a modern data platform using a number of cutting-edge technologies, and in a multi cloud environment,
Youll get the opportunity to spread your knowledge and skills across multiple areas, with involvement in a range of different functional areas.
As the business grows, we want our staff to grow with us, so therell be plenty of opportunity to learn and upskill in areas such as data pipelines, data integrations, data preparation, data models, analytical and reporting marts.
Also, whilst work is often following business requirements and design concepts , youll play a huge part in the continuous development and maturing of design patterns and automation process for others to follow .
Accountabilities And Main Responsibilities
In this role, you will be delivering solutions and patterns through Agile methodologies as part of a squad. Youll be collaborating with customers, partners and peers, and will help to identify data requirements. Wed also rely on you to:
- Help break down large problems into smaller iterative steps
- Contribute to defining the prioritization of your squads backlog
- Build out the modern data platform (data pipelines, data integrations, data preparation, data models, analytical and reporting marts) based on business requirements using agreed design patterns
- Help determine the most appropriate tool, method and design pattern in order to satisfy the requirement
- Proactively suggest improvements where they see issues
- Learn how to prepare our data in order to surface it for use within APIs
- Learn how to Document, support, manage and maintain the modern data platform built within your squad
- Learn how to provide guidance and training to downstream consumers of data on how best to use the data in our platform
- Learn how to support and build new data APIs
- Contribute to evangelising and educating within Sanne about the better use and value of data
- Comply with all Sanne policies
- Any other duties in the scope of the role that the company requires.
Technical Skills:
- Data Warehousing and Data Modelling
- Data Lakes (AWS Lake Formation, Azure Data Lake)
- Cloud Data Warehouses (AWS Redshift, Azure Synapse, Snowflake)
- ETL/ELT/Pipeline tools (AWS Glue, Azure Data Factory, FiveTran, Stitch)
- Data Message Bus/Pub Sub systems (AWS SNS & SQS Azure ASQ, Kafka, RabbitMQ)
- Data Programming languages (SQL, Python, Scala, Java)
- Cloud Workflow Service (AWS Step Functions, Azure Logic Apps, Camuda)
- Interactive Query Services (AWS Athena, Azure DL Analytics)
- Event and schedule management (AWS Lambda Functions, Azure Functions)
- Traditional Microsoft BI Stack (SQLServer, SSIS, SSAS, SSRS)
- Reporting and visualisation tools (Power BI, QuickSight, Mode)
- NoSQL & Graph DBs (AWS Neptune, Azure Cosmos, Neo4j) NoSQL & Graph DBs (AWS Neptune, Azure Cosmos, Neo4j) (Desirable)
- API Management (Desirable)
- Excellent communication and interpersonal skills
- Critical Thinking and research capabilities
- Strong problem-solving skills
- Ability to plan, and manage your own work loads
- Work well on own initiative as well as part of a bigger team
- Working knowledge of Agile Software Development Lifecycles
