DEPARTMENT: D&A
PURPOSE OF THE JOB:
We are a leading retail company with a strong focus on customer satisfaction and innovation. We are currently looking for a talented and motivated data engineer to join our dynamic team. In this role, you will have the opportunity to work on exciting projects, helping us define and solve real business problems and see solutions through to the end by using data and applying state-of-the-art machine learning operations principles. The Data Engineer is responsible for and specializes in the development of the architecture and data platform that ensures the data is ready for analytical usage, thus supporting both the data scientists and data analysts. The data engineer develops the foundation for various data operations and does analysis on the data.
KEY RESPONSIBILITIES:
- Data Exploration and data quality assurance
- Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality
- Acquire data from primary or secondary data sources and maintain databases/data systems
- Filter and clean data by reviewing other reports and performance indicators to locate and correct code problems
- Development, construction, and maintenance of data architectures.
- Conducting testing on large scale data platforms.
- Handling error logs and building robust data pipelines
- Ability to handle raw and unstructured data.
- Provide recommendations for data improvement, quality, and efficiency of data.
- Ensure and support the data architecture utilized by data scientists and analysts.
- Development of data processes for data modeling, mining, and data production
- Analyze data through descriptive statistics
- Design and build production grade minimum viable products such as dashboards
- Communicating results with the team and business using data visualization
- Design and build production grade minimum viable products such as dashboards and published notebooks for business users, linked to the feature stores, scoring engines and optimisation models.
- Feature engineering
- Standardization or Scaling of the data
- Create Value Transformation like Square or Cube
- Design and build the feature stores for model training
- Operationalize feature stores for model training
- Work with management to prioritize business and information needs
- Locate and define new process improvement opportunities
- Presentations to management
- Identify the goal and frame the business problem.
- Gather information on resources, constraints, assumptions, risks.
- Prepare Analytical Goals and Benefit to business
- Understanding business and information flow
- BSc in Mathematics, Economics, Computer Science Management or Statistics, National Diploma in Information Technology, BEng.
- 3-4 years of relevant experience within the field of data engineering/software development.
- Knowledge of core computing concepts like programming and algorithms to build robust data systems.
- Knowledge of application development and working of APIs;
- 3- 4 years of programming
- Proficient in Visualisation
- Cloud tools: (preferably on GCP) :
- Big Query, Composer/Airflow, AI Platform, Kubernetes, Dataflow, Pub/Sub, Cloud (Functions, Scheduler, SQL, Storage), Compute Engine, Monitoring
- Knowledge of the following will be advantageous:
- Git workflow, Agile management (JIRA, Shortcut, Trello), Slack, CI/CD, Docker, SQL, Python, JavaScript, Google Workspaces, Visualization tools: QlikView / Datastudio / Grafana
