CloudFactory

Global leader in workforce solutions for machine learning and business process optimization | Creating meaningful work for talented people in developing nations

https://www.cloudfactory.co.ke/

Data Engineer – CloudFactory – Roles and Responsibilities

As a member of CloudFactory’s engineering team, the Data Engineer – CloudFactory will be tasked with building and growing a world-class distributed workforce management platform that will connect one million people in developing countries to basic computer work.

Technical

Help to design and develop a data estate that is performant, accessible, secure, scalable, maintainable, and extensible.
Help to implement true CI/CD using Github Actions, etc.
Design and develop EDW using Snowflake, DBT.
Design and develop AWS Data Lake using S3, Athena, Snowflake
Design and develop data ingestion pipelines using SnowPipe, FiveTran, etc.
Model EDW entities and ensure all data is complete, accurate, timely, and documented within Data Dictionary and Ubiquitous Terms documents.
Work towards the implementation of a true Self-Service BI platform.
Implement good practices to ensure that Git is used in all circumstances, This includes reviewing others’ work and having your work reviewed and approved by colleagues.

Governance & Data Protection

Ensure that all work follows best security practices and fully adheres to GDPR, PECR, and other data regulations.
Ensure that all work follows the correct approval and sign-off process before it is pushed into Production.
Ensure that all work is documented and if needed – has a runbook in order to allow the business to continually support it.
Where required, ensure that a PIA (Privacy Impact Assessment) is completed.
Work with others in the team to keep the Data Dictionary and Ubiquitous Language complete and up to date.

Process

Follow existing processes and work to improve/identify gaps in these processes.
Ensure the correct SDLC promotion processes are followed.
Follow the correct sign-off processes to ensure that only approved releases are deployed into Production.
Ensure that all AWS development follows CI/CD processes and is repeatable.
Ensure that the AWS “Well-Architected Framework” is adhered to. See https://aws.amazon.com/architecture/well-architected/?wa-lens-whitepapers.sort-by=item.additionalFields.sortDate&wa-lens-whitepapers.sort-order=desc

Team/People

Evangelise about the Data Team and estate across the business.
Build relationships with members of the Data Team and the wider Engineering Team.
Work closely and collaboratively with all members of the Data Team and wider Engineering Team.
Work closely with and learn from tech and team leads and challenge proposed solutions with your own ideas.

Data Engineer – CloudFactory – Requirements

Behavioral Skills

Ability to work across global teams and working with different cultures across different time zones with strong communication and collaboration skills
Tendency to go above and beyond to make things work; manage own and others work to meet the deadline and assist other team members in their deliverables
Ability to identify solutions and make the complex simple.
Ability to breakdown complex problems into simple solutions

Technical Skills

Some experience and knowledge of a coding language such as Python.
Good experience and knowledge of the SQL query language.
Some understanding of star schemas and data warehouse concepts..
Some knowledge of AWS tools and technologies (i.e. Lambda, S3, SQS, SNS, DDB, RDS etc).
Beneficial – ETL and ELT experience – both batch and microservices-led.
Beneficial – Some Snowflake experience.