Hiring?Get vetted candidates in 48 hours. Pay only when you hire.Get vetted candidates in 48 hours.
P
Data Engineer
Paystack
Posted 1 months agoNigeria or South Africa
Location
Nigeria or South Africa
Job Type
Full-time
Experience
Mid-Level
Category
Engineering
Job Description
About Paystack
Paystack's mission is to accelerate digital payments on the continent of Africa. Over 200K of some of the most renowned businesses in Nigeria, as well as a growing number of merchants in South Africa, Ghana, and Kenya use Paystack's modern payments gateway. They include the likes of Qatar Airways, MTN, Burger King, UPS, AXA Mansard Insurance, FilmHouse Cinemas, Smile Communications, Air Peace Airlines, Crocs, Under Armour, Richemont Lifestyle Group (RLG), and many others.
In just over 10 years, our growing team has introduced a steady cadence of innovation within the online payments landscape, such as automated recurring payments, the ability for customers to pay directly from their bank accounts, automated chargebacks, and much more. We've been acquired by Stripe and are doubling down on the mission to help build out Africa's payments infrastructure.
Paystack builds technology to help Africa's best businesses grow - from new startups, to market leaders launching new business models. We make it easy for businesses to accept secure payments from multiple local and global payment channels, and then we provide tools to help you retain existing customers, and acquire new ones.
ProGigFinder Press
Fifty silent applications, or one that lands.
Pick any job above and pay $1.49 with MoMo or Card. In 60 seconds you get a CV written for that role: it passes the screening software and reads the way a recruiter expects.
Get my CV for this job: $1.49
Responsibilities
- Key Responsibilities:
- Data Pipeline Development: Design, develop, and maintain robust data pipelines using ETL and ELT methodologies to process and integrate data from various sources into a data lake, a central data warehouse, operational data stores, analytical data marts and various application interfaces.
- Streaming Data Processing: Implement and manage real-time data streaming solutions utilising Kafka, Debezium, Kafka Connect.
- Workflow Orchestration: Build, schedule and maintain custom workflows using Apache Airflow to ensure timely and accurate data processing and delivery.
- Database Management: Work with a variety of database technologies, including relational databases (MySQL, PostgreSQL), NoSQL databases (MongoDB) and analytical/big data systems (Redshift, BigQuery, SingleStore).
- Infrastructure as Code: Employ tools like Terraform, Kubernetes, and Helm to manage and provision infrastructure efficiently.
- CI/CD Implementation: Develop and maintain continuous integration and deployment pipelines to streamline development processes.
- Testing and Quality Assurance: Conduct unit and integration testing to ensure high code quality, data integrity and system reliability.
- Collaboration: Engage with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data needs and deliver solutions.
- Documentation: Maintain clear and comprehensive documentation of data processes, workflows and systems.
- Monitoring and Support: Monitoring system performance and addressing faults and failures in production systems as part of an on-call rotation ****
- Skills and Experience:
- Educational Background: Bachelor's degree in Computer Science, Engineering or a related field.
- Programming Skills: Proficiency in Python is essential. JavaScript and Scala development experience is advantageous.
- Data Engineering Experience: Minimum of 3 years of experience in data engineering roles, with a focus on building and managing data pipelines.
- Software Engineering Experience: Minimum of 2 years experience in software and/or application development roles (can be concurrent with data engineering experience)
- Streaming Technologies: Hands-on experience with Kafka, Debezium, and Kafka Connect.
- Data pipeline orchestration tools: Proficiency in a data pipeline orchestration tool or suitable workflow orchestration tool like Apache Airflow (preferred), Databricks, Dagster or Airbyte.
- Database Expertise: Strong understanding and hands-on experience working with various database technologies, including MySQL, PostgreSQL, MongoDB and Redshift (BigQuery and SingleStore advantageous)
- Infrastructure Tools: Experience with Terraform, Kubernetes, and Helm for infrastructure management.
- Cloud Computing: Solid knowledge of cloud computing concepts, with experience in AWS services being advantageous.
- SQL Proficiency: Ability to write complex SQL queries across different dialects.
- Testing Practices: Familiarity with unit and integration testing methodologies.
- CI/CD Pipelines: Experience in setting up and maintaining CI/CD pipelines.
- Data Science Tools: Exposure to analytical systems and basic data science tooling. Familiarity with basic machine learning and analytical modelling concepts advantageous.
- BI Reporting Platforms: Exposure to self-service reporting tools like Tableau, Looker and DOMO.
- Soft Skills:
- Communication: Good verbal and written communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.
- Team Collaboration: Demonstrated ability to work collaboratively within a team and across departments.
- Adaptability: Comfortable working in a fast-paced environment with changing priorities, technologies and tooling. Life-long learners will do well here.
- Problem-Solving: Strong analytical and problem-solving skills.
- Company Core Values:
- Transparency: We encourage open sharing of work, seeking feedback and having honest conversations promptly.
- Clear Communication: We simplify ideas, communicate directly and confirm understanding to ensure clarity.
- Kindness: We value positive vibes, generosity and empathy, both within the team and with customers.
- High Standards: We insist on delivering consistent excellence, taking ownership and striving to be domain experts.
- Pursue Growth and Learning: We view every situation as a learning opportunity, encouraging experimentation and continuous improvement.
- Embrace the Mission: We find joy in our work and recognise the company's broader impact.
