Middle Data engineer IRC274101

Description

Our client is focused on developing a robust and versatile data ingestion pipeline and associated schema designed to efficiently and accurately collect, process, analyze, and manage diverse data types from various sources in real-time or near real-time.This pipeline will automate and enhance data workflows, ensure data quality, and support advanced analytical capabilities including NLP, Face Recognition, and OCR.

As a Middle Data Engineer on the project, you will play a crucial role in managing deployment, infrastructure, automation, and monitoring. You will be instrumental in setting up and maintaining CI/CD pipelines, managing cloud resources, ensuring system stability and performance, and implementing robust logging and alerting mechanisms for the client platform.If you seek a challenge and want to impact the way the world distributes products from manufacturers to store shelves, we invite you to join our team.

Requirements

• Strong experience in data pipeline development and ETL/ELT processes.
• Proficiency with Apache Airflow for workflow orchestration.
• Hands-on experience with object storage solutions, preferably MinIO.
• Expertise in SQL and database management, specifically PostgreSQL.
• Experience with graph databases like Neo4j.
• Familiarity with vector databases such as Qdrant.
• Ability to work with large, diverse datasets and ensure data integrity.
• Solid expertise in SQL and relational DBs
• Experience in database design and optimization
• Experience with NoSQL DBs (MongoDB, Cosmos, etc.) for handling unstructured and semi-structured data
• Contributing to release management following the best CI/CD practices
  
  
  

Job responsibilities

• Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading diverse datasets.
• Implement ETL/ELT processes to cleanse, validate, and enrich raw data into query-optimized formats.
• Orchestrate data workflows using Apache Airflow, including scheduling jobs and managing dependencies.
• Manage and optimize data storage solutions in MinIO (object storage), PostgreSQL (relational data).
• Ensure data integrity, quality, and compliance throughout the data lifecycle.
• Collaborate with cross-functional teams to understand data requirements and deliver data solutions that enable advanced analytics and AI/ML initiatives.
• Troubleshoot and resolve data-related issues, ensuring high availability and performance of data systems.
  
  
  

What we offer

Culture of caring. At GlobalLogic, we prioritize a culture of caring. Across every region and department, at every level, we consistently put people first. From day one, you’ll experience an inclusive culture of acceptance and belonging, where you’ll have the chance to build meaningful connections with collaborative teammates, supportive managers, and compassionate leaders.

Learning and development. We are committed to your continuous learning and development. You’ll learn and grow daily in an environment with many opportunities to try new things, sharpen your skills, and advance your career at GlobalLogic. With our Career Navigator tool as just one example, GlobalLogic offers a rich array of programs, training curricula, and hands-on opportunities to grow personally and professionally.

Interesting & meaningful work. GlobalLogic is known for engineering impact for and with clients around the world. As part of our team, you’ll have the chance to work on projects that matter. Each is a unique opportunity to engage your curiosity and creative problem-solving skills as you help clients reimagine what’s possible and bring new solutions to market. In the process, you’ll have the privilege of working on some of the most cutting-edge and impactful solutions shaping the world today.

Balance and flexibility. We believe in the importance of balance and flexibility. With many functional career areas, roles, and work arrangements, you can explore ways of achieving the perfect balance between your work and life. Your life extends beyond the office, and we always do our best to help you integrate and balance the best of work and life, having fun along the way!

High-trust organization. We are a high-trust organization where integrity is key. By joining GlobalLogic, you’re placing your trust in a safe, reliable, and ethical global company. Integrity and trust are a cornerstone of our value proposition to our employees and clients. You will find truthfulness, candor, and integrity in everything we do.

About GlobalLogic

GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world’s largest and most forward-thinking companies. Since 2000, we’ve been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.