תיאור משרה:

 

We are looking for Senior Data Engineer to join our innovative COMBATAI programThe program aim is to produce world-class capabilities for structured & un-structured data, AI lifecycle management that will introduce a new way to empower value from data & AIWe're looking for a skilled Data Engineer to join our team. You'll work with bid data lakehouse & warehouse, focusing on Data pipeline automation methods. Your role involves technical leading of data structure for contextual pipelines, and SQL knowledge is a plus :Key ResponsibilitiesBuild smart logic data pipelines for processing data, manipulating, and deriving insights from dataCollaborate with data stewards to define and build metadata schemesImplement data FAAS and workflows for example: metadata generator, metadata auto-extraction, validation procedures according to schemes and taxonomy, format conversions and moreDevelop tools for audit, observability, and compliance analyticsDate feature Analyzing raw data and organizing it for Interpreting trends and patternsConducting complex data analysis and reporting on resultsPreparing data for prescriptive and predictive modelingExploring ways to enhance data quality and reliability, data fusion from different sourcesDeveloping analytical tools, algorithms and programsCollaborating with data scientists and architects on several projects  :RequirementsBSc in computer science or Data scienceFive years of hands-on experience as lead big data engineerExperience in developed and deployed Big Data & AI solutions with tools and frameworks such as Apache Nifi, Kafka, Spark, Graph DB, NoSQL DB, Airflow, microservices, S3 object storageCoding: Python, Bash:AdvantagesExperience with concepts of managing metadata & Taxonomy beside big data objectsExperience with public cloud Big Data & AI servicesFamiliar with machine learning concepts