Java software engineer with GCP experience and DataFlow Sme.

March 23 2021
Industries Airspace, Aviation, National Defence
Categories Aerospace, Aviation, Airport, Computer Engineering, Software Engineering, Data Centre, Warehousing, Cloud, Engineering, Information technology, Information Technology, IT, Systems, Software, Telecom, Programming, Development
Vaudreuil-Dorion, QC

Java software engineer with GCP experience and DataFlow SME.

Location : Canada
Job Type : Temp/Contract
Hours : Full Time
Required Years of Experience : 4+
Required Education : Bachelor degree (BS in Computer Science) or similar
Travel : No
Relocation : No

Job Description :
GAL AeroStaff, which provides specialized staffing and recruiting services in the Business & Commercial aviation industry, is presently seeking a Java software engineer with GCP experience and DataFlow SME. based out in Canada. GAL AeroStaff is an internationally approved OEM service provider/supplier. We offer competitive salaries, as well as a professional and ethical work environment



Responsabilities




  • Design and develop data processing pipelines using Java/Apache Beam and Cdap and deploying into Google cloud in Google GCP Dataflow.

  • Developed various custom Cdap plugins in Java for transformations. Developed data pipelines monitoring system using Cdap Rest APi & Stackdriver API, Alerts from Java application.

  • Design and develop a solution to keep data in sync from on-premises Hadoop cluster to GCP cloud. A data change in Mapr clusters copies the data in near real-time to GCP and triggers a job in GCP.

  • Migrating Hive based data transformation pipelines from on-premises Mapr Hadoop clusters to Google GCP Dataproc clusters and converting it to Spark pipelines using Pyspark

  • Developed POC using various technologies Apache Beam/Spark/Cdap/DataFusion based data pipelines on GCP

  • Explore cloud-based new technologies/products for Marketing Analytics and Data Science team and provide data engineering consultancy


Required Qualifications :




Apply now!

Jobs.ca network

#