Databricks Engineer Job at United Software Group Inc, Canada

TXMzR0c5eXBJcGw1Ymk2RnJXbGdwWjlTS0E9PQ==
  • United Software Group Inc
  • Canada

Job Description

Job Title: Databricks/PySpark Consultant

Location : Halifax, NX (100% Permanent remote work accepted from anywhere in US; however we’d prefer EST and CST based resources)

Duration : 12+ Months Contract

Teams Meeting Interview

Job Description:

Note : Customer is looking for senior candidates. They actually need someone having architectural experience,

Key Responsibilities

  • Design, develop, and optimize data pipelines and ETL/ELT processes using Azure Databricks and PySpark.
  • Integrate and orchestrate data workflows using Azure Data Factory (ADF).
  • Implement data modeling and analytics solutions with Azure Synapse Analytics.
  • Collaborate with data architects and analysts to ensure alignment with business requirements.
  • Optimize performance of Spark jobs and data pipelines for large-scale data processing.
  • Ensure best practices for data governance, security, and compliance within the Azure ecosystem.
  • Troubleshoot and resolve issues related to data ingestion, transformation, and integration.

Required Skills & Qualifications

  • 8+ years of Proven hands-on experience with Azure Databricks and PySpark for big data processing.
  • 5+ years of experience of Azure Data Factory (ADF) for pipeline orchestration and integration.
  • 3+ years of Experience with Azure Synapse Analytics for data warehousing and advanced analytics.
  • 5+ years of experience in Python and SQL for data manipulation and transformation.
  • 8+ years of exposure to data modeling, ETL processes, and performance tuning.
  • 3+ years of experience working with CI/CD, Git, and DevOps practices in a data engineering context.
  • 5+ years of experience in problem-solving, communication, and collaboration skills.

Job Tags

Permanent employment, Contract work, Remote work,

Similar Jobs

UPMC

Neonatal Nurse Practitioner or Neonatal Physician Assistant - UPMC Altoona Job at UPMC

 ...Advanced Practice Provider Neonatal Care Team at UPMC Altoona!University of Pittsburgh Physicians are seeking a Full-Time Neonatal Nurse Practitioner or Neonatal Physician Assistant to join our Pediatric Neonatology team at UPMC Altoona! This position will support the eight-... 

Stability Healthcare

Travel Operating Room RN (Circulator & First Assistant) Job at Stability Healthcare

 ...weeks ~40 hours per week ~ Shift: 8 hours, evenings ~ Employment Type: Travel Stability Healthcare is looking for a OR RN contract position in San Antonio, TX. Also known as perioperative nurses, operating room (OR) nurses provide care for patients before,... 

CNO Financial Group

Senior Director, Actuary, ALM - REMOTE Job at CNO Financial Group

 ...salary is determined by factors including relevant work experience, skills and location. This position is bonus eligible. CNO Financial Group is hiring a Senior Managing Actuary ALM to work with 40|86, Actuarial and Finance to be responsible for the companys asset... 

Retail

Team Members, Toronto Eaton Centre Job at Retail

Description Reporting to the Store Manager the Team Member is accountable for delivering excellent community member service in our efforts to spread global wellness. Through day to day functions the Team Members contributes to maintaining and strengthening Sajes culture...

株式会社HIKE

社内インフラエンジニア Job at 株式会社HIKE

 ...20256HIKE HoldingsMBOPHD 5...