Senior/Lead Big Data Engineer_

Nr ref.: 8311/JP

Misją Connectis jest rozwój ludzi i firm poprzez budowanie zespołów Developerów dziś oraz inwestowanie w Liderów jutra. Connectis_ to ponad 20 lat doświadczenia w branży, 350 Specjalistów IT i tysiące zrealizowanych projektów.

Opis stanowiska

  • Working within a modern Azure Cloud data platform (particular focus on Azure Databricks / Spark and Azure Synapse (former Azure DWH))
  • Designing, building and supporting data processing pipelines built in PySpark & SparkSQL (process orchestration in Azure Data Factory or Apache Airflow)
  • Providing ongoing supporting and coaching to Data Engineering and Science teams in Spark area – promoting best practices, performance tuning & optimization (Databricks clusters sizing, PySpark/SparkSQL performance)

Wymagania

  • Years (5+) of professional experience in data engineering and/or data warehousing
  • Understanding of Relational Databases, Data Warehouse modeling techniques, ETL/ELT processing and Big Data technologies
  • Practical experience in building data pipelines on large data sets Hands-on experience in both Spark and SQL environments (Databricks and Azure DWH/Synapse/MS SQL Server or close equivalents)
  • Experience in Linux environment, bash scripting and data processing command line tools such as grep/awk/sed, regular expressions etc.
  • Experience working in agile methodologies (SCRUM) Git version control Agile mindset and willingness to learn new tools
  • Working knowledge of Python, PySpark & SQL (Scala would be a plus)
  • Fluent English (all project communication in English)

 

 

Nice to have:

  • Past experience with Spark Structured streaming and related technologies (Azure Event Hub/Event Grid/Kafka),experience with R and SparkR
  • Experience in managing Apache Airflow, experience with CI/CD tools (preferably Azure DevOps)

Oferujemy

  • Remote work
  • Many interesting technology meetups
  • Individual care on the Connectis_ side
  • Medical package and sports card

Benefity

Pakiet medyczny
Pakiet sport
Komputer
Konferencje
Wynagrodzenie do negocjacji
Lokalizacja Warszawa/zdalnie
Obszar
Start ASAP
Język angielski - zaawansowany
Forma zatrudnienia B2B
Potrzebujesz więcej informacji?

Codziennie od 9:00 do 17:00

czekamy na Twoje pytania 22 222 5000