Project description:
Integrating a second-layer SCADA system for renewable assets into the Databricks environment. You will ensure data integrity and reliability by designing and executing automated checks on incoming data streams and recording results in Azure DevOps.
Responsibilities:
- Develop and execute automated data and integration tests in Databricks (Python, PySpark, SQL)
- Conduct end-to-end pipeline testing from SCADA through AMQP into Databricks, including verification of transformation logic and compliance with defined business rules
- Capture, store, and prepare test results using Azure services (Data Lake, Blob Storage, SQL)
- Integrate automated tests into CI/CD pipelines with Azure DevOps, collaborating closely with DevOps and Cloud teams to ensure continuous validation and monitoring
- Work closely with Data Engineers, SCADA specialists, and Cloud architects, ensuring full doc
Technologies:
- Python, PySpark/SQL
- Databricks
- Azure Services for storing and prepare results
- Azure DevOps
- SCADA
What we offer
Growth
- Technical certification support (training and conference budget)
- Clear career path
- On-the-job trainings, workshops and conferences
- Challenging global projects
Innovation
- Dynamic, result-focused work environment
- Incentive to act independently
- Challenging global projects for corporate customers
Wellness
- Private health insurance for you and for your family
- Multisport membership cards for you and for your family
- Sports-friendly approach with company-sponsored running and football events
- Height adjustable desk and a comfortable chair
Atmosphere
- Celebrating of work anniversaries, birthdays and employee awards
- Kudos – way to thank you for friendly help
- Team hangouts – we go out together for board games, wine tasting or to escape room
- Flat structure in the company = high decision-making (but also responsibility)
