Big Data Infrastructure Consultant
Uni Systems, Italy
Job Description
At Uni Systems, we are working towards turning digital visions into reality. We are continuously growing and we are looking for a Full Stack Developer to join our UniQue Ispra team.
Our client in Ispra, Italy is one of the biggest science hubs in Europe.
It employs approximately 3.000 top-notch researchers, engineers and scientists eager to resolve challenges and provide knowledge in order to support EU policies in a variety of areas from agriculture and food security, to environment and climate change, as well as nuclear safety, security and growth.
At the moment, they are looking for a Big Data Infrastructure Consultant to support the data analysis services team within data analysis and infrastructure services in the data management activities. In this context the consultant shall contribute to the data access through APIs and micro-services.
What will you be bringing to the team?
- System integration for big data:
Apply best practices of distributed architectures in a container based infrastructure. In that context, the consultant shall contribute to the design, implementation and documentation of APIs and microservices. The micro-services also include data processing to which the expert shall contribute.
The expert might also be asked to assess and test the interoperability and portability of ICT solutions and methodologies in a big data environment.
Furthermore, the expert may be asked to develop a forward vision on the challenges of the 'Big Data era'. This will entail the surveying and monitoring of data analytics and reference development environments from existing open source data and software as well as the fostering of the adoption of open source and FAIR data analytics to secure transparency, reproducibility, and accountability while easing the path from new applications and service prototyping to operational services.
- Complex big data infrastructure:
To secure continued hands-on expertise and knowledge required for a sound technical evaluation of big data solutions, the consultant shall support the BDAP project team in further developing the services of the European's Institution Big Data Analytics Platform (BDAP) with focus on open source cloud native technologies while taking into account the applicable security requirements.
- Systematic data download workflows:
The specialist shall work on the definition an implementation of systematic data download workflows for the European Institution's Big Data Analytics Platform (BDAP).
The workflows should include various tasks performed by micro-services: data download, data validation, data processing, metadata creation and storage. The specialist shall define and implement the various microservices, containerize them, integrate and orchestrate them through a workflow engine (Argo Workflows).
The consultant shall also contribute to the drafting of technical reports with the required level of confidentiality.
- Documentation of work:
Besides the production of technical reports regarding the shaping, implementation and evaluation of systematic data download workflows, the consultant shall document procedures, routines, scripts, configurations, and results in the data analysis and infrastructure services' project wiki and code repositories. All developments and deployments should be conducted in a programmatic way to secure that the different environments can be replicated and evolved in a controlled way.
Requirements
What do you need to succeed in this position?
- Master Degree + 6 years of experience
- Technical knowledge about cloud platforms, ideally regarding geospatial data
- Hands-on experience with cloud platforms and open source software for cloud native technologies, e.g. Docker and Kubernetes.
- Experience in design and implementation of micro-services and REST APIs.
- Experience in workflows definition and implementation.
- Experience with Linux environments, including scripting and automated system/service deployment.
- Experience with at least one data intensive application domain.
- Very good knowledge of English.
At Uni Systems, we are providing equal employment opportunities and banning any form of discrimination on grounds of gender, religion, race, color, nationality, disability, social class, political beliefs, age, marital status, sexual orientation or any other characteristics. Take a look atour Diversity, Equality amp; Inclusion Policyfor more information.