Software Engineer, LLM Evaluation – English
Role description
Software Engineer, LLM Evaluation – English
Job Specs
Category: Software Engineering/Development, Artificial Intelligence
Region: Netherlands: Remote
Depends on experience
Hiring type: Contracting
Duration: 1 year with view to extension
ASAP
Job ref.
Description
Our client’s mission is to empower people, build community, and bring the world closer together. Through their apps and services, they are building a different kind of company that connects people worldwide and provides meaningful ways to share what matters most.
We are looking for a Software Engineer, LLM Evaluation, to join their team in the Netherlands remotely.
Job Profile: The successful candidate will work closely with researchers and engineers to evaluate model performance, conduct data‑driven analyses, and contribute to research initiatives related to LLM pretraining and evaluation.
Responsibilities
- Analyse and evaluate large language models and their performance across various tasks and benchmarks
- Execute benchmark evaluations and generate performance metrics and insights
- Conduct quantitative and qualitative data analysis to support research objectives
- Contribute to the design, implementation, and validation of new evaluation methodologies
- Support research initiatives related to LLM pretraining and model evaluation
- Develop and maintain machine learning and deep learning systems and tooling
- Build research workflows and experimental frameworks using Python and PyTorch
- Enable rapid experimentation and support the execution of research initiatives
- Collaborate with researchers and engineers within a multidisciplinary team environment
Qualifications
- Must be fluent in English, both written and spoken
- Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, or a related technical discipline. Master’s degree in Computer Science, Machine Learning, AI, or a related discipline is desirable; a PhD in a relevant field would be considered a strong advantage
- 3–5 years of experience working with large language models, including pretraining and evaluation
- Strong hands‑on programming experience in Python
- Proven experience building and maintaining ML/DL systems and research infrastructure
- Experience developing machine learning and deep learning solutions using PyTorch
- Publications in machine learning, natural language processing, artificial intelligence, or related fields are a plus
- Experience working with transformer architectures, large language models, and/or multimodal models is an advantage
- Ability to write clean, efficient, and production-quality code
- Strong interest in model evaluation, experimentation, and data analysis
- Demonstrated scientific curiosity and problem‑solving skills
Benefits
- 25 holidays per annum
- Pension plan
- Opportunity to work alongside experienced researchers and engineers on cutting‑edge AI initiatives
- Exposure to state‑of‑the‑art methodologies and technologies within the LLM ecosystem
- Collaborative and research‑driven environment in a technologically advanced office
To learn more about the role, please visit our website.
#J-18808-Ljbffr
