Kacper Wiśniewski
Senior LLM Engineer
Wrocław, Poland · UTC+1 · 11 years of experience
Kacper focuses on measurable model behavior, reliable retrieval, and production evaluation.
Request a shortlistWhy this profile fits
Evaluation before optimization
Defines datasets, graders, and regression checks so model changes are measured against real failure cases.
Reliable retrieval and tool use
Improves grounding, structured outputs, tool permissions, and recovery behavior for production workflows.
Cost and latency control
Treats task success, response time, and model spend as one production system rather than separate concerns.
Relevant toolkit
Python
LangGraph
Claude API
OpenAI GPT API
PostgreSQLDelivery evidence
Selected work
Eval Harness & Regression Gate
B2B SaaS support automationDesigned a golden-dataset eval harness with LLM-as-judge rubrics calibrated to human labels, wired into CI to block any prompt or model change that regressed quality. Cut hallucinated answers 58% and eliminated post-deploy rollbacks over two quarters.
RAG Evaluation Framework
Legal-tech document searchBuilt a retrieval evaluation framework with labeled query sets and automated recall, precision, and faithfulness scoring on every index change. Surfaced chunking and embedding regressions early and lifted answer relevance (NDCG@10) by 27%.
Relevant experience
Career timeline
Lead LLM Evaluation Engineer
2023–PresentConfidential AI Developer Platform (Series B) · Wrocław, Poland (Remote)
- Designed and owned the company-wide LLM evaluation platform on Braintrust and LangSmith, covering 30+ features and gating every prompt, model, and RAG change behind automated CI quality checks.
Senior Machine Learning Engineer
2020–2023B2B SaaS Search Scale-up · Wrocław, Poland
- Led migration of keyword search to a RAG system over pgvector, lifting answer relevance (NDCG@10) 27% and deflecting 45% of tier-1 support conversations.
Machine Learning / NLP Engineer
2017–2020Fintech Analytics Scale-up · Kraków, Poland
- Built NLP classification and entity-extraction pipelines for financial documents, reaching 94% F1 and automating a process that previously took analysts 12 hours per batch.
Hiring this role?
See how Kacper fits the LLM Engineer role.
Skill depth
Show experience in context.
Relevant experience by skill
See the complete skill index
Languages
Frameworks
Libraries/APIs
Tools
Paradigms
Platforms
Storage
Other
Education
Formal background
MSc, Computer Science (Machine Learning & NLP)
AGH University of Science and Technology, Kraków · 2013–2015 · Graduated with distinction
BSc, Computer Science
Wrocław University of Science and Technology · 2010–2013
Credentials
Certifications
Databricks Certified Generative AI Engineer Associate
Databricks · 2022 · Certified
Google Cloud Professional Machine Learning Engineer
Google Cloud · 2025 · Certified
Communication
Languages
Polish
Native
English
Fluent
German
Conversational
More examples
Similar LLM Engineer profiles.
Aravind Balakrishnan
Senior LLM Engineer
Senior LLM engineer, 13 years, specializing in RAG retrieval quality and eval-driven model behavior.
Henrique Vasconcelos
Senior LLM Engineer
Senior LLM engineer, 9 years, specializing in structured outputs and tool calling.
Tomás Ferreyra
Senior LLM Engineer
Senior LLM engineer specializing in model routing and cost control.