Workster is partnering with a leading global customer experience company to recruit Polish-speaking AI Data Evaluation Specialists for an innovative artificial intelligence project based in Lisbon, Portugal.

If you’re passionate about AI, language technology, and data quality, this is a unique opportunity to contribute to the development of next-generation Large Language Models (LLMs). You’ll work with cutting-edge AI systems by reviewing, evaluating, and improving AI-generated responses, helping create safer, more accurate, and more reliable conversational experiences for millions of users worldwide.

Your Role

  • Review and evaluate AI-generated responses using detailed annotation and quality guidelines
  • Label and annotate language model outputs with accuracy and consistency
  • Assess AI-generated content for quality, relevance, factual accuracy, safety, and policy compliance
  • Apply complex annotation guidelines across a wide variety of scenarios and content types
  • Identify inconsistencies, edge cases, and potential model issues, escalating when appropriate
  • Analyze structured text, datasets, and AI-generated content to support model improvement
  • Deliver high-quality work while consistently meeting productivity, quality, and accuracy targets
  • Maintain strict confidentiality and comply with all project standards, security requirements, and internal procedures
  • Contribute directly to the ongoing training and improvement of advanced AI systems through detailed evaluations and feedback
  • 24/7 rotating schedule (Monday to Sunday) with 2 rotational days off

Your Qualifications

  • Native-level fluency in Polish
  • Good command of English (B2 level or above)
  • Strong understanding of Generative AI, Large Language Models (LLMs), and AI evaluation methodologies
  • Previous experience in AI data annotation, LLM evaluation, Reinforcement Learning from Human Feedback (RLHF), Supervised Fine-Tuning (SFT), Quality Assurance, Content Labeling, Trust & Safety, or similar analytical roles is highly desirable
  • Excellent analytical thinking and attention to detail
  • Ability to interpret and consistently apply complex written guidelines and policies
  • Strong reading comprehension and written communication skills
  • Comfortable working with structured text, datasets, and AI-generated content
  • Ability to work efficiently in a fast-paced, quality-driven environment while meeting performance targets
  • Professional, objective, and capable of handling confidential information with discretion

The Offer

  • Office-based position in Lisbon, Portugal
  • Comprehensive onboarding and specialized training in AI evaluation processes
  • Opportunity to work on one of the world’s most advanced artificial intelligence projects
  • International and multicultural working environment
  • Career development opportunities within the rapidly growing AI industry
  • Private health insurance
  • Meal allowance
  • Ongoing learning and professional development
  • Modern office facilities with a collaborative team culture
  • Unique opportunity to shape the future of generative AI and language technologies