All openings
Consulting Delivery·Full-time·3–6 years
AI Quality, Governance & Security Analyst
Help clients use AI responsibly by building quality checks, governance controls, evidence trails, and security-aware operating practices around production AI workflows.
Remote — Global Full-time Competitive — based on experience
About the role
Kresto Consulting deploys AI systems into operational workflows where accuracy, auditability, and trust matter. This role exists to ensure those systems are not merely functional; they are controlled, explainable enough for their use case, and safe to operate in real client environments.
As an AI Quality, Governance & Security Analyst, you will define evaluation methods, test AI outputs, identify failure modes, document controls, and work with delivery teams to build practical human-review and escalation processes. You will help clients answer the hard but necessary questions: What can the system do? Where can it fail? Who reviews it? What evidence is retained?
This is not a policy-only role. You will work directly with solution, engineering, and implementation teams to make governance operational rather than theoretical.
What you will do
- Create test plans, evaluation rubrics, and acceptance criteria for AI-enabled workflows
- Assess output quality, accuracy, consistency, traceability, and failure modes before client release
- Define human-review, escalation, logging, retention, and access-control requirements
- Document AI system purpose, data flows, known limitations, controls, and operating guidance
- Support security and privacy questionnaires, client due diligence, and governance discussions
- Conduct periodic quality checks and recommend corrective actions based on observed performance
- Partner with engineers to improve prompts, evaluations, guardrails, and monitoring
- Build reusable governance templates and quality standards across Kresto engagements
What we are looking for
- 3+ years in AI quality assurance, information security, data governance, risk, compliance, technical audit, or a related discipline
- Strong analytical ability and comfort working with structured test cases and evidence
- Understanding of LLM risks including hallucination, prompt injection, privacy, and human oversight
- Familiarity with data protection, access control, audit logging, or enterprise security principles
- Ability to write clear, concise documentation for technical and non-technical audiences
- Pragmatic mindset: able to design controls that fit operational reality rather than create bureaucracy
- Fluent written and spoken English is mandatory
- Ability to work independently in a remote global team
Nice to have
- —Experience with ISO 27001, SOC 2, NIST AI RMF, or comparable frameworks
- —Experience in regulated or project-critical industries
- —Hands-on testing of LLM applications, retrieval systems, or automated decision-support workflows
- —Privacy, security, or internal-audit certification
- —Experience supporting enterprise vendor risk assessments
Role summary
Department
Consulting Delivery
Location
Remote — Global
Type
Full-time
Experience
3–6 years
Compensation
Competitive — based on experience