Key Responsibilities: ------------------------- * Key Responsibilities * AI Assurance Architecture * Architect platforms and frameworks for AI assurance evaluation and benchmarking * Design systems for LLM agent and RAG evaluation across functional non functional and risk dimensions * Define architectural patterns for Responsible AI bias detection explainability and safety validation * Build reusable assurance components supporting Business Assurance Risk Assurance and Reliability * Security Reliability Governance * Architect AI testing and validation for security privacy prompt injection and adversarial robustness * Integrate red teaming threat simulation and chaos style validation for AI systems * Define governance mechanisms for model usage auditability traceability and compliance * Ensure AI systems meet enterprise standards for resilience fault tolerance and observability * Platform Engineering Enablement * Design AI assurance platforms supporting automated test execution reporting and insights * Enable integration with CI CD pipelines to enforce AI quality gates * Collaborate with QE engineering teams to embed AI assurance into the SDLC * Mentor teams on AI risk identification and mitigation from an engineering perspective * Core Platforms Frameworks Tooling * LLM and AI evaluation frameworks PromptFoo DeepEval custom LLM evaluation harnesses * Prompt RAG and agent validation tooling prompt testing frameworks retrieval accuracy validators agent workflow evaluators * Responsible AI and model risk tooling Fairlearn SHAP Explainable AI libraries toxicity and bias scanners * Security and adversarial testing tools for AI systems PyRIT Garak * AI red teaming and threat simulation frameworks automated red team scripts adversarial test suites for LLMs and agents * AI assurance automation and QE frameworks Galileo * Observability for AI behavior and drift Langfuse Arize Evidently custom telemetry dashboards * Client Orientation Leadership * Partner with product and engineering teams to identify AI Assurance opportunities and shape roadmaps * Support client workshops RFPs and solution presentations * Mentor engineers on AI ML Gen AI best practices and emerging technologies * Translate complex AI concepts into business friendly narratives Technical Requirements: --------------------------- * Must Have Qualifications * 13 years of experience in software engineering with 3 years in AI with strong architecture ownership * Hands on expertise in AI ML systems LLM evaluation and assurance frameworks * Experience with AI red teaming model risk management or AI audit tooling * Strong understanding of Responsible AI AI risks and governance principles * Experience with security testing adversarial testing and reliability engineering * Proficiency in Python automation frameworks and cloud platforms * Good to Have Skills * Knowledge of regulatory or compliance considerations for AI systems * Exposure to performance engineering chaos engineering or resilience testing for AI * Contributions to internal platforms frameworks or standards Preferred Skills: --------------------- Domain->Strategy->Communication Design->Design strategy for the respective domain,Domain->Information Security->Governance Risk and Compliance, Audits , Workflow,Foundational->Governance_COBIT->Assurance,Technology->AI-Generative AI->Conversational AI Platform,Technology->AI-Generative AI->Generative AI - Basic,Technology->AI-Generative AI->Prompt Engineering,Technology->AI-Responsible AI->Responsible AI