Clinical AI Evaluation Specialist

Job Description We are seeking licensed, experienced clinical professionals to serve as Clinical AI Evaluation Specialists. In this fully remote, highly flexible role, you will apply your deep hands-on clinical background to evaluate, test, and refine advanced artificial intelligence models designed for healthcare applications. This position involves zero direct patient care, shift work, or on-call duties. Key Responsibilities AI Model Evaluation & Rubric Creation: Test prompts on active AI models, analyze output accuracy, and build detailed grading rubrics to evaluate model performance against complex healthcare standards. Domain Validation: Ensure AI models safely and accurately handle domain-specific clinical terminology, documentation rules, compliance standards, and real-world workflow logic. Clinical Documentation Analysis: Review and critique clinical documentation at an individual contributor practitioner level, grounding decisions in current clinical guidelines and patient context. Independent Execution: Work self-sufficiently using provided AI tools, detailed project instructions, and analytical support resources. Qualifications & Requirements Licensure: Active or recent clinical license as a Registered Nurse (RN), Nurse Practitioner (NP), Physician Assistant (PA), Medical Doctor (MD), Doctor of Osteopathic Medicine (DO), Pharmacist (PharmD), or an allied health license with independent clinical judgment (e.g., Respiratory Therapy, Physical Therapy, Midwifery). Clinical Experience: 5+ years of hands-on, direct clinical practice at an individual contributor level (recent or current experience preferred; managerial-only experience does not qualify). Education: Bachelor's, Master's, or PhD strongly preferred (or equivalent advanced practical domain experience). Work Ethic & Autonomy: Strong attention to detail, stamina for deep analytical work, and the ability to operate independently without direct supervision. Technical Setup: Personal, reliable laptop/computer equipped with high-speed internet. Location: Must reside in the United States, Canada, or United Kingdom. Work Schedule & Modality Modality: 100% Fully Remote. Schedule: Flexible hours and days. Commitment: Minimum of 5+ hours per week required (10+ hours per week strongly preferred). Perks & Highlights Complete schedule flexibility-work whenever and wherever fits your lifestyle. Direct involvement in cutting-edge healthcare AI development, serving as a standout addition to your resume. No patient care, no call, and no rigid shifts. For California Applicants: We will consider for employment all qualified Applicants, including those with criminal histories, in a manner consistent with the requirements of applicable federal, state and local laws, including the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance (FCIHO), Los Angeles Fair Chance Ordinance for Employers (ULAC), The San Francisco Fair Chance Ordinance (FCO) , and the California Fair Chance Act (CFCA). This position is subject to a background check based on its job duties, which may include patient care, working with vulnerable populations, access to financial and confidential information, driving, working with heavy machinery, or working in a warehouse or laboratory environment. Due to these job duties, this position has a significant impact on the business operations and reputation, as well as the safety and well-being of individuals who may be cared for as part of the job position or who may interact with staff or clients.