Lead Software Engineer in Test – Agentic AI (Remote - US)
Company shown to members 🔒 · Remote · Location not stated
🔒 See the company and apply✨ Tailor my resume for this job🎤 Practise the interview question, free
✓ Straight from the company's own careers page. No middlemen, no “we'll send you a check for equipment”. How we check jobs
✓ Sent 100 applications and heard nothing? Make this one count: tailor your resume to this ad in one tap, free.
✨ Not sure this is the one? Get your remote job matches in 30 seconds →
First recorded by Psst: 9 November 2026. The company's name and the apply link are for Psst members, from $4.99 for 7 days.
Practise the first interview question for this job
If the company replies, the first call usually starts with a question like this. Vague answers are the easiest to reject. Write yours and get an AI score in under a minute.
The 20-second read
- Experience
- 8+ years required
- Where you can live
- Location not stated
- Skills and tools
- sql, javascript, typescript, java, aws, gcp
- Degree or licence
- Mentioned (bachelor)
- Job type
- Full-time
Read from the ad by Psst's rules. Check the full ad below before applying.
Job details
- Where you can work
- Location not stated
- Employment type
- Full-time
- Advertised pay
- Not stated by the employer
- Experience
- 8+ years required
- Employer publication date
- 25 June 2026
- First recorded by Psst
- 9 November 2026
- Last checked at the source
- 10 November 2026
About the job
Why join the company?
We are a leader in specialty healthcare technology solutions.
We’re committed to hiring and retaining talent, which is why we invest in our employees through competitive pay, a generous bonus structure, great healthcare, a comprehensive wellness program, and many other benefits.
If you are a software engineer, finance or accounting professional, customer support specialist, or a business development expert with a passion for healthcare technology (just to name a few), we want to hear from you.
We are an equal opportunity employer with a commitment to diversity. All individuals, regardless of personal characteristics are encouraged to apply. If you are a candidate in need of assistance or an accommodation in the application process, please contact [link for members]
Essential Functions
•
In addition to adhering to the company’s conduct and behavioral standards, responsibilities include:
- Design, develop, and maintain scalable automation frameworks and regression test suites across UI, API, backend, and AI-driven applications.
- Lead quality engineering initiatives for Agentic AI systems, including LLM workflows, RAG pipelines, autonomous agents, and distributed orchestration platforms.
- Validate complex AI agent behaviors such as multi-agent coordination, tool calling, prompt chaining, memory retention, conversational workflows, and fallback/retry mechanisms.
- Develop and execute functional, integration, regression, end-to-end, and performance testing strategies across cloud-native and distributed environments.
- Collaborate with Software Engineers, Product Managers, and DevOps teams to integrate testing throughout the software development lifecycle.
- Build reusable automation solutions to validate APIs, orchestration layers, asynchronous workflows, enterprise integrations, and AI-driven business processes.
- Design and maintain robust test environments, CI/CD integrations, cloud execution platforms, and automation pipelines to improve efficiency and reliability.
- Establish and drive best practices related to code quality, maintainability, scalability, dependency injection, observability, and testability.
- Lead validation efforts for AI workflow resiliency, distributed system reliability, data consistency, and orchestration performance under failure and edge-case conditions.
- Mentor QA engineers and contribute to the development of scalable QA standards, automation strategies, reporting practices, and continuous improvement initiatives.
Minimum Requirements
•
Bachelor’s degree in Computer Science, MIS, Engineering, or a related technical field.
•
8+ years of experience in Software Development or Software Engineer in Test (SDET/SET) roles.5+ years of hands-on experience developing automation frameworks using JavaScript/TypeScript or Java.
•
Strong experience with automation tools such as Playwright, Cypress, Postman, K6, or equivalent frameworks.2+ years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows.
•
Hands-on experience validating agent orchestration workflows, conversational AI applications, multi-step reasoning systems, and AI-driven business processes.
•
Experience testing AI orchestration frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar technologies.
•
Strong expertise in API testing, backend integration testing, UI automation, and end-to-end workflow validation.
•
Solid understanding of automation architecture, design patterns, anti-patterns, dependency injection, and inversion of control principles.
•
Experience working in cloud-native and containerized environments (Azure, AWS, GCP, Docker, Kubernetes).Proven experience integrating automated test suites into CI/CD pipelines and distributed cloud environments.
•
Experience validating asynchronous, event-driven, and distributed systems with a focus on resiliency and fault tolerance.
•
Strong analytical, debugging, troubleshooting, and problem-solving skills.
•
Excellent organizational, communication, and stakeholder collaboration skills in Agile environments.
•
Experience with AI observability, telemetry, RAG pipeline validation, vector databases, or prompt evaluation frameworks (preferred).
Preferred Qualifications
•
Experience with Azure DevOps, GitHub Actions, or similar CI/CD platforms.
•
Experience with AI observability tools, telemetry platforms, and prompt evaluation frameworks.Experience validating RAG pipelines, vector databases, and AI response accuracy.
•
Knowledge of healthcare systems, HIPAA compliance, and regulated enterprise environments.
•
Experience with event-driven architectures, messaging systems, and distributed data platforms (including SQL Server).
•
Prior experience leading QA teams, mentoring engineers, and implementing enterprise test reporting and quality metrics practices.
Work Environment/Physical Demands
- Remote work
- Activities require a significant amount of work in front of a computer monitor.
Total Rewards
Generous annual bonus opportunity
401(k) with Employer Match
Flexible Time Off: take time off when you need it without worrying about available hours
11 paid holidays
Your Day Your Way - Celebrate a day of cultural or social significance to you
Access to Linkedin Learning courses for professional development
Insurance: Choice of Medical, Dental, and Vision plans
Health Savings Account with employer match
Flexible Spending Account
100% Company-Paid Parental leave (After 6 months with the company)
Company-Paid Life Insurance and Short/Long Term Disability Insurance
The company Luminary Peer Recognition Program
Employee Assistance Program with free counseling sessions available
Corporate Discounts on Retail, Travel, and Entertainment
🔒 See the company and apply✨ Tailor my resume for this jobPrepare with the Apply kitMore remote software engineer jobs →
- Open 5 months
- No pay listed
A resume that gets past the first 6 seconds
Recruiters skim. The Psst resume coach fixes what makes them skip you, line by line, and one tap tailors it to the exact job you're applying to.
- 1Every line checkedWeak starts, clichés and AI-sounding phrases flagged, with a 3-question fix.
- 2Tailored to the jobSee which skills and tools from the ad your resume is missing. Try it free on one of your matches.
- 3Ready for remoteTime zone, tools and proof you work on your own, the things remote teams look for.
- 4Looks the partOne page, 6 designs, 8 colours and a matching cover letter, free to build. Premium downloads the PDF and Word files.
Example with made-up details. The score is the builder's own check, not an ATS rating or a promise of an interview.