QA Engineer

About the company AI

The company AI is building a next-generation AI recruiting and talent intelligence platform that helps organizations turn hiring into a measurable, repeatable, and intelligent science. Our ARBI platform supports sourcing, screening, evaluation, sequencing, scheduling, recruiter workflow automation, and explainable candidate assessment for commercial enterprises, public-sector agencies, higher education, staffing organizations, and workforce-development teams.

The company is also building Athena, an AI-powered career readiness and candidate-assistance experience that helps users improve resumes, prepare for interviews, receive rubric-based feedback, and navigate the job search process with personalized AI support.

We are a fast-moving startup operating at the intersection of GenAI, HR technology, recruiting operations, career services, workflow automation, and enterprise AI deployment. As ARBI and Athena scale across customers, we need a senior quality leader who can make product quality measurable, automated, auditable, and trusted.

Role Overview

We are hiring a Senior QA Engineer / Quality Engineering Lead to own quality assurance, testing strategy, product validation, and release readiness across ARBI and Athena. This is a hands-on senior role for someone who can test deeply, build automated quality systems, validate AI workflows, improve developer quality practices, and become the central authority for product quality.

This role is not limited to executing test cases. You will audit the current QA landscape, define the testing strategy, build regression and automation frameworks, validate AI behavior, pressure-test edge cases, and create the release confidence needed for a rapidly evolving SaaS platform.

You will work closely with engineering, product, design, customer success, and founders to make sure new features, bug fixes, experiments, integrations, data workflows, and AI-driven experiences ship with clarity, reliability, auditability, accessibility, and user trust.

What You’ll Do (Key Responsibilities)

• Build the Quality Strategy and QA Operating System

• Perform a deep audit of the current QA setup across ARBI, Athena, frontend flows, backend services, APIs, data workflows, integrations, AI pipelines, and release processes.

• Define a company-wide QA strategy across short-term stabilization, mid-term automation, and long-term quality engineering maturity.

• Design a scalable test architecture using test pyramid principles, shift-left testing, smoke testing, regression testing, release gates, exploratory testing, and risk-based coverage.

• Define clear QA responsibilities between developers, QA, product, design, customer success, and release owners.

• Establish a practical quality operating rhythm: test plans, release checklists, defect triage, severity definitions, sign-off workflows, and quality metrics.

• Own Product QA for ARBI and Athena

• Validate implementation against requirements, designs, copy, acceptance criteria, user stories, and customer-specific workflows.

• Test UI, UX, business logic, responsiveness, edge cases, error states, empty states, loading states, accessibility, and validation messages.

• Perform exploratory, smoke, regression, and release-candidate testing before launches.

• Validate recruiter workflows including candidate search, matching, scoring, resume parsing, outreach sequencing, scheduling, recruiter dashboards, candidate profiles, and ATS/CRM-style workflows.

• Validate Athena workflows including resume support, interview preparation, rubric-based feedback, candidate assistance, AI-generated recommendations, and user-facing guidance.

• Build and Modernize Test Automation

• Take ownership of automated frontend, API, integration, and end-to-end test coverage using Cypress, Playwright, Pytest, Postman/Newman, or equivalent tools.

• Create reliable automated regression suites for critical ARBI and Athena workflows, including authentication, permissions, candidate pipelines, analytics, notifications, integrations, and admin experiences.

• Integrate tests deeply into CI/CD pipelines so failures are visible, actionable, and tied to release confidence.

• Improve test reliability, execution speed, data setup, fixture management, and maintainability.

• Introduce AI-assisted testing practices where useful, while maintaining clear human judgment and repeatable test evidence.

• Validate Backend, API, Data, and Workflow Reliability

• Test robust REST APIs, Python/FastAPI services, backend business logic, asynchronous workflows, and data-processing pipelines.

• Validate PostgreSQL, Redis, OpenSearch, Celery, Temporal, containerized services, deployment pipelines, and AWS-hosted environments from a QA perspective.

• Create API and integration test coverage for imports, exports, webhooks, permissions, search, scoring, candidate data, customer-specific configuration, and workflow automation.

• Test with realistic and large-scale datasets to uncover performance, latency, search relevance, data integrity, and resilience issues.

• Establish baseline performance, load, and reliability testing using JMeter, k6, Locust, or similar tools.

• QA AI, LLM, and Evaluation Workflows

• Validate AI-assisted recruiting workflows for accuracy, consistency, explainability, hallucination risk, bias risk, prompt adherence, rubric alignment, and human-in-the-loop behavior.

• Test AI scoring, candidate summaries, recommendations, interview feedback, resume analysis, and knowledge-retrieval experiences across normal, adversarial, and edge-case inputs.

• Create repeatable evaluation datasets and test harnesses to measure AI quality over time.

• Validate guardrails, fallback behavior, citations, confidence indicators, data boundaries, audit trails, and customer-specific configuration.

• Partner with product and engineering to define what “good” means for AI-generated outputs and how release readiness should be measured.

• Own Release Readiness and Quality Visibility

• Create clear, structured, reproducible bug reports with screenshots/videos, environment details, severity, expected behavior, actual behavior, impact, and reproduction steps.

• Retest fixed issues, validate root-cause resolution, and prevent regressions.

• Communicate QA status clearly before release: passed, failed, blocked, passed with known issues, or requires founder/product decision.

• Build dashboards and reporting for test coverage, defect trends, regression health, release risk, performance baselines, and customer-impacting quality issues.

• Help developers produce testable, high-quality code by establishing testing standards, review practices, and shared quality expectations.

What Success Looks Like (First 6–12 Months)

• A clear QA audit, quality strategy, and implementation roadmap are delivered and accepted by engineering and product leadership.

• A comprehensive ARBI and Athena test case portfolio exists across critical user journeys, backend services, data workflows, AI behavior, integrations, and release gates.

• Automated test coverage increases meaningfully across the highest-risk frontend, API, integration, and regression areas.

• CI/CD quality gates are established so test results directly inform release decisions.

• Baseline performance, load, security-oriented, and resilience testing practices are introduced for critical workflows.

• Release readiness becomes transparent, repeatable, and trusted by founders, engineering, product, customer success, and customers.

• You become the central authority for product quality and a mentor to developers on testing best practices.

Qualifications (Required)

• Experience in Quality Assurance, Quality Engineering, Software Test Engineering, or a related product-quality role.

• Experience testing SaaS web products, B2B platforms, workflow systems, data-heavy applications, or AI-enabled products.

• Strong manual QA skills: exploratory testing, edge-case discovery, cross-browser validation, responsive web testing, clear bug reporting, and release sign-off.

• Hands-on automated testing experience with Cypress, Playwright, Selenium, Pytest, Postman/Newman, RESTAssured, or similar tools.

• Strong backend engineering fluency, especially with Python, FastAPI, REST APIs, PostgreSQL, Redis, OpenSearch, Celery, Temporal, containers, AWS services, and deployment pipelines.

• Experience validating large-scale datasets, data pipelines, integrations, asynchronous workflows, search/indexing behavior, and workflow automation.

• Experience integrating automated tests into CI/CD pipelines and improving reliability, speed, observability, and developer feedback loops.

• Strong systems thinking, sound quality architecture judgment, and the ability to move from ambiguity to an executable QA roadmap.

• Ability to operate in startup-style environments with high ownership, speed, accountability, and mentoring responsibility.

• Ability and willingness to relocate to the Northern VA / Washington, DC area, or work closely with the team during Eastern Time business hours if remote.

Preferred Qualifications

• Prior experience as a Staff QA Engineer, Lead QA Engineer, Quality Engineering Lead, SDET Lead, Founding QA Engineer, or QA Architect.

• Experience testing AI/LLM products, evaluation workflows, RAG systems, AI copilots, agentic workflows, or human-in-the-loop decision systems.

• Experience in HR tech, recruiting platforms, ATS/CRM systems, career services, assessment platforms, or workforce development products.

• Experience with A/B testing, feature flags, analytics validation, event tracking, access control, subscription/billing flows, or customer-specific configurations.

• Experience with performance and load testing tools such as JMeter, k6, Locust, or similar platforms.

• Familiarity with security, privacy, accessibility, SOC 2-style controls, federal deployment expectations, or auditability requirements.

• Comfort working directly with founders, customers, customer success, and implementation teams to translate real-world feedback into quality improvements.

Current / Target Tech Environment

• Frontend: ReactJS / modern web application interfaces.

• Backend: Python, FastAPI, REST APIs, PostgreSQL, Redis, OpenSearch, Celery, Temporal, and asynchronous workflow services.

• Infrastructure: AWS, Docker, containers, CI/CD pipelines, observability tooling, and production SaaS deployment practices.

• Testing: Cypress / Playwright-style E2E testing, API testing, integration testing, regression suites, performance testing, and AI-assisted QA workflows.

Compensation & Benefits

• Base Salary: Competitive and commensurate with experience, seniority, location, and ability to own quality across the product.

• Performance / Equity: Potential performance incentives and/or equity participation based on final offer structure.

• Benefits: Medical, dental, vision, PTO, and company-supported professional growth.

• Learning: Support for relevant courses, conferences, certifications, technical books, and quality engineering communities.

• Flexibility: Remote-first / hybrid flexibility with strong preference for Northern VA / DC-area alignment and occasional in-person collaboration.

Equal Opportunity

The company is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.