Software Engineer in Test

Job description

What you’ll doTest Infrastructure & Automation• Design, build, and maintain scalable test automation frameworks for API, UI, and end-to-end testing across Xometry’s suite of products• Integrate automated test suites into CI/CD pipelines to enable fast, reliable feedbackloops for development teams• Build and maintain test observability tooling including dashboards, flakiness tracking,and failure triage automation• Partner with engineers to define testability requirements and embed quality practicesearly in the software development lifecycle Agentic Test Authoring• Design and maintain agent-readable context files (CLAUDE.md, AGENTS.md, etc.) thatGive AI coding agents the knowledge they need to write correct, idiomatic E2E tests• Write reusable agent skills and rules that encode your team’s testing conventions, pageobject patterns, assertion strategies, and fixture management - so agents produce teststhat pass code review• Build and maintain agent workflows (e.g., Claude Code agents, Cursor rules) that canTake a feature description or ticket and produce a working E2E test suite with minimalhuman intervention• Define skills, test scaffolding, and guardrails that constrain agent output to yourframework conventions and prevent common failure modes (flaky selectors, missingteardown, improper auth state)• Evaluate and iterate on agent-generated test quality - tracking pass rates, flakiness, andmaintainability of agent-authored tests vs. human-authored tests• Collaborate with engineers to identify high-value test coverage gaps that can be closedthrough agentic generation, and build the pipelines to close them continuously• Stay current on the agentic coding tooling landscape (Claude Code, Cursor, CopilotWorkspace, etc.) and bring a point of view on what belongs in our stack What you’ll bringRequired• 4+ years of software engineering experience with a test infrastructure or automationfocus• Proficiency in one or more of: Python, Java, TypeScript, or Go• Strong experience building and maintaining UI automation frameworks (Playwright,Cypress, or Selenium)• Solid API testing experience (REST, GraphQL)• Experience integrating tests into CI/CD systems (GitHub Actions, Jenkins, CircleCI, etc.)• Familiarity with containerization and test environment orchestration (Docker,Kubernetes)• Strong software engineering fundamentals - you write test code the same way you’dwrite production code• Hands-on experience with AI coding agents (Claude Code, Cursor, Copilot, or similar) -you use these tools daily and understand their capabilities and failure modes• Experience writing structured context files, rules, or instructions that shape agentbehavior - you know the difference between an agent that writes a test and an agent that writes a good test Nice to have• Experience with performance or load testing (k6, Locust, Gatling)• Exposure to observability tooling (Datadog, Grafana, PagerDuty, Coralogix)• Familiarity with chaos engineering or fault injection testing• Experience in marketplace, e-commerce, or B2B SaaS environments• Background building self-healing or AI-assisted test tooling• Experience scaling test infrastructure across multiple teams or product areas• Contributions to open-source testing tools or agent frameworks