Evaluation Engineer, Part-Time

You’ll build the test sets and evaluation code that measure how accurately our automated checks, and our expert reviewers, reach their decisions.

What you’ll do

• Build and maintain test sets with answer keys that measure how accurately our automated checks, and our expert reviewers, reach their decisions.
• Write evaluation code in Python.
• Compute agreement statistics (Cohen’s kappa, Krippendorff’s alpha) and error analyses, and write short, clear reports on what went wrong and why.

Your code is reviewed by our CTO. You work in a scoped environment, with access limited to what each assignment needs.

You have

• Current enrollment in, or recent completion of, a master’s or PhD program in computer science, statistics, data science or a related field.
• Strong Python and testing discipline.
• Working knowledge of inter-rater agreement and basic statistics.
• Clear technical writing.

Pay

• $60–$95 per hour (W-2).
• $60–$75 for master’s students; $75–$95 for PhD candidates and PhD holders.
• Pay is hourly, for all time worked. It is never per piece.
• Quality bonus: an extra 15% of your pay for an assignment’s hours when your accuracy on its answer-keyed check items is at least 95% and none of your decisions on it is reversed in our second review. It is paid with your regular pay on the next payday.
• Hard-to-fill assignments may carry a premium of up to 25%, stated in the offer before you accept it.
• The range is the pay we reasonably expect to offer. Everyone at the same level starts at the same rate.
• Benefits: paid sick leave as required by law; workers’ compensation; reimbursement of necessary remote-work expenses. Any other benefits are stated in the offer.

How the work is offered

• Work is offered per assignment when customer projects are funded; there are no guaranteed hours; you choose which assignments to accept. When there is no assignment, there is no work and no pay obligation.
• Each offer states the task, an estimated time, the deadline and your hourly rate. Accept or decline it with no penalty.
• You are paid for every hour you work on an accepted assignment, including any time beyond the estimate, with a minimum of one paid hour per accepted assignment.
• Training, the qualification assessment and quality-check tasks are paid at your rate.
• Part-time W-2 employment, not gig or 1099 work: your employer is I Am Violet Inc. (dba Metis Foundry), and you are paid through our payroll provider. Variable hours, with no on-call duty and no scheduled shifts.
• Where we are: we are building our expert pool now. Onboarding and the paid qualification assessment begin when the first customer projects are funded, and we will tell you honestly when that is.

Location and eligibility

• Must reside in and perform all work from within the United States; U.S. work authorization required.
• You must be authorized to work in the United States for the full period of each assignment. We do not sponsor visas.
• F-1 students may apply only with CPT or OPT authorization that covers this work. Please check with your international student office first.
• If you hold a university appointment, fellowship or assistantship, you are responsible for confirming that it allows outside paid work. Metis work must not use university time, equipment or confidential information.
• A background check is run only for roles or assignments that need one, after a conditional offer and with your written authorization.

Export-controlled assignments (none offered today). Some future assignments may involve export-controlled (ITAR/EAR) technical data. Those assignments will be offered only to U.S. persons as defined in 22 CFR 120.62: U.S. citizens and nationals, lawful permanent residents, and people granted refugee or asylum status. The reason is that Metis does not hold export licences to share such data with anyone else. We check this only for those assignments, separately from employment-eligibility verification. All other assignments are open to everyone who meets the requirements above.

How we verify

• We check your degree or enrollment, and your professional credentials, with the bodies that issue them. We do this only with your written consent.
• We verify your identity.
• You take a paid qualification assessment in your field; the passing score is 95%.
• All work happens inside the secure Metis portal: no downloads, no personal AI tools on customer material, and no direct contact with customers.
• Your decisions are delivered under a Metis reviewer ID together with your verified credentials. Your name is shown only with your written consent.