Senior Software Engineer - AI Evaluation / Coding Agents (Remote) | $100–$150/Hour | Contractor

The remote Senior Software Engineer - AI Evaluation job is designed for experienced developers with at least five years of hands-on software engineering experience and strong code-review skills. The contractor will work with AI coding agents, assess their performance on real-world software projects, and provide technical feedback that helps improve how AI models understand and generate code.

PROJECT DETAILS

The project focuses on evaluating AI coding models using actual software repositories rather than building conventional production applications. Engineers will examine code generated by AI agents, check whether implementations solve the intended problems, identify technical weaknesses, and explain where an approach succeeds or falls short.

The work also involves developing evaluation criteria, reviewing data, and building the infrastructure needed to support AI training and assessment. Engineers will collaborate with researchers and other developers to turn their technical judgment into consistent feedback that can be used to improve coding models.

This is a fully remote independent contractor engagement available exclusively to candidates in North America. The project is expected to run for approximately three months, with an immediate start. Contractors should be available for 20–40 hours per week and maintain at least six hours of daily overlap with Pacific Time. Explore more remote WFA jobs for additional work-from-anywhere opportunities.

APPLY HERE

JOB INFORMATION

• Responsibilities: Evaluate AI-generated code across real-world repositories, design and refine evaluation rubrics, review and label technical data, identify implementation errors, maintain data generation and evaluation pipelines, and communicate findings to researchers and engineering teams.

• Qualifications: At least five years of hands-on software engineering experience, familiarity with substantial codebases, strong technical judgment, excellent code-review skills, and the ability to explain implementation quality clearly.

• Technical Skills: Python, TypeScript, JavaScript, Go, Java, GitHub, CI/CD, open-source development, LLMs, code analysis, and code reviews. Experience with modern AI coding tools is also expected.

• Language and Schedule: Strong written communication skills are required. Availability of 20–40 hours per week is expected, including at least six hours of daily overlap with Pacific Time.

• Contract Type: Independent contractor, fully remote, with an expected project duration of approximately three months.

COMPENSATION

• Pay: $100–$150 per hour. Applicants are asked to provide their specific hourly rate expectation.

• Referral Bonus: $500.

• Openings: The number of available positions is not specified.

• Payment frequency and payment method should be confirmed during the application process.

WHO SHOULD APPLY

This position is particularly relevant to senior software engineers who regularly work with production codebases and have the experience to distinguish between code that simply runs and code that is genuinely reliable, maintainable, and technically sound. Candidates should be comfortable reviewing complex implementations, identifying subtle problems, and explaining their reasoning in writing.

Experience with LLM evaluation, AI coding agents, reinforcement learning from human feedback (RLHF), preference data, evaluation rubric design, or model post-training is an advantage. However, these specializations are not mandatory, as the listing places greater emphasis on practical software engineering experience and strong technical judgment.

Professionals exploring AI Training Jobs can also review related opportunities available through RemoteWFAJobs.

HOW TO APPLY

1. Review the responsibilities, qualifications, and working requirements to determine whether the opportunity matches your experience.

2. Prepare relevant professional experience, examples of previous work, and any other information requested in the application.

3. Submit your application through APPLY HERE and confirm any outstanding details during the hiring process.

The stated evaluation process includes three stages:

• AI interview lasting approximately 25 minutes.

• Practical code and AI evaluation exercise lasting approximately 30 minutes.

• Hiring manager interview lasting approximately 20 minutes.

The practical exercise focuses on reviewing AI-generated code rather than solving competitive programming problems or algorithm puzzles.

FAQ

Is previous AI evaluation experience required?

No. Candidates must have at least five years of hands-on software engineering experience, but previous experience with LLM evaluation, coding agents, RLHF, preference data, or post-training is preferred rather than mandatory.

Is a degree necessary?

The listing does not specify a mandatory university degree or educational qualification. Practical software engineering experience is the stated primary requirement.

How many openings are available?

The number of available positions is not specified in the original listing.

CLOSING

This remote contractor position is suited to senior developers who can bring practical engineering judgment to AI model evaluation. The work combines code reviews, technical assessments, evaluation framework development, and collaboration with AI researchers over an expected three-month engagement.

For more opportunities, browse RemoteWFAJobs remote positions and explore the available AI Training Jobs.

#SeniorSoftwareEngineer #AIEvaluation #PythonJobs #CodingAgents #RemoteJobs #RemoteContractor #WorkFromHome

Share :