Communicate with vendors

Possibly build dashboard

Communication between engineers and annotation team internally and externally. SO the accuracy of the annotation is important.

Responsibilities

1. Annotation Lead

Own the end-to-end quality and coordination of annotation workflows across concurrent streams.

• Ensure annotation outputs consistently meet the quality bar; identify and resolve systematic error patterns (label inconsistencies, missed edge cases, boundary errors)
• Serve as the primary interface between ML engineers and the external annotation vendor — ensuring timely and crisp communication on annotation tasks including clarifications, guideline alignment, and QA feedback
• Work with engineers to establish and maintain annotation guidelines, protocols, and gold-set standards; verify ground truth sets against engineer intent before large-scale annotation begins
• Identify and implement pre-annotation tooling or workflow improvements that increase annotator efficiency and reduce error rates
• Track inter-annotator agreement, QA pass rates, and rejection patterns over time; use findings to drive continuous guideline improvement

2. System Performance Monitoring

Develop a deep understanding of where and how the system struggles, and keep engineering informed.

• Build intuition on hard corner cases and edge conditions where the system fails — through ongoing review of annotation outputs, rejection patterns, and model validation results
• Communicate regularly with engineering to surface failure patterns, flag emerging issues, and play an active role in driving identified issues to resolution
• Track pipeline health metrics by building and maintaining whatever tooling is appropriate (SQL queries, scripts, spreadsheets, or lightweight dashboards)

What We're Looking For

• Able to pick up and use tools such as SQL, Python, dashboard platform to pull, summarize, and track metrics.
• Strong analytical eye — able to spot quality issues and failure patterns in structured label data
• Clear, crisp communicator; comfortable as the connective tissue between engineering and an external vendor
• Organized and self-directed; able to maintain tracking systems and keep multiple workstreams moving without close oversight
• Work in the US business hours

Requirements:

• A Bachelor’s Degree with 3-4 years of experience or a Master’s Degree with 2 years’ experience in Computer Science, Electrical Engineering, or a related field.
• Core Skills: General Software Engineering skills with 3+ years of programming experience in python and the surrounding tooling ecosystem along with familiarity in linux and expertise in infrastructure, cloud and/or MLOps,.
• Personal Attributes: Team player, good communication skills, self starter.
• Strong teamwork and communication skills to collaborate with cross-functional teams, including ML and software engineers.
• Nice to Have: Experience building MLOps pipelines for deep learning based perception solutions on AWS or GCP