Software Engineer II - Autonomy Data
- Role
- Data Engineering
- Experience
- Mid
- Employment
- Full-time
Open to US only. Set where you work from to check your eligibility.
No BS summary
Level 2 autonomy data engineer with 4+ years of data engineering experience, or 2+ with a master's. Must be a U.S. citizen and strong in Python, SQL, cloud data infrastructure, IaC, and production data pipelines for time-series/binary data.
Core skills
Required skills
Optional skills
About The Company:
At Torc, we have always believed that autonomous vehicle technology will transform how we travel, move freight, and do business. A leader in autonomous driving since 2007, Torc has spent over a decade commercializing our solutions with experienced partners. Now a part of the Daimler family, we are focused solely on developing software for automated trucks to transform how the world moves freight. Join us and catapult your career with the company that helped pioneer autonomous technology, and the first AV software company with the vision to partner directly with a truck manufacturer.
Meet The Team:
Torc is hiring an Autonomy Data Engineer Level 2 to help design, build and operate the data infrastructure that powers our autonomy program. You will build the pipelines, storage systems, and tooling that turn raw vehicle sensor logs into the curated, structured datasets that our perception, planning and simulation engineers depend on.
This is a high-ownership role on a lean team. Moving large scale sensor data reliably from vehicles operating in demanding environments and making it quickly available for model training is a difficult and high-impact problem to solve.
What You'll Do:
- Data Lake and Ingestion Pipeline
- Contribute to the design and organization of the program’s data lake, including schema definitions, partitioning strategy and metadata indexing.
- Build and maintain end-to-end pipelines that ingest high-bandwidth sensor logs from vehicles into cloud storage with high reliability and tolerant of ad-hoc and intermittent connectivity mechanisms.
- Implement data validation and integrity checks that can detect corrupted information, missing sensors, and inconsistent calibration prior to the data being processed by downstream systems.
- Implement retention, tiering and lifecycle policies for data to balance storage costs with development value.
- Dataset Curation and Labeling Infrastructure
- Build tooling to query raw logs to produce curated training and evaluation datasets.
- Build automation to run cost-effective pseudo-labeling workflows at the scale of data ingest.
- Implement data quality and model performance metrics that are used to direct labeling effort toward the highest-value examples.
- Autonomy Data Visualization
- Deploy and maintain data visualization tooling to support log review, annotation QA, and autonomy debugging workflows.
- Build integrations between the visualization tooling and the data lake so engineers can navigate from a dataset entry or model failure directly to the origin log data
- Work with autonomy engineers to define and surface custom visualization panels and implement metrics for analyzing unstructured operating environments.
- Build dashboards that provide the autonomy engineers visibility into data coverage by terrain type, operating environment and geographic region.
- Cross-functional Collaboration
- Establish and document data contracts between the data services and model training consumers.
- Partner with perception, planning and embedded engineers across the data lifecyle: from shaping the logging schemas and collection triggers to defining the dataset interfaces that supply model training and evaluation.
- Follow and help evolve data engineering standards, best practices, and tooling choices for an innovative and fast-paced team.
- Contribute to the data roadmap and surface findings to senior technical leadership.
What You'll Need to Succeed:
- Bachelor’s degree in Computer Science, Computer Engineering, Software Engineering, Electrical Engineering or a related field with 4+ years of data engineering experience or a Master’s with 2+ years.
- Strong proficiency in Python and SQL, with demonstrated ability to build production-quality data pipelines
- Experience with cloud data infrastructure (AWS preferred: S3, Glue Athena, redshift, or equivalent) and infrastructure-as-code tools (Terraform, Cloud Formation).
- Solid understanding of data partitioning strategies and columnar storage formats (Parquet, Orc, etc.)
- Experience building and operating data pipelines that process time-series and binary data.
- Proven ability to evaluate and integrate open-source tooling when appropriate versus building from scratch.
- Good instincts for delivering data quality through first-class implementations of monitoring, validation and lineage tracking.
Bonus points!
- Experience with autonomous vehicles, robotics, or other sensor-driven autonomous systems.
- Deep experience with Foxglove or Rerun beyond basic playback, e.g. building custom extensions or integrating them into a structured log review or annotation QA workflow.
- Familiarity with the MCAP CLI and/or python library and experience converting MCAP data to columnar data formats for further querying and processing.
- Experience with data curation for ML training, e.g. diversity sampling, pseudo-labeling, and dataset versioning.
U.S. Citizenship Requirement:
This position requires access to information and systems that are restricted under U.S. law. Accordingly, only U.S. citizens are eligible for this role. This requirement is based on applicable government regulations and is not related to immigration status discrimination.
Perks of Being a Full-time Torc’r
Torc cares about our team members and we strive to provide benefits and resources to support their health, work/life balance, and future. Our culture is collaborative, energetic, and team focused. Torc offers:
- A competitive compensation package that includes a bonus component and stock options
- 100% paid medical, dental, and vision premiums for full-time employees
- 401K plan with a 6% employer match
- Flexibility in schedule and generous paid vacation (available immediately after start date)
- Company-wide holiday office closures
- AD+D and Life Insurance
At Torc, we’re committed to building a diverse and inclusive workplace. We celebrate the uniqueness of our Torc’rs and do not discriminate based on race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, veteran status, or disabilities.
Even if you don’t meet 100% of the qualifications listed for this opportunity, we encourage you to apply.
Our compensation reflects the cost of labor across several geographic markets. Pay is based on a number of factors and may vary depending on job-related knowledge, skills, and experience. Torc's total compensation package will also include our corporate bonus and stock option plan. Dependent on the position offered, sign-on payments, relocation, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits.
Job ID: R-102913
What you'll do
- Contribute to the design and organization of the program’s data lake, including schema definitions, partitioning strategy, and metadata indexing.
- Build and maintain end-to-end pipelines that ingest high-bandwidth sensor logs from vehicles into cloud storage with high reliability and tolerance of ad-hoc and intermittent connectivity mechanisms.
- Implement data validation and integrity checks to detect corrupted information, missing sensors, and inconsistent calibration before downstream processing.
- Implement retention, tiering, and lifecycle policies for data to balance storage costs with development value.
- Build tooling to query raw logs to produce curated training and evaluation datasets.
- Build automation to run cost-effective pseudo-labeling workflows at the scale of data ingest.
- Implement data quality and model performance metrics used to direct labeling effort toward the highest-value examples.
- Deploy and maintain data visualization tooling to support log review, annotation QA, and autonomy debugging workflows.
- Build integrations between visualization tooling and the data lake so engineers can navigate from a dataset entry or model failure directly to the origin log data.
- Work with autonomy engineers to define and surface custom visualization panels and implement metrics for analyzing unstructured operating environments.
- Build dashboards that give autonomy engineers visibility into data coverage by terrain type, operating environment, and geographic region.
- Establish and document data contracts between data services and model training consumers.
- Partner with perception, planning, and embedded engineers across the data lifecycle, from shaping logging schemas and collection triggers to defining dataset interfaces for model training and evaluation.
- Follow and help evolve data engineering standards, best practices, and tooling choices for an innovative and fast-paced team.
- Contribute to the data roadmap and surface findings to senior technical leadership.
What they require
- Bachelor’s degree in Computer Science, Computer Engineering, Software Engineering, Electrical Engineering, or a related field with 4+ years of data engineering experience, or a Master’s with 2+ years.
- Strong proficiency in Python and SQL, with demonstrated ability to build production-quality data pipelines.
- Experience with cloud data infrastructure, AWS preferred: S3, Glue Athena, Redshift, or equivalent, and infrastructure-as-code tools such as Terraform or CloudFormation.
- Solid understanding of data partitioning strategies and columnar storage formats such as Parquet or ORC.
- Experience building and operating data pipelines that process time-series and binary data.
- Proven ability to evaluate and integrate open-source tooling when appropriate versus building from scratch.
- Good instincts for delivering data quality through first-class implementations of monitoring, validation, and lineage tracking.
- Only U.S. citizens are eligible for this role because the position requires access to information and systems restricted under U.S. law.
- Preferred: Experience with autonomous vehicles, robotics, or other sensor-driven autonomous systems.
- Preferred: Deep experience with Foxglove or Rerun beyond basic playback, such as building custom extensions or integrating them into a structured log review or annotation QA workflow.
- Preferred: Familiarity with the MCAP CLI and/or Python library and experience converting MCAP data to columnar data formats for further querying and processing.
- Preferred: Experience with data curation for ML training, such as diversity sampling, pseudo-labeling, and dataset versioning.
Benefits
- Competitive compensation package that includes a bonus component and stock options.
- 100% paid medical, dental, and vision premiums for full-time employees.
- 401K plan with a 6% employer match.
- Flexibility in schedule and generous paid vacation available immediately after start date.
- Company-wide holiday office closures.
- AD+D and Life Insurance.
- Corporate bonus and stock option plan.
- Depending on the position offered, sign-on payments, relocation, and other forms of compensation may be provided.
- Full range of medical, financial, and/or other benefits.
Torc develops autonomous driving software for automated trucks and is part of the Daimler family. It has been a leader in autonomous driving since 2007.