Senior Software Engineer - Go/MongoDB
- Role
- Backend
- Experience
- Senior
- Employment
- Full-time
Open to IT, HR, PL, ES only. Set where you work from to check your eligibility.
No BS summary
Senior Go engineer for open-source MongoDB operational tooling. Needs production Go, concurrency, distributed systems/data consistency, hands-on MongoDB, and open-source Git/PR workflows. Remote role tied to EMEA / listed countries Italy, Croatia, Poland, and Spain.
Core skills
Required skills
Optional skills
MongoDB Tools Team
Team: MongoDB Tools (Product and Engineering)
Location: Remote
Projects: percona/percona-clustersync-mongodb and percona/percona-backup-mongodb
About the team
The MongoDB Tools Team builds Percona's open source operational tooling for MongoDB. Two projects sit at the center of what we do. Percona ClusterSync for MongoDB (PCSM) clones and continuously replicates data between clusters. Percona Backup for MongoDB (PBM) is a distributed, low-impact backup and restore solution for replica sets and sharded clusters. Both are written in Go, both are Apache 2.0 licensed, and both are built fully in the open.
This role sits primarily on PCSM, which is younger and moving fast, so you will have real influence over how it takes shape. You will also work across into PBM. The two tools share many hard problems: cluster topology, the oplog and change streams, consistency across shards, and performance in very large production clusters. Backup and restore experience is a real advantage here, not just a box to tick.
The projects
- PCSM (primary focus): initial data cloning followed by continuous change replication over MongoDB Change Streams, for both replica sets and sharded clusters. Still pre-1.0 and evolving quickly.
- PBM (secondary): consistent backup and restore with point-in-time recovery, using oplog capture to stay consistent across replica sets and sharded clusters, with S3-compatible and filesystem storage. Driven by pbm-agent processes on each node and a pbm CLI. Mature and widely deployed in production.
What you will work on
Primary, on PCSM
- The core replication engine: initial collection cloning followed by continuous change capture over MongoDB Change Streams, with correct handling of resume tokens, ordering, and resumability after failures.
- Correctness and fault tolerance at scale: recovering cleanly from network drops, primary elections, and restarts without losing or duplicating changes, and reasoning carefully about the delivery guarantees we can honestly promise.
- Sharded cluster support: replicating across shards, dealing with the realities of chunk migrations and balancer activity, and keeping the target consistent.
- Namespace filtering and automatic index management, plus the edge cases that show up with DDL, TTL, and index differences between source and target.
- Performance and throughput: parallelizing the clone, applying backpressure, and keeping memory and connection use sane against large clusters with great change volume.
- The CLI and HTTP API that drive and observe a sync, and the metrics and logging that let an operator trust what is happening.
Also across PBM
- Consistent backup, restore, and point-in-time recovery across replica sets and sharded clusters, using physical or logical type of the backup.
- Backup storage: integrating reliably with main cloud object storage (S3, GCS, Azure Blob Storage...) and remote filesystems, and handling the throughput and failure modes that show up at scale.
- The pbm-agent and pbm CLI, and the control-collection state in MongoDB that coordinates them across the cluster.
Shared across both
- Working in the open: pull requests, code review, JIRA, and the community forum.
- Release quality: tests, packaging, and the CI and security scanning that gate every change.
What Have You Done:
- Strong Go experience in production, with real fluency in concurrency: goroutines, channels, context cancellation, worker pools, and backpressure. You have debugged a race condition that only showed up under load, and you know how you found it.
- Solid grounding in distributed systems and data consistency. You can talk clearly about at-least-once versus exactly-once, idempotency, ordering, and what it takes to make a stateful process resumable.
- Hands-on MongoDB knowledge: change streams, the oplog, resume tokens, replica sets, and sharding. You do not need to have built replication before, but you should understand why it is hard.
- Comfort building and operating command-line tools and HTTP APIs, and instrumenting them with metrics and structured logs.
- A habit of writing tests that catch real problems, and comfort working across a mixed toolchain.
- Experience working in the open: Git and pull request workflows, giving and taking code review well, and communicating clearly in writing with contributors you have never met in person.
Nice to have
- Prior work on database internals, CDC pipelines, ETL, or data migration tooling.
- Familiarity with the Prometheus style of metrics and observability.
- Experience with golangci-lint, vulnerability scanning (for example Trivy), and deb and rpm packaging.
- Background maintaining or contributing to an open source project with an external community.
- Exposure to MongoDB sharded clusters at large scale, where balancer behavior and backup interaction stop being theoretical.
How we work
Both projects live on GitHub, contributions go through pull requests and code review, and we track work in JIRA. We care about keeping open source open, so the default is that the work you do here is public and stays that way.
Why Percona?
At Percona, we believe an open world is a better world. Our mission is to enable everyone to innovate freely, by providing the best open source database software, support, and services. We make databases and applications run better through a unique combination of expertise and open source software built with the community for you. Our technical teams are experts in MySQL, MongoDB, PostgreSQL, and MariaDB.
Percona is proud to be a remote-only and globally dispersed workforce – we have colleagues in more than 50 countries! We offer a collaborative, highly-engaged culture where your ideas are welcome and your voice is heard.
Our staff receives generous benefits including flexible work hours and various paid time off programs, all your equipment for your remote office, funds for career development (external training, certifications, conferences), ongoing connectivity allowances, and the opportunity to participate in our equity incentive plan. We also have benefits that support a healthy work/life balance such as The Percona Adventure Team, Work-from-Anywhere, FlowDays, FryDays, and overall flexibility. We also support being socially responsible through our PAVE volunteering program and Women Transforming Technology.
If you love the idea of working with a high-growth tech company that is one of the best in the business and known globally as a leader in the open-source database space, let’s talk!
Connect with us and stay up to date on our latest news and developments by following us on LinkedIn and Twitter. We look forward to connecting with you!
What you'll do
- Work primarily on Percona ClusterSync for MongoDB (PCSM), with additional work on Percona Backup for MongoDB (PBM).
- Build the core replication engine for initial collection cloning followed by continuous change capture over MongoDB Change Streams.
- Handle resume tokens, ordering, and resumability after failures correctly.
- Improve correctness and fault tolerance at scale, including recovery from network drops, primary elections, and restarts without losing or duplicating changes.
- Reason about delivery guarantees for stateful replication processes.
- Support sharded cluster replication, including chunk migrations, balancer activity, and target consistency.
- Work on namespace filtering and automatic index management, including DDL, TTL, and index-difference edge cases.
- Improve performance and throughput by parallelizing cloning, applying backpressure, and managing memory and connection use against large clusters.
- Build and maintain the CLI and HTTP API used to drive and observe syncs.
- Add metrics and logging that let operators trust sync behavior.
- Work on consistent backup, restore, and point-in-time recovery across replica sets and sharded clusters using physical or logical backups.
- Integrate backup storage reliably with cloud object storage and remote filesystems.
- Handle backup storage throughput and failure modes at scale.
- Work on pbm-agent, pbm CLI, and MongoDB control-collection state that coordinates cluster backup processes.
- Work in the open through pull requests, code review, JIRA, and the community forum.
- Contribute to release quality through tests, packaging, CI, and security scanning.
What they require
- Strong production Go experience with real fluency in concurrency, including goroutines, channels, context cancellation, worker pools, and backpressure.
- Experience debugging race conditions that appeared under load.
- Solid grounding in distributed systems and data consistency.
- Ability to discuss at-least-once versus exactly-once, idempotency, ordering, and making a stateful process resumable.
- Hands-on MongoDB knowledge, including change streams, the oplog, resume tokens, replica sets, and sharding.
- Understanding of why MongoDB replication is hard, even if the candidate has not built replication before.
- Comfort building and operating command-line tools and HTTP APIs.
- Comfort instrumenting tools and APIs with metrics and structured logs.
- Habit of writing tests that catch real problems.
- Comfort working across a mixed toolchain.
- Experience working in the open with Git and pull request workflows.
- Ability to give and receive code review well.
- Ability to communicate clearly in writing with contributors the candidate has never met in person.
- Preferred: Prior work on database internals, CDC pipelines, ETL, or data migration tooling.
- Preferred: Background maintaining or contributing to an open source project with an external community.
- Preferred: Exposure to MongoDB sharded clusters at large scale, where balancer behavior and backup interaction stop being theoretical.
Benefits
- Remote-only and globally dispersed workforce.
- Collaborative, highly engaged culture where ideas are welcome and the employee's voice is heard.
- Flexible work hours.
- Various paid time off programs.
- Equipment for remote office.
- Funds for career development, including external training, certifications, and conferences.
- Ongoing connectivity allowances.
- Opportunity to participate in equity incentive plan.
- Work/life balance support through The Percona Adventure Team, Work-from-Anywhere, FlowDays, FryDays, and overall flexibility.
- PAVE volunteering program.
- Women Transforming Technology program.
Percona provides open source database software, support, and services, helping databases and applications run better through expertise and community-built open source software. Its technical teams work with MySQL, MongoDB, PostgreSQL, and MariaDB.