When engineers talk about scaling systems, the conversation usually revolves around databases, caches, or Kubernetes.
Rarely does anyone talk about the queue.
But if you're building an email platform, the queue is the product.
Every OTP, password reset, invoice, notification, or marketing campaign eventually becomes a queued job. And once you're processing thousands-or millions-of these jobs, reliability stops being a feature and becomes the entire business.
At Keplars, our email delivery pipeline has been powered by BullMQ for a long time.
It's fast, mature, reliable, and honestly one of the best queueing libraries in the Node.js ecosystem.
So why are we migrating to Restate.dev?
The answer isn't because BullMQ failed us.
It's because our architecture evolved.
BullMQ Was Never the Problem
BullMQ solved exactly what it promised to solve.
It gave us queues, retries, delayed jobs, priorities, concurrency controls, and a robust execution engine that helped us build a reliable email pipeline.
For many applications, that's more than enough.
But building an email infrastructure platform introduces challenges that aren't really "queue problems."
They're distributed systems problems.
As our platform grew, our queue slowly became surrounded by infrastructure.
We started building systems for:
- Retry policies
- Crash recovery
- Idempotency
- Stateful workflows
- Long-running executions
- Scheduled operations
- Provider failover
- Duplicate prevention
- Distributed coordination
None of those features were technically part of BullMQ.
They were things we had to build around it.
And eventually, we realized something.
We were spending almost as much time building the infrastructure around the queue as we were building the actual email platform.
The Moment It Clicked
One question kept coming up during architecture discussions:
"If this worker crashes halfway through the workflow... what happens?"
It sounds like a simple question.
But answering it isn't.
You have to know:
- Which steps already completed
- Which API calls succeeded
- Which emails were already sent
- Which database writes happened
- Which retries are still pending
- Whether we can safely resume
- Whether this creates duplicate emails
Every distributed system eventually has to answer these questions.
We just realized we didn't want to answer them manually anymore.
Why Restate.dev Felt Different
When we started experimenting with Restate, what stood out wasn't performance.
It was the programming model.
Instead of thinking in terms of queue jobs, retries, timers, distributed state, and recovery mechanisms...
...you simply write a workflow.
Every step is durably persisted.
If a process crashes after step seven out of ten, execution doesn't restart from the beginning.
It simply continues from step eight.
No custom recovery code.
No rebuilding state.
No "did this already execute?" logic scattered throughout the application.
That changes how you design software.
Our Email Pipeline Looks Different Now
Instead of individual jobs triggering other jobs, we can express the entire lifecycle as a single durable workflow.
Something like this:
Each step becomes durable.
Each transition is tracked.
Recovery is automatic.
The workflow itself becomes the source of truth.
Less Infrastructure. More Product.
One of the biggest benefits isn't speed.
It's simplicity.
Previously, we spent engineering effort on questions like:
- How retries should be stored
- How to resume interrupted jobs
- How to prevent duplicate execution
- How to coordinate workers across multiple instances
- How to maintain workflow state during failures
Those are important problems.
They're also problems we'd rather not own.
Restate moves many of those responsibilities into the runtime.
That means our engineering team spends more time improving email infrastructure and less time maintaining workflow infrastructure.
What This Means for Our Customers
Most Keplars users probably won't notice this migration.
And honestly...
That's exactly how infrastructure should work.
There won't be a shiny new dashboard.
There won't be a flashy button.
Instead, they'll experience:
- Better reliability
- More predictable email processing
- Improved recovery from failures
- Fewer duplicate sends
- Faster rollout of new platform capabilities
Sometimes the biggest improvements are completely invisible.
The Bigger Lesson
One thing I've learned while building infrastructure products is this:
The tools that get you from 0 to 1 aren't always the tools that get you from 1 to 100.
BullMQ helped us build an incredible foundation.
We're grateful for it.
But architecture isn't about choosing the "best" technology.
It's about choosing the right abstraction for where your product is today.
For us, durable workflows have become a better abstraction than traditional queue jobs.
Looking Ahead
Infrastructure migrations rarely make headlines.
Customers don't wake up excited because you replaced your queue engine.
But these are often the decisions that determine whether a platform can scale for the next five years.
BullMQ helped us build a reliable foundation.
Restate.dev gives us a simpler model for building resilient, stateful workflows while reducing the operational complexity that comes with managing distributed systems.
We're excited to build the next generation of Keplars on top of it.
If you're building distributed systems, workflow engines, or large-scale queueing infrastructure, I'd love to hear what challenges you've faced and whether you've explored durable execution models like Restate.