Jacob Paris
← Back to all content

Offload work to a background process in Node JS

Javascript is a single threaded language, so the moment you start running some intensive task, it will block the event loop and prevent the rest of your application from working.

while (10 > 5) {
console.log(
"This will never end and will cause the browser tab to freeze or the Node.js process to stall indefinitely.",
)
}

While most applications won't have any forever-tasks that stall it completely, functions that take a long time to run can degrade the user experience elsewhere in the app.

This can cause dropped network requests, slow page loads, and other issues.

If it's javascript's bottleneck that a single javascript process can only run one thread at a time, then the solution must be to use multiple javascript processes to run different parts of our application.

By offloading the work to a separate process, we can keep our main application running and responsive while the background tasks run.

The simplest setup is to use a singleton module to spawn a new Node.js process that runs another javascript file.

import { spawn } from "child_process"
import { join } from "path"
let subprocess: ReturnType<typeof spawn> | null = null
declare global {
var subprocess: ReturnType<typeof spawn> | null
}
if (!global.subprocess) {
global.subprocess = spawn(
"node",
join(__dirname, "subprocess.js"),
{
stdio: "inherit",
shell: true,
detached: true,
},
)
}
subprocess = global.subprocess
export default subprocess

This will successfully offload the work to a separate process, but to be a practical solution for most developers, there are many more things to consider.

  • How do we communicate between the two processes?
  • How do we handle errors?
  • How do we run a typescript file?
  • How do we handle the subprocess crashing?

Rather than solving these one by one, we can use a battle-tested library like BullMQ that supports background processes natively to handle all of these concerns.

Professional headshot

Hi, I'm Jacob

Hey there! I'm a developer, designer, and digital nomad with a background in lean manufacturing.

About once per month, I send an email with new guides, new blog posts, and sneak peeks of what's coming next.

Everyone who subscribes gets access to the source code for this website and every example project for all my tutorials.

Stay up to date with everything I'm working on by entering your email below.

Unsubscribe at any time.