How to run background jobs?

@smorks@lemmy.ca

lemmy uses clockwerk i believe, there’s a few out there for rust.

@SuddenlyBlowGreen@lemmy.world

let mut scheduler = Scheduler::with_tz(chrono::Utc);

scheduler.every(10.minutes()).plus(30.seconds()).run(|| println!("Periodic task"));

scheduler.every(1.day()).at("3:20 pm").run(|| println!("Daily task")); 

scheduler.every(Tuesday).at("14:20:17").and_every(Thursday).at("15:00").run(|| println!("Biweekly task"));

Damn, that a really ingenious and intuitive use of the builder pattern.

Kudos to the devs!

@nothingness@lemmy.world

Scheduler

what library is it from?

@SuddenlyBlowGreen@lemmy.world

The clokwerk crate.

peterprototypes

@SuddenlyBlowGreen @nothingness I recently needed that, but fuzzy. Like I want a task ran anywhere between 1PM and 2PM. Can this be accomplished with the clockwerk crate?

@SuddenlyBlowGreen@lemmy.world

You could schedule a task at 1 PM, then generate a random number between 0-60 inside that task, wait that many minutes, then launch the actual task.

@ikornaselur@lemm.ee

I looked into this last week for recurring tasks in an actix-web project and ended up picking fang, but haven’t really battle tested it… But seems to do what I need. I’m using cron tasks with Postgres as the backend to run a recurring task every minute.

@kevincox@lemmy.ml

For my project I just run them. I’m using async so it is just spawning a task in the background, but you can do the same with threads. Don’t underestimate the number of threads that you can run on a modern computer.

If you want some sort of throttling you can stuff tasks into a queue and just run N background threads pulling them off an processing them.

If you need durability then you start to have more trouble. This is where I would start looking at a library. IDK if there are any libraries that handle logging, retries and similar, but if not you can probably get the basics down pretty easily.

@nothingness@lemmy.world

For my project I just run them.

How would you “just run” a task every 30 minutes? Every 5 hours? Once a day?

@kevincox@lemmy.ml

std::thread::spawn(|| {
    loop {
        std::thread::sleep(std::time::Duration::from_secs(30*60));
        do_job();
    }
});

Works pretty well. Maybe add a bit of code to crash the whole process on panic or some other logging. Wastes a few KiB of memory per loop but probably not a major issue. Doing this with async will waste only the tiniest amount of memory.

Vest

@kevincox How do you stop the job? Do you use channels like in Go?

@kevincox@lemmy.ml

It depends. Sometimes you can just put an exit call at the end of main to kill the thread. If you want to attempt graceful shutdown then usually I just use a boolean shutdown flag. Then the loop becomes while !shutdown.get() {

@nothingness@lemmy.world

what if one of the calls crashes? how would you re-run it?

@kevincox@lemmy.ml

Best option is probably to add a wrapper around the thread that re-spawns it. But you can also just catch panics in the loop.

@nothingness@lemmy.world

How would you re-run it multiple times then? An internal should be progressively greater. How would you terminate it if it continues to produce an exception?

vaalla

Can you use async in your project?

If Yes, you can spawn a task that will listen on a channel. If you need to run them in parallel probably you can find a mpmc channel.

If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

Don’t know any crate just for this.

@nothingness@lemmy.world

If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

How would you do it every 30 minutes? Every 5 hours? Once a day?

How to run background jobs?

How to run background jobs?

Rust Programming