Fix for takeLock & optionally forcing

[rysi3k]

1. Fix for call job.lockKey in scripts.js (it is function, but called as property)
2. Possibility to take lock without checking key existance

[manast]

Thanks for the PR. Could you please motivate why you would like to force taking a lock? it could potentially have a lot of side effects.

[rysi3k]

Hello,
My scenario:

1. Job is processing on remote server, my code only supervising it (requests for status every 15s)
2. Supervisor process is restarted/crashed
3. On process start I need to check if on remote server job is still processing (request for status)
4. If processing is in progress, I take lock of job and run standard requests every 15s

In normal implementation, job will be stalled on bull, but on remote server will be continued. I dont want to restart it.

Regards

[manast]

In the scenario above, I do not get why the job is stalled, you wrote that the supervisor process crashes, but not the remote server processing the job, right?

[rysi3k]

Yes, supervisor where bull works crashes. So, there is different queue.token, this causes job stalled.

[manast]

If the process running bull crashes, the jobs will picked up later by other bull workers, I still do not fully understand your use case :/

[rysi3k]

Hello. I missed important part. My `process` function is defined as: 1. Start request to remote server 2. Checks every 15s to get result 3. If remote server will return "done", job is done, bull is notified. So once started job will process independly from bull. If bull crashes and whole process will start from begining it will also send `start` request which cause reset job on remote side. Regards

[manast]

Can you explain the point 4 a bit more? (If processing is in progress, I take lock of job and run standard requests every 15s)

[rysi3k]

Here is my simplified code, it could resolve any confusions.

queue.on('failed', () => {
    worker.setCurrentJobId(0);
}).on('completed', () => {
    worker.setCurrentJobId(0);
});
start();
   
async start() {  

    // check if some job is not done
    if (worker.currentJobId) {
        const job = await queue.getJob(jobId);
        if (job) {
            // if some job is found, process and wait until end
            await resumeProcessing(job);
        }
        worker.setCurrentJobId(0);
    }

    // normal processing method
    queue.process(async(job) => {
        await remoteServer.start(job.data);
        worker.setCurrentJobId(job.id);

        return await checkStatus(job);
    });
}

async resumeProcessing(job) {
    job.takeLock(true);
    // process method needs uniq name if we dont want to use this handler anymore, lockKey looks good for it
    queue.process(job.name = job.lockKey(), async(job) => {
        return await checkStatus(job);
    });
    await queue.processJob(job);
}

async checkStatus(job) {
    const res = await remoteServer.getStatus(job.data);
    
    if (!isJobDone(res)) {
        // wait next 15s
        return new Promise((resolve, reject) => {
            setTimeout(() => this.checkStatus(job).then(resolve).catch(reject), 15000);
        });
    } else {
        return 'done';
    }
}

[manast]

I have studied the code above and I do not understand why it is needed, it looks as if you have implemented a similar mechanism of what already exists in bull to handle stalled jobs. A stalled job will be picked up by a idle worker, just remember to have a large enough value in the maxStalledCount setting: https://github.com/OptimalBits/bull/blob/master/REFERENCE.md#queue

[rysi3k]

I have different "process" method in normal processing (start + checkStatus) and durign resuming (only checkStatus).
In built in stalled method it is not possible to achive this or I'm miss something?

[manast]

ok, so your use case is to run a different process function when re-processing a stalled job?

[rysi3k]

Hmmm I think yes, it could be called like this

[manast]

In that case I think I have a better solution. I think we can add a "status" flag to the job object that is passed to the processor, so depending on the flag you can run one code or another, it should simplify your code a lot, what do you think?

[rysi3k]

Your Idea sounds good and should resolve my problems.
BTW dont forget to fix takeLock method, ther is bug in call lockKey method ;)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.