Skip to content

decoding queue overflow #25

Open
Open

Description

@madpsy

Seeing the following errors on a fairly constant basis:

2022-07-24 09:00:00,130 ERROR 1764 [KIWIC-20+] decoding queue overflow; dropping one file 2022-07-24 09:00:00,130 ERROR 1764 [FLYDOG-60~] decoding queue overflow; dropping one file 2022-07-24 09:00:00,141 ERROR 1764 [RASP-160~] decoding queue overflow; dropping one file 2022-07-24 09:00:00,141 ERROR 1764 [RASP-160!] decoding queue overflow; dropping one file 2022-07-24 09:00:00,142 ERROR 1764 [KIWIC-15!] decoding queue overflow; dropping one file 2022-07-24 09:00:00,143 ERROR 1764 [RASP-630!] decoding queue overflow; dropping one file 2022-07-24 09:00:00,144 ERROR 1764 [KIWIA-80~] decoding queue overflow; dropping one file 2022-07-24 09:00:00,145 ERROR 1764 [KIWIC-10~] decoding queue overflow; dropping one file 2022-07-24 09:00:00,146 ERROR 1764 [KIWIB-30+] decoding queue overflow; dropping one file 2022-07-24 09:00:00,147 ERROR 1764 [KIWIA-80!] decoding queue overflow; dropping one file 2022-07-24 09:00:00,147 ERROR 1764 [FLYDOG-40~] decoding queue overflow; dropping one file 2022-07-24 09:00:00,148 ERROR 1764 [KIWIC-20~] decoding queue overflow; dropping one file 2022-07-24 09:00:00,148 ERROR 1764 [KIWIB-12!] decoding queue overflow; dropping one file

What I'd like to understand is how the queue works and how it decides there is an overflow. I suspect the logic is based on if it can decode everything within the cycle {i.e. 15s for FT8). In other words is the CPU going to fail to decode in time.

Is this easily tweakable? I would be willing to bet that it wouldn't run away and build up an excessive queue as it's not unsual to have a a very busy cycle(s) followed by less busy cycle(s), thus the queue would catch up within, say, two minutes or less.

Activity

  1. lazywalker commented on Jul 25, 2022

    @lazywalker
    Owner

    Try to tweak these two values within settings.py

    DECODER_QUEUE = {
        'maxsize': 10,
        'workers': 3
    }
    

    the value maxsize can set the size of queue, workers means how many decoders working at the sometime, more workers, faster queue consuming and more cpu resources will be taken.

    for example, tune the maxsize to 100

  2. madpsy commented on Jul 25, 2022

    @madpsy
    Author

    Perfect thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions