Sitelet https://github.com/resemble-ai/Resemblyzer/issues/69
Skip to content

Async and parallel processing #69

Description

@frankiedrake

First, sorry if this question is addressed here wrong, but I should try.
The original question was asked here, I described the problem in detail.
In general: I have an application where some audios are being processed. There's function that embeds audio file which internally calls preprocess_wav function.
When processing requests synchronously (using flask), everything is working fine, but in async-way processing (via rabbitmq using amqp-storm), my gunicorn workers are sometimes being restarted. Adding a debug points show that it breaks at the preprocess_wav call and more precisely at trim_long_silences call.
What I've noticed is that there isn't high CPU or memory usage (logged it with psutil), also this even happens with the only one worker, so this tells me that probably this is not a resource usage issue but probably some blocking operations.
Could you suggest me what can I do to solve this or which way I get the details on why this actually happen?
Thank you.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions