RuntimeError: This event loop is already running in python

Question or problem about Python programming:

I think I’m getting this error because my code calls asyncio.get_event_loop().run_until_complete(foo()) twice. Once from foo() and second time from function called by foo(). My question is then: why should this be a problem? Why should I even care that this loop is running?

There was an edit made to this question which, I think, obscured it (some people prefer to follow rules without understanding them, thus an “illegal” word was removed from the title). Unfortunately, this creates confusion.

I’m not surprised by the fact that the error is raised. I can trace it back to the asyncio source and see that the authors of this library wanted to do it this way, there’s no mystery there. The puzzling part is in the reason the authors of the library decided it’s illegal to ask from event loop to run some function to completion when the loop is already running.

We can reduce the problem to just two such calls, and through case analysis we will see that these are the three possibilities:

Now, is there any sane behavior which would address all three cases? To me, it is obvious that there is, or, perhaps are multiple sane behaviors possible here. For example:

Now, I can understand that this behavior may not be something that everyone would want. But, since this library decided to give programmers control over starting / stopping the event loop, it should also meet the consequences of such decisions. Making it an error to start the same loop multiple times precludes library code from ever doing this, which reduces the quality and usefulness of libraries utilizing asyncio (which is indeed the case with, for example, aiohttp).

How to solve the problem:

Solution 1:

I got the issue resolved by using the nest_async

pip install nest_asyncio

and adding below lines in my file.

import nest_asyncio
nest_asyncio.apply()

Solution 2:

Event loop running – is an entry point of your async program. It manages running of all coroutines, tasks, callbacks. Running loop while it’s running makes no sense: in some sort it’s like trying to run job executor from same already running job executor.

Since you have this question, I guess you may misunderstand a way how asyncio works. Please, read this article – it’s not big and gives a good introduction.

Upd:

There’s absolutely no problem in adding multiple things to be ran by event loop while this loop is already running. You can do it just by awaiting for it:

await coro()  # add coro() to be run by event loop blocking flow here until coro() is finished

or creating a task:

asyncio.ensure_future(coro())  # add coro() to be run by event loop without blocking flow here

As you can see you don’t need call event loop’s methods to make something being ran by it.

Event loop’s method such as run_forever or run_until_complete — are just a ways to start event loop in general.

run_until_complete(foo()) means: “add foo() to be ran by event loop and run event loop itself until foo() isn’t done”.

Solution 3:

I’m writing this down not to patronize, but to explain how we can handle the situation where simply queueing async functions and awaiting their results synchronously while the event loop is running, doesn’t work.

run_until_complete is not for running any number of arbitrary async functions synchronously, it is for running the main entry point of your entire async program. This constraint is not immediately apparent from the docs.

Since libraries like aiohttp will queue it’s own entry point to run as a server and block the loop’s synchronous operations using run_until_complete or run_forever, the event loop will already be running and you won’t be able to run independent synchronous operations on that event loop and wait for it’s result within that thread.

That being said, if you have to queue an async operation into a running event loop from within a sync context and get it’s result like a regular function, that may not be possible. Your best bet is to pass in a synchronous callback to be called once the async operation finishes. That will of course slow down your event loop.

Another way of handling the situation is to execute your code within startup and cleanup callbacks of the async http library you’re using. Here’s a sample of how you may accomplish this.

Solution 4:

Just add this bunch of code in the beginning

!pip install nest_asyncio
import nest_asyncio
nest_asyncio.apply()

Solution 5:

Using nest_asyncio didn’t work for me, because then aiohttp started complaining with

• RuntimeError: Timeout context manager should be used inside a task

Instead I decided to replace all calls to asyncio.run with calls to this asyncio_run:

def asyncio_run(future, as_task=True):
    """
    A better implementation of `asyncio.run`.

    :param future: A future or task or call of an async method.
    :param as_task: Forces the future to be scheduled as task (needed for e.g. aiohttp).
    """

    try:
        loop = asyncio.get_running_loop()
    except RuntimeError:  # no event loop running:
        loop = asyncio.new_event_loop()
        return loop.run_until_complete(_to_task(future, as_task, loop))
    else:
        nest_asyncio.apply(loop)
        return asyncio.run(_to_task(future, as_task, loop))


def _to_task(future, as_task, loop):
    if not as_task or isinstance(future, Task):
        return future
    return loop.create_task(future)

A secondary goal was to be able to think of asyncio.run as promise.resolve from the JS world, or Task.Wait from the .NET world.

Hope this helps!