No bullshit IT
Back to blog
•
14 min read

Test client in python tests

fastpi django testing python wsgi asgi
Table of Contents

It always begins with other problem

Look, it is always like this. Someone asks one question, you find it interesting. One question leads to another, and before you know it, boom! You’re writing a blog post about something completely different.

In this case, the problem we started with was, believe it or not, “how do I clean up the database between tests?” Easy, just drop it and recreate it. Like you always did. Never thought more about it.

You give this as an answer to your friend, quick easy win, you’ll remind him next time you are in the pub he owes you a beer for something that he could’ve googled in 30 seconds.

“Ok, that makes sense. I come from the Rails world where it is done automatically”. Django does that as well. And this intrusive thought starts to settle in. Takes off its coat, sits comfortably on the couch, and whispers in your ear “Exactly Maciek, how the fuck does Django do that?”

But that will be the next blog post. Let’s set the stage first and talk about something we rarely think about. The test client. The one you call with client.get in tests.

What test client is?

We will use FastAPI as an example. Don’t worry if you don’t know FastAPI it won’t matter, I promise.

So this is an example from official fastAPI docs

from fastapi import FastAPI
from fastapi.testclient import TestClient

app = FastAPI()


@app.get("/")
async def read_main():
    return {"msg": "Hello World"}


client = TestClient(app)


def test_read_main():
    response = client.get("/")
    assert response.status_code == 200
    assert response.json() == {"msg": "Hello World"}

source

We can immediately say things about what the testing client is, without deeper thought. Maybe they’re true, maybe not. Let’s look professional and write something down. We’ll verify this later.

  1. the client is used to make requests to our application in tests
  2. we don’t have to provide app url
  3. it looks like a normal HTTP client

Very important side note first

Very important side note

Look, it has to happen. We’ll dig into some source code. Don’t be afraid. I know, I know it is scary.

Jokes aside, for quite a long time I was afraid of source code. I thought it is too complicated, too much to understand, and I won’t be able to do it. I’m to dump to understand code that is good enough to be open source. Really. It can be daunting task to clone the repo and start reading it. I’d better find something on stack overflow!

You know what worst can happen? You won’t understand it. Each time you dive into someone else source code you grow. Grow by getting more comfortable in new environments. You get to see new patterns and solution. You find an answer yourself to a question instead of reading this blog post. You start to know what to look for.

Some will say that figuring stuff out yourself is the best way to learn. Maybe. Being able not know something, ask question and figure that out is an awesome feeling.

So much of our time as developers is spent on reading code. Practice this skill. Many times it is much easier to read source then to find someone who know what code does and still works at your company. I hope by reading this you’ll see that it is not that scary.

FastAPI test client

The docs say FastAPI’s test client is a wrapper around httpx. httpx is an HTTP client (go look at its page, you won’t find anything about testing at first glance there). We’re getting somewhere, it is just an HTTP client.

Also, the FastAPI docs say that its test client is just a wrapper over Starlette’s one. So we should look there.

from starlette.testclient import TestClient as TestClient  # noqa

source

Reading Starlette’s test client source code may seem overwhelming. 800 lines of code, some ASGI, async stuff, what the hell is anyio.streams.stapled even? Let’s try to make more sense of it and dismantle it a bit.

You quickly find tha there are plenty of stuff related to websocket. We don’t care right now. Let’s skip that.

Start with TestClient class. It has methods like get, post, put, delete, patch, head, options which you are familiar with if ever used any http client.

def get(  # type: ignore[override]
    self,
    url: httpx._types.URLTypes,
    *,
    params: httpx._types.QueryParamTypes | None = None,
    headers: httpx._types.HeaderTypes | None = None,
    cookies: httpx._types.CookieTypes | None = None,
    auth: httpx._types.AuthTypes | httpx._client.UseClientDefault = httpx._client.USE_CLIENT_DEFAULT,
    follow_redirects: bool | httpx._client.UseClientDefault = httpx._client.USE_CLIENT_DEFAULT,
    timeout: httpx._types.TimeoutTypes | httpx._client.UseClientDefault = httpx._client.USE_CLIENT_DEFAULT,
    extensions: dict[str, Any] | None = None,
) -> httpx.Response:
    return super().get(
        url,
        params=params,
        headers=headers,
        cookies=cookies,
        auth=auth,
        follow_redirects=follow_redirects,
        timeout=timeout,
        extensions=extensions,
    )

source

Looks scary right? Right. Is it? Not really. Take a closer look. It just calls super().get(...) which is httpx.Client.get. Nothing more. The only thing I can think of for why it is done this way is to make sure that if httpx.Client.get changes, the TestClient.get interface stays the same.

We can also ignore __enter__ and __exit__ methods as we don’t care about the context manager, it wasn’t in our simple example. The only interesting method left in TestClient is __init__.

def __init__(self, app: ASGIApp, base_url: str = "http://testserver", client: tuple[str, int] = ("testclient", 50000), ...) -> None:  # abbreviated

    # abbreviated

    self.app = asgi_app
    self.app_state: dict[str, Any] = {}
    transport = _TestClientTransport(
        self.app,
        portal_factory=self._portal_factory,
        raise_server_exceptions=raise_server_exceptions,
        root_path=root_path,
        app_state=self.app_state,
        client=client,
    )
    if headers is None:
        headers = {}
    headers.setdefault("user-agent", "testclient")
    super().__init__(
        base_url=base_url,
        headers=headers,
        transport=transport,
        follow_redirects=follow_redirects,
        cookies=cookies,

source

I cut out a couple of lines here and there, for brevity. Again, it mostly passes stuff from init to the super() call. It adds its own transport!

Transport

So what is the transport? Httpx docs is good place to start.

HTTPX’s Client also accepts a transport argument. This argument allows you to provide a custom Transport object that will be used to perform the actual sending of the requests.

That is it. So the HTTP client knows what HTTP methods are, how to handle cookies, headers, etc. Transport, on the other hand, knows nothing about that. It is only interested in how to send requests and receive responses.

Wait, what? Isn’t HTTP always sent over TCP or something like that? Well yes, normally it is. This is a separation of concerns. Like in real life transport. You know you have to put the proper sticker on your package to send it to a friend, but you don’t care how it is delivered. That is what transport does. Don’t like comparisons to real life? Ok.

Transport is responsible for handling socket connections, TLS, timeouts. Want to use HTTP/2? Just change the transport layer. The HTTP client should not care about that.

class BaseTransport:

    def handle_request(self, request: Request) -> Response:
        raise NotImplementedError(
            "The 'handle_request' method must be implemented."
        )  # pragma: no cover

    def close(self) -> None:
        pass

source

This is the source code of httpx’s BaseTransport. It checks out with what we talked about a minute ago. We have to implement a method that takes a request and returns a response.

WSGI

Our test client takes ASGI app as an argument and passes it to the _TestClientTransport.

class TestClient(httpx.Client):
    def __init__(
            self,
            app: ASGIApp,
    ...

    transport = _TestClientTransport(self.app, ...)

source

So what is ASGI? Let’s take a step back to look at WSGI, it will be easier to understand because ASGI is just an adaptation for the async world (kinda).

The idea of WSGI is to provide an interface between the web server and web application. Separation of concerns again. Let the framework (like FastAPI) handle application logic. It doesn’t have to know anything about sockets, connections, SSL, ports, IPs and any of that stuff. It only cares about getting a request and returning a response.

WSGI is not magic, it is really simple. Let’s look at the simplest WSGI app.

def app(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-type', 'text/plain')]
    start_response(status, response_headers)
    return ['Hello world!\n'] # can also yield

So, it is a function that takes environ and start_response. environ is a dictionary with request data, like headers, method, path, etc. It has well defined keys in it. start_response is a function and well, it start response. To return response, we start_response with status and headers and return iterable with bytes that are body of http response.

For more info you should go to the WSGI specs PEP-3333

Why is it split into two parts instead of just returning status, headers and body at once? This way the web server can start sending the response to the client before the application is done processing the request. To be honest I don’t think you’ll encounter it that much in the wild. One example is returning a very big file. A WSGI app can read a file chunk by chunk from disk and yield each chunk to the client.

There are WSGI equivalents in other languages. For example, Ruby has rack which is widely used.

class HelloWorld
  def call (env)
    [200, {"content-type" => "text/html; charset=utf-8"}, ["Hello World"]]
  end
end

run HelloWorld.new

source

As you can see, Ruby ditched the separate start_response and return value.

Did you ever use any of these:

$ gunicorn -w 4 main:app
$ uvicorn main:app

What you did is you pointed the web server (gunicorn/uvicorn) to a WSGI or ASGI app.

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

That is the same, but not from CLI.

$ fastapi run main.py

Same thing, look at the source code here

Even python manage.py runserver operates on a WSGI application but it is not that straightforward to show with one URL. But don’t take my word for that! Start here and figure that out on your own!

ASGI

So ASGI is just a way to do the same thing in the async world. It is not that complicated, it is just a different interface.

async def application(scope, receive, send):
    event = await receive()
    ...
    await send({"type": "websocket.send", ...})

source

It is based on async send and receive functions that pass events instead of the return value of a function. ASGI is a superset of WSGI so ASGI can handle a WSGI app with some adapter layer. I don’t feel like you need to know more right now. Having a grasp of WSGI and the concept of separation is enough as this post is getting long already.

More on ASGI here: https://asgi.readthedocs.io/en/latest/index.html

Back to the transport

We have all the pieces now:

  • Transport knows how to send requests and receive responses
  • Starlette’s test client implements its own transport which takes an ASGI app as an argument
  • An ASGI app can handle requests and return responses

So the test client is like any other ordinary HTTP client. The only difference is that it doesn’t use an address and port to communicate with the server. It implements a transport that communicates with the ASGI app directly. Not over a network hop.

Let’s go back to our initial assumptions from the beginning.

  • The client is used to make requests to our application in tests
  • It looks like a normal HTTP client

Yes! That is the idea. And it is just a normal HTTP client. Heck, if we want, we can start our app (using Uvicorn for example), use a normal HTTP client in tests, point it to our server and it will work exactly the same!

  • we don’t have to provide app url

Now we know exactly why. There is no URL, there is no address because there is no web server in between. The custom transport allows us to communicate directly between the HTTP client and our app (using ASGI). One side note, it is not really true that there’s no URL, because many features rely on the Host header, but in this context it can mostly stay hardcoded. And it is here

With all this knowledge I strongly encourage you to go to https://github.com/encode/starlette/blob/master/starlette/testclient.py and check out how the custom transport communicates with the ASGI app. Don’t worry about that portal stuff which seems out of this world. The only thing you need to know is that calling through the portal solves the problem with multiple event loops.

Notice in ASGI there is a way to get extra data from inside the app that is not strictly an HTTP response. This is a so-called ASGI extension.

elif message["type"] == "http.response.debug":
    template = message["info"]["template"]
    context = message["info"]["context"]

No magic?

Yup, not at all. But I want to emphasize one thing.

Since there is no magic, your request is processed in exactly the same way it would be if you used a normal HTTP client, curl, wget or browser. For FastAPI, your dependencies are resolved in the same manner as in a normal request. That means you have two separate database connections, one that you created for the test, and one that the app created.

This is the reason why you can’t just rollback the transaction you used in the test and have changes done by the app rolled back. Also the reason why you have to commit the transaction in the test before calling the app endpoint in tests - so the app can see changes done by the other transaction, the one in the test.

Now it may be a bit clearer why even though the problem started as “database in tests” it is connected to test clients.

Django test client

Now we have a strong understanding of how the simplest test client works, we can dive into the Django test client. But this is material for the next blog post :)

Don’t expect anything vastly different there, it uses a WSGI client too. It will allow us to start digging deeper to figure out how Django can automatically revert all database changes.

Disclaimer: at the moment of writing this post I didn’t check if my hunch on how it is done is correct. If not I’ll revise this post with short summary what is different in Django’s approach.

Summary

Quite a journey, huh? We started with question what is this testing client and went trough http clients, transport, WSGI and ASGI.

But the more important thing is we figured it out by ourselves by reading source code!

Key takeaways:

  • in FastAPI, the client used in tests is just a normal HTTP client
  • an HTTP client doesn’t have to send requests over the network
  • transport is the layer that is responsible for delivering messages, the protocol client is responsible for what goes in the message
  • WSGI is the interface between the web server and application
  • we can clone a repo from GitHub and check what is going on inside
  • by reading source code we pick up many things along the way
SQLAlchemy sessions in FastAPI