Gunicorn from scratch

When you create a new Django project with django-admin startproject, it automatically creates a wsgi.py file for you. This is a module containing a WSGI application object called application that can be called by any type of WSGI server. Here is how this works.

The simplest WSGI application

WSGI: stands for Web Server Gateway Interface. It is an interface specification by which server and application communicate. The application provides a function which the server calls for each request:

def application(environ, start_response):
    ...

environ is a Python dictionary containing the CGI-defined environment variables plus a few extras.
start_response is a callback by which the application returns the HTTP headers.
```
start_response(status, response_headers)
```
status is an HTTP status string (e.g., “200 OK”)
response_headers is a list of 2-tuples, the HTTP headers in key-value format

The application function then returns an iterable of body chunks, that need to be bytes, e.g. [b"<html>Hello, world!</html>"]. So the simplest WSGI application can be:

def application(environ, start_response):
    start_response("200 OK", [])
    return [b"<html>Hello, world!</html>"]

Gunicorn

Normally during development you use python manage.py runserver to start a web server, but it is not the recommended way in production^[1]. Basically it is also a WSGI application that has some additional features such as reloading on code changes, but it has not gone through security audits or performance tests.

Gunicorn stands for ‘Green Unicorn’ and is a Python WSGI HTTP Server for UNIX. It has no dependencies and can be installed with pip or poetry:

poetry add gunicorn

Assuming you have a project structure like this:

.
├── manage.py
└── project
    ├── __init__.py
    ├── asgi.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

you can run

gunicorn project.wsgi:application

inside your project directory and it will spawn two processes that you can see when you call ps aux | grep gunicorn. One of them is the parent process that controlls all the workers, the other one is the process for the worker. By default it will only spawn one worker and listen to HTTP requests on 127.0.0.1:8000.

Configure Gunicorn

Gunicorn detects if there is any file called gunicorn.conf.py inside the directory where it is executed, or you can specify a different path with the -c flag. The simplest way is to place it next to your manage.py file:

.
├── gunicorn.conf.py    <--- create here
├── manage.py
└── project
    ├── __init__.py
    ├── asgi.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

Multiple workers

You can spawn multiple gunicorn processes that will serve your application to make use of the computing power of your server, based on the number of CPUs the machine has. To find out the number of processors on your UNIX machine, you can execute grep -c ^processor /proc/cpuinfo. Gunicorn suggests to run 2-4 x $(NUM_CORES) of workers^[2]. The reasoning behind it is that, while one of the processes is occupied with handling I/O or waiting for a response from the database, the other one can take over and handle a new request.

Here is how the gunicorn.conf.py file might look like:

import multiprocessing

workers = multiprocessing.cpu_count() * 2 + 1

Listen to different IPs

By default the Gunicorn process will listen for http request on 127.0.0.1:8000. You can configure Gunicorn to listen to different ports, or even to a public IP address. You can easily test this by finding out your computer’s IP address inside your local network with ifconfig and set the bind address in the gunicorn.conf.py file like this:

bind = "192.168.x.y:8000"

and navigate from your phone to your computer with this address. Ports smaller than 1024 require additional configuration.

In case you plan to use a proxy server like Nginx on the same machine, you might consider using a socket for proxying requests to your Gunicorn server.

bind = "unix:/path/to/myproject.sock"

This has slight performance advantages, since they can avoid some checks and operations (like routing)^[3].

You can even specify multiple bind addresses as a list, which is especially useful if you want to serve your application through HTTPS.

bind = [
    "192.168.x.y:80",
    "192.168.x.y:443"
]

Serving requests will require an SSL certificate, which deserves another post to be discussed in detail.

Run Gunicorn as a background process

To run the Gunicorn process in the background, you need to configure Supervisor or Systemd. There are plenty articles online that explain how to configure both of them with Gunicorn, but after doing the reasearch I consider Supervisor to be the more versatile and simpler one to configure, even though Systemd is probably already installed on your Linux distribution.

In the next post I describe how to run Gunicorn as a background process with both Supervisor and Systemd, so you should go ahead and make your own decision which one you like more.