Setup a Web Server with automatic HTTPS¶

Introduction¶

In this tutorial we will cover how to setup a Virtual Machine on Jetstream 2 with the Web Server Caddy to serve either static HTML files or a Django application via HTTPS using SSL certificates automatically generated via Let’s Encrypt.

Caddy is a relatively new Web Server which is significantly easier to setup and use than NGINX or Apache, it also has automatic HTTPS support built-in, therefore it can automatically request and renew SSL certificates from Let’s Encrypt for any domain, either the domain automatically supplied by Jetstream or another domain whose DNS a user can manage.

The simpler application is to serve static HTML pages, which could be created either directly or by a static website generator tool like Hugo or JupyterBook.

The next step is to serve dynamic applications, in this tutorial we take as an example a Python web application developed with the Django web framework, however a similar setup can easily be adapted to other types of applications.

Create a Ubuntu 22 Virtual Machine¶

First of all, login to Exosphere and create a Ubuntu 22.04 Instance, at least a small size. In the “Advanced Options”, confirm that “Install operating system updates?” is Yes and disable Guacamole.

Wait for the instance to be online, click on it in Exosphere to find the address, which should be of the form:

instancename.xxx0000000.projects.jetstream-cloud.org

where instancename is the name you gave to the instance and xxx000000 is the allocation id.

Now you should be able to SSH to the instance from your local machine using:

ssh ubuntu@instancename.xxx0000000.projects.jetstream-cloud.org

Install the web server¶

Follow the instructions to install Caddy on Ubuntu. You can copy-paste all the lines at once.

At this point, the web server should already be running, point your browser to:

http://instancename.xxx0000000.projects.jetstream-cloud.org

Notice this is not https yet, you should see the Caddy welcome page.

Configure a custom domain¶

If you have a custom domain for your website, you first need to go in the DNS configuration and create a CNAME record that points to instancename.xxx0000000.projects.jetstream-cloud.org. If your domain is yourdomain.com, you can create a CNAME named www that points to instancename.xxx0000000.projects.jetstream-cloud.org.

Once the CNAME is setup, it could take a few hours to propagate. After a while you should be able to connect to:

http://www.yourdomain.com

and see the Caddy welcome page.

Enable HTTPS¶

Next we would like to secure the connection using the “Automatic HTTPS” capability of Caddy. We need to edit the Caddy configuration file:

/etc/caddy/Caddyfile

and replace :80 with your domain name, i.e. instancename.xxx0000000.projects.jetstream-cloud.org or www.yourdomain.com.

Then reload the configuration with:

sudo systemctl reload caddy

Caddy takes care automatically of handling the certificates, now connect again with your browser and you should be redirected to the HTTPS version of the Caddy welcome page.

Serve static files¶

Caddy is already set up to serve static files from the /usr/share/caddy directory. Confirm by checking that when you run:

cat /etc/caddy/Caddyfile` 

It looks something like this:

instancename.xxx0000000.projects.jetstream-cloud.org {
    # Set this path to your site's directory.
    root * /usr/share/caddy

    # Enable the static file server.
    file_server

    # Another common task is to set up a reverse proxy:
    # reverse_proxy localhost:8080

    # Or serve a PHP site through php-fpm:
    # php_fastcgi localhost:9000
}

The important lines are root * /usr/share/caddy and fileserver.

Check the contents of /usr/share/caddy:

ls -l /usr/share/caddy

You should see an index.html file owned by root:

-rw-r--r-- 1 root root 18630 Dec  8 00:28 index.html

Change the ownership of the index.html file to the ubuntu user:

sudo chown ubuntu:ubuntu /usr/share/caddy/index.html

Check permissions:

ls -l /usr/share/caddy

It should now look something like this:

-rw-r--r-- 1 ubuntu ubuntu 18630 Dec  8 00:28 index.html

This is owned by ubuntu so we can modify it without sudo and is globally readable so the caddy user can access it.

Next let’s update the index.html file:

echo "<h1>Works</h1>" > /usr/share/caddy/index.html

Finally check via browser. We should now only see Works instead of the Caddy welcome page.

Deploy a Django App¶

Django is the most popular Python web framework, we can deploy it in production using Caddy and Gunicorn.

First of all let’s use the Django version included in Ubuntu (otherwise we could create a virtualenv or a conda environment:

sudo apt install python-is-python3 python3-django 

Next we create the base folder for this application:

sudo mkdir /var/www/djangoapp
sudo chown ubuntu:ubuntu /var/www/djangoapp

I have created a very simple Django Hello World app suitable for testing, but any Django app should work, notice that for a production instance, we will also want to install PostgreSQL as back-end for Django, but that is not covered here.

cd /var/www/djangoapp
git clone https://github.com/zonca/django-hello-world mysite

This app only has a single view in the root URL that prints Hello World and a static file.

Edit /var/www/djangoapp/mysite/mysite/settings.py, set your domain in ALLOWED_HOSTS, and create a long random string in SECRET_KEY.

Note: You can generate a random string using the following command:

openssl rand -hex 30

Next we should collect all the static files (mostly images and CSS from Django Admin) so that they can be handled by the web server directly:

cd /var/www/djangoapp/mysite
python manage.py collectstatic

The most complex part of the setup is to configure Gunicorn, which is a pure Python HTTP server that can serve our Django website securely and efficiently.

sudo apt install gunicorn

The Gunicorn setup comes from the DigitalOcean tutorial, check there for more details.

In summary we create 2 files related to systemd so that it can manage the Gunicorn process.

/etc/systemd/system/gunicorn.socket contains:

[Unit]
Description=gunicorn socket

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

While /etc/systemd/system/gunicorn.service:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/var/www/djangoapp/mysite
ExecStart=gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          mysite.wsgi:application

[Install]
WantedBy=multi-user.target

Finally we enable the service and check that it works properly, see the Digital Ocean post for troubleshooting:

sudo systemctl start gunicorn.socket
sudo systemctl enable gunicorn.socket
sudo systemctl status gunicorn.socket

The last step is to configure Caddy (editing /etc/caddy/Caddyfile) to serve both static files and reverse-proxy Gunicorn:

instancename.xxx000000.projects.jetstream-cloud.org {
        root * /var/www/djangoapp/mysite

        route {
                file_server /static/*
                reverse_proxy unix//run/gunicorn.sock
        }
}

Then reload the configuration with:

sudo systemctl reload caddy

When you reload your browser on the site you should see:

Hello World ✔️