What is Gatsby?

Gatsby is a framework built on React and GraphQL that can be used to build static or dynamic web applications. In this tutorial we are going to use the Gatsby framework to build the front end of a Todo app using GraphQL to query and mutate data from a decoupled Django back end API.

Django API

For the back end we are going to be using Django, Graphene, Postgresql to serve our GraphQL endpoints for our Todo app front end. You can clone the Django back end from my Gitlab repo here.

This Django app is built using Docker and Docker Compose which makes the process spinning up a fully functional back end API simple.

You will only need a basic knowledge of Git, Docker and Django to clone, build and spin up our Todo API where all our data will be stored.

If you are interested in how this Django API was built you can check out my seven part Django tutorial here.

You will need the following installed on your local host whether it be Windows or Mac:

Setting Up Django API

We start by cloning, building and configuring the Django API:

# Clone Django backend from Gitlab repo
git clone https://gitlab.com/ronleeson/todo-api-docker-django-graphql.git

# Rename directory to todo_app
mv todo-api-docker-django-graphql todo_app

cd todo_app

# If you are using VSCode launch IDE
code .

# Build Django app
docker-compose build server

# Spin up database
docker-compose up -d database

# Make migrations and migrate user db
docker-compose run server ./manage.py makemigrations
docker-compose run server ./manage.py migrate

# Spin up server
docker-compose up -d server

# Make and run migrations for users
docker-compose run server ./manage.py makemigrations
docker-compose run server ./manage.py migrate

# Create superuser
docker-compose run server ./manage.py createsuperuser

Once the app is up we should be able to access:

Our directory structure should look like so:

  • todo_app (directory root of the project)frontend (directory)Dockerfile
    • postgres-data
    • server (the Django Graphene app we created in Part 1)
    • docker-compose.yml
    • .env
    • .gitignore
    • Makefile

Now that we have the Django API up and running we can concentrate on setting up the Gatsby front end app.

# 1) Install Gatsby CLI on local computer
npm install -g gatsby-cli

# 2 Change directory to root of our project
cd /todo_app

# 3) Generate Gatsby starter site
gatsby new frontend https://github.com/gatsbyjs/gatsby-starter-hello-world

# 4) Remove the Gatsby git resources since we will use our git installation at the root of our project
rm -rf frontend/.git
rm -rf frontend/.gitignore

# 5) Create our frontend Dockerfile script
touch frontend/Dockerfile

# 6) Create a directory for Nginx which will serve our Gatsby site in production mode
mkdir nginx

# 7) Create a Dockerfile and nginx.conf files for Nginx
touch nginx/Dockerfile
touch nginx/nginx.conf

# 8) Update Docker Compose file adding frontend and nginx sevices
vi docker-compose.yml

# 9) Create a new Docker Compose file for development
touch docker-compose-dev.yml

This is how our current directory structure should look:

  • todo_app (directory root of the project)
    • frontend (directory)
      • Dockerfile
    • nginx (directory)
      • Dockerfile
      • nginx.conf
    • server (the Django Graphene app we created in Part 1)
    • docker-compose.yml
    • docker-compose-dev.yml
    • .env
    • .gitignore
    • Makefile

Populate Frontend Files

  1. We start by installing the Gatsby CLI on our local computer which is the main entry point for setting Gatsby up on our computer.
  2. Change directory to the root of our todo_app.
  3. We created a new Gatsby project using the Gatsby Hello World starter site which we renamed to frontend.
  4. When Gatsby generated its own git repo which we don’t need since we already have our git repo at the root of our project. We also deleted the .gitignore from the frontend directory.
  5. Create our frontend Dockerfile to configure the Gatsby env.
  6. Create a directory for nginx.
  7. Create a nginx/Dockerfile and nginx/nginx.conf file to configure Nginx server for our production build.
  8. Update the docker-compose.yml file by adding the frontend and nginx services. We will eventually be deploying our Gatsby build to production using nginx which will serve our site using SSR, (server side rendering), which comes with the Gatsby build. Nginx
  9. Create a new docker-compose-dev.yml for configuring our development environment.

Lets begin by populating frontend Dockerfile.

Frontend Dockerfile

# frontend/Dockerfile

# 1)
FROM node:16-alpine

# 2)
EXPOSE 8000

# 3)
WORKDIR /frontend

# 4)
RUN apk update && apk add python3 g++ make && rm -rf /var/cache/apk/*

# 5)
RUN npm install -g gatsby-cli

# 6)
COPY . .

# 7)
RUN npm install

# 8) Building Gatsby generating public directory that can be copied to Nginx service in production
CMD [ "gatsby", "build" ]
  1. Using node:16-alpine image we build our node environment. I’m specifying node version 16 here because Gatsby requires a minimum of node 14 for Gatsby 3.14.4 which is the latest version at the time of writing this blog. I found that Gatsby
  2. Open a port on the frontend container.
  3. Set the working directory with the WORKDIR Docker command. Doing this in one of the first layers is very important otherwise when we run npm install Gatsby will be installed at the root of the container instead of the working directory frontend. When we try running Gatsby commands such as Gatsby build, an error will be thrown saying that Gatsby is not installed. Declaring the WORKDIR early fixes this problem.
  4. Since Node.js is built with node-gyp which is written in Python we need Python installed. You can read more about this Python requirement for GYP here.
  5. Install Gatsby CLI.
  6. Copy the contents of the frontend directory on the host machine into the frontend container that we are building. Note that in step 3 when we set the working directory with WORKDIR, Docker creates the frontend directory for us and cd into the frontend directory so all the subsequent commands are executing in the frontend directory.
  7. Install all the packages defined in the package.json file.
  8. Using the Dockerfile CMD we define the default command to set Gatsby into production mode which will serve the production build from the /www/public.

Docker Compose

# docker-compose.yml

version: '3'
 
services:
    database:
        container_name: postgres
        image: postgres:latest
        environment:
            - POSTGRES_DB=$DATABASE_NAME
            - POSTGRES_USER=$DATABASE_USER
            - POSTGRES_PASSWORD=$DATABASE_PASSWORD
            - POSTGRES_PORT=$DATABASE_PORT
        volumes:
            #- ./server/initial.sql:/docker-entrypoint-initdb.d/initial.sql
            - ./server/db_backups/backup.sql:/docker-entrypoint-initdb.d/backup.sql
            - postgres-data:/var/lib/postgresql/data
 
    server:
        build: ./server/
        container_name: dj02
        working_dir: /var/www/server
        ports:
            - '$SERVER_PORT:$SERVER_PORT'
        volumes:
            - ./server/:/var/www/server
        command: python manage.py runserver 0.0.0.0:$SERVER_PORT
        environment:
            - DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
            - PGHOST=$DATABASE_HOST
        env_file:
            - .env
        depends_on:
            - database

    
    # 1)
    frontend:
        # 2)
        build:
            context: ./frontend
            dockerfile: Dockerfile
        # 3)
        ports:
            - 8000:8000
        # 4)
        volumes:
          - ./frontend/:/frontend
        # 5)
        container_name: frontend
        # 6)
        depends_on:
          - server

volumes:
    postgres-data:

With our additions we are defining the frontend service for the development mode of our project.

  1. Name of service.
  2. Tell docker-compose where the Dockerfile is for the frontend service.
  3. Map the outside port 8000 to port 8000 the container’s port.
  4. Map the /frontend files from the frontend container onto our host machine which allows us to edit and save files from our host machine dynamically updating the files in the frontend container and triggers a browser hot-reload.
  5. Give the container the name frontend
  6. Using depends_on tell the frontend to wait for the server to finish launching
Development Docker Compose Override

To run Gatsby in development mode we override the docker-compose file with the docker-compose-dev file.

# docker-compose-dev.yml
version: '3'

services:
  frontend:
    # 1)
    command: gatsby develop --host 0.0.0.0 --port 8000
  1. The command: develop –host 0.0.0.0 –port 8000 sets Gatsby in development mode

Build the Frontend

Now we can build the frontend container:

docker-compose build frontend
Notes on How I Built the Gatsby Frontend Dockerfile

I was not able to find much documentation on how to use Docker with Gatsby. Some of the Dockerfile solutions I did find didn’t work for me. I did find one solution by Ari Palo that did work but seemed a bit complex so I decided to write my own with the intention of serving the Gatsby build with Nginx as a separate service.

Building this Dockerfile was the most challenging part of this tutorial for me. I ran across many gotchas in the process. For example, Gatsby 3.14.4 will not work with the latest version of Node hence my use of node 16 which is compatible with Gatsby 3.14.4. As Gatsby keeps evolving this will probably change in the future.

Trouble Shooting Tips!

What proved invaluable for me in developing and sifting out all the errors was to use the Docker exec command to gain shell access to the frontend container and run each of the commands above in the shell of the container. This is the command I used to gain shell access to the running frontend container:

docker exec -it frontend ash

Running the commands manually in the container allowed me to catch two subtle issues that docker-compose logs frontend for the frontend container missed:

  1. The WORKDIR command has to run in one of the first layers so Gatsby will install in the frontend directory as opposed to the root of the container.
  2. While running npm install I would get the error that node-gyp could not find Python so I needed to install Python before running npm install.
  3. Manually running gatsby build directly in the container flagged two issues
    1. Gatsby requiring a minimum of node 14 for this version of Gatsby thus I was able through trial and error find the most recent version of node alpine that is compatible with Gatsby.
    2. Gatsby was not installed in the working frontend directory.

As I mentioned earlier, as Gatsby continues to evolve the issues I encountered will change so if you, dear reader, in the process of following this tutorial in the future run into problems, I encourage you to start with a bare bones version of a Dockerfile and manually run the each command incrementally so you can catch the errors as they occur with more verbose documentation than docker-compose logs frontend might give you.

Here’s an example of a bare-bones Dockerfile to begin with:

FROM node:16-alpine

EXPOSE 8000

WORKDIR /frontend

CMD tail -f /dev/null

Note the CMD tail -f /dev/null command. If we don’t have a Docker command specified the container will build itself and shut itself down so using CMD tail -f /dev/null keeps the container up and running allowing us shell access.

From here spin up the container and start manually running subsequent commands such as installing python, installing gatsby-cli, etc.

docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

# gain shell access to the container
# docker exec -it [name of container] ash
docker exec -it frontend ash

Once we are able to hit Gatsby’s development frontend with http://localhost:8000 we can move on with building the Nginx service which will serve our Gatsby production build.

Nginx Dockerfile
# nginx/Dockerfile

# 1)
FROM nginx:alpine

# 2)
EXPOSE 80

# 3)
COPY nginx.conf /etc/nginx/conf.d/default.conf
  1. We pull the nginx alpine image to build the Nginx service
  2. Expose port 80 of the Docker container
  3. Copy our nginx.conf file to the Docker container’s nginx conf.d directory where we tell Nginx where to serve the frontend build files from
Configure Nginx
# nginx/nginx.conf

server {
  root /frontend/public;

  location / {
    index index.html;
  }
}

Next we update docker-compose.yml with the nginx service:

Docker Compose
# docker-compose.yml

version: '3'

services:
  nginx:
    build:
      context: ./nginx
      dockerfile: Dockerfile
    volumes:
      - ./frontend/public:/frontend/public
    ports:
      - 8080:80
    depends_on: 
      - frontend
      
  database:
    container_name: postgres
    image: postgres:13
    env_file: 
      - .env 
    environment: 
      - POSTGRES_DB=$DATABASE_NAME
      - POSTGRES_USER=$DATABASE_USER
      - POSTGRES_PASSWORD=$DATABASE_PASSWORD
      - POSTGRES_PORT=$DATABASE_PORT
    volumes:
      - ./server/initial.sql:/docker-entrypoint-initdb.d/initial.sql
      - ./postgres-data:/var/lib/postgresql/data 

  server:
    container_name: server
    build: ./server/
    ports:
      - $SERVER_PORT:$SERVER_PORT
    volumes:
      - ./server/:/server
    command: python manage.py runserver 0.0.0.0:$SERVER_PORT
    environment: 
      # DJANGO_SETTINGS_MODULE necessary for using pytest
      - DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
      - PGHOST=$DATABASE_HOST
    env_file: 
      - .env
    depends_on: 
      - database 

  frontend:
    build: ./frontend
    ports:
      - 8000:8000
    volumes:
      - ./frontend/src:/frontend/src
    container_name: frontend
    depends_on:
      - server
    
volumes:
  postgres-data:

With our additions we are defined the nginx service which will use in production mode of our project. More on that later.

Development Docker Compose Override

To run Gatsby in development mode we override the docker-compose file with the docker-compose-dev file.

# docker-compose-dev.yml
version: '3'

services:
  frontend:
    # 1)
    command: gatsby develop --host 0.0.0.0 --port 8000
  1. The command: develop –host 0.0.0.0 –port 8000 sets Gatsby in development mode

Booting Up the Development Environment for Gatsby Frontend

To bring Gatsby up in development mode we need only run:

docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

… which should allow you to hit http://localhost:8000 to view the frontend index page.

If you were to edit the copy in the frontend index.js file:

vi frontend/src/pages/index.js
# Edit the copy, close and save

The changes should show up immediately in the browser without having to do a manual refresh of your browser. Is this not cool?

You should also be able to access the backend server we built in Part 1 at http://localhost:5555/admin:

If you get an error on this page or you can’t log in you need to run Django migrations and create a super user:

docker-compose run server ./manage.py makemigrations
docker-compose run server ./manage.py migrate
docker-compose run server ./manage.py createsuperuser

Booting Up the Production Environment for Gatsby Frontend and the Django Backend Server

To boot up, build and deploy run:

docker-compose build frontend
docker-compose up -d

This command invokes the frontend Dockerfile which will run the Gatsby build command to compile the site into files that can be delivered to a web browser later.

You should be able to access the production version being served by Nginx at http://localhost:8080.

As well you should be able to access the Django server as well at http://localhost:5555/admin.

To power down the production environment run:

docker-compose down

Trouble Shooting

If you have trouble booting up all the docker services successfully with docker-compose up -d or the development override command docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d you can always try booting up each service separately because sometimes the Docker Compose depends_on attribute doesn’t always work so the server may boot up before Postgres service is ready to accept connections.

docker-compose up -d database
docker-compose up -d server
docker-compose up -d nginx

# for the Gatsby development server
docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

After running these commands you should be able to hit each of the different services this app is using via the browser:

Since the docker compose commands are a bit cumbersome we can update our Makefile with shortcut commands for the frontend development and production processes:

# Makefile


pytest:
  docker-compose run server pytest

test:
  docker-compose run server ./manage.py test

build:
  docker-compose build

reboot:
  docker-compose down && docker-compose up -d

prod:
  docker-compose up -d

dev:
  docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

prod-down:
  docker-compose down

dev-down:
  docker-compose -f docker-compose.yml -f docker-compose-dev.yml down

up-non-daemon:
  docker-compose up

start:
  docker-compose start

stop:
  docker-compose stop

down:
  docker-compose down

restart:
  docker-compose stop && docker-compose start

restart-frontend:
  docker-compose stop frontend && docker-compose start frontend

restart-server:
  docker-compose stop server && docker-compose start server

shell-server:
  docker exec -ti server bash

shell-frontend:
  docker exec -ti frontend bash

shell-db:
  docker exec -ti postgres bash

log-server:
  docker-compose logs server

log-db:
  docker-compose logs postgres

collectstatic:
  docker exec server /bin/sh -c "python manage.py collectstatic --noinput"

migrations:
  docker exec server /bin/sh -c "python manage.py makemigrations; python manage.py migrate"

Now if we want to boot up the production environment we need only run:

make prod

To power down simply:

make prod-down

To power up the development environment we run:

make dev

To power down the development environment:

make dev-down

Now that we have both development and production environments for Gatsby operational as well as the Django Graphene server operational — we can push on to the actual development of building out our frontend user interface.

How do we know we haven’t broken anything with our server while setting up our frontend development environment. We could manually test the Django admin, and / or run ALL the Django queries and mutations in Django’s GraphiQL UI. Ugh!

Testing the Django Graphene Server

But have we broken anything in the backend in the process of building the frontend? Easy enough to find out. We need only to run our unit tests.

docker-compose run server pytest

Voila! Our tests will tell us what and where if we have any failing tests. In my case all my tests passed. This gives me enormous confidence moving forward with the development of our frontend site.

We will continue in our next chapter using TDD for Gatsby.

Part 2 >