Setting Up Our Dev Environment

Note that we are building on the Django Graphene project from Part 1 which will provide our backend for the Gatsby frontend to interface with.

This chapter on setting up a Dockerized Gatsby environment is based on Aripalo’s elegant solution on Github.

Setting Up the Directory Structure

# 1) Install Gatsby CLI on local computer
npm install -g gatsby-cli

# 2) Generate Gatsby starter site
gatsby new frontend https://github.com/gatsbyjs/gatsby-starter-hello-world

# 3) Remove the Gatsby git resources since we will use our git installation at the root of our project
rm -rf frontend/.git
rm -rf frontend/.gitignore

# 4) Remove node_modules from local computer since we will be installing the node modules on the Docker frontend container
rm -rf frontend/node_modules

# 5) Create our frontend Dockerfile script
touch frontend/Dockerfile

# 6) Create a directory for Nginx
mkdir nginx

# 7) Create a Dockerfile and nginx.conf files for Nginx
touch nginx/Dockerfile
touch nginx/nginx.conf

# 8) Create a new Docker Compose files for production and development
touch docker-compose.yml
touch docker-compose-dev.yml

# 9) Initialize git repository
git init

This is how our current directory structure should look:

todo_gatsby-django-graphene (directory root of the project)
- frontend (directory)
  - Dockerfile
- nginx (directory)
  - Dockerfile
  - nginx.conf
- server (the Django Graphene app we created in Part 1)
- docker-compose.yml
- docker-compose-dev.yml
- .env
- .gitignore
- Makefile

Populate Frontend Files

Our goal is to build a todo application with Gatsby as our frontend which will interface with the Django Graphene server we built in Part 1 of this tutorial. We start by installing the Gatsby CLI on our local computer which is the main entry point for setting Gatsby up on our computer.
We created a new Gatsby project using the Gatsby Hello World starter site which we renamed to frontend.
When Gatsby generated its own git repo which we don’t need since we already have our git repo at the root of our project. We also deleted the .gitignore from the frontend directory.
Create our frontend Dockerfile to configure the Gatsby env.
Create a directory for nginx.
Create a nginx/Dockerfile and nginx/nginx.conf file to configure Nginx server for our production build.
Create two Docker Compose files, one for deploying our Gatsby build to nginx which will serve our site using SSR, (server side rendering), which comes with the Gatsby build.
Initialize our git repository for this project.

Lets begin by populating frontend Dockerfile.

Frontend Dockerfile

# frontend/Dockerfile

# 1)
FROM node:16-alpine

# 2)
EXPOSE 8000

# 3)
WORKDIR /frontend

# 4)
RUN apk update &amp;&amp; apk add python3 g++ make &amp;&amp; rm -rf /var/cache/apk/*

# 5)
RUN npm install -g gatsby-cli

# 6)
COPY . .

# 7)
RUN npm install

# 8) Building Gatsby generating public directory that can be copied to Nginx service in production
CMD [ "gatsby", "build" ]

Using node:16-alpine image we build our node environment. I’m specifying node version 16 here because Gatsby requires a minimum of node 14 for Gatsby 3.14.4 which is the latest version at the time of writing this blog. I found that Gatsby
Open a port on the frontend container.
Set the working directory with the WORKDIR Docker command. Doing this in one of the first layers is very important otherwise when we run npm install Gatsby will be installed at the root of the container instead of the working directory frontend. When we try running Gatsby commands such as Gatsby build, an error will be thrown saying that Gatsby is not installed. Declaring the WORKDIR early fixes this problem.
Since Node.js is built with node-gyp which is written in Python we need Python installed. You can read more about this Python requirement for GYP here.
Install Gatsby CLI.
Copy the contents of the frontend directory on the host machine into the frontend container that we are building. Note that in step 3 when we set the working directory with WORKDIR, Docker creates the frontend directory for us and cd into the frontend directory so all the subsequent commands are executing in the frontend directory.
Install all the packages defined in the package.json file.
Using the Dockerfile CMD we define the default command to set Gatsby into production mode which will serve the production build from the /www/public.

Now we can build the frontend container:

docker-compose build frontend

Notes on How I Built the Gatsby Frontend Dockerfile

I was not able to find much documentation on how to use Docker with Gatsby. Some of the Dockerfile solutions I did find didn’t work for me. I did find one solution by Ari Palo that did work but seemed a bit complex so I decided to write my own with the intention of serving the Gatsby build with Nginx as a separate service.

Building this Dockerfile was the most challenging part of this tutorial for me. I ran across many gotchas in the process. For example, Gatsby 3.14.4 will not work with the latest version of Node hence my use of node 16 which is compatible with Gatsby 3.14.4. As Gatsby keeps evolving this will probably change in the future.

Trouble Shooting Tips!

What proved invaluable for me in developing and sifting out all the errors was to use the Docker exec command to gain shell access to the frontend container and run each of the commands above in the shell of the container. This is the command I used to gain shell access to the running frontend container:

docker exec -it frontend ash

Running the commands manually in the container allowed me to catch two subtle issues that docker-compose logs frontend for the frontend container missed:

The WORKDIR command has to run in one of the first layers so Gatsby will install in the frontend directory as opposed to the root of the container.
While running npm install I would get the error that node-gyp could not find Python so I needed to install Python before running npm install.
Manually running gatsby build directly in the container flagged two issues
1. Gatsby requiring a minimum of node 14 for this version of Gatsby thus I was able through trial and error find the most recent version of node alpine that is compatible with Gatsby.
2. Gatsby was not installed in the working frontend directory.

As I mentioned earlier, as Gatsby continues to evolve the issues I encountered will change so if you, dear reader, in the process of following this tutorial in the future run into problems, I encourage you to start with a bare bones version of a Dockerfile and manually run the each command incrementally so you can catch the errors as they occur with more verbose documentation than docker-compose logs frontend might give you.

Here’s an example of a bare-bones Dockerfile to begin with:

FROM node:16-alpine

EXPOSE 8000

WORKDIR /frontend

CMD tail -f /dev/null

Note the CMD tail -f /dev/null command. If we don’t have a Docker command specified the container will build itself and shut itself down so using CMD tail -f /dev/null keeps the container up and running allowing us shell access.

From here spin up the container and start manually running subsequent commands such as installing python, installing gatsby-cli, etc.

docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

# gain shell access to the container
# docker exec -it [name of container] ash
docker exec -it frontend ash

Once we are able to hit Gatsby’s development frontend with http://localhost:8000 we can move on with building the Nginx service which will serve our Gatsby production build.

Nginx Dockerfile

# nginx/Dockerfile

# 1)
FROM nginx:alpine

# 2)
EXPOSE 80

# 3)
COPY nginx.conf /etc/nginx/conf.d/default.conf

We pull the nginx alpine image to build the Nginx service
Expose port 80 of the Docker container
Copy our nginx.conf file to the Docker container’s nginx conf.d directory where we tell Nginx where to serve the frontend build files from

Configure Nginx

# nginx/nginx.conf

server {
  root /frontend/public;

  location / {
    index index.html;
  }
}

Next we update docker-compose.yml with our new frontend:

Docker Compose

# docker-compose.yml

version: '3'
 
services:
    database:
        container_name: postgres
        image: postgres:latest
        environment:
            - POSTGRES_DB=$DATABASE_NAME
            - POSTGRES_USER=$DATABASE_USER
            - POSTGRES_PASSWORD=$DATABASE_PASSWORD
            - POSTGRES_PORT=$DATABASE_PORT
        volumes:
            #- ./server/initial.sql:/docker-entrypoint-initdb.d/initial.sql
            - ./server/db_backups/backup.sql:/docker-entrypoint-initdb.d/backup.sql
            - postgres-data:/var/lib/postgresql/data
 
    server:
        build: ./server/
        container_name: dj02
        working_dir: /var/www/server
        ports:
            - '$SERVER_PORT:$SERVER_PORT'
        volumes:
            - ./server/:/var/www/server
        command: python manage.py runserver 0.0.0.0:$SERVER_PORT
        environment:
            - DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
            - PGHOST=$DATABASE_HOST
        env_file:
            - .env
        depends_on:
            - database

    
    # 1)
    frontend:
        # 2)
        build:
            context: ./frontend
            dockerfile: Dockerfile
        # 3)
        ports:
            - 8000:8000
        # 4)
        volumes:
          - ./frontend/:/frontend
        # 5)
        container_name: frontend
        # 6)
        depends_on:
          - server

volumes:
    postgres-data:

With our additions we are defining the frontend service for the development mode of our project.

Name of service.
Tell docker-compose where the Dockerfile is for the frontend service.
Map the outside port 8000 to port 8000 the container’s port.
Map the /frontend files from the frontend container onto our host machine which allows us to edit and save files from our host machine dynamically updating the files in the frontend container and triggers a browser hot-reload.
Give the container the name frontend
Using depends_on tell the frontend to wait for the server to finish launching

Development Docker Compose Override

To run Gatsby in development mode we override the docker-compose file with the docker-compose-dev file.

# docker-compose-dev.yml
version: '3'

services:
  frontend:
    # 1)
    command: gatsby develop --host 0.0.0.0 --port 8000

The command: develop –host 0.0.0.0 –port 8000 sets Gatsby in development mode

Booting Up the Development Environment for Gatsby Frontend

To bring Gatsby up in development mode we need only run:

docker-compose -f docker-compose.yml -f docker-compose-dev.yml up

… which should allow you to hit http://localhost:8000 to view the frontend index page.

If you were to edit the copy in the frontend index.js file:

vi frontend/src/pages/index.js
# Edit the copy, close and save

The changes should show up immediately in the browser without having to do a manual refresh of your browser. Is this not cool?

You should also be able to access the backend server we built in Part 1 at http://localhost:5555/admin:

If you get an error on this page or you can’t log in you need to run Django migrations and create a super user:

docker-compose run server ./manage.py makemigrations
docker-compose run server ./manage.py migrate
docker-compose run server ./manage.py createsuperuser

Booting Up the Production Environment for Gatsby Frontend and the Django Backend Server

To boot up, build and deploy run:

docker-compose build frontend
docker-compose up -d

This command invokes the frontend Dockerfile which will run the Gatsby build command to compile the site into files that can be delivered to a web browser later.

You should be able to access the production version being served by Nginx at http://localhost:8080.

As well you should be able to access the Django server as well at http://localhost:5555/admin.

To power down the production environment run:

docker-compose down

Trouble Shooting

If you have trouble booting up all the docker services successfully with docker-compose up -d or the development override command docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d you can always try booting up each service separately because sometimes the Docker Compose depends_on attribute doesn’t always work so the server may boot up before Postgres service is ready to accept connections.

docker-compose up -d database
docker-compose up -d server
docker-compose up -d nginx

# for the Gatsby development server
docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

After running these commands you should be able to hit each of the different services this app is using via the browser:

Production frontend: http://localhost:8080
Development frontend, (if using the docker compose override): http://localhost:8000
Django admin: http://localhost:5555/admin
Django graphql playground: http://localhost:5555/graphql

Since the docker compose commands are a bit cumbersome we can populate our Makefile with shortcut commands for development and production processes:

# Makefile

dev:
  docker-compose up -d 

prod:
  docker-compose -f docker-compose.yml -f docker-compose-prod.yml up -d

prod-down:
  docker-compose down --remove-orphans

# Production Build
prod-build:
  docker-compose -f docker-compose.yml -f docker-compose-prod.yml build

Now if we want to boot up the production environment we need only run:

make prod

To power down simply:

make prod-down

To power up the development environment we run:

make dev

To power down the development environment:

make dev-down

Now that we have both development and production environments for Gatsby operational as well as the Django Graphene server operational — we can push on to the actual development of building out our frontend user interface.

How do we know we haven’t broken anything with our server while setting up our frontend development environment. We could manually test the Django admin, and / or run ALL the Django queries and mutations in Django’s GraphiQL UI. Ugh!

Testing the Django Graphene Server

But have we broken anything in the backend in the process of building the frontend? Easy enough to find out. We need only to run our unit tests.

docker-compose run server pytest

Voila! Our tests will tell us what and where if we have any failing tests. In my case all my tests passed. This gives me enormous confidence moving forward with the development of our frontend site.

We will continue in our next chapter using TDD for Gatsby.

< Previous: Chapter 7

Next: Chapter 9 >

Tutorials in Web Development

TDD with Gatsby, Django & Docker Part 2, Chapter 08 — Setup Gatsby Development Environment