Using Docker Compose with Nx Monorepo For Multi Apps Development

Introduction

The concept and toolings of Monorepo are gaining a fare share of momentum in recent years. Today, I am going to take some time to explain how we can streamline multi-apps development works by leveraging the power of Monorepo and Container Orchestration tool.

You can find the working sample in this GitHub repo. The tools that I am using are Nx (Monorepo) and Docker Compose (Container Orchestration).

If you run into any issues or questions about this topic, please feel free to reach out to me. I will be glad to connect with you on this. Trust me, I know the pain of setting this whole thing up. It took me well over a weekend due to my initial lack of Docker knowledge. Haha..

Topics Coverage

Please feel free to skip to the sections of your interest.

  1. Why use Docker-Compose with Nx during development time?
  2. Docker-compose Configurations for Multi Apps in Nx
  3. How to make Hot Reload works for Nx in Docker?
  4. Outcomes

Why use Docker-Compose with Nx during development time?

The reason might differ from one engineer to another.

For myself and the works that I am involved in day to day, there are a lot of context switching required. I am mostly switching and working on 4 different applications’ development at the same time. Now, imagine the following situation…

You are currently developing app-1, then suddenly there is an urgent request that would need you to hop on to fix something in app-2. What do you normally do? Exit the multiple bashs/shells running the apis, frontend for app-1 and then spin up another bunch for app-2… Right? Then, suddenly you realize some ports are clashing, and damn you forgot to turn off one of the api… oh.. and what are the cli commands to run app-2 again? Haha..

The list of issue goes on. Hence, the gigantic context switching and a direct jeopardize to productivity. This problem is mainly due to not having the capability to gain a single cohesive view of all “components” required to run an application.

Well, with some help from docker-compose, the game truly changes for me 😎 Because… Now, I can view everything through Docker Desktop’s dashboard. It’s a one stop station. Spinning applications up and down is just a matter of a few clicks. Clean!

Sample Docker Desktop’s Dashboard View

Of course, docker-compose is only of the many solutions out there for this problem. There are some great devs whom I know of, favor solutions like ConEmu, etc. as well. And that’s fine too. Also we can’t deny the fact that, anything runs in a virtual environment will be slower due to additional layering.

Docker-compose Configurations for Multi Apps in Nx

The following example is based upon this GitHub repo here. It might be a good idea for you to clone/fork the repo to follow and study along. At this point in time, I have created a total of 4 apps in the Nx workspace.

  • 2 React apps (app-1 and app-2)
  • 1 Angular app (app-3)
  • 1 Express api (api-1)

In Nx, all “servable” applications are called apps regardless of frontend or api. Hence, the 4.

Alright, now the objective here is to spin up all apps in one go with just a simple “docker-compose up” command. The following are the 4 files that are crucial for this Docker setup.

1. .dockerignore

This is much like the .gitignore file, except it is used by Docker to ignore files/folders in the “context” of a Docker build. And “context” in docker-compose is the root path for the build process. Bear in mind, it can be a different directory than the target Dockerfile. This was one confusing aspect when I first started working with Docker.

Anyway, here is my .dockerignore file

node_modules

Not a mistake, here I am just excluding node_modules from my build context. Reason is to avoid Docker to COPY my entire node_modules from code base into the image build, which can be terribly slow.

2. Dockerfile

FROM node:17-alpine3.12

WORKDIR /app

COPY . .

RUN npm install

CMD ["npm", "run", "start", "app-1"]

Dockerfile is a build step definition file to tell Docker how to build an image, step by step.

I am simply copying everything from my code base into the build context except the node_modules which is exempted via the .dockerignore file already. After that, we have to run “npm install” to reinstall the dependencies for the image.

Since we are using a monorepo, the source files to build every app will be the same, i.e the monorepo itself. Only difference being the entrypoints to individual applications. Hence, we only need one Dockerfile to build all the “services” defined in our docker-compose file later.

3. docker-compose.base.yml

version: '3'

services:
  nx-app-base:
    restart: always
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - DEV_PLATFORM=DOCKER
    volumes:
      - ./:/app
      - mono-node-modules:/app/node_modules

I created a base docker-compose.base.yml file to consolidate the common configurations into one file, so we can have a cleaner docker-compose.yml later.

Here, notice that the build section is telling Docker that the build context is the current directory and use the Dockerfile at this directory. If your Dockerfile is in a different location, you can specify the relative path to it here, e.g. ./.docker/Dockerfile.

The environment section is for us to specify environment variables. For our case, we are using Node image. Hence, the values defined here will go into process.env.[the_environment_variable_name], i.e. the Node environment variables. Here, we defined an environment variable called “DEV_PLATFORM”. This value will be used to override a Webpack config file in following next section for the hot reload feature to work.

Lastly, the volumes section defined 2 very important things.

  • ./:/app
    • Maps our source codes to the container’s /app directory (which is defined as the working directory in our Dockerfile)
    • This will enable our host machine and the containers to share the same source files, enabling file monitoring and hot reloads later on
  • mono-node-modules:/app/node_modules
    • mono-node-modules will be defined in our docker-compose.yml later as the shared volume. This is for all containers to share the same node_modules volume
    • In a way, this will speed up “npm install” after the first service, because we now only need to install the node dependencies once
    • For our case, it makes total sense because in monorepo, dependencies are shared

4. docker-compose.yml

version: '3'

services:
  # React App
  app-1:
    extends:
      file: docker-compose.base.yml
      service: nx-app-base
    command: npm run app-1:dev:docker
    ports:
      - 4201:4200

  # React App
  app-2:
    extends:
      file: docker-compose.base.yml
      service: nx-app-base
    command: npm run app-2:dev:docker
    ports:
      - 4202:4200

  # Angular App
  app-3:
    extends:
      file: docker-compose.base.yml
      service: nx-app-base
    command: npm run app-3:dev:docker
    ports:
      - 4203:4200

  # Express App
  api-1:
    extends:
      file: docker-compose.base.yml
      service: nx-app-base
    command: npm run api-1:dev:docker
    ports:
      - 4310:3333 # API Entry port
      - 4311:4311 # Server Debugging port

volumes:
  mono-node-modules: null


In this docker-compose.yml file, there are 4 “services” defined corresponding to the 4 Nx apps in our workspace. As you can see, each of these services has an “extends” property that extends the base service defined in our docker-compose.base.yml file in earlier section.

Also note that mono-node-modules is defined as the shared volume in the last section, which is then consumed by individual services as they extend the base service definition.

How to make Hot Reload works for Nx in Docker?

This actually depends on the OS you’re running on. In my case, my containers are running as Linux containers. On the other hand, I am developing on a Windows machine.

That is a problem for node-based apps that rely on Webpack’s HMR. By default, if no extra configuration is included, the HMR detects changes via the OS’s filesystem. For my case, Linux and Windows run on different filesystems. Hence, the hot reload issue.

In order to make the hot reload works for Nx in Docker, be it React, Angular, Express, any other node-based apps, we then need to override its webpack configs to explicitly tells Webpack that we want to use polling mechanism instead of listening to the filesystem.

For React and Express apps

To override the webpack config for React and Express apps in Nx, we need to first update the app’s project.json or in older version of Nx, the root workspace.json file or the architect/schematic file to instruct the builder that we want to use a custom Webpack config file.

This is illustrated here in the repo

{
  "root": "apps/app-1",
  "sourceRoot": "apps/app-1/src",
  "projectType": "application",
  "targets": {
    "build": {
        ...
        "webpackConfig": ".webpack/react-dev.config.js"
      },
....

Next step would then be the custom Webpack config file itself. The full sample can be found here.

const nrwlConfig = require("@nrwl/react/plugins/webpack.js");

module.exports = (config, context) => {
  nrwlConfig(config); // first call it so that it @nrwl/react plugin adds its configs

  if (process.env.DEV_PLATFORM === 'DOCKER') {
    // Make Hot Module Reload (HMR) works
    // Use polling mechanism to handle Filesystem disparities among diff OS
    config.watchOptions = {
      aggregateTimeout: 500,
      poll: 1000,
    }

    // Handle WebSocket port binding when Docker Host:Container port is different
    config.devServer = {
      ...config.devServer,
      client: {
        webSocketURL: "auto://0.0.0.0:0/ws",
      },
    };
  }

  return config;
};

Note that we’re using the environment variable “DEV_PLATFORM” here defined in our docker-compose.base.yml earlier on. It will then include the polling options only when the value is equal to “DOCKER”.

For Angular app

For Angular app, there is no straight forward mechanism to override its Webpack config. Though it is possible with the help of another tool like @angular-builders/custom-webpack, I certainly wouldn’t prefer to do so. That’s mostly because that is yet another blackbox itself. A blackbox on top of the already obscure blackbox? Hmm… Not a fan seriously.

However, we can still turn on the Webpack’s polling option in Angular with its cli parameter fortunately. So, we can do it with just

nx serve app-3 --host=0.0.0.0 --poll 5000

I tried this on both Nx v12 and v13. While Nx v13 (with Angular 13) is still working fine from the looks of it, it isn’t entirely the case with v12. Somehow, changes were detected constantly and rebuilds kept going on and on, causing infinite reloads… Didn’t bother to figure that out anyway. Escaping hard from Angular! 😆

Outcomes

Okay! So, finally here is how my Docker Desktop dashboard looks like after all the hard works. All 4 apps are up and running fine. I can turn them on and off via the dashboard itself. Cool!

Docker Desktop’s Dashboard outlook

Also, we can take a look at the Inspect tab of these 4 containers. Their node_modules are all pointing towards the mono-node-modules volume that we have defined earlier on! Niceeeee!

The container’s node_modules is pointing at the shared mono-node-modules volume that we’ve defined earlier on

The apps are then accessible at the exposed ports, i.e. 4201, 4202, 4203. Here is how my app-1 looks! And yea, when I change my source files, it gets updated and reloads with no prob! Truly hottttt! 😆

app-1 showcase

Conclusion

Been playing extensively with Docker and docker-compose close to a month now. And man, I’ve never looked back since! It’s such a helpful tool. Getting to do a “context switch” much faster, means that I can be a lot more productive in my day to day job!

Though there’s a serious uphill learning curve there about how Docker works in general and all those unfamiliar configs and commands involved, once you get it, you get it! Just don’t give up! It’s beneficial to both your productivity in a long run and of course, your value as a tech folk in the market today! 😆

Also worth mentioning, it was especially tiring after getting it works with sample projects and then… it hits a whole other bunch of probs when it comes to corporate proxies, private Npm, Nuget source, etc 🙁 Definitely one of the worst things to solve in corporate IT environments haha… If you too run into a similar situation as such and felt helpless, drop me a message, I might just have the answer for you.

Anyway, some of those topics/toolings like Monorepo, Microservice, Microfrontend, Containerization, Container Orchestration, Docker, Docker-Compose are somewhat cohesive, and often complement each other in ways that help improve software development flows. Hack, they even unlock tonnes of possibilities in software architectures. It’s good to study and research a bit along those lines. Such a great time to be in the tech field now more than ever seriously.


Here is the January 2022 subscription from Apartment Coffee. Really love this batch of Ethiopian beans. Floral and juicy, totally my kind of beans! Love it!

January subscription from Apartment Coffee. Diligent beans to kick start the year!

Resources

  1. https://github.com/DriLLFreAK100/nx-docker-compose-sample
  2. https://en.wikipedia.org/wiki/Monorepo
  3. https://microservices.io/
  4. https://micro-frontends.org/
  5. https://nx.dev/
  6. https://webpack.js.org/concepts/hot-module-replacement/