Speed up Docker for Mac with docker-sync

Speed up Docker for Mac with docker-sync

Running web app on Docker on MacOS can be quite slow, especially when you run multiple containers with file sync from the host machine to containers. To help solve this problem when developing on MacOS, one of the solutions we can use is a tool called docker-sync.

This post will guide you step-by-step on how to set up your local machine.

WARNING: You should have experience with Docker and Docker Compose in advance, in order to understand the post.

Why is it so slow?

The root of the problem lies in the OS file system layer between Docker and Mac. On Linux, Docker can mount directories and files directly, whereas, on Mac, Docker has to pass the request to Mac (osxfs) for each file read/write.

There are two dimensions to filesystem performance: throughput and latency. Throughput is the rate at which data can be processed. Latency is the time it takes for a file system call (e.g. write to that file) to complete.

Throughput is not the problem. Modern systems with SSDs can achieve throughput up to at least a few GBs/second. OSXFS has a limit throughput of 250 MB/s. While this is significantly slower than the throughput of a SSD, this typically is not a bottleneck as most applications don’t read or write that much data each second.

That leaves us with latency. Most block-based filesystems (block-based meaning the data on the disk is stored in blocks) have a latency of around 10μs (microseconds). OSXFS is about 13x slower (130μs microseconds). While it is still incredibly fast, when put to the test in a typical web app workload, like npm install, it all adds up and becomes incredibly slow.

Pre-requisites

  1. Docker and Docker Compose must be installed
  2. A Docker image of your project
  3. Ruby is installed

Setup

Step 1. Install docker sync

gem install docker-sync --no-rdoc --no-ri
docker pull eugenmayer/unison:2.51.2.1

Step 2. Create docker-sync.yml in the main directory of your project and include the following config in the file:

version: "2"
syncs:
  app-code-sync:
    # Source directory
    src: './src'
    # This will exclude the files/folders you specify.
    sync_excludes: [
      '.git'
    ]
  sync_host_ip: '127.0.0.1'
  sync_host_port: 10872

Step 3. Create docker-compose-dev.yml in the main directory

Imagine you have docker_compose.yml like this and you want to sync files folder in ./src with docker-sync

version: '3.5'

networks:
  laravel:

services:
  php:
    build:
      context: ./docker/php
    image: 'php:alpine'
    container_name: kutia_php
    ports:
      - '9000:9000'
    volumes:
      - ./src:/src
    networks:
      - laravel

First of all, remove the line – ./src:/src (it causes the container to slow down). Then add the following config to the docker-compose-dev.yml file

version: '3.5'

networks:
  laravel:

services:
  php:
    build:
      context: ./docker/php
    image: 'php:alpine'
    container_name: kutia_php
    ports:
      - '9000:9000'
    volumes:
      - app-code-sync:/src:nocopy
    networks:
      - laravel

volumes:
  app-code-sync:
    external: true

Step 4. Start docker-sync

docker-sync start

If you want to stop, then replace start with stop . To clean up the log and reset docker-sync, use clean .

Step 5. Start containers with Docker Compose

docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d

To check if everything is running properly, run docker-compose ps and check for the names of the container with app-code-sync and kutia_php . If they exist, then hooray! Your app is now up and running with docker-sync.