Speed up Docker for Mac with docker-sync
Running web app on Docker on MacOS can be quite slow, especially when you run multiple containers with file sync from the host machine to containers. To help solve this problem when developing on MacOS, one of the solutions we can use is a tool called docker-sync.
This post will guide you step-by-step on how to set up your local machine.
WARNING: You should have experience with Docker and Docker Compose in advance, in order to understand the post.
Why is it so slow?
The root of the problem lies in the OS file system layer between Docker and Mac. On Linux, Docker can mount directories and files directly, whereas, on Mac, Docker has to pass the request to Mac (osxfs) for each file read/write.
There are two dimensions to filesystem performance: throughput and latency. Throughput is the rate at which data can be processed. Latency is the time it takes for a file system call (e.g. write to that file) to complete.
Throughput is not the problem. Modern systems with SSDs can achieve throughput up to at least a few GBs/second. OSXFS has a limit throughput of 250 MB/s. While this is significantly slower than the throughput of a SSD, this typically is not a bottleneck as most applications don’t read or write that much data each second.
That leaves us with latency. Most block-based filesystems (block-based meaning the data on the disk is stored in blocks) have a latency of around 10μs (microseconds). OSXFS is about 13x slower (130μs microseconds). While it is still incredibly fast, when put to the test in a typical web app workload, like npm install, it all adds up and becomes incredibly slow.
Pre-requisites
- Docker and Docker Compose must be installed
- A Docker image of your project
- Ruby is installed
Setup
Step 1. Install docker sync
gem install docker-sync --no-rdoc --no-ri
docker pull eugenmayer/unison:2.51.2.1
Step 2. Create docker-sync.yml
in the main directory of your project and include the following config in the file:
version: "2"
syncs:
app-code-sync:
# Source directory
src: './src'
# This will exclude the files/folders you specify.
sync_excludes: [
'.git'
]
sync_host_ip: '127.0.0.1'
sync_host_port: 10872
Step 3. Create docker-compose-dev.yml
in the main directory
Imagine you have docker_compose.yml
like this and you want to sync files folder in ./src
with docker-sync
version: '3.5'
networks:
laravel:
services:
php:
build:
context: ./docker/php
image: 'php:alpine'
container_name: kutia_php
ports:
- '9000:9000'
volumes:
- ./src:/src
networks:
- laravel
First of all, remove the line – ./src:/src (it causes the container to slow down). Then add the following config to the docker-compose-dev.yml
file
version: '3.5'
networks:
laravel:
services:
php:
build:
context: ./docker/php
image: 'php:alpine'
container_name: kutia_php
ports:
- '9000:9000'
volumes:
- app-code-sync:/src:nocopy
networks:
- laravel
volumes:
app-code-sync:
external: true
Step 4. Start docker-sync
docker-sync start
If you want to stop, then replace start
with stop
. To clean up the log and reset docker-sync, use clean
.
Step 5. Start containers with Docker Compose
docker-compose -f docker-compose.yml -f docker-compose-dev.yml up -d
To check if everything is running properly, run docker-compose ps
and check for the names of the container with app-code-sync
and kutia_php
. If they exist, then hooray! Your app is now up and running with docker-sync.