I decided to keep my new laptop clean. Don’t want to install any DB, framework, or DevOps software. And then I found out about Docker and how easy it is to work with it. This introduction follows my first steps and the reason for implementing a development environment based on Docker Compose.
A while ago I got a new laptop. I decided to keep it clean and don’t want to install every DB, framework, or any other thing on it. So I was working on a nice solution for this purpose. At first, I had a look at vagrant because I already know it…
It is nice and I can do many things with it. So I start using it again. BUT the problem was that I cannot convert easily from development to production. Now I got a problem and I remembered why I stop using vagrant a while ago…
BUT THEN something happens. My brain starts remembering about this container stuff… WHICH could also be used in a production environment. AWESOME!
And then I was like,… SHIT, I am using windows…. DOH… can I use docker? And how?.. So I was thinking about my PC which has a docker environment and within that, some containers are simpler PCs and have all the development stuff on them. And I can create containers within containers and everything is managed by Docker. THIS WAS SO NICE IN MY BRAIN.
And after that, I came up with a graphic that looks nice. I know everything is wrong but I had this idea to have an environment like this...
Then I want to start with Docker... BUT SUDDENLY…..
The first problem was that I never really used docker before and does not know where to start. So I tried to find tutorials and I found many of them, really a lot… And I learned about the things I can do with Docker.
After an initial search, I decided which things I want to do first: I want to find out how a beginner can install his own docker development environment WITHOUT having ANY knowledge about Docker, Docker-Compose, etc…
Unfortunately, no tutorial showed everything and explained everything I want to know. So I watched videos and researched on GitHub to gather all information I need to build my own personal docker development environment which can easily be transferred to production...
Within the next steps, I will explain what I found and how everyone can achieve this. I try to explain everything as simply as possible. So every person who has any experience will ask me WHY. But I will not answer….
2. Installing Docker-Desktop
The first thing was very easy. I downloaded Docker Desktop for Windows (here) and install it on my System. I am using Windows Pro so there were no problems installing (could be problems while using Home Edition).
3. Now I need something
After installation, I switched to https://docs.docker.com/get-started/ and try to follow the instructions mentioned there. For me, it was the wrong place to start. I got more questions while following the tutorial than they could answer. So I tried another approach.
Very important to know is that I heard people talking about docker-compose and how much better it is as standard Dockerfile because you can put Docker container into services...
With this in mind, I started a google search to find docker-compose files. I want to extract information/commands/settings from working docker-compose services.
4. Weird Problems…
I learned many things while searching for docker-compose files. Nothing of them was really interesting for developers who only want to develop a NODEJS-backend for example. So I won't mention them...
Because I am working with windows and have more than one HDD I learned a very important thing!
Docker-Desktop is installed within the
C:/Users folder which I really find annoying... So I have to move it… The problem was…many websites suggest you can switch it within Docker-Desktop settings...
BUT NOONE SAID THAT ONLY WORKS IF YOU USE THE OLD HYPER-V Mode which does not exist in my Docker-Desktop environment…
Because I am using a new release of Docker-Desktop it uses the WSL now (Windows Subsystem For Linux). And because of this images are stored within this subsystem. To move them you have to move the WSL save path.
To make things short you have to do, go to a windows shell (you can use VSCode and add an extension called Remote — WSL) and do:
- check which wsl images you have installed:
$> wsl -l -v NAME STATE VERSION docker-desktop Running 2 docker-desktop-data Running 2
2. Export, Unregister, and reimport into a new location (docker-desktop)
$> wsl --export docker-desktop docker-desktop.tar $> wsl --unregister docker-desktop $> wsl --import docker-desktop X:\NEWLOCATIONHERER\ docker-desktop.tar --version 2
I suggest moving all images to an HDD which has much space on it. You have to redo step 2 for every image you find in step 1. (on my machine it was docker-desktop and docker-desktop-data)
5. Finally start doing stuff
So I created a new folder (named it timetracking) and downloaded the docker-compose.yml file (this one) into it. After that I switched to a terminal and run:
$> docker-compose up -d
the first time I used without -d and then I stays open and I could not use the terminal anymore. so I canceled it and started it with this parameter (it means
AND NOW MAGIC... I’ve got my first dockerized app running on my machine.. HOOOOOORAAAY … but how can I access it?
To access the app I switched to a browser and entered http://localhost:8001 because I saw within the docker-compose file that at some point there was an option that says:
ports: - 8001:80
Because I am a good engineer I know that this means port 8001 is 80 within docker!
Another way to visit the app is to open it with docker-desktop. This is possible because after you execute
docker-compose up -d and everything works fine the new container is visible there and you can simply press Open in Browser EASY!
And then I was like… I'M A DOCKER EXPERT…!
And shortly after that, I was wondering… where is the database stored ..why can’t I find it?? WHAT IS HAPPENING?
So I had a look at the docker-compose file and found within the
sqldb-service this part:
volumes: - mysql:/var/lib/mysql
This leads to the creation of a new folder within my timetracking folder which was called kimai2_db and contained the whole important MySQL data.
So now I am safe to work on it (earlier I destroyed it so many times and was frustrated because my data wasn’t saved as I want to…)
The last thing I did was prune the docker-compose file because I found it full of unimportant stuff which I did not need on my local machine for my time tracking stuff and came up with:
version: '3.5' services: sqldb: image: mysql:5.7 environment: - MYSQL_DATABASE=kimai - MYSQL_USER=kimai - MYSQL_PASSWORD=kimai - MYSQL_ROOT_PASSWORD=kimai volumes: - ./kimai2_db:/var/lib/mysql command: --default-storage-engine innodb restart: unless-stopped nginx: image: tobybatch/nginx-fpm-reverse-proxy ports: - 8001:80 volumes: - public:/opt/kimai/public:ro restart: unless-stopped depends_on: - kimai kimai: image: kimai/kimai2:latest environment: - APP_ENV=prod - TRUSTED_HOSTS=localhost - ADMINMAILemail@example.com - ADMINPASS=changemeplease - DATABASE_URL=mysql://kimai:kimai@sqldb/kimai - TRUSTED_HOSTS=nginx,localhost,127.0.0.1 volumes: - public:/opt/kimai/public restart: unless-stopped volumes: public:
And now this docker container is ready to use. If I want to copy it to another machine I only zip the timetracking folder put it on a USB stick and “reinstall” it (by executing
docker-compose up -d).
The best thing is that I don’t have to reinstall everything, exporting importing the database, etc….
6. Make my Dev-Env work!
Now I want my own docker container! And the first thing I do was copy stuff from the time tracking one.
That’s how my first prototype of the docker-compose looked:
version: '3' services: mysql: environment: MYSQL_DATABASE: nodeappdb MYSQL_PASSWORD: nodeapppw MYSQL_ROOT_PASSWORD: hellosupersafepasswordforroot MYSQL_USER: nodeappuser image: mysql:5.7 volumes: - ./db:/var/lib/mysql
I extracted the DB part, and change the name into
mysql and changed the
Then the real work begins. I wanted to have a dev environment where I can develop node apps. So I needed a node container... Normally it is possible to add another service into the docker-compose file which is based on any node image and you have your dev env.
But there was a big problem for me. I wanted to have an environment where I can develop on my own machine which is directly transported into the docker container without rebuilding the image or doing something within a terminal.
So I searched and try to gather information about this
live reload thing.
After some time I finally finished my docker-compose file which will be explained in detail to everyone!
version: '3.4' services: mysql: environment: MYSQL_DATABASE: nodeappdb MYSQL_PASSWORD: nodeapppw MYSQL_ROOT_PASSWORD: hellosupersafepasswordforroot MYSQL_USER: nodeappuser image: mysql:5.7 command: "--lower_case_table_names=1 --max-allowed-packet=33554432" networks: - default volumes: - ./db:/var/lib/mysql node: ports: - 3000:3000 healthcheck: test: ["CMD-SHELL", "curl -f http://localhost:3000/healthcheck || exit 1"] interval: 30s timeout: 10s retries: 2 start_period: 10s environment: - NODE_ENV=development - BASEAPI_ENV=development - BASEAPI_PORT=3000 build: . depends_on: - mysql command: /command.sh volumes: - ./src/:/usr/src/app/ - ./node_modules:/usr/src/app/node_modules - ./command.sh:/command.sh networks: - default networks: default:
This is the MySQL container defined earlier. I made two additions. The first thing is a
command which is used to define settings for MySQL if you connect. The other thing is
networks which is only used to put different services into the same network so that they can communicate with each other
This line defines the node service
As mentioned earlier this line is used to make the port available outside of the docker environment. This enables the functionality to access the machine from outside the container (my local machine) through http://localhost:3000
This time I wanted to have a health check. It is an automated script which is executed how I define
Contains the command (
test) which should be executed. The other parameters are used for defining an interval (
interval) when the command should be executed, after how many seconds it should timeout (
timeout) and how often it could fail (
retries) before it stops running. Also, there is a starting time (
start_period) after which the server is started
This part defines environment variables that can be used within the container. I used three different variables which I normally use on my local machine. They can be named whatever you want.
This is a very interesting part. While researching I found out that you can use a Dockerfile to build a personalized image which can then be used in a docker-compose file. With this line, I inform docker-compose about a Dockerfile within the same directory that should be used to build the image (as you maybe noticed no
image is used in
Another really important part of the file. This line specifies that the `node` container should start AFTER MySQL is started. This is really important if you have a node app that connects to the MySQL server after starting. In my app after starting the node app, it synchronizes the models with the database which isn’t possible if the MySQL services are not started. This would result in an error and not starting container
A really important command which is executed for my environment which I will explain later
Start of volume creation
Maps the local subfolder src to /usr/src/app within the docker container
Maps the local subfolder
/usr/src/app/node_modules. This is done to have the node_modules within the docker container but not within the local src folder (awesome magic!!)
Maps the command.sh to the container so it is used there
Uses the same network as the MySQL container. It would do the same if we leave it out but I put it in if I want to have more services in the future which should be in different networks
important because the network was used earlier
As mentioned in the explanation above I used
build: . within the docker-compose file which indicates that there is a Dockerfile to build the image which is used for the node-container. This is the file I used (will also be explained in detail):
FROM node:8.6 # Install app dependencies RUN mkdir /build-dir WORKDIR /build-dir COPY ./src/package.json /build-dir RUN npm install -g nodemon RUN npm install # Create app directory RUN mkdir -p /usr/src/app WORKDIR /usr/src/app RUN ln -s /build-dir/node_modules node_modules # Bundle app source COPY ./src/ /usr/src/app/ EXPOSE 3000 CMD [ "npm", "run", "debug" ]
This is the base image that I want to use. If this is the only line within the Dockerfile I would have used
image: node:8.6 directly within the docker-compose file
Just a comment
This line creates a new folder within the container
The working directory for the following commands is set to
The package.json file HAS TO BE within the
./src folder on the local machine is copied to the
build-dir within the container
Two commands will be executed within the container. The first one installs
nodemon globally. The other one will install every other package from the package.json.
nothing important / comment
The app directory will be created within the container
The working directory for the following commands is set to
This IMPORTANT step is used to create a soft link between the
node_modules folder within the
/usr/src/app folder and the
node_modules folder within
build-dir where npm install put in all use packages by the app
Just a comment
src folder will be copied into
Exposes the port 3000
This command starts the node server. It executes
npm run debug. This will only work if the package.json contains a script called debug. This HAS TO BE there. And because we use nodemon it should be:
nodemon — legacy-watch server.js where server.js is your node-app-entrypoint
At last, I have this weird
command.sh file which is used in node-service within docker-compose:
#!/bin/bash cp -r /build-dir/node_modules/ /usr/src/app/ exec npm run debug
This command is really simple but is used to copy
node_modules and every subfolder to the
/usr/src/app. I have done this because I don’t want to have node_modules within the local
src folder while developing.
7. THE DEV ENV IS FINALLY READY!
Now I can save these files into a new folder and put my node code within src and it works. And if I change something within the files they will be directly affected within the docker-container. THIS IS REALLY AWESOME!
8. Closing Notes
I LOVE DOCKER!
I am not 100% sure if this document is perfectly researched and if every command/settings I used is necessary. But it works and is a good starting point for further improvement of docker!
One really important thing I want to add is that my local
src folder is mapped to
/usr/src/app/ which I found annoying because in my head it is
/usr/src/src/ and I will change it as soon as finishing this sentence!