Imagine you need to deploy a microservices application across 10 production servers. Would you manually SSH into each machine, install Docker, copy configuration files, and hope everything works the same way? That's not just tedious — it's a recipe for human error and inconsistent environments.
This project tackles two critical challenges that every DevOps engineer faces:
- Configuration Management — How do you automate server setup consistently across multiple machines?
- Service Discovery — How do services find and communicate with each other when their IP addresses can change?
The goal was to build a fully automated infrastructure using Ansible for configuration management and Consul for service discovery, simulating a real-world production environment.
The first part of the project focused on automating remote node configuration using Ansible. The infrastructure consists of three virtual machines:
┌────────────────┐ SSH ┌───────────────┐
│ manager01 │ ────────────► │ node01 │
│ (Ansible Ctrl) │ │ (Docker/App) │
└────────────────┘ └───────────────┘
│
│ SSH
▼
┌────────────────┐
│ node02 │
│ (Apache + DB) │
└────────────────┘
- manager01 — The control node running Ansible
- node01 — Hosts the microservice application via Docker Compose
- node02 — Runs Apache web server and PostgreSQL database
The second part implemented service discovery using Consul, allowing services to find each other dynamically without hardcoded IP addresses:
┌─────────────────┐ ┌──────────────────┐ ┌────────────────┐
│ consulserver │ │ api │ │ db │
│ │ │ (hotel-service) │ │ (PostgreSQL) │
│ │ │ | │ │ | │
│(Consul Server) │ │ ▼ │ │ ▼ │
│ │ │ +(Consul Agent) │ ◄──► │ +(Consul Agent)│
│ │ │ │ │ │
└─────────────────┘ └──────────────────┘ └────────────────┘
▲ ▲ ▲
│ │ │
└──────────────────────────┴────────────────────────┘
(Client-Server Communication)
Service Registration & Health Checks
- consulserver — Runs Consul in server mode, providing the central service registry (UI available on port 8500)
- api — Hosts the hotel microservice with Consul client agent; registers itself and discovers the database service via Consul
- db — Runs PostgreSQL with Consul client agent; registers the database service for discovery, communicates with Consul server for health checks
The foundation began with Vagrant to create reproducible virtual machines. Each VM runs Ubuntu 24.04 with 4GB RAM and 2 CPUs, connected via a private network:
config.vm.define "manager01" do |manager|
manager.vm.hostname = "manager01"
manager.vm.network "private_network", ip: "192.168.56.10"
end
config.vm.define "node01" do |node|
node.vm.hostname = "node01"
node.vm.network "private_network", ip: "192.168.56.11"
node.vm.network "forwarded_port", guest: 8081, host: 8081
node.vm.network "forwarded_port", guest: 8087, host: 8087
end
config.vm.define "node02" do |node|
node.vm.hostname = "node02"
node.vm.network "private_network", ip: "192.168.56.12"
endFor Ansible to work without interactive password prompts, we generated an SSH key on the manager and distributed it to all nodes:
# Generate SSH key
ssh-keygen -t ed25519 -f /home/vagrant/.ssh/id_ed25519 -N ""
# 3. Copy to Node 01
sshpass -p "vagrant" ssh-copy-id -o StrictHostKeyChecking=no -i /home/vagrant/.ssh/id_ed25519.pub vagrant@192.168.56.11
# 4. Copy to Node 02
sshpass -p "vagrant" ssh-copy-id -o StrictHostKeyChecking=no -i /home/vagrant/.ssh/id_ed25519.pub vagrant@192.168.56.12This enables passwordless SSH access, which is essential for Ansible automation.
The inventory file defines our hosts and groups:
[managers]
manager01 ansible_host=192.168.56.10
[appnodes]
node01 ansible_host=192.168.56.11
[dbnodes]
node02 ansible_host=192.168.56.12
[all:vars]
ansible_user=vagrant
ansible_python_interpreter=/usr/bin/python3Before running any playbooks, we verify Ansible can communicate with all nodes using the ping module:
ansible all -m pingThis simple test confirms SSH connectivity and Python availability on all target machines.
In the first part of the project, we deploy microservices using Docker Compose on node01. The Docker role installs Docker, and the application role builds and runs the containers.
The second part takes a different approach — instead of containers, we deploy the hotel service as a native Java application directly on the API node. This demonstrates both containerized and traditional deployment patterns:
- Part 1 (Docker): Microservices run in containers via Docker Compose
- Part 2 (Native): Hotel service runs directly as a Java process (java -jar)
This contrast showcases Ansible's flexibility in managing different types of workloads.
The first major playbook installs Docker on the application node. Instead of using shell scripts, we leverage Ansible's apt module for idempotent package installation:
- name: Update apt package cache
apt:
update_cache: yes
cache_valid_time: 3600
- name: Install required packages
apt:
name:
- ca-certificates
- curl
- gnupg
- name: Add Docker repository
apt_repository:
repo: "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu {{ ansible_distribution_release }} stable"
- name: Install Docker packages
apt:
name:
- docker-ce
- docker-ce-cli
- containerd.io
- docker-buildx-plugin
- docker-compose-plugin
- name: Add vagrant user to docker group
user:
name: vagrant
groups: docker
append: yesThe Docker role ensures consistent Docker installation across any number of nodes by running the same declarative configuration.
Once Docker is installed, the application role handles deploying the microservice:
- name: Build the microservice from services folder
command: docker compose build
args:
chdir: /home/vagrant/services
- name: Deploy the microservice
command: docker compose up -d
args:
chdir: /home/vagrant/servicesThis role copies the Docker Compose file and source code to the remote node and builds/runs the containers.
The project required creating three reusable Ansible roles:
Deploys the microservice application using Docker Compose on node01.
Installs and configures the Apache web server on node02:
- name: Install Apache
apt:
name: apache2
state: present
- name: Start Apache service
service:
name: apache2
state: started
enabled: yesInstalls PostgreSQL, creates a database, and adds sample records:
- name: Install PostgreSQL
apt:
name: postgresql
state: present
- name: Create database
postgresql_db:
name: hotels_db
- name: Create user
postgresql_user:
name: "{{ db_user }}"
password: "{{ db_password }}"
db: hotels_db
- name: Create sample table with data
community.postgresql.postgresql_query:
queries:
- CREATE TABLE IF NOT EXISTS hotels (id SERIAL PRIMARY KEY, name VARCHAR(100), city VARCHAR(100));
- INSERT INTO hotels (name, city) VALUES ('Grand Hotel', 'Paris'), ('Seaside Resort', 'Miami'), ('Mountain Lodge', 'Denver');The final playbook assigns roles strategically:
- name: Install Docker on Application Node
hosts: node01
roles:
- docker
- application
- name: Configure web & Database server on node02
hosts: node02
roles:
- apache
- postgresThis demonstrates Ansible's ability to apply different configurations to different machines from a single playbook.
The Consul server runs in agent mode as a server, providing the central service registry:
datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
node_name = "consul-server"
server = true
bootstrap_expect = 1
ui_config {
enabled = true
}
client_addr = "0.0.0.0"
bind_addr = "192.168.56.11"
advertise_addr = "192.168.56.11"
connect {
enabled = true
}
ports {
http = 8500
dns = 8600
}Note on Consul Connect: The connect block enables Consul Connect (service mesh), which provides automatic mTLS between services via Envoy sidecar proxies. In this implementation, Connect is enabled in the configuration but not fully utilized — services communicate directly rather than through sidecar proxies. This represents an enhancement opportunity for production deployments requiring encrypted service-to-service communication.
The server listens on port 8500 for the web UI and API, and port 8600 for DNS-based service discovery.
Clients (api and db nodes) run in client mode and connect to the server:
datacenter = "dc1"
data_dir = "/opt/consul"
server = false
retry_join = ["192.168.56.11"]
bind_addr = "{{ GetInterfaceIP \"enp0s8\" }}"
advertise_addr = "{{ GetInterfaceIP \"enp0s8\" }}"
connect {
enabled: true
}Note on Consul Connect: As with the server configuration, Consul Connect is enabled here to support future mTLS-based service communication via Envoy sidecar proxies. Currently, services communicate directly without encryption.
The template uses Consul's interpolation to dynamically bind to the correct network interface on each machine.
The install_db role installs PostgreSQL and creates the hotels database:
- name: Install PostgreSQL
apt:
name: postgresql
state: present
- name: Start PostgreSQL
service:
name: postgresql
state: started
enabled: yes
- name: Create database
postgresql_db:
name: hotels_dbThe install_hotels_service role handles deploying the Java application directly (not containerized):
- name: Copy hotel service source
synchronize:
src: files/hotel-service/
dest: /opt/hotel-service/
- name: Install OpenJDK
apt:
name: openjdk-8-jdk
state: present
- name: Set environment variables
lineinfile:
path: /etc/environment
line: "{{ item }}"
loop:
- "POSTGRES_HOST=127.0.0.1"
- "POSTGRES_PORT=5432"
- "POSTGRES_DB=hotels_db"
- "POSTGRES_USER=postgres"
- "POSTGRES_PASSWORD=password"
- name: Build the application
command: ./mvnw -DskipTests package
args:
chdir: /opt/hotel-service
- name: Create systemd service for hotel service
copy:
dest: /etc/systemd/system/hotel-service.service
content: |
[Unit]
Description=Hotel Service
After=network.target postgresql.service
[Service]
Type=simple
User=vagrant
WorkingDirectory=/opt/hotel-service
ExecStart=/usr/bin/java -jar /opt/hotel-service/target/hotel-service-0.0.1-SNAPSHOT.jar
Restart=on-failure
[Install]
WantedBy=multi-user.target
- name: Start and enable hotel service
systemd:
name: hotel-service
state: started
enabled: yes
daemon_reload: yesNote on PostgreSQL Connection: Currently, the application connects directly to the database using the fixed IP 127.0.0.1. With full Consul integration, this would instead use Consul's DNS-based service discovery (e.g., POSTGRES_HOST=db.service.consul) to dynamically resolve the database address. This is a planned enhancement for production environments.
The final playbook orchestrates everything:
- name: Install Consul Server
hosts: ConsulServer
roles:
- install_consul_server
- name: Install Consul Client
hosts: api, db
roles:
- install_consul_client
- name: Install Postgresql on db machine
hosts: db
roles:
- install_db
- name: Setup api machine
hosts: api
roles:
- install_hotels_service- Ansible Ping — Verified SSH connectivity to all nodes
- Docker Deployment — Confirmed microservices started successfully on node01
- Postman Tests — Ran Newman to execute API tests against the deployed services
- Apache Verification — Accessed the web server through the browser
- PostgreSQL Verification — Connected to the database and queried sample data
- Consul UI — Accessed at http://localhost:8500 to view registered services
- CRUD Operations — Tested Create, Read, Update, Delete on the hotel service API
- Service Health — Verified services were properly registered in Consul's catalog
Ansible's declarative nature means playbooks can be run multiple times safely. If Docker is already installed, the docker role simply reports "ok" instead of reinstalling.
Breaking playbooks into reusable roles (docker, application, apache, postgres, install_consul_server, install_consul_client, install_db, install_hotels_service) makes the codebase modular and maintainable.
All configuration is version-controlled. Need to recreate the infrastructure? Just run vagrant up and ansible-playbook — everything deploys automatically.
Consul provides the foundation for dynamic microservice architectures. Services register themselves, and other services discover them via DNS or HTTP API, eliminating hardcoded dependencies.
Problem: Ansible needs passwordless SSH access to remote nodes.
Solution: Created shell scripts (key.sh, key+1.sh) that generate SSH keys and distribute them using ssh-copy-id during VM provisioning.
Problem: Installing Docker manually on each VM is error-prone.
Solution: Used Ansible's apt module with proper repository configuration to ensure consistent Docker installation across all nodes.
Problem: PostgreSQL must be running before the hotel service tries to connect.
Solution: Used Ansible's role dependency system and handlers for proper service startup ordering.
Problem: Consul agents needed to bind to the correct network interface on different VMs.
Solution: Used Consul's template syntax {{ GetInterfaceIP "enp0s8" }} to dynamically determine the correct IP address for each node.
Problem: The hotel service needed database connection details without hardcoding IP addresses.
Solution: Used Ansible's lineinfile module to set environment variables (POSTGRES_HOST, POSTGRES_PORT, etc.) that the application reads at runtime.
Problem: Running a Java application in the background using Ansible's daemonize is unreliable and doesn't integrate with system startup.
Solution: Created a systemd service unit file to properly manage the Java application lifecycle, enabling automatic restart on failure and startup on boot.
Problem: The database connection currently uses a hardcoded IP address (127.0.0.1), defeating the purpose of Consul's service discovery.
Solution (Future Enhancement): For production, the application would use Consul's DNS (e.g., db.service.consul) to dynamically discover the database, allowing the database to move without reconfiguring the application.
| Category | Technology |
|---|---|
| Virtualization | Vagrant, VirtualBox |
| Configuration Management | Ansible |
| Service Discovery | Consul |
| Container Runtime | Docker |
| Orchestration | Docker Compose |
| Web Server | Apache |
| Database | PostgreSQL |
| Build Tool | Maven |
| Runtime | OpenJDK 8 |
This project demonstrates the essential skills required for modern DevOps engineering. From manually configuring servers to fully automated, idempotent infrastructure as code, the journey covers the complete lifecycle of deploying and managing distributed applications.
Key takeaways include understanding Ansible's declarative model for configuration management, learning how Consul enables dynamic service discovery, and appreciating the importance of infrastructure as code for reproducible deployments.
These skills form the foundation for any cloud-native or DevOps role, where automation, scalability, and reliability are paramount.