I use watchtower to update my containers but the Portainer agent keeps to have issues when updated. Whenever watchtower updates the container, it keeps restarting with the following error message:
agent_1 | 2021/08/05 12:22:25 [ERROR] [main,docker] [message: Unable to retrieve local agent IP address] [error: Error: No such container: 6c9fd4504fb5]
I use the following docker command to add the agent:
docker run -d \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/volumes:/var/lib/docker/volumes \
-v /:/host \
-v portainer_agent_data:/data \
--restart always \
-e EDGE=1 \
-e EDGE_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXX \
-e EDGE_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXX \
-e CAP_HOST_MANAGEMENT=1 \
-e EDGE_INSECURE_POLL=1 \
portainer/agent:latest
The problem is with watchtower, as it clones the container configs included hostname when upgrading the container.
When upgrade/re-creating the container, by default the docker daemon will assign a new hostname to the container and update internal dns, but the watchtower won't update the container hostname after the container upgrade so the agent will try to look up the old hostname and it would be getting nxdomain result by docker internal dns.
The fix is to assign a container name and a container hostname so that they stay the same and not causing watchtower issues.
docker run -d \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/volumes:/var/lib/docker/volumes \
-v /:/host \
-v portainer_agent_data:/data \
--restart always \
-e EDGE=1 \
-e EDGE_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXX \
-e EDGE_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXX \
-e CAP_HOST_MANAGEMENT=1 \
-e EDGE_INSECURE_POLL=1 \
--name portainer_edge_agent \
--hostname portainer_edge_agent \
portainer/agent:latest portainer/agent:latest