I needed to move my private Docker registry to a new server, and I decided to take the opportunity to improve my setup. Instead of just rsync-ing the files, I chose to use an S3-compatible storage backend (MinIO) for managing Docker registry files. I already had MinIO in place, and it made backups much easier to manage.

The challenge? I needed to clone a large number of repositories with multiple tags, and doing it manually would be tedious and time-consuming. To solve this, I created a simple Bash script that uses the Docker Registry API, jq, curl, and skopeo to automate the process.

Here’s the Bash script I used. It fetches repositories and tags from the source registry and migrates them to the target registry using the powerful skopeo tool.

#!/bin/bash

SOURCE_REGISTRY="source-registry.xxxxx.xxx"
TARGET_REGISTRY="target--registry.xxxxx.xxx"
USERNAME="XXXXXXXX"
PASSWORD="XXXXXXX"

# Fetch all repositories from the source registry
REPOS=$(curl -s -u ${USERNAME}:${PASSWORD} https://${SOURCE_REGISTRY}/v2/_catalog | jq -r '.repositories[]')

# Loop through each repository
for repo in $REPOS; do
    echo "Processing repository: $repo"

    # Fetch all tags for the repository
    TAGS=$(curl -s -u ${USERNAME}:${PASSWORD} https://${SOURCE_REGISTRY}/v2/${repo}/tags/list | jq -r '.tags[]')

    # Loop through each tag
    for tag in $TAGS; do
        SOURCE_IMAGE="${SOURCE_REGISTRY}/${repo}:${tag}"
        TARGET_IMAGE="${TARGET_REGISTRY}/${repo}:${tag}"

        echo "Migrating ${SOURCE_IMAGE} to ${TARGET_IMAGE}"

        # Copy the image using Skopeo
        skopeo copy --all \
            --src-creds ${USERNAME}:${PASSWORD} \
            --dest-creds ${USERNAME}:${PASSWORD} \
            docker://${SOURCE_IMAGE} \
            docker://${TARGET_IMAGE}

        if [ $? -ne 0 ]; then
            echo "Error migrating ${SOURCE_IMAGE}"
        else
            echo "Successfully migrated ${SOURCE_IMAGE}"
        fi
    done
done

echo "Migration completed!"

How it's works:

• Fetching Repositories: The script uses the Docker Registry HTTP API to list all repositories from the source registry.
• Fetching Tags for Each Repository: It queries all available tags for each repository.
• Using skopeo to Copy Images: skopeo directly transfers images between registries without the need to pull and push them manually.
• Error Handling: Logs success and failure for each image, ensuring transparency in the migration process.

Used tools:

• Docker Registry HTTP API: Documentation
• jq: A lightweight and flexible command-line JSON processor for parsing and manipulating API responses. Homepage | GitHub
• skopeo: A powerful tool for managing container images Homepage | Installation Guide

Quick way of having docker image with integrated apple social identity provider. 

FROM maven:3.5-jdk-11 AS builder

RUN git clone https://github.com/BenjaminFavre/keycloak-apple-social-identity-provider.git
WORKDIR /keycloak-apple-social-identity-provider
RUN mvn package 

FROM jboss/keycloak:16.1.1
COPY --chown=jboss:root --from=builder /keycloak-apple-social-identity-provider/target/apple-social-identity-provider-1.0.2.jar /opt/jboss/keycloak/standalone/deployments/

MQTT is a lightweight, publish-subscribe network protocol that transports messages between devices. Golang has a very nice client which can be easily used for the purpose.

For example here is how to send data to my public free MQTT Broker - xMQTT

package main

import (
	"fmt"
	mqtt "github.com/eclipse/paho.mqtt.golang"
	"log"
)

func main(){
	var broker = "broker.xmqtt.net"
	var port = 8883

	opts := mqtt.NewClientOptions()
	opts.AddBroker(fmt.Sprintf("tcps://%s:%d", broker, port))
	opts.OnConnect = func(client mqtt.Client) {
		log.Println("Connected")
	}
	client := mqtt.NewClient(opts)
	if token := client.Connect(); token.Wait() && token.Error() != nil {
		panic(token.Error())
	}

	for i := 0; i < 100; i++ {
		t := client.Publish("/random/test",0,false,fmt.Sprintf("event_id %d",i))
		t.Wait()
	}

	client.Disconnect(5)
	log.Println("Events sent")
} 

Haproxy is an awesome load balancer for TCP and HTTP connections. In this short post we will see how to secure bind by source IP address Configuration of HaProxy to allow and reject connections by IP Address: For doing this we will use ACL to check source IP and based on it we will decide how to proceed.
In my use case I want to publish RabbitMQ management interface to few public users. It will be served through different from 80/443 port. So my config looks like:

listen port_5672
  bind :15672
  mode tcp
  acl network_allowed src 1.1.1.1 2.2.2.2.2
  tcp-request connection reject if !network_allowed
  server rmq_rmq1_1 127.0.0.1:25672 check

Note: RabbitMQ management interface is running on port 25672
So as you can see I’m adding new listener on port 15672 which backend will be 127.0.0.1:25672. Interesting part here is ACL conditions

acl network_allowed src 1.1.1.1 2.2.2.2.2
tcp-request connection reject if !network_allowed

It’s a pretty straightforward and self-explanatory. Of course this can be used in frontend section as well.
For example:

frontend www
  bind *:80
  mode tcp
  acl network_allowed src 1.1.1.1 2.2.2.2
  tcp-request connection reject if !network_allowed
  use_backend backend_server_original

Happy codding!

Developing in go typacly involve installing a lot of packages. A common scenario is from time to time to update all of them to their latest versions. This can be done easily with

go get -u all