Fix panic when using MySQL for events storage and no table created yet

Add built-in cron feature.

CHANGELOG.md

@@ -0,0 +1,24 @@
+## Changelog
+
+### 0.7.1 (2018-07-18)
+
+* Fix panic when using MySQL for events storage and no table created yet.
+
+### 0.7 (2018-07-04)
+
+* When using MySQL for events storage, do not leak connections.
+* Last events were not shown when viewing a repo of non-default namespace.
+* Support repos with slash in the name.
+* Enable Sonatype Nexus compatibility.
+* Add `base_path` option to the config to run UI from non-root.
+* Add built-in cron feature for purging tags task.
+
+### 0.6 (2018-05-28)
+
+* Add MySQL along with sqlite3 support as a registry events storage.
+  New config settings `event_database_driver`, `event_database_location`.
+* Bump Go version and dependencies.
+
+### 0.5 (2018-03-06)
+
+* Initial public version.

Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.10.0-alpine3.7 as builder
+FROM golang:1.10.3-alpine as builder
 
 ENV GOPATH /opt
 
@@ -10,8 +10,9 @@ ADD glide.* /opt/src/github.com/quiq/docker-registry-ui/
 RUN cd /opt/src/github.com/quiq/docker-registry-ui && \
     /opt/bin/glide install
 
+ADD events /opt/src/github.com/quiq/docker-registry-ui/events
 ADD registry /opt/src/github.com/quiq/docker-registry-ui/registry
-ADD main.go /opt/src/github.com/quiq/docker-registry-ui/
+ADD *.go /opt/src/github.com/quiq/docker-registry-ui/
 RUN cd /opt/src/github.com/quiq/docker-registry-ui && \
     go test -v ./registry && \
     go build -o /opt/docker-registry-ui github.com/quiq/docker-registry-ui

README.md

@@ -11,6 +11,7 @@
 * Automatically discover an authentication method (basic auth, token service etc.)
 * Caching the list of repositories, tag counts and refreshing in background
 * Event listener of notification events coming from Registry
+* Store events in sqlite or MySQL database
 * CLI option to maintain the tags retention: purge tags older than X days keeping at least Y tags
 
 No TLS or authentication implemented on the UI web server itself.
@@ -24,7 +25,7 @@ The configuration is stored in `config.yml` and the options are self-descriptive
 
 ### Run UI
 
-    docker run -d -p 8000:8000 --read-only -v /local/config.yml:/opt/config.yml:ro \
+    docker run -d -p 8000:8000 -v /local/config.yml:/opt/config.yml:ro \
         --name=registry-ui quiq/docker-registry-ui
 
 To run with your own root CA certificate, add to the command:
@@ -35,6 +36,9 @@ To preserve sqlite db file with event notifications data, add to the command:
 
     -v /local/data:/opt/data
 
+You can also run the container with `--read-only` option, however when using using event listener functionality
+you need to ensure the sqlite db can be written, i.e. mount a folder as listed above.
+
 ## Configure event listener on Docker Registry
 
 To receive events you need to configure Registry as follow:
@@ -52,9 +56,33 @@ To receive events you need to configure Registry as follow:
             - application/octet-stream
 
 Adjust url and token as appropriate.
+If you are running UI from non-root base path, e.g. /ui, the URL path for above will be `/ui/api/events`.
+
+## Using MySQL instead of sqlite3 for event listener
+
+To use MySQL as a storage you need to change `event_database_driver` and `event_database_location`
+settings in the config file. It is expected you create a database mentioned in the location DSN.
+Minimal privileges are `SELECT`, `INSERT`, `DELETE`.
+You can create a table manually if you don't want to grant `CREATE` permission:
+
+	CREATE TABLE events (
+		id INTEGER PRIMARY KEY AUTO_INCREMENT,
+		action CHAR(4) NULL,
+		repository VARCHAR(100) NULL,
+		tag VARCHAR(100) NULL,
+		ip VARCHAR(15) NULL,
+		user VARCHAR(50) NULL,
+		created DATETIME NULL
+	);
 
 ### Schedule a cron task for purging tags
 
+To delete tags you need to enable the corresponding option in Docker Registry config. For example:
+
+    storage:
+      delete:
+        enabled: true
+
 The following example shows how to run a cron task to purge tags older than X days but also keep
 at least Y tags no matter how old. Assuming container has been already running.
 
@@ -64,6 +92,14 @@ You can try to run in dry-run mode first to see what is going to be purged:
 
     docker exec -t registry-ui /opt/docker-registry-ui -purge-tags -dry-run
 
+Alternatively, you can schedule the purging task with built-in cron feature:
+
+    purge_tags_keep_days: 90
+    purge_tags_keep_count: 2
+    purge_tags_schedule: '0 10 3 * * *'
+
+Note, the cron schedule format includes seconds! See https://godoc.org/github.com/robfig/cron
+
 ### Debug mode
 
 To increase http request verbosity, run container with `-e GOREQUEST_DEBUG=1`.

config.yml

@@ -1,5 +1,7 @@
 # Listen interface.
 listen_addr: 0.0.0.0:8000
+# Base path of Docker Registry UI.
+base_path: /
 
 # Registry URL with schema and port.
 registry_url: https://docker-registry.local
@@ -19,6 +21,12 @@ event_listener_token: token
 # Retention of records to keep.
 event_retention_days: 7
 
+# Event listener storage.
+event_database_driver: sqlite3
+event_database_location: data/registry_events.db
+# event_database_driver: mysql
+# event_database_location: user:password@tcp(localhost:3306)/docker_events
+
 # Cache refresh interval in minutes.
 # How long to cache repository list and tag counts.
 cache_refresh_interval: 10
@@ -32,7 +40,11 @@ admins: []
 # Debug mode. Affects only templates.
 debug: true
 
-# CLI options.
 # How many days to keep tags but also keep the minimal count provided no matter how old.
 purge_tags_keep_days: 90
 purge_tags_keep_count: 2
+# Enable built-in cron to schedule purging tags in server mode.
+# Empty string disables this feature.
+# Example: '25 54 17 * * *' will run it at 17:54:25 daily.
+# Note, the cron schedule format includes seconds! See https://godoc.org/github.com/robfig/cron
+purge_tags_schedule: ''

events/event_listener.go

@@ -0,0 +1,190 @@
+package events
+
+import (
+	"database/sql"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"os"
+	"strings"
+
+	"github.com/hhkbp2/go-logging"
+	"github.com/quiq/docker-registry-ui/registry"
+	// 🐒 patching of "database/sql".
+	_ "github.com/go-sql-driver/mysql"
+	_ "github.com/mattn/go-sqlite3"
+	"github.com/tidwall/gjson"
+)
+
+const (
+	schemaSQLite = `
+	CREATE TABLE events (
+		id INTEGER PRIMARY KEY AUTOINCREMENT,
+		action CHAR(4) NULL,
+		repository VARCHAR(100) NULL,
+		tag VARCHAR(100) NULL,
+		ip VARCHAR(15) NULL,
+		user VARCHAR(50) NULL,
+		created DATETIME NULL
+	);
+`
+)
+
+// EventListener event listener
+type EventListener struct {
+	databaseDriver   string
+	databaseLocation string
+	retention        int
+	logger           logging.Logger
+}
+
+type eventData struct {
+	Events []interface{} `json:"events"`
+}
+
+// EventRow event row from sqlite
+type EventRow struct {
+	ID         int
+	Action     string
+	Repository string
+	Tag        string
+	IP         string
+	User       string
+	Created    string
+}
+
+// NewEventListener initialize EventListener.
+func NewEventListener(databaseDriver, databaseLocation string, retention int) *EventListener {
+	return &EventListener{
+		databaseDriver:   databaseDriver,
+		databaseLocation: databaseLocation,
+		retention:        retention,
+		logger:           registry.SetupLogging("events.event_listener"),
+	}
+}
+
+// ProcessEvents parse and store registry events
+func (e *EventListener) ProcessEvents(request *http.Request) {
+	decoder := json.NewDecoder(request.Body)
+	var t eventData
+	if err := decoder.Decode(&t); err != nil {
+		e.logger.Errorf("Problem decoding event from request: %+v", request)
+		return
+	}
+	e.logger.Debugf("Received event: %+v", t)
+	j, _ := json.Marshal(t)
+
+	db, err := e.getDababaseHandler()
+	if err != nil {
+		e.logger.Error(err)
+		return
+	}
+	defer db.Close()
+
+	now := "DateTime('now')"
+	if e.databaseDriver == "mysql" {
+		now = "NOW()"
+	}
+	stmt, _ := db.Prepare("INSERT INTO events(action, repository, tag, ip, user, created) values(?,?,?,?,?," + now + ")")
+	for _, i := range gjson.GetBytes(j, "events").Array() {
+		// Ignore calls by docker-registry-ui itself.
+		if i.Get("request.useragent").String() == "docker-registry-ui" {
+			continue
+		}
+		action := i.Get("action").String()
+		repository := i.Get("target.repository").String()
+		tag := i.Get("target.tag").String()
+		// Tag is empty in case of signed pull.
+		if tag == "" {
+			tag = i.Get("target.digest").String()
+		}
+		ip := strings.Split(i.Get("request.addr").String(), ":")[0]
+		user := i.Get("actor.name").String()
+		e.logger.Debugf("Parsed event data: %s %s:%s %s %s ", action, repository, tag, ip, user)
+
+		res, err := stmt.Exec(action, repository, tag, ip, user)
+		if err != nil {
+			e.logger.Error("Error inserting a row: ", err)
+			return
+		}
+		id, _ := res.LastInsertId()
+		e.logger.Debug("New event added with id ", id)
+	}
+
+	// Purge old records.
+	var res sql.Result
+	if e.databaseDriver == "mysql" {
+		stmt, _ := db.Prepare("DELETE FROM events WHERE created < DATE_SUB(NOW(), INTERVAL ? DAY)")
+		res, _ = stmt.Exec(e.retention)
+	} else {
+		stmt, _ := db.Prepare("DELETE FROM events WHERE created < DateTime('now',?)")
+		res, _ = stmt.Exec(fmt.Sprintf("-%d day", e.retention))
+	}
+	count, _ := res.RowsAffected()
+	e.logger.Debug("Rows deleted: ", count)
+}
+
+// GetEvents retrieve events from sqlite db
+func (e *EventListener) GetEvents(repository string) []EventRow {
+	var events []EventRow
+
+	db, err := e.getDababaseHandler()
+	if err != nil {
+		e.logger.Error(err)
+		return events
+	}
+	defer db.Close()
+
+	query := "SELECT * FROM events ORDER BY id DESC LIMIT 1000"
+	if repository != "" {
+		query = fmt.Sprintf("SELECT * FROM events WHERE repository='%s' ORDER BY id DESC LIMIT 5", repository)
+	}
+	rows, err := db.Query(query)
+	if err != nil {
+		e.logger.Error("Error selecting from table: ", err)
+		return events
+	}
+	defer rows.Close()
+
+	for rows.Next() {
+		var row EventRow
+		rows.Scan(&row.ID, &row.Action, &row.Repository, &row.Tag, &row.IP, &row.User, &row.Created)
+		events = append(events, row)
+	}
+	return events
+}
+
+func (e *EventListener) getDababaseHandler() (*sql.DB, error) {
+	firstRun := false
+	schema := schemaSQLite
+	if e.databaseDriver == "sqlite3" {
+		if _, err := os.Stat(e.databaseLocation); os.IsNotExist(err) {
+			firstRun = true
+		}
+	}
+
+	// Open db connection.
+	db, err := sql.Open(e.databaseDriver, e.databaseLocation)
+	if err != nil {
+		return nil, fmt.Errorf("Error opening %s db: %s", e.databaseDriver, err)
+	}
+
+	if e.databaseDriver == "mysql" {
+		schema = strings.Replace(schema, "AUTOINCREMENT", "AUTO_INCREMENT", 1)
+		rows, err := db.Query("SELECT * FROM events LIMIT 1")
+		if err != nil {
+			firstRun = true
+		}
+		if rows != nil {
+			rows.Close()
+		}
+	}
+
+	// Create table on first run.
+	if firstRun {
+		if _, err = db.Exec(schema); err != nil {
+			return nil, fmt.Errorf("Error creating a table: %s", err)
+		}
+	}
+	return db, nil
+}

example/.gitignore

@@ -0,0 +1 @@
+/data

example/README.md

@@ -0,0 +1,25 @@
+# Docker Registry UI Example
+
+Start the local Docker Registry and UI.
+
+```bash
+$ docker-compose up -d
+```
+
+As an example, push the docker-registry-ui image to the local Docker Registry.
+
+```bash
+$ docker tag quiq/docker-registry-ui localhost/quiq/docker-registry-ui
+$ docker push localhost/quiq/docker-registry-ui
+The push refers to repository [localhost:5000/quiq/docker-registry-ui]
+ab414a599bf8: Pushed
+a8da33adf86e: Pushed
+71a0e0a972a7: Pushed
+96dc74eb5456: Pushed
+ac362bf380d0: Pushed
+04a094fe844e: Pushed
+latest: digest: sha256:d88c1ca40986a358e59795992e87e364a0b3b97833aade5abcd79dda0a0477e8 size: 1571
+```
+
+Then you will find the pushed repository 'quiq/docker-registry-ui' in the following URL.
+http://localhost/ui/quiq/docker-registry-ui

example/config/httpd.conf

@@ -0,0 +1,79 @@
+LoadModule mpm_event_module modules/mod_mpm_event.so
+LoadModule headers_module modules/mod_headers.so
+
+LoadModule authn_file_module modules/mod_authn_file.so
+LoadModule authn_core_module modules/mod_authn_core.so
+LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
+LoadModule authz_user_module modules/mod_authz_user.so
+LoadModule authz_core_module modules/mod_authz_core.so
+LoadModule auth_basic_module modules/mod_auth_basic.so
+LoadModule access_compat_module modules/mod_access_compat.so
+
+LoadModule log_config_module modules/mod_log_config.so
+
+LoadModule proxy_module modules/mod_proxy.so
+LoadModule proxy_http_module modules/mod_proxy_http.so
+
+LoadModule unixd_module modules/mod_unixd.so
+
+<IfModule unixd_module>
+    User daemon
+    Group daemon
+</IfModule>
+
+ServerAdmin you@example.com
+
+ErrorLog /proc/self/fd/2
+
+LogLevel warn
+
+<IfModule log_config_module>
+    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
+    LogFormat "%h %l %u %t \"%r\" %>s %b" common
+
+    <IfModule logio_module>
+      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
+    </IfModule>
+
+    CustomLog /proc/self/fd/1 common
+</IfModule>
+
+ServerRoot "/usr/local/apache2"
+
+Listen 80
+
+<Directory />
+    AllowOverride none
+    Require all denied
+</Directory>
+
+<VirtualHost *:80>
+
+  ServerName myregistrydomain.com
+
+  Header always set "Docker-Distribution-Api-Version" "registry/2.0"
+  Header onsuccess set "Docker-Distribution-Api-Version" "registry/2.0"
+
+  ProxyRequests     off
+  ProxyPreserveHost on
+
+  # no proxy for /error/ (Apache HTTPd errors messages)
+  ProxyPass /error/ !
+
+  ProxyPass        /v2 http://registry:5000/v2
+  ProxyPassReverse /v2 http://registry:5000/v2
+
+  <Location /v2>
+    Order deny,allow
+    Allow from all
+  </Location>
+
+  ProxyPass /ui/ http://registry-ui:8000/ui/
+  ProxyPassReverse /ui/ http://registry-ui:8000/ui/
+
+  <Location /ui>
+    Order deny,allow
+    Allow from all
+  </Location>
+
+</VirtualHost>

example/config/registry-ui.yml

@@ -0,0 +1,46 @@
+# Listen interface.
+listen_addr: 0.0.0.0:8000
+# Base path of Docker Registry UI.
+base_path: /ui
+
+# Registry URL with schema and port.
+registry_url: http://registry:5000
+# Verify TLS certificate when using https.
+verify_tls: true
+
+# Docker registry credentials.
+# They need to have a full access to the registry.
+# If token authentication service is enabled, it will be auto-discovered and those credentials
+# will be used to obtain access tokens.
+# registry_username: user
+# registry_password: pass
+
+# Event listener token.
+# The same one should be configured on Docker registry as Authorization Bearer token.
+event_listener_token: token
+# Retention of records to keep.
+event_retention_days: 7
+
+# Event listener storage.
+event_database_driver: sqlite3
+event_database_location: data/registry_events.db
+# event_database_driver: mysql
+# event_database_location: user:password@tcp(localhost:3306)/docker_events
+
+# Cache refresh interval in minutes.
+# How long to cache repository list and tag counts.
+cache_refresh_interval: 10
+
+# If users can delete tags. If set to False, then only admins listed below.
+anyone_can_delete: false
+# Users allowed to delete tags.
+# This should be sent via X-WEBAUTH-USER header from your proxy.
+admins: []
+
+# Debug mode. Affects only templates.
+debug: true
+
+# CLI options.
+# How many days to keep tags but also keep the minimal count provided no matter how old.
+purge_tags_keep_days: 90
+purge_tags_keep_count: 2

example/docker-compose.yml

@@ -0,0 +1,31 @@
+version: "2.3"
+
+services:
+  httpd:
+    image: httpd:2.4
+    ports:
+      - "80:80"
+    volumes:
+      - "./config/httpd.conf:/usr/local/apache2/conf/httpd.conf:ro"
+
+  registry:
+    image: registry:2
+    ports:
+      - "5000"
+    volumes:
+      - "./data/registry:/var/lib/registry"
+    healthcheck:
+      test: ["CMD", "wget", "-s", "localhost:5000/v2/"]
+      interval: 5s
+      timeout: 10s
+
+  registry-ui:
+    image: quiq/docker-registry-ui:latest
+    ports:
+      - "8000"
+    volumes:
+      - "./data/registry-ui:/opt/data"
+      - "./config/registry-ui.yml:/opt/config.yml:ro"
+    depends_on:
+      registry:
+        condition: service_healthy

glide.lock

@@ -1,66 +1,70 @@
-hash: 29246065eafa5aaec8848881fb6c99995e91a3f9b0082724db71c71590937e29
+hash: fea96c473a02b07acc1d600ee0f71c6a5143f34e5eb04a4c5e3e14378fca46f0
-updated: 2018-02-19T16:47:40.847725+02:00
+updated: 2018-06-09T09:03:51.972089+09:00
 imports:
 - name: github.com/CloudyKit/fastprinter
   version: 74b38d55f37af5d6c05ca11147d616b613a3420e
 - name: github.com/CloudyKit/jet
   version: 2b064536b25ab0e9c54245f9e2cc5bd4766033fe
 - name: github.com/dgrijalva/jwt-go
-  version: a539ee1a749a2b895533f979515ac7e6e0f5b650
+  version: 06ea1031745cb8b3dab3f6a236daf2b0aa468b7e
-- name: github.com/grafana/grafana
+- name: github.com/go-sql-driver/mysql
-  version: c4683f1ae85a9c80dfd04fce09318c466885f3c0
+  version: 64db0f7ebe171b596aa9b26f39a79f7413a3b617
-  subpackages:
-  - pkg/cmd/grafana-cli/logger
 - name: github.com/hhkbp2/go-logging
-  version: 1bf77adfece4a2018ac4bcc84e1f20509157a534
+  version: 377ba05d98977baa2a0d9e13f13aac2f3a47ac4d
 - name: github.com/hhkbp2/go-strftime
   version: d82166ec6782f870431668391c2e321069632fe7
 - name: github.com/labstack/echo
-  version: b338075a0fc6e1a0683dbf03d09b4957a289e26f
+  version: 6d227dfea4d2e52cb76856120b3c17f758139b4e
   subpackages:
   - middleware
 - name: github.com/labstack/gommon
-  version: 779b8a8b9850a97acba6a3fe20feb628c39e17c1
+  version: 0a22a0df01a7c84944c607e8a6e91cfe421ea7ed
   subpackages:
   - bytes
   - color
   - log
   - random
 - name: github.com/mattn/go-colorable
-  version: 3fa8c76f9daed4067e4a806fb7e4dc86455c6d6a
+  version: efa589957cd060542a26d2dd7832fd6a6c6c3ade
 - name: github.com/mattn/go-isatty
-  version: fc9e8d8ef48496124e79ae0df75490096eccf6fe
+  version: 6ca4dbf54d38eea1a992b3c722a76a5d1c4cb25c
 - name: github.com/mattn/go-sqlite3
-  version: 6c771bb9887719704b210e87e934f08be014bdb1
+  version: 323a32be5a2421b8c7087225079c6c900ec397cd
 - name: github.com/moul/http2curl
-  version: 4e24498b31dba4683efb9d35c1c8a91e2eda28c8
+  version: 9ac6cf4d929b2fa8fd2d2e6dec5bb0feb4f4911d
 - name: github.com/parnurzeal/gorequest
   version: a578a48e8d6ca8b01a3b18314c43c6716bb5f5a3
 - name: github.com/pkg/errors
-  version: c605e284fe17294bda444b34710735b29d1a9d90
+  version: 816c9085562cd7ee03e7f8188a1cfd942858cded
+- name: github.com/robfig/cron
+  version: b41be1df696709bb6395fe435af20370037c0b4c
 - name: github.com/tidwall/gjson
-  version: 87033efcaec6215741137e8ca61952c53ef2685d
+  version: 01f00f129617a6fe98941fb920d6c760241b54d2
 - name: github.com/tidwall/match
-  version: 173748da739a410c5b0b813b956f89ff94730b4c
+  version: 1731857f09b1f38450e2c12409748407822dc6be
 - name: github.com/valyala/bytebufferpool
   version: e746df99fe4a3986f4d4f79e13c1e0117ce9c2f7
 - name: github.com/valyala/fasttemplate
   version: dcecefd839c4193db0d35b88ec65b4c12d360ab0
 - name: golang.org/x/crypto
-  version: 4d70248d17d12d1edb7153434a74001c1540938b
+  version: 1a580b3eff7814fc9b40602fd35256c63b50f491
   subpackages:
   - acme
   - acme/autocert
 - name: golang.org/x/net
-  version: 02ac38e2528ff4adea90f184d71a3faa04b4b1b0
+  version: dfa909b99c79129e1100513e5cd36307665e5723
   subpackages:
   - publicsuffix
 - name: golang.org/x/sys
-  version: cd2c276457edda6df7fb04895d3fd6a6add42926
+  version: 7c87d13f8e835d2fb3a70a2912c811ed0c1d241b
   subpackages:
   - unix
+- name: google.golang.org/appengine
+  version: b1f26356af11148e710935ed1ac8a7f5702c7612
+  subpackages:
+  - cloudsql
 - name: gopkg.in/yaml.v2
-  version: 3b4ad1db5b2a649883ff3782f5f9f6fb52be71af
+  version: 5420a8b6744d3b0345ab293f6fcba19c978f1183
 testImports:
 - name: github.com/jtolds/gls
   version: 9a4a02dbe491bef4bab3c24fd9f3087d6c4c6690

glide.yaml

@@ -3,16 +3,19 @@ import:
 - package: github.com/CloudyKit/jet
   version: v2.1.2
 - package: github.com/labstack/echo
-  version: v3.2.6
+  version: v3.3.5
   subpackages:
   - middleware
 - package: github.com/parnurzeal/gorequest
   version: v0.2.15
 - package: github.com/hhkbp2/go-logging
 - package: github.com/tidwall/gjson
-  version: v1.0.6
+  version: v1.1.0
 - package: github.com/mattn/go-sqlite3
-  version: 1.6.0
+  version: 1.7.0
+- package: github.com/go-sql-driver/mysql
+- package: github.com/robfig/cron
+  version: ~1.1.0
 testImport:
 - package: github.com/smartystreets/goconvey
   version: 1.6.2

main.go

@@ -3,7 +3,6 @@ package main
 import (
 	"flag"
 	"fmt"
-	"io"
 	"io/ioutil"
 	"net/http"
 	"net/url"
@@ -13,25 +12,31 @@ import (
 	"github.com/CloudyKit/jet"
 	"github.com/labstack/echo"
 	"github.com/labstack/echo/middleware"
+	"github.com/quiq/docker-registry-ui/events"
 	"github.com/quiq/docker-registry-ui/registry"
+	"github.com/robfig/cron"
 	"github.com/tidwall/gjson"
 	"gopkg.in/yaml.v2"
 )
 
 type configData struct {
-	ListenAddr           string   `yaml:"listen_addr"`
+	ListenAddr            string   `yaml:"listen_addr"`
-	RegistryURL          string   `yaml:"registry_url"`
+	BasePath              string   `yaml:"base_path"`
-	VerifyTLS            bool     `yaml:"verify_tls"`
+	RegistryURL           string   `yaml:"registry_url"`
-	Username             string   `yaml:"registry_username"`
+	VerifyTLS             bool     `yaml:"verify_tls"`
-	Password             string   `yaml:"registry_password"`
+	Username              string   `yaml:"registry_username"`
-	EventListenerToken   string   `yaml:"event_listener_token"`
+	Password              string   `yaml:"registry_password"`
-	EventRetentionDays   int      `yaml:"event_retention_days"`
+	EventListenerToken    string   `yaml:"event_listener_token"`
-	CacheRefreshInterval uint8    `yaml:"cache_refresh_interval"`
+	EventRetentionDays    int      `yaml:"event_retention_days"`
-	AnyoneCanDelete      bool     `yaml:"anyone_can_delete"`
+	EventDatabaseDriver   string   `yaml:"event_database_driver"`
-	Admins               []string `yaml:"admins"`
+	EventDatabaseLocation string   `yaml:"event_database_location"`
-	Debug                bool     `yaml:"debug"`
+	CacheRefreshInterval  uint8    `yaml:"cache_refresh_interval"`
-	PurgeTagsKeepDays    int      `yaml:"purge_tags_keep_days"`
+	AnyoneCanDelete       bool     `yaml:"anyone_can_delete"`
-	PurgeTagsKeepCount   int      `yaml:"purge_tags_keep_count"`
+	Admins                []string `yaml:"admins"`
+	Debug                 bool     `yaml:"debug"`
+	PurgeTagsKeepDays     int      `yaml:"purge_tags_keep_days"`
+	PurgeTagsKeepCount    int      `yaml:"purge_tags_keep_count"`
+	PurgeTagsSchedule     string   `yaml:"purge_tags_schedule"`
 }
 
 type template struct {
@@ -39,8 +44,9 @@ type template struct {
 }
 
 type apiClient struct {
-	client *registry.Client
+	client        *registry.Client
-	config configData
+	eventListener *events.EventListener
+	config        configData
 }
 
 func main() {
@@ -71,6 +77,15 @@ func main() {
 	if err != nil {
 		panic(err)
 	}
+	// Normalize base path.
+	if a.config.BasePath != "" {
+		if !strings.HasPrefix(a.config.BasePath, "/") {
+			a.config.BasePath = "/" + a.config.BasePath
+		}
+		if strings.HasSuffix(a.config.BasePath, "/") {
+			a.config.BasePath = a.config.BasePath[0 : len(a.config.BasePath)-1]
+		}
+	}
 
 	// Init registry API client.
 	a.client = registry.NewClient(a.config.RegistryURL, a.config.VerifyTLS, a.config.Username, a.config.Password)
@@ -80,78 +95,54 @@ func main() {
 
 	// Execute CLI task and exit.
 	if purgeTags {
-		registry.PurgeOldTags(a.client, purgeDryRun, a.config.PurgeTagsKeepDays, a.config.PurgeTagsKeepCount)
+		a.purgeOldTags(purgeDryRun)
 		return
 	}
+	// Schedules to purge tags.
+	if a.config.PurgeTagsSchedule != "" {
+		c := cron.New()
+		task := func() {
+			a.purgeOldTags(purgeDryRun)
+		}
+		if err := c.AddFunc(a.config.PurgeTagsSchedule, task); err != nil {
+			panic(fmt.Errorf("Invalid schedule format: %s", a.config.PurgeTagsSchedule))
+		}
+		c.Start()
+	}
 
 	// Count tags in background.
 	go a.client.CountTags(a.config.CacheRefreshInterval)
 
+	if a.config.EventDatabaseDriver != "sqlite3" && a.config.EventDatabaseDriver != "mysql" {
+		panic(fmt.Errorf("event_database_driver should be either sqlite3 or mysql"))
+	}
+	a.eventListener = events.NewEventListener(a.config.EventDatabaseDriver, a.config.EventDatabaseLocation, a.config.EventRetentionDays)
+
 	// Template engine init.
-	view := jet.NewHTMLSet("templates")
-	view.SetDevelopmentMode(a.config.Debug)
-	view.AddGlobal("registryHost", u.Host)
-	view.AddGlobal("pretty_size", func(size interface{}) string {
-		var value float64
-		switch i := size.(type) {
-		case gjson.Result:
-			value = float64(i.Int())
-		case int64:
-			value = float64(i)
-		}
-		return registry.PrettySize(value)
-	})
-	view.AddGlobal("pretty_time", func(datetime interface{}) string {
-		return strings.Split(strings.Replace(datetime.(string), "T", " ", 1), ".")[0]
-	})
-	view.AddGlobal("parse_map", func(m interface{}) string {
-		var res string
-		for _, k := range registry.SortedMapKeys(m) {
-			res = res + fmt.Sprintf(`<tr><td style="padding: 0 10px; width: 20%%">%s</td><td style="padding: 0 10px">%v</td></tr>`, k, m.(map[string]interface{})[k])
-		}
-		return res
-	})
 	e := echo.New()
-	e.Renderer = &template{View: view}
+	e.Renderer = setupRenderer(a.config.Debug, u.Host, a.config.BasePath)
 
 	// Web routes.
-	e.Static("/static", "static")
+	e.Static(a.config.BasePath+"/static", "static")
-	e.GET("/", a.viewRepositories)
+	e.GET(a.config.BasePath+"/", a.viewRepositories)
-	e.GET("/:namespace", a.viewRepositories)
+	e.GET(a.config.BasePath+"/:namespace", a.viewRepositories)
-	e.GET("/:namespace/:repo", a.viewTags)
+	e.GET(a.config.BasePath+"/:namespace/:repo", a.viewTags)
-	e.GET("/:namespace/:repo/:tag", a.viewTagInfo)
+	e.GET(a.config.BasePath+"/:namespace/:repo/:tag", a.viewTagInfo)
-	e.GET("/:namespace/:repo/:tag/delete", a.deleteTag)
+	e.GET(a.config.BasePath+"/:namespace/:repo/:tag/delete", a.deleteTag)
-	e.GET("/events", a.viewLog)
+	e.GET(a.config.BasePath+"/events", a.viewLog)
 
 	// Protected event listener.
-	p := e.Group("/api")
+	p := e.Group(a.config.BasePath + "/api")
 	p.Use(middleware.KeyAuthWithConfig(middleware.KeyAuthConfig{
 		Validator: middleware.KeyAuthValidator(func(token string, c echo.Context) (bool, error) {
 			return token == a.config.EventListenerToken, nil
 		}),
 	}))
-	p.POST("/events", a.eventListener)
+	p.POST("/events", a.receiveEvents)
 
 	e.Logger.Fatal(e.Start(a.config.ListenAddr))
 }
 
-// Render render template.
-func (r *template) Render(w io.Writer, name string, data interface{}, c echo.Context) error {
-	t, err := r.View.GetTemplate(name)
-	if err != nil {
-		panic(fmt.Errorf("Fatal error template file: %s", err))
-	}
-	vars, ok := data.(jet.VarMap)
-	if !ok {
-		vars = jet.VarMap{}
-	}
-	err = t.Execute(w, vars, nil)
-	if err != nil {
-		panic(fmt.Errorf("Error rendering template %s: %s", name, err))
-	}
-	return nil
-}
-
 func (a *apiClient) viewRepositories(c echo.Context) error {
 	namespace := c.Param("namespace")
 	if namespace == "" {
@@ -159,7 +150,6 @@ func (a *apiClient) viewRepositories(c echo.Context) error {
 	}
 
 	repos, _ := a.client.Repositories(true)[namespace]
-
 	data := jet.VarMap{}
 	data.Set("namespace", namespace)
 	data.Set("namespaces", a.client.Namespaces())
@@ -185,7 +175,8 @@ func (a *apiClient) viewTags(c echo.Context) error {
 	data.Set("repo", repo)
 	data.Set("tags", tags)
 	data.Set("deleteAllowed", deleteAllowed)
-	data.Set("events", registry.GetEvents(repo))
+	repoPath, _ = url.PathUnescape(repoPath)
+	data.Set("events", a.eventListener.GetEvents(repoPath))
 
 	return c.Render(http.StatusOK, "tags.html", data)
 }
@@ -201,7 +192,7 @@ func (a *apiClient) viewTagInfo(c echo.Context) error {
 
 	sha256, infoV1, infoV2 := a.client.TagInfo(repoPath, tag, false)
 	if infoV1 == "" || infoV2 == "" {
-		return c.Redirect(http.StatusSeeOther, fmt.Sprintf("/%s/%s", namespace, repo))
+		return c.Redirect(http.StatusSeeOther, fmt.Sprintf("%s/%s/%s", a.config.BasePath, namespace, repo))
 	}
 
 	var imageSize int64
@@ -261,7 +252,7 @@ func (a *apiClient) deleteTag(c echo.Context) error {
 		a.client.DeleteTag(repoPath, tag)
 	}
 
-	return c.Redirect(http.StatusSeeOther, fmt.Sprintf("/%s/%s", namespace, repo))
+	return c.Redirect(http.StatusSeeOther, fmt.Sprintf("%s/%s/%s", a.config.BasePath, namespace, repo))
 }
 
 // checkDeletePermission check if tag deletion is allowed whether by anyone or permitted users.
@@ -281,13 +272,18 @@ func (a *apiClient) checkDeletePermission(user string) bool {
 // viewLog view events from sqlite.
 func (a *apiClient) viewLog(c echo.Context) error {
 	data := jet.VarMap{}
-	data.Set("events", registry.GetEvents(""))
+	data.Set("events", a.eventListener.GetEvents(""))
 
 	return c.Render(http.StatusOK, "event_log.html", data)
 }
 
-// eventListener listen events from registry.
+// receiveEvents receive events.
-func (a *apiClient) eventListener(c echo.Context) error {
+func (a *apiClient) receiveEvents(c echo.Context) error {
-	registry.ProcessEvents(c.Request(), a.config.EventRetentionDays)
+	a.eventListener.ProcessEvents(c.Request())
 	return c.String(http.StatusOK, "OK")
 }
+
+// purgeOldTags purges old tags.
+func (a *apiClient) purgeOldTags(dryRun bool) {
+	registry.PurgeOldTags(a.client, dryRun, a.config.PurgeTagsKeepDays, a.config.PurgeTagsKeepCount)
+}

registry/client.go

@@ -38,7 +38,7 @@ func NewClient(url string, verifyTLS bool, username, password string) *Client {
 		password:  password,
 
 		request:   gorequest.New().TLSClientConfig(&tls.Config{InsecureSkipVerify: !verifyTLS}),
-		logger:    setupLogging("registry.client"),
+		logger:    SetupLogging("registry.client"),
 		tokens:    map[string]string{},
 		repos:     map[string][]string{},
 		tagCounts: map[string]int{},
@@ -69,7 +69,7 @@ func NewClient(url string, verifyTLS bool, username, password string) *Client {
 			c.logger.Warn("No token auth service discovered from ", c.url)
 			return nil
 		}
-	} else if strings.HasPrefix(authHeader, "Basic") {
+	} else if strings.HasPrefix(strings.ToLower(authHeader), "basic") {
 		c.request = c.request.SetBasicAuth(c.username, c.password)
 		c.logger.Info("It was discovered the registry is configured with HTTP basic auth.")
 	}

registry/common.go

@@ -8,8 +8,8 @@ import (
 	"github.com/hhkbp2/go-logging"
 )
 
-// setupLogging configure logging.
+// SetupLogging configure logging.
-func setupLogging(name string) logging.Logger {
+func SetupLogging(name string) logging.Logger {
 	logger := logging.GetLogger(name)
 	handler := logging.NewStdoutHandler()
 	format := "%(asctime)s - %(name)s - %(levelname)s - %(message)s"

registry/event_listener.go

@@ -1,132 +0,0 @@
-package registry
-
-import (
-	"database/sql"
-	"encoding/json"
-	"fmt"
-	"net/http"
-	"strings"
-	"time"
-
-	"github.com/grafana/grafana/pkg/cmd/grafana-cli/logger"
-	// 🐒 patching of "database/sql".
-	_ "github.com/mattn/go-sqlite3"
-	"github.com/tidwall/gjson"
-)
-
-const (
-	dbFile = "data/registry_events.db"
-	schema = `
-	CREATE TABLE IF NOT EXISTS events (
-		id INTEGER PRIMARY KEY AUTOINCREMENT,
-		action CHAR(4) NULL,
-		repository VARCHAR(100) NULL,
-		tag VARCHAR(100) NULL,
-		ip VARCHAR(15) NULL,
-		user VARCHAR(50) NULL,
-		created DATETIME NULL
-	);
-`
-)
-
-type eventData struct {
-	Events []interface{} `json:"events"`
-}
-
-// EventRow event row from sqlite
-type EventRow struct {
-	ID         int
-	Action     string
-	Repository string
-	Tag        string
-	IP         string
-	User       string
-	Created    time.Time
-}
-
-// ProcessEvents parse and store registry events
-func ProcessEvents(request *http.Request, retention int) {
-	logger := setupLogging("registry.event_listener")
-	decoder := json.NewDecoder(request.Body)
-	var t eventData
-	if err := decoder.Decode(&t); err != nil {
-		logger.Errorf("Problem decoding event from request: %+v", request)
-		return
-	}
-	logger.Debugf("Received event: %+v", t)
-	j, _ := json.Marshal(t)
-
-	db, err := sql.Open("sqlite3", dbFile)
-	if err != nil {
-		logger.Error("Error opening sqlite db: ", err)
-		return
-	}
-	defer db.Close()
-
-	_, err = db.Exec(schema)
-	if err != nil {
-		logger.Error("Error creating a table: ", err)
-		return
-	}
-
-	stmt, _ := db.Prepare("INSERT INTO events(action, repository, tag, ip, user, created) values(?,?,?,?,?,DateTime('now'))")
-	for _, e := range gjson.GetBytes(j, "events").Array() {
-		// Ignore calls by docker-registry-ui itself.
-		if e.Get("request.useragent").String() == "docker-registry-ui" {
-			continue
-		}
-		action := e.Get("action").String()
-		repository := e.Get("target.repository").String()
-		tag := e.Get("target.tag").String()
-		// Tag is empty in case of signed pull.
-		if tag == "" {
-			tag = e.Get("target.digest").String()
-		}
-		ip := strings.Split(e.Get("request.addr").String(), ":")[0]
-		user := e.Get("actor.name").String()
-		logger.Debugf("Parsed event data: %s %s:%s %s %s ", action, repository, tag, ip, user)
-
-		res, err := stmt.Exec(action, repository, tag, ip, user)
-		if err != nil {
-			logger.Error("Error inserting a row: ", err)
-			return
-		}
-		id, _ := res.LastInsertId()
-		logger.Debug("New event added with id ", id)
-	}
-
-	// Purge old records.
-	stmt, _ = db.Prepare("DELETE FROM events WHERE created < DateTime('now',?)")
-	res, _ := stmt.Exec(fmt.Sprintf("-%d day", retention))
-	count, _ := res.RowsAffected()
-	logger.Debug("Rows deleted: ", count)
-}
-
-// GetEvents retrieve events from sqlite db
-func GetEvents(repository string) []EventRow {
-	var events []EventRow
-	db, err := sql.Open("sqlite3", dbFile)
-	if err != nil {
-		logger.Error("Error opening sqlite db: ", err)
-		return events
-	}
-	defer db.Close()
-
-	query := "SELECT * FROM events ORDER BY id DESC LIMIT 1000"
-	if repository != "" {
-		query = fmt.Sprintf("SELECT * FROM events WHERE repository='%s' ORDER BY id DESC LIMIT 5", repository)
-	}
-	rows, err := db.Query(query)
-	if err != nil {
-		logger.Error("Error selecting from table: ", err)
-		return events
-	}
-
-	for rows.Next() {
-		var row EventRow
-		rows.Scan(&row.ID, &row.Action, &row.Repository, &row.Tag, &row.IP, &row.User, &row.Created)
-		events = append(events, row)
-	}
-	rows.Close()
-	return events
-}

registry/tasks.go

@@ -34,7 +34,7 @@ func (p timeSlice) Swap(i, j int) {
 
 // PurgeOldTags purge old tags.
 func PurgeOldTags(client *Client, purgeDryRun bool, purgeTagsKeepDays, purgeTagsKeepCount int) {
-	logger := setupLogging("registry.tasks.PurgeOldTags")
+	logger := SetupLogging("registry.tasks.PurgeOldTags")
 	// Reduce client logging.
 	client.logger.SetLevel(logging.LevelError)
 

template.go

@@ -0,0 +1,75 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"net/url"
+	"strings"
+
+	"github.com/CloudyKit/jet"
+	"github.com/labstack/echo"
+	"github.com/quiq/docker-registry-ui/registry"
+	"github.com/tidwall/gjson"
+)
+
+// Template Jet template.
+type Template struct {
+	View *jet.Set
+}
+
+// Render render template.
+func (r *Template) Render(w io.Writer, name string, data interface{}, c echo.Context) error {
+	t, err := r.View.GetTemplate(name)
+	if err != nil {
+		panic(fmt.Errorf("Fatal error template file: %s", err))
+	}
+	vars, ok := data.(jet.VarMap)
+	if !ok {
+		vars = jet.VarMap{}
+	}
+	err = t.Execute(w, vars, nil)
+	if err != nil {
+		panic(fmt.Errorf("Error rendering template %s: %s", name, err))
+	}
+	return nil
+}
+
+// setupRenderer template engine init.
+func setupRenderer(debug bool, registryHost, basePath string) *Template {
+	view := jet.NewHTMLSet("templates")
+	view.SetDevelopmentMode(debug)
+
+	view.AddGlobal("basePath", basePath)
+	view.AddGlobal("registryHost", registryHost)
+	view.AddGlobal("pretty_size", func(size interface{}) string {
+		var value float64
+		switch i := size.(type) {
+		case gjson.Result:
+			value = float64(i.Int())
+		case int64:
+			value = float64(i)
+		}
+		return registry.PrettySize(value)
+	})
+	view.AddGlobal("pretty_time", func(datetime interface{}) string {
+		d := strings.Replace(datetime.(string), "T", " ", 1)
+		d = strings.Replace(d, "Z", "", 1)
+		return strings.Split(d, ".")[0]
+	})
+	view.AddGlobal("parse_map", func(m interface{}) string {
+		var res string
+		for _, k := range registry.SortedMapKeys(m) {
+			res = res + fmt.Sprintf(`<tr><td style="padding: 0 10px; width: 20%%">%s</td><td style="padding: 0 10px">%v</td></tr>`, k, m.(map[string]interface{})[k])
+		}
+		return res
+	})
+	view.AddGlobal("url_decode", func(m interface{}) string {
+		res, err := url.PathUnescape(m.(string))
+		if err != nil {
+			return m.(string)
+		}
+		return res
+	})
+
+	return &Template{View: view}
+}

templates/base.html

@@ -15,7 +15,7 @@
                 <h2>Docker Registry UI</h2>
             </div>
             <div style="float: right">
-                <a href="/events">Event Log</a>
+                <a href="{{ basePath }}/events">Event Log</a>
             </div>
             <div style="clear: both"></div>
 

templates/event_log.html

@@ -14,7 +14,7 @@
 
 {{block body()}}
 <ol class="breadcrumb">
-    <li><a href="/">Home</a></li>
+    <li><a href="{{ basePath }}/">Home</a></li>
     <li class="active">Event Log</li>
 </ol>
 
@@ -39,7 +39,7 @@
                 {{end}}
                 <td>{{ e.IP }}</td>
                 <td>{{ e.User }}</td>
-                <td>{{ e.Created }}</td>
+                <td>{{ e.Created|pretty_time }}</td>
             </tr>
         {{end}}
     </tbody>

templates/repositories.html

@@ -8,12 +8,14 @@
             "stateSave": true
         });
         $('#namespace').on('change', function (e) {
-            window.location = '/' + this.value;
+            window.location = '{{ basePath }}/' + this.value;
         });
-        if (window.location.pathname == '/') {
+        namespace = window.location.pathname;
+        namespace = namespace.replace("{{ basePath }}", "");
+        if (namespace == '/') {
             namespace = 'library';
         } else {
-            namespace = window.location.pathname.split('/')[1]
+            namespace = namespace.split('/')[1]
         }
         $('#namespace').val(namespace);
     });
@@ -23,15 +25,19 @@
 {{block body()}}
 <div style="float: right">
     <select id="namespace" class="form-control input-sm" style="height: 36px">
-        <option value="" disabled>-- Namespace --</option>
         {{range namespace := namespaces}}
         <option value="{{ namespace }}">{{ namespace }}</option>
         {{end}}
     </select>
 </div>
+<div style="float: right">
+    <ol class="breadcrumb">
+        <li class="active">Namespace</li>
+    </ol>
+</div>
 
 <ol class="breadcrumb">
-    <li><a href="/">Home</a></li>
+    <li><a href="{{ basePath }}/">Home</a></li>
 </ol>
 
 <table id="datatable" class="table table-striped table-bordered">
@@ -44,7 +50,7 @@
     <tbody>
         {{range repo := repos}}
             <tr>
-                <td><a href="/{{ namespace }}/{{ repo }}">{{ repo }}</a></td>
+                <td><a href="{{ basePath }}/{{ namespace }}/{{ repo|url }}">{{ repo }}</a></td>
                 <td>{{ tagCounts[namespace+"/"+repo] }}</td>
             </tr>
         {{end}}

templates/tag_info.html

@@ -4,11 +4,11 @@
 
 {{block body()}}
 <ol class="breadcrumb">
-    <li><a href="/">Home</a></li>
+    <li><a href="{{ basePath }}/">Home</a></li>
     {{if namespace != "library"}}
-    <li><a href="/{{ namespace }}">{{ namespace }}</a></li>
+    <li><a href="{{ basePath }}/{{ namespace }}">{{ namespace }}</a></li>
     {{end}}
-    <li><a href="/{{ namespace }}/{{ repo }}">{{ repo }}</a></li>
+    <li><a href="{{ basePath }}/{{ namespace }}/{{ repo }}">{{ repo|url_decode }}</a></li>
     <li class="active">{{ tag }}</li>
 </ol>
 <table class="table table-striped table-bordered">

templates/tags.html

@@ -1,7 +1,7 @@
 {{extends "base.html"}}
 
 {{block head()}}
-<script type="text/javascript" src="/static/bootstrap-confirmation.min.js"></script>
+<script type="text/javascript" src="{{ basePath }}/static/bootstrap-confirmation.min.js"></script>
 <script type="text/javascript" src="https://cdn.datatables.net/plug-ins/1.10.16/sorting/natural.js"></script>
 <script type="text/javascript">
     $(document).ready(function() {
@@ -29,11 +29,11 @@
 
 {{block body()}}
 <ol class="breadcrumb">
-    <li><a href="/">Home</a></li>
+    <li><a href="{{ basePath }}/">Home</a></li>
     {{if namespace != "library"}}
-    <li><a href="/{{ namespace }}">{{ namespace }}</a></li>
+    <li><a href="{{ basePath }}/{{ namespace }}">{{ namespace }}</a></li>
     {{end}}
-    <li class="active">{{ repo }}</li>
+    <li class="active">{{ repo|url_decode }}</li>
 </ol>
 
 <table id="datatable" class="table table-striped table-bordered">
@@ -46,9 +46,9 @@
         {{range tag := tags}}
         <tr>
             <td>
-                <a href="/{{ namespace }}/{{ repo }}/{{ tag }}">{{ tag }}</a>
+                <a href="{{ basePath }}/{{ namespace }}/{{ repo }}/{{ tag }}">{{ tag }}</a>
                 {{if deleteAllowed}}
-                <a href="/{{ namespace }}/{{ repo }}/{{ tag }}/delete" data-toggle="confirmation" class="btn btn-danger btn-xs pull-right" role="button">Delete</a>
+                <a href="{{ basePath }}/{{ namespace }}/{{ repo }}/{{ tag }}/delete" data-toggle="confirmation" class="btn btn-danger btn-xs pull-right" role="button">Delete</a>
                 {{end}}
             </td>
         </tr>
@@ -78,7 +78,7 @@
                 {{end}}
                 <td>{{ e.IP }}</td>
                 <td>{{ e.User }}</td>
-                <td>{{ e.Created }}</td>
+                <td>{{ e.Created|pretty_time }}</td>
             </tr>
         {{end}}
     </tbody>