mirror of
https://github.com/containers/skopeo.git
synced 2025-09-28 05:25:48 +00:00
(skopeo copy) and (skopeo sync) now support --sign-by-sigstore=param-file, using the containers-sigstore-signing-params.yaml(5) file format. That notably adds support for Fulcio and Rekor signing. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
235 lines
8.8 KiB
Markdown
235 lines
8.8 KiB
Markdown
# ksuid [](https://goreportcard.com/report/github.com/segmentio/ksuid) [](https://godoc.org/github.com/segmentio/ksuid) [](https://circleci.com/gh/segmentio/ksuid.svg?style=shield)
|
|
|
|
ksuid is an efficient, comprehensive, battle-tested Go library for
|
|
generating and parsing a specific kind of globally unique identifier
|
|
called a *KSUID*. This library serves as its reference implementation.
|
|
|
|
## Install
|
|
```sh
|
|
go get -u github.com/segmentio/ksuid
|
|
```
|
|
|
|
## What is a KSUID?
|
|
|
|
KSUID is for K-Sortable Unique IDentifier. It is a kind of globally
|
|
unique identifier similar to a [RFC 4122 UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier), built from the ground-up to be "naturally"
|
|
sorted by generation timestamp without any special type-aware logic.
|
|
|
|
In short, running a set of KSUIDs through the UNIX `sort` command will result
|
|
in a list ordered by generation time.
|
|
|
|
## Why use KSUIDs?
|
|
|
|
There are numerous methods for generating unique identifiers, so why KSUID?
|
|
|
|
1. Naturally ordered by generation time
|
|
2. Collision-free, coordination-free, dependency-free
|
|
3. Highly portable representations
|
|
|
|
Even if only one of these properties are important to you, KSUID is a great
|
|
choice! :) Many projects chose to use KSUIDs *just* because the text
|
|
representation is copy-and-paste friendly.
|
|
|
|
### 1. Naturally Ordered By Generation Time
|
|
|
|
Unlike the more ubiquitous UUIDv4, a KSUID contains a timestamp component
|
|
that allows them to be loosely sorted by generation time. This is not a strong
|
|
guarantee (an invariant) as it depends on wall clocks, but is still incredibly
|
|
useful in practice. Both the binary and text representations will sort by
|
|
creation time without any special sorting logic.
|
|
|
|
### 2. Collision-free, Coordination-free, Dependency-free
|
|
|
|
While RFC 4122 UUIDv1s *do* include a time component, there aren't enough
|
|
bytes of randomness to provide strong protection against collisions
|
|
(duplicates). With such a low amount of entropy, it is feasible for a
|
|
malicious party to guess generated IDs, creating a problem for systems whose
|
|
security is, implicitly or explicitly, sensitive to an adversary guessing
|
|
identifiers.
|
|
|
|
To fit into a 64-bit number space, [Snowflake IDs](https://blog.twitter.com/2010/announcing-snowflake)
|
|
and its derivatives require coordination to avoid collisions, which
|
|
significantly increases the deployment complexity and operational burden.
|
|
|
|
A KSUID includes 128 bits of pseudorandom data ("entropy"). This number space
|
|
is 64 times larger than the 122 bits used by the well-accepted RFC 4122 UUIDv4
|
|
standard. The additional timestamp component can be considered "bonus entropy"
|
|
which further decreases the probability of collisions, to the point of physical
|
|
infeasibility in any practical implementation.
|
|
|
|
### Highly Portable Representations
|
|
|
|
The text *and* binary representations are lexicographically sortable, which
|
|
allows them to be dropped into systems which do not natively support KSUIDs
|
|
and retain their time-ordered property.
|
|
|
|
The text representation is an alphanumeric base62 encoding, so it "fits"
|
|
anywhere alphanumeric strings are accepted. No delimiters are used, so
|
|
stringified KSUIDs won't be inadvertently truncated or tokenized when
|
|
interpreted by software that is designed for human-readable text, a common
|
|
problem for the text representation of RFC 4122 UUIDs.
|
|
|
|
## How do KSUIDs work?
|
|
|
|
Binary KSUIDs are 20-bytes: a 32-bit unsigned integer UTC timestamp and
|
|
a 128-bit randomly generated payload. The timestamp uses big-endian
|
|
encoding, to support lexicographic sorting. The timestamp epoch is adjusted
|
|
to March 5th, 2014, providing over 100 years of life. The payload is
|
|
generated by a cryptographically-strong pseudorandom number generator.
|
|
|
|
The text representation is always 27 characters, encoded in alphanumeric
|
|
base62 that will lexicographically sort by timestamp.
|
|
|
|
## High Performance
|
|
|
|
This library is designed to be used in code paths that are performance
|
|
critical. Its code has been tuned to eliminate all non-essential
|
|
overhead. The `KSUID` type is derived from a fixed-size array, which
|
|
eliminates the additional reference chasing and allocation involved in
|
|
a variable-width type.
|
|
|
|
The API provides an interface for use in code paths which are sensitive
|
|
to allocation. For example, the `Append` method can be used to parse the
|
|
text representation and replace the contents of a `KSUID` value
|
|
without additional heap allocation.
|
|
|
|
All public package level "pure" functions are concurrency-safe, protected
|
|
by a global mutex. For hot loops that generate a large amount of KSUIDs
|
|
from a single Goroutine, the `Sequence` type is provided to elide the
|
|
potential contention.
|
|
|
|
By default, out of an abundance of caution, the cryptographically-secure
|
|
PRNG is used to generate the random bits of a KSUID. This can be relaxed
|
|
in extremely performance-critical code using the included `FastRander`
|
|
type. `FastRander` uses the standard PRNG with a seed generated by the
|
|
cryptographically-secure PRNG.
|
|
|
|
*_NOTE:_ While there is no evidence that `FastRander` will increase the
|
|
probability of a collision, it shouldn't be used in scenarios where
|
|
uniqueness is important to security, as there is an increased chance
|
|
the generated IDs can be predicted by an adversary.*
|
|
|
|
## Battle Tested
|
|
|
|
This code has been used in production at Segment for several years,
|
|
across a diverse array of projects. Trillions upon trillions of
|
|
KSUIDs have been generated in some of Segment's most
|
|
performance-critical, large-scale distributed systems.
|
|
|
|
## Plays Well With Others
|
|
|
|
Designed to be integrated with other libraries, the `KSUID` type
|
|
implements many standard library interfaces, including:
|
|
|
|
* `Stringer`
|
|
* `database/sql.Scanner` and `database/sql/driver.Valuer`
|
|
* `encoding.BinaryMarshal` and `encoding.BinaryUnmarshal`
|
|
* `encoding.TextMarshal` and `encoding.TextUnmarshal`
|
|
(`encoding/json` friendly!)
|
|
|
|
## Command Line Tool
|
|
|
|
This package comes with a command-line tool `ksuid`, useful for
|
|
generating KSUIDs as well as inspecting the internal components of
|
|
existing KSUIDs. Machine-friendly output is provided for scripting
|
|
use cases.
|
|
|
|
Given a Go build environment, it can be installed with the command:
|
|
|
|
```sh
|
|
$ go install github.com/segmentio/ksuid/cmd/ksuid
|
|
```
|
|
|
|
## CLI Usage Examples
|
|
|
|
### Generate a KSUID
|
|
|
|
```sh
|
|
$ ksuid
|
|
0ujsswThIGTUYm2K8FjOOfXtY1K
|
|
```
|
|
|
|
### Generate 4 KSUIDs
|
|
|
|
```sh
|
|
$ ksuid -n 4
|
|
0ujsszwN8NRY24YaXiTIE2VWDTS
|
|
0ujsswThIGTUYm2K8FjOOfXtY1K
|
|
0ujssxh0cECutqzMgbtXSGnjorm
|
|
0ujsszgFvbiEr7CDgE3z8MAUPFt
|
|
```
|
|
|
|
### Inspect the components of a KSUID
|
|
|
|
```sh
|
|
$ ksuid -f inspect 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
|
|
|
REPRESENTATION:
|
|
|
|
String: 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
|
Raw: 0669F7EFB5A1CD34B5F99D1154FB6853345C9735
|
|
|
|
COMPONENTS:
|
|
|
|
Time: 2017-10-09 21:00:47 -0700 PDT
|
|
Timestamp: 107608047
|
|
Payload: B5A1CD34B5F99D1154FB6853345C9735
|
|
```
|
|
|
|
### Generate a KSUID and inspect its components
|
|
|
|
```sh
|
|
$ ksuid -f inspect
|
|
|
|
REPRESENTATION:
|
|
|
|
String: 0ujzPyRiIAffKhBux4PvQdDqMHY
|
|
Raw: 066A029C73FC1AA3B2446246D6E89FCD909E8FE8
|
|
|
|
COMPONENTS:
|
|
|
|
Time: 2017-10-09 21:46:20 -0700 PDT
|
|
Timestamp: 107610780
|
|
Payload: 73FC1AA3B2446246D6E89FCD909E8FE8
|
|
|
|
```
|
|
|
|
### Inspect a KSUID with template formatted inspection output
|
|
|
|
```sh
|
|
$ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
|
2017-10-09 21:00:47 -0700 PDT: B5A1CD34B5F99D1154FB6853345C9735
|
|
```
|
|
|
|
### Inspect multiple KSUIDs with template formatted output
|
|
|
|
```sh
|
|
$ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' $(ksuid -n 4)
|
|
2017-10-09 21:05:37 -0700 PDT: 304102BC687E087CC3A811F21D113CCF
|
|
2017-10-09 21:05:37 -0700 PDT: EAF0B240A9BFA55E079D887120D962F0
|
|
2017-10-09 21:05:37 -0700 PDT: DF0761769909ABB0C7BB9D66F79FC041
|
|
2017-10-09 21:05:37 -0700 PDT: 1A8F0E3D0BDEB84A5FAD702876F46543
|
|
```
|
|
|
|
### Generate KSUIDs and output JSON using template formatting
|
|
|
|
```sh
|
|
$ ksuid -f template -t '{ "timestamp": "{{ .Timestamp }}", "payload": "{{ .Payload }}", "ksuid": "{{.String}}"}' -n 4
|
|
{ "timestamp": "107611700", "payload": "9850EEEC191BF4FF26F99315CE43B0C8", "ksuid": "0uk1Hbc9dQ9pxyTqJ93IUrfhdGq"}
|
|
{ "timestamp": "107611700", "payload": "CC55072555316F45B8CA2D2979D3ED0A", "ksuid": "0uk1HdCJ6hUZKDgcxhpJwUl5ZEI"}
|
|
{ "timestamp": "107611700", "payload": "BA1C205D6177F0992D15EE606AE32238", "ksuid": "0uk1HcdvF0p8C20KtTfdRSB9XIm"}
|
|
{ "timestamp": "107611700", "payload": "67517BA309EA62AE7991B27BB6F2FCAC", "ksuid": "0uk1Ha7hGJ1Q9Xbnkt0yZgNwg3g"}
|
|
```
|
|
|
|
## Implementations for other languages
|
|
|
|
- Python: [svix-ksuid](https://github.com/svixhq/python-ksuid/)
|
|
- Ruby: [ksuid-ruby](https://github.com/michaelherold/ksuid-ruby)
|
|
- Java: [ksuid](https://github.com/ksuid/ksuid)
|
|
- Rust: [rksuid](https://github.com/nharring/rksuid)
|
|
- dotNet: [Ksuid.Net](https://github.com/JoyMoe/Ksuid.Net)
|
|
|
|
## License
|
|
|
|
ksuid source code is available under an MIT [License](/LICENSE.md).
|