Demo on k3s
A local Kubernetes in Docker, on which the server is deployed one way or another: one pod, three pods that share their state, or the same objects without the chart. Docker is the only prerequisite.
demo/k3s/demo.sh cluster-up # k3s in Docker; again: resumes and repairs
demo/k3s/demo.sh deploy cluster # the chart, 3 replicas
demo/k3s/demo.sh check cluster # plays the scenario and checks it
demo/k3s/demo.sh deploy single --preset example --with-s3
demo/k3s/demo.sh list # modes, deployed or not, ports
demo/k3s/demo.sh remove single # its namespace only
demo/k3s/demo.sh cluster-down # every cfg-demo-* container, volume, network
deploy prints the addresses and the accounts, with passwords drawn at each
deployment and hashed by the image’s hash-password; they stay in
demo/k3s/.work/<mode>/env.sh. The steps: Multi-instance scenario.
After an interrupted run or a docker restart cfg-demo-k3s, cluster-up
waits for the API server again, recreates the tools container and restarts
the console port-forwards of the deployed modes.
Modes
A mode is the topology: demo/k3s/modes/<mode>/. Each lives in its
namespace cfg-demo-<mode>, its release is called gate, and the modes
coexist on their own host ports.
| Mode | What | SFTP | Explorer | Console | S3 console |
|---|---|---|---|---|---|
cluster | the chart, 3 replicas, ban.backend: configmap, shared Secret, cluster channel, control door, cluster.unreadyWhen: auto | 32122 | 32180 | 32181 | 32190 |
single | the chart, 1 replica, bans in a file of a volume | 32222 | 32280 | 32281 | 32290 |
ansible | the playbook, the objects of cluster | 32322 | 32380 | 32381 | 32390 |
The explorer is http://127.0.0.1:<port>/files/, the console
http://127.0.0.1:<port>/ (a kubectl port-forward to gate-control), the
S3 console http://127.0.0.1:<port>/rustfs/console/.
CFG_DEMO_PORT_BASE (32000) moves every port, CFG_DEMO_BIND (127.0.0.1)
the address; both are read by cluster-up.
| Mode file | Role |
|---|---|
mode.env | DESCRIPTION, SLOT (ports: base + 100 x slot), KIND (helm or ansible), TOPOLOGY (cluster or single), REPLICAS |
values.yaml | the chart’s values (KIND=helm) |
check.sh | its scenario, run by demo.sh check <mode> |
Another mode is another directory: a slot not taken (1 to 4), its values.
Presets
A preset is the content, the same in every mode: demo/k3s/presets/<name>/,
or any directory (--preset DIR, or PRESET=DIR). Every file is optional;
without the file, presets/default/ serves.
| File | Effect |
|---|---|
config.toml | the configuration, in place of presets/default/config.toml |
accounts.env | ADMIN_PASSWORD, USER_PASSWORD: the passwords of admin and demo, in place of drawn ones |
users.toml | [[users]] added to the configuration, hashed beforehand |
values.yaml | chart values over the mode’s (helm modes) |
data/files/... | files of the local backend (its root), after those of presets/default/data/ |
data/archive/... | objects of the bucket, under the backend’s prefix (with S3) |
bans.json | [{"ip", "protocol", "minutes"}]: bans in force from the start |
grants.json | [{"username", "role", "hours", "reason"}]: temporary access granted at deployment |
preset.env | WITH_S3=1: as --with-s3 |
The mode adds what is its own, in every configuration: backend (ConfigMap)
or persist_file (file) under each [sftp.ban], [api.ban],
[admin.ban], and [cluster]; the chart passes the shared Secret, the
revocations, the grants and the peers to the server. A preset’s
config.toml leaves out [cluster] and, under the ban, session and grants
tables, backend, persist_file, key_file, secret_name and the
ConfigMap names. Before anything is deployed, the configuration goes through
config check of the image, with the mode’s variables, and the values
through helm template.
| Placeholder | Value |
|---|---|
{{ADMIN_PASSWORD_HASH}}, {{USER_PASSWORD_HASH}} | the hashes of the two passwords |
{{HOST_KEY}}, {{FILES_ROOT}} | /keys/host_ed25519 (Secret gate-host-keys), /data (the volume shared by the pods) |
{{S3_ENDPOINT}}, {{S3_BUCKET}}, {{S3_PREFIX}} | http://s3:9000, archive, gate/ |
{{S3_ACCESS_KEY_FILE}}, {{S3_SECRET_KEY_FILE}} | the keys of the Secret demo-s3, mounted as files |
lines between #{{S3_BEGIN}} and #{{S3_END}} | kept with S3 only |
presets/example/ uses every file: fixed passwords, an account partner,
the DEMO environment in green, a file and an object, a ban, an access, S3.
The S3 store
Without S3, the local backend alone: enough to drop and fetch files, and
the lightest. --with-s3 adds RustFS (rustfs/rustfs:1.0.3, pulled by k3s
on first use) and the archive backend (/archive): its keys, drawn at
deployment, are printed with the accounts, and its web console shows the
objects as they arrive. Measured here: a server pod holds about 5 MiB,
RustFS about 60 MiB.
What is built
| Piece | In the demo |
|---|---|
| k3s | the cfg-demo-k3s container (rancher/k3s:v1.34.1-k3s1), without Traefik, ServiceLB or metrics-server |
| image | craftogether/craft-file-gate:v<appVersion> (the chart’s appVersion, the latest release); LOCAL_IMAGE=craft-file-gate:dev imports a local image |
| volumes | demo-files (the local backend, shared by the pods: one node), demo-state (ban files of single), written through the demo-seed pod |
| tools | the cfg-demo-tools container (sftp, curl, jq) in the node’s network: it reaches each pod |
Short delays, for the demo, in presets/default/config.toml:
[server.backend_probe]
interval_secs = 5 # a storage outage is seen in about 7 s
timeout_secs = 2
failures_before_down = 1
[admin.session]
reread_interval_secs = 1 # likewise [admin.grants]
The cluster mode adds unready_after_secs = 10 and ready_after_secs = 5
under [cluster].