jack/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: jack:fba7eb68ea70f2fec3bd1a4ae06d68fbe365e682
Branches Tags
jack:master
..
compare: jack:3b875f57d2b5fb3fb839eea7a95c1e19be0dbeb8
Branches Tags
jack:master

12 commits
fba7eb68ea ... 3b875f57d2

Author SHA1 Message Date
jack
3b875f57d2 fix: disable discord-bot and walava-web until images exist in registry
Some checks failed
CI/CD / syntax-check (push) Successful in 3m0s
Details
CI/CD / deploy (push) Failing after 1m39s
Details 
These custom images (discord-bot, walava-web) are built by their own
repos' CI/CD and pushed to git.walava.io registry. On a fresh server
Forgejo hasn't run yet so images don't exist — bootstrap chicken/egg.
Re-enable after Forgejo is up and images are pushed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 06:26:14 +07:00
jack
f4688ed8be fix: disable outline-mcp until image is built and pushed to registry 
outline-mcp uses git.walava.io/jack/outline-mcp:latest which doesn't
exist in Forgejo registry yet (Forgejo itself wasn't running).
Comment out the service; re-enable after building the image.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 06:05:30 +07:00
jack
7aa5574098 chore: remove mon from inventory, update server descriptions 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 04:29:26 +07:00
jack
f704ede1cd chore: rename servers to main and tools in Timeweb 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 04:27:23 +07:00
jack
8a3aaa2fca feat: Terraform infra-as-code + delete mon server + fix S3/Outline 
Terraform: imported main (7004701) + tools (7076013) into state,
destroyed mon (7076015, 188.225.79.34). State: No changes.

S3: fix endpoint s3.timeweb.cloud → s3.twcstorage.ru (actual Timeweb
endpoint), remove AWS_S3_ACL=private (Timeweb doesn't support per-object
ACLs — was causing Outline upload failures).

Vault: added vault_timeweb_token.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 04:26:33 +07:00
jack
862eac5f11 feat: add Terraform config for Timeweb Cloud infrastructure 
Manages main + tools servers and S3 buckets (walava-backup, walava-outline).
Includes mon server resource for import + destroy workflow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 04:15:27 +07:00
jack
fde51352d7 feat: migrate monitoring to tools server, fix Outline S3 uploads 
Monitoring stack (Prometheus, AlertManager, Grafana, Loki, Uptime Kuma)
moved from main to tools server. Prometheus now scrapes main exporters
over network (ip_main:9100/8080). Promtail pushes logs to ip_tools:3100.
Traefik routes for dash/status.walava.io updated to ip_tools. discord-bot
PROMETHEUS_URL updated to http://ip_tools:9090.

Outline S3 fix: remove AWS_S3_ACL=private (Timeweb doesn't support
per-object ACLs — caused upload failures). Add CORS configuration task
for browser-side presigned uploads.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 04:10:28 +07:00
jack
d6015b76a3 fix: add proxy network to Outline and n8n for outbound internet access 
Outline needs proxy network for SMTP (Resend) and S3 (Timeweb).
n8n needs proxy network for external API calls in workflows.
Both were only on backend (internal:true) so DNS/TCP to internet was blocked.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 03:54:19 +07:00
jack
036b80501f ci: retrigger deploy after image retag 2026-03-27 03:33:33 +07:00
jack
521c806ed9 docs: update STATUS.md — reflect walava.io migration and service layout 2026-03-27 03:11:54 +07:00
jack
36be9fb33d chore: remove SMTP relay, clean up tools role after Outline/n8n migration to main 
- Remove smtp-relay (postfix) container — Outline now on main, uses Resend directly
- Remove UFW port 1025 rule (SMTP relay no longer needed)
- Remove postfix-relay from image pull list
- Clean up tools role: remove Outline/n8n/env.j2, simplify tasks/main.yml
- tools docker-compose now empty (pending monitoring migration)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 03:10:56 +07:00
jack
489791403c feat: migrate Outline + n8n to main server, rename S3 buckets to walava-* 
- Add Outline, outline-db, outline-redis, n8n, outline-mcp containers to main docker-compose
- Add env.outline.j2 template with Resend SMTP and S3 (walava-outline bucket)
- Update Traefik routes: wiki → outline:3000, auto → n8n:5678 (local, not cross-server)
- Rename S3 buckets: visual-backup → walava-backup, visual-outline → walava-outline
- Extend backup.sh.j2: add Outline DB, n8n, Plane MinIO to backup scope
- Add outline_image, n8n_image, outline_mcp_image to services/defaults
- Remove Authelia config deployment tasks from configs.yml
- Add outline-internal and n8n-internal networks to docker-compose

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 03:04:54 +07:00
34 changed files with 17726 additions and 679 deletions
Show stats Expand all files Collapse all files

128
docs/STATUS.md
View file

@ -1,7 +1,7 @@
# Статус инфраструктуры # Статус инфраструктуры
> Обновляй этот файл при каждом значимом изменении. > Обновляй этот файл при каждом значимом изменении.
> Последнее обновление: 2026-03-23 > Последнее обновление: 2026-03-27
--- ---
@ -9,8 +9,10 @@
| Сервер | IP | Роль | Состояние | | Сервер | IP | Роль | Состояние |
|--------|----|------|-----------| |--------|----|------|-----------|
| **main** | 87.249.49.32 | Основные сервисы + мониторинг | ✅ Работает | | **main** | 87.249.49.32 | Все продуктовые сервисы + мониторинг | ✅ Работает |
| **tools** | 85.193.83.9 | Wiki + автоматизация + почта | ✅ Работает | | **tools** | 85.193.83.9 | Вспомогательные сервисы (пусто, ожидает мониторинг) | ✅ Работает |
> mon (188.225.79.34) — планируется к отключению.
--- ---
@ -20,95 +22,91 @@
| Сервис | Домен | Статус | Заметки | | Сервис | Домен | Статус | Заметки |
|--------|-------|--------|---------| |--------|-------|--------|---------|
| Traefik | — | ✅ | Реверс-прокси, TLS wildcard `*.csrx.ru` через Cloudflare DNS-01 | | Traefik | — | ✅ | Реверс-прокси, TLS wildcard `*.walava.io` через Cloudflare DNS-01 |
| Vaultwarden | vault.csrx.ru | ✅ | Менеджер паролей | | Vaultwarden | vault.walava.io | ✅ | Менеджер паролей |
| Forgejo | git.csrx.ru | ✅ | Git-сервер, SSH на порту 2222 | | Forgejo | git.walava.io | ✅ | Git-сервер, SSH на порту 2222 |
| Forgejo Actions | — | ✅ | CI/CD runner, деплой через push в master | | Forgejo Actions | — | ✅ | CI/CD runner, деплой через push в master |
| Plane | plane.csrx.ru | ✅ | Управление проектами | | Plane | plane.walava.io | ✅ | Управление проектами |
| Grafana | dash.csrx.ru | ✅ | Дашборды мониторинга | | Outline Wiki | wiki.walava.io | 🔄 Переезд (CI в процессе) | SMTP: Resend через walava.io |
| n8n | auto.walava.io | 🔄 Переезд (CI в процессе) | Workflow автоматизация |
| outline-mcp | — | 🔄 Переезд | MCP сервер для Claude |
| discord-bot | — | ✅ | Деплой-нотификации в Discord |
| walava-web | walava.io | ✅ | Лендинг (заглушка) |
| Grafana | dash.walava.io | ✅ | Дашборды мониторинга |
| Prometheus | — | ✅ | Сбор метрик, 30 дней хранения | | Prometheus | — | ✅ | Сбор метрик, 30 дней хранения |
| Loki + Promtail | — | ✅ | Сбор логов | | Loki + Promtail | — | ✅ | Сбор логов |
| AlertManager | — | ✅ | Алерты в Telegram | | AlertManager | — | ✅ | Алерты в Telegram |
| CrowdSec | — | ✅ | IDS, банит злоумышленников | | CrowdSec | — | ✅ | IDS, банит злоумышленников |
| Authelia | auth.csrx.ru | ✅ | 2FA SSO, защищает traefik dashboard и plane/god-mode | | Uptime Kuma | status.walava.io | ✅ | Публичная страница статуса |
| Uptime Kuma | status.csrx.ru | ✅ | Публичная страница статуса | | Бэкап | — | ✅ | Каждые 6 часов → S3 `walava-backup/data/`, 7 дней |
| Бэкап | — | ✅ | Каждые 6 часов (00/06/12/18) → S3 `visual-backup/data/`, 7 дней |
### Tools-сервер (tools, 85.193.83.9) ### Tools-сервер (tools, 85.193.83.9)
| Сервис | Домен | Статус | Заметки | Outline и n8n **переехали на main**. Сервер ожидает переноса мониторинга.
|--------|-------|--------|---------|
| Outline | wiki.csrx.ru | ✅ | Wiki, аутентификация через email magic link |
| n8n | n8n.csrx.ru | ✅ | Автоматизация workflow |
| docker-mailserver | mx.csrx.ru | ✅ | Postfix + Dovecot, порты 25/465/587/993, DKIM+SPF+DMARC |
| SnappyMail | mail.csrx.ru | ✅ | Веб-клиент почты |
### Почта (@csrx.ru) ---
**Аккаунты:** ## Почта
| Аккаунт | Назначение | Используется **Resend** (resend.com) для исходящей почты.
|---------|-----------| - Домен `walava.io` верифицирован в Resend
| noreply@csrx.ru | Системные письма (Outline magic link) | - Отправитель: `noreply@walava.io`
| admin@csrx.ru | Администратор | - Outline шлёт magic link напрямую через `smtp.resend.com:587`
| jack@csrx.ru | Личный | - API ключ: в vault как `vault_resend_api_key`
**Архитектура почты:** **Входящая почта не настроена** (нет MX, не нужна).
```
Входящая: internet → port 25 → mx.csrx.ru (85.193.83.9) → Postfix → Dovecot → IMAP
Исходящая: Outlook/SnappyMail → port 587 (STARTTLS) → mx.csrx.ru → Postfix → internet
Outline: outline контейнер → mailserver:587 (Docker mail-internal сеть) → noreply@
```
**Настройки почтового клиента:** ---
- IMAP: `mx.csrx.ru` порт 993 (SSL/TLS)
- SMTP: `mx.csrx.ru` порт 587 (STARTTLS) или 465 (SSL/TLS)
- Веб-клиент: https://mail.csrx.ru (SnappyMail)
**DNS-записи для почты** (все должны быть в Cloudflare): ## S3 (Timeweb Object Storage)
```
A mx → 85.193.83.9 DNS-only (НЕ proxied!)
A mail → 87.249.49.32 Proxied (webmail через Traefik)
MX @ 10 → mx.csrx.ru.
TXT @ → "v=spf1 mx -all"
TXT _dmarc → "v=DMARC1; p=quarantine; rua=mailto:admin@csrx.ru; ..."
TXT mail._domainkey → "v=DKIM1; k=rsa; p=<KEY>" (генерируется при деплое)
CNAME autoconfig → mx.csrx.ru. для Thunderbird autodiscover
CNAME autodiscover → mx.csrx.ru. для Outlook autodiscover
```
**rDNS (PTR-запись)** — настроить в панели Timeweb: | Bucket | Назначение |
`85.193.83.9 → mx.csrx.ru` (критично для доставки в Gmail/Yandex!) |--------|-----------|
| `walava-backup` | Бэкапы (каждые 6 часов, 7 дней хранения) |
**Автообновление TLS-сертификата:** | `walava-outline` | Файлы Outline (вложения, изображения) |
- certbot renew cron: каждый день в 03:15 и 15:15
- deploy-hook: после обновления автоматически перезагружает Postfix+Dovecot
--- ---
## CI/CD ## CI/CD
- Репозиторий: `git.csrx.ru/jack/infra` - Репозиторий: `git.walava.io/jack/infra`
- Триггер: push в `master` запускает `ansible-playbook playbooks/deploy.yml` + `playbooks/tools.yml` - Триггер: push в `master` запускает `ansible-playbook playbooks/deploy.yml` + `playbooks/tools.yml`
- Runner: `act_runner` на основном сервере - Runner: `act_runner` на main-сервере
- **Правило**: все изменения только через git, никаких ручных правок на сервере - **Правило**: все изменения только через git, никаких ручных правок на сервере
--- ---
## Известные проблемы ## Бэкап (что входит)
| Проблема | Статус | | Данные | Метод |
|----------|--------| |--------|-------|
| PTR-запись 85.193.83.9 → mx.csrx.ru | ⏳ Настроена в Timeweb, обновляется 3–24 ч | | Forgejo DB | pg_dump → gzip |
| Tools-сервер не бэкапится | ⚠️ outline-db, n8n, mailserver/config не входят в бэкап | | Forgejo data | tar volume |
| Tools-сервер вне мониторинга | ⚠️ Prometheus не скрейпит tools-сервер | | Plane DB | pg_dump → gzip |
| SnappyMail домен csrx.ru не настроен | ⚠️ Нужно в админке: IMAP mailserver:993, SMTP mailserver:587 | | Plane MinIO | tar volume |
| Outline DB | pg_dump → gzip |
| n8n workflows | tar volume |
| Vaultwarden | tar volume |
| Uptime Kuma | tar volume |
| Traefik acme.json | в составе volumes |
Расписание: 00:00, 06:00, 12:00, 18:00 UTC. Хранение: 7 дней.
--- ---
## Сети Docker ## Известные проблемы / TODO
| Проблема | Статус |
|----------|--------|
| Outline + n8n переезд на main | 🔄 CI задеплоен, ожидаем старт контейнеров |
| Authelia всё ещё запущена | ⚠️ Нужно удалить после деплоя (remove_orphans уберёт) |
| Мониторинг переезд на tools | ⏳ Следующий шаг |
| Отключить mon-сервер | ⏳ После переноса мониторинга |
---
## Сети Docker (main)
### Основной сервер
- `proxy` — публичная, только для Traefik (нужна для ACME) - `proxy` — публичная, только для Traefik (нужна для ACME)
- `backend` — internal, Traefik ↔ сервисы - `backend` — internal, Traefik ↔ сервисы
- `forgejo-db` — internal, Forgejo ↔ PostgreSQL - `forgejo-db` — internal, Forgejo ↔ PostgreSQL
@ -116,11 +114,5 @@ CNAME autodiscover → mx.csrx.ru. для Outlook autodiscover
- `plane-internal` — internal, все компоненты Plane - `plane-internal` — internal, все компоненты Plane
- `runner-jobs` — публичная, для job-контейнеров CI/CD - `runner-jobs` — публичная, для job-контейнеров CI/CD
- `monitoring` — internal, стек мониторинга - `monitoring` — internal, стек мониторинга
- `authelia-internal` — internal, Authelia ↔ Redis
### Tools-сервер
- `front` — публичная, для port binding хоста
- `outline-internal` — internal, Outline ↔ DB ↔ Redis - `outline-internal` — internal, Outline ↔ DB ↔ Redis
- `n8n-internal` — internal, n8n изоляция - `n8n-internal` — internal, n8n изоляция
- `mail-internal` — internal, Outline → mailserver (SMTP без auth)
- `webmail-internal` — internal, SnappyMail изоляция

1
inventory/group_vars/all/main.yml
View file

@ -48,6 +48,7 @@ discord_bot_token: "{{ vault_discord_bot_token }}"
discord_bot_app_id: "{{ vault_discord_bot_app_id }}" discord_bot_app_id: "{{ vault_discord_bot_app_id }}"
discord_bot_public_key: "{{ vault_discord_bot_public_key }}" discord_bot_public_key: "{{ vault_discord_bot_public_key }}"
outline_mcp_api_key: "{{ vault_outline_mcp_api_key }}" outline_mcp_api_key: "{{ vault_outline_mcp_api_key }}"
timeweb_token: "{{ vault_timeweb_token }}"
# Server IPs (used for cross-server Traefik routing) # Server IPs (used for cross-server Traefik routing)
ip_main: "87.249.49.32" ip_main: "87.249.49.32"

327
inventory/group_vars/all/vault.yml
View file

@ -1,146 +1,183 @@
$ANSIBLE_VAULT;1.1;AES256 $ANSIBLE_VAULT;1.1;AES256
64373338333466646131363639303563393130303436613766383337383937336663643765636563 61333332643965336563306337363339366563626363333837343831316137346335343031653735
6438303632663737356166633739383065363535653234380a316266613236313532353361356339 3139333134386564376566616637613761386663346337320a613937373264653435636334633432
39343137333435333734623665393033383136383932306634323563623135626235633430333232 63373735386239383034343935336235646432316364366361323664393062383233393336343434
3166623031613436650a356538646633343266323533653466623961363231303730333161313332 3238623361636136310a666262363065346234636630316264613662336261336234356135333662
62353133336462636564643835613361303039343737333938343162336566303737336534376364 37313233396265313961316135376334356662303161643132623734303864313765303264313934
65646236633934623937393963653166356331393137333730306663333133326631393163386361 62636265356161343635626337396466623766323839336266353063316333336534313138663336
62363065653232623634383032376165366330343533386266656334623334623333366332313065 61303834306233363739326237376339616638323162333631346565366232343966663037396161
36666133383535383537663339656162616638396464653238313732646435376537626335663563 39373437353732393765306230373864356135666630613930306335343466643435326630373831
64656237353263376466303632343432613065373061353632346539353562623064616534656466 64663363646334306339363265363162323561333531343932303034633331316262356533666363
34663534303838396165333761313337393261386433376631313066393765613232633535313863 63303431386130386536616436393764666136323636643431336565623537393638646632363762
31643535653431646564613232323538363061343664396332373565303531383438383665656233 34383731336163656236303937346466323332616433613938663134363661323538363963393533
31383731323633353635373232613036633433376636386537353136396336373663326635353838 38646338303464303765623433376163663232653931353034346633656336346530663864396465
36396234396436636565646638303761633061653761366238656266323533303466343666326365 31313864333730356434343066636331663838343635633235623162393963366436336437396363
64336164326566646630643361613264636533373330646630386266663965353934633765326434 62393761386636366664316436323535343032356537363833333962303761663563623833396638
64646162306365303065303830613636346232333964633035373463333630386132396632313035 66343033306237376437633238656634346264613561336335306432353635313363356333303561
62623139393430633665333034663661313965663134373534623166333435343132346662393361 35663839366434303366376266663033616364306636643437663464343866336466356133366135
32643332623731386535373366633563333961326632356265633330313839616463313834636164 63646233303037663563616661316533366264366535636333316333633437333036613039626636
37626330313465316364313334623031383531393132303563303263646637383838623863633566 38356363303761643630653964616136303566626363343436633834346230356430363132643637
37383036623030653630343235393336616435346231363338386334653732633634353565396638 32633761333564616639383463613662373163386264353935396332623065633366306330386266
61306565623166333332376439636636333133333934363631613166366664303362636362326534 39356665666530333331613632303461663764356638303234623031623731613431636666343366
37323561313538343466666161313937353630396162333361343437383537363966383730373364 30396132323536363838653730656263366165633964666463316236346562666639323339313333
61363733646237623761386436653165616136633131316538323266666262373761663066653934 61613962646232356264643932663962366266373332333938366230636232353630396532373161
63313266396232383662663735333530393633616637653466383734326636663137336462383134 34633230386162626535633032353330613337626237623937646165636132666630323038623233
35353536353763616531623433653764623535623464363432613635663137363738323231353939 35363032643936306230376165613632633061353430386366636238653034313038356366323064
64653363396634613737376462343139316337366234653639386335656462643661353764646363 39643961623136343130646666646630346638653136326365386437336466636263333238666266
62636537326639373665616134363837633237613734383761346662363931346634323161383364 39373861376261393365373238623662626137313862326333333135343730376439336434363633
36613236646163396636383036323566373664393963623961366634333337353833643439313565 34303230373933346563656363613034393535646539356562353635663065383137643337386564
32313631333433613139313533306436346334383239366337303865336562313235643734366332 62376230373334626239313661633835333062656432633931333735653736636238663331613165
30323530643034303336336531363433626431346464303562396366333336386561313964373364 34343538663264353536656661643265383865333665656363326166326239666136646435616130
35333037353030383861663165366534396637343634653239653732663138346566653035386135 33343730386232336232363561363439383666313762613933666235363665356264386161656631
34333732336263643532333133323063613363393037313639663966373938393762316663333066 30386331663232653537653036653531346265386239326134323066316139656434356232303938
35393463613232303338386535333935646466623162623531653663666235383263383461656462 66633562313037363066363332353262623831326263316339376436333833313231363662336432
39663162373537653735653536396164366330616164613561663363323463313634626632383964 32333961616231303162383233666562636164633735326338323363323965323436623537366232
37303134663764643163653062366537383630376463333363313839366134313535653866626332 30643533383733333532383766366265363636366136653332666634336263306563643638373739
63386435653065306231616532616563336330333235303562393731613434366438643038623234 32313937303832353236656437626662306132323566393763643665633736376436356235616332
66313038623137306666643762393234356332333532316333353266346163303036393366383238 62653838356162323066613565323835313263366134666132393136306264306337353834646334
35383232636164633132643339623936633732663966613961613964333631653433343731333962 38346166383539383864386362393932353164636239383366383664636632313534353866643663
64656239303631386461663036393039326330393435613562356263363461363261366430393736 63613338333466653365663262313561313763323035316638376666643035616663646664313061
39336536393834636336643837663038623863373362306364393166326135343430356437313435 64663535376432656439636534653431373133616239373161653134303039373163633866303233
63643064343438373661306363306233623563363061613032386165303262303533363433383062 31363539643264323434393535353233303435626230303636646364646331356266396239373438
39653166646534633733333335356464646339323961653038383165363663313738643363656663 38376638383339626139363863646565316433373634633132616633383438663534306665373166
34633664343833656332643663323036303665616265633463323462363330303333363836306331 34633935373430653666666238633634323132313664373562653931313131633564363437626161
62613937363936633533633965323530373761376438303061643063316162336566613934393562 61346263386235626465316561393064646333326165633338393037363732323936336439363564
65653463376233353662336536303731613632623836623263616435353235656232313438663236 32366532663539316638366439646634313665623365343566623232376161346439623433336665
64663265623431333831383866316237663237333235363733623739343134306636366366613737 34613964303366396266373835356333393433343461633765663537366261313965626133346638
62616430666233646131353463396236316366613430663231643435653161366562353265356632 64326131616439306532366162616239343234616563393436343062373933313762316236386666
65613164376437353566643435623366323266336635666336373465303936646665656135346135 30626162376335313738663566386138323564326134623563663134326662626362393362666236
38663938633063346164623962383733333239623565336539663531646330366536363336643032 38643237323062363433303833616533383437636262663762363266383962333936623032373334
30376562663364303864376330326638336131656362313264346361343236386664376338333963 34323138613463623537643262663662356534313031666564383761326133613530363665613933
34356236346534336233626361386231613761386135303335303233303730323939383735643138 62393432346231373063366238326361306161306235393333366532306263333636616536353363
63393433363734346436366436633064643532613335613062306231343163393962343031613062 30646165613738656332363337383031326539393462393365613766346438323539636535366137
30303962666434616235613062376339636433636466633935303837343937613239663161353038 36383334383038343539666562343137623030663935663639303333643933633566373264326330
61663130396262636339316161313636636339343033303132373733336663643433613161653361 38313037346635663430643238323863363831393064346436636463383738643832336362373537
31316237653461336335626138646139376264643733336434326339623330373337333761656333 63333064396433666666653935306339613062373539386665386264623462653535303933396263
65353837323563623561663865663664356534626433373934393263353234626630383738656362 34336337626462353265336633346439316330303639323563613561346532376530376530666238
31333338663363366531646230393663646462613164326366356238373137363230343561623934 63626436326339336436303232666665623632343433306362306661366565306436613765656539
65353963643336376231613130626466353735396166373165643965313464346562323362306161 63373764313835353437653535333935626136613563396662336162623237613431623463663561
62353830346130326261323963613938333136393665343566656636653939306466316432653431 34643936393030303938386339306435326561663062316539623661323861303330653765613065
66393466393164346237366365393039353563373861303134656662646364626562633538316531 63323532666139336631343839316638616132636366353438366566396664393561333330323861
62346161653038303836643932656563363230613732373662396133353736323138336162343936 31383236316336643238653132326362343235343534633032376564356463633539396331646330
34383334333031383537366533313161373937313261306563653431363961383563346565646333 61663232616534626130613331396161373538396430383963346262623230326638396338376631
66656433663538373038646266373538313365656532616634333861303139616236663765353532 32343630353734656163343931396238653731316562356433656335636339616231323763633033
32346638633434376232666635306331633363623639383464303434633936653162356264373766 65666539386134646561366264376333656361386362636330363434343865383336313535383736
31376431633261346230363938396437643938343637646564643966346261303932613563646166 32633465663230356566303134633131383565366163653333613433633035616565393733306466
61643263396163313735326239353561636333323765653734393132623062346630643730366235 33626534383436366162633831306231663361363836363034393462303861373862626165326163
35623535373039623933383131373032646330383764616565613364646431343631646430383538 34656233636161336534633562333263346639636133663235626436393966666438653739386436
30376535376261633738326238393634656433316234333432303439393137653362383139353466 39336364313238303737643762356530333537623565663361646233663336376533623233373738
64323965633337343161323165376531363066356432303832393065663639653363313531373436 36336164343935633533376463373437663830303965383134383431303463376265316663663862
35323130636161626565666632626633393834363136633337623839323939623464653330343963 38663964653133616464646231353435353863613131633139393331356634643334656133623834
33336161333865613634303334336436336561653338633439396335396635363832333064376637 33656230343465343035656532633939336339353934623466663537363163626463633737303738
30656665363434666439373033633462613136353339623730376238643435616532633563383764 62346563313061343661356564326565363531353464653135353763333038326333313434306132
39363666313732333030333338616532636463623335633366333235333961636136636436336534 63656535623064396530663932656434666661663763666535613661333038306362653031363533
37353063393763646339613938366132396565313833613066303664666165646439623832336432 65373234386130313634613934653836353434363432653463376464643935643836376237636236
34383762623735306233386566363230643063653635383636666639303637316131383163356133 65323261373334613630343865393039396262336230663030323335363730646361633363613131
66363530383734313136646633613166323761393531356537666231613339353066303332373462 32363664613862653035383436666161363366396161313437643662363635613532373765666237
63663233316631653234383234656262346238373762616138363130386664383133356239656161 30303261333363373663616535343436306530346666323036326365616334383134383435306637
38313638333231653066646165636231306239353766313437336634346664336330616465343430 32653935363962393363306137373861383461346439633030323935636564623264303638353539
62373661326237393666363737396630643034356666663338346664643837303331613961376634 32366363616636383337313763393765316637656638356365373765306639336332363631383734
64633761653436343135356364363362306563643536656437663836613766643763623334333361 39393131393765383165326662633438366330633632343665653735313032643037653430363761
65636366663339636438346631616239393865653138656262653632386238303566333762616634 36646365623835343466313764636236623936666330643730386238353565646565656664623861
61623931663466383736306563653231333234633963386333323939316439306461653064373662 36333438306161666232613839356131306166376434336431323036383634383539306134323038
65616638343930636636633230366131356536333236306339363562383063383035326635346335 32333666663237643865643461373538626264656335373534303236616435303039346661633338
65653462626538666364626635663331393263386630323235326334613830653432613334306461 66363238386233303532633431666263323235613335626361326461353466613661616433613736
37353232346337363034653633313565336565333934633062623136623062663262386331373862 38636331663635396636303931353338333437393936373631366130366632313037363262613333
32303730313034396337633132303531353436643662646564343635386163643538623935613661 30303261633263306661633862326665313338326433373033333639656663393564343536373063
32643836666438323535656332336339323333373634363664383866393765646430393364653563 64336464353937393238346331313166326238303033663838306339303463643364333330396435
33376634316137363066393337646336623065623636643862393534316630303562633761666639 36313139636338323233353262373864643463386464343037333733343236303132366236353231
65326233626435616531383939323035663333626134303631336532643136383938303839323639 62316533613734313038633461323864303862616339356236313030623963376432376365366166
34316261613761373934396335323763336663306264373431626134313935343062333930336133 34303564333863326265373066346233363964633061626335623636373839633366336266343161
39656161373532343934323263316666666430316462636439653762656236656635613531643731 33323866353937653835396534346235646236326563386234386465333839333464653462633264
33386265623531353335376636623564633234616465653166363830363531393933393163353033 62396163353039613436363866623938326164323638346531653533326239636562353334366564
62323237346263633032613161363831303963356432313534313832386138393335336166643436 34353235303838353765333636626261383462373539646230303332653364346635616232313131
30353762363434373836353966396265666166636561346436373934333339626637646662613261 33636636616439666134383536383965363532333635663232323063623031666562653461646132
39643664303034353364333538356536666163623538656261613265633839316163303732646637 66353530373466363731633435653732663337373434323436306539376632376538373636613561
38363465303362303566343165623364376532626137353237333165343162363537333237646230 62393631343639623736306336376237396239363232303732653434666366396162346463373864
35373464333365663163303634333439333938643334393136643437303064396631323331643662 34386435663438353031376639613530356437633665653266336436616539333262646362623836
62386534303630373236343730626562323738313561326339633061356534323862323033356539 65646537663031616466303065646137633333623932666534326139373064633834393666366661
39316162656435663133663533646331393636613037643866303534646166646662613664663561 32343065386362343331383637333766313330383532326132373338386435353463616233323633
62383035313831333736653831333739356535623864666165343362373933336366326264396331 65666362663638663938643133333631636539623864653864366263303830366230306537663162
63653837623433396636323265356165396437316538656533363064393033353061626463636533 63396236636634616238353863323332626565393139663537636435646132323864303539316364
39636463646233363233376365323731636433663765326232613335356234626635663538343061 36353034333664316338383131316335333133303930633030376238616164633264303662653363
39613837656661363439346662386563653361613435626163376232306635376537373931646637 62376361376637373466666534663136363639383262343235366366303530343361663764333066
64313537356431626466303165646538323234303065363163323431663962323030623263623233 34346130396365386232313365646138353733376430373066323561363235633236636533303433
34636662366266653538323337656662633938616437323862343064353533306437656136323939 33623839343532326163663865313332366663643861323737616362636233343963613934383437
34376137333135343333383633326465373164643636313239343365306237316238323534373239 63336565323935303337336434303561356566616537666330323039346531623039656162636538
33383434313033633337386438613134326430306536643666656534326538396166656265346531 36353232346633653135393266313865663532373834613663386261666535363965653366643666
30613434343334383135376337383034373365313762663131396234323330663565666431383264 38626563353765356435653534356630663730313932323739623866373335333338646434326539
63396430663733646337656164336630386164373964376439663465626165656632623635333766 34323561313237633935643364646162633834616466653138373639326464393837666630386236
36396439316538323530363266303366326230366564616639613738623463623835313264353561 65653066346361616131626238616564303033623966303230343638353464373036333863373465
32633065646161633462346634393737616333333566353630666565656431303162353633646138 63313562386237623532323432663562333862626262656630303032656565613635313034303566
62643634316263313034383063643438396537393361373632323739336262393639666537646266 38656563393934663364366333316431653563643963383838396366333338323236343164626363
34313466313461663664623430363634383236636330376165633430303665613261346631643938 66323131363361643031343366393034373131623631643265383532633864623362613566316565
39326632383565346663303937653138623433643038386131343435366361393137353062346562 39373538626364613163333836376666643764386330616564376364626362373962623838613533
62316337616435623762313630643966343836353163356534666363346239363638303031383231 64386438666365656163373065653832326132343535353238363165663461623264343831343037
33363830376337656537356635303636323037623763306136363761353037623137643832356562 35323433613832643934343633376136663733653465653762313963636535373539356565383937
33333937656239613562643661343634613230386130353439323139313965393266376565656338 66383538643435643461616337333864336166663030353762316630613536656466303761353531
31663438336538316663623939363633316363656661646162363365303065653766313161663334 35666539376633383236396535336335303764663435373633373335383863376466313633393830
33326236643039333862363034306432326634626330373862653761616238616435363233616337 37323638633437373966343935623536353161316366623431376565306666616662316561316130
36653532316362643366333566616664353938363032613766656235386536343737313231613334 36336239663232316262363462343431323664373330643361656464343938646330373631623565
64363661333664616337633565336137343361323131363034346437343265343634356139336263 33306365653037656232326461343336623134643433666561346164626236383565653061383961
33626261623864613039306335386536656636303238316265313863616134306239616661656133 34363734633237643064376337336163623035316630646631666564653634323339656164363063
32303036376431646238663337303737616232666130623730383166646265333062303263666463 30363165643663613138623537363234313939653461353130366366336662663236323131303239
65363164316634613231323065353331363035353335386662636334373930646437376239353531 35316462316132633330366634366533336563623730633664316564393263633436306463353639
36323361323661363739373665373838346138346662386235353136653230393939393332653638 32383031386638313533613665623966363630636637656164363736353937303162373839623339
38623866623461646330376233333837653334333665393665396261653065623835313831323936 35393834393235363966613032643935663139653733316261366566623566396565636636663136
36366438623563373937663233666433666363353132373333613734393330663133313966363162 30306433386533363038633666383862343064313338363835623030316466653564366562393733
31356364643830353630646263313162346664353736383236333235633838366633643636353032 32343761623234353665313061353330623666333037393633316333303436373530653666646632
38383633323438383433366632343433353633633537366231613537333938396438643732383530 33393637363333643638333666623430623963373963303739653261356461393933646533623035
37323665316435633961323230396265353930343537366666323034333235616431376138386536 63343936616233353035626565633439376339386237396433663933633335316131333834623264
65386432623530323935633938393161633430663063303837353565616561396434333063346462 35353866336564623166323236663733333532333465633661663666316664626239653631643232
35616366633438343330663232303366333662356364373964346439343961396561613963396238 32393135316533393464636461636163373762363638333439376335643237383262663131383032
37643430376130386366316533316561323466333061616266643962336331633761343935613337 36656134666130633237383733333532646365323131626430653031326363626438336436656135
30613235316130626165343232656165646536383531383430366365616466353939346638656561 66303430373230386166396132316530316536646165666633386164313061376439653663326363
38373638313864663862653634613631313361363161666135333532666430373764333233333938 35376634343735393735396365366239643865356231633530313865643438633934656664356366
62313330383337316566363531373236373834613166356538653037663137336131663138333039 61306136623635663165366637376565366461383363656365623136353533663963623766376334
62333634326434353736363662346530353939316464313364373135323765383232323135353065 63363661303262353939653366383931623235643632663035353431356434396230623366666439
63393163623937366135356233653866656236366133366530393864366165373436333235376165 39623265343763303030366133323263356261383361646662386565363238306437653732616232
62333763323237343133323538613832363938353431316566373966393365346337653638303139 32303461373239353737666638313130373765626462376137363162386430333762623663663934
39633337633733316465376161636134616637313733303663373766383664333030393431636534 62383331396331323961303461623130663931303537353831303664306564643866623739356566
32373362366264626462396366346236343635643865316562353532353166653636623736333232 65643631643739313039333337646162653065366462383938303163636132356263386461323535
64636563643332363534363733303662633331613664653532346138333334316561646636376465 62626436336464333961633535333164363532616163353337633430653063626336386635633331
30656266383536333165353234393665376235646132386630346466633466346464386361666363 30313862356165613833316662613764316139363335633833303864656434383062616434643864
31353036356435393039363162323062303831613138306131643936373266323065663865356235 66646233333937393462336236643161663135313163613664376665633235306233353561313435
39636465303762386533626231383732636138363863346439346536323663656366353139653661 30383363613330306131303262646130633032386531333565643833313566636231323130626135
3931 30383831306136373537346532383866326638663263333737643737363535653037653163346536
35663331356330313665643337383639366535616333663233356165366363626530346538623636
62393261333938333031396138303965303762353838333038643864626439633935393538633164
34353333393535363939616366376335353435366161313563396632333665363538323962366362
64353631393730303162653131306130376131323665626339636130613137343665303036383862
31383066353238396131336538656133626335636635643366313838323635323537323638303162
33353466383738353637623263633564383236633434373939313936393638303862623536363035
63656232376562373963633865346536373532636334656238356131656435343333386233363132
61636433613866373664363138323764356538343363613232626539366237666263333165383566
63663033336266663266333335636264666431306435636133363331623364336432616166346130
64303437316633306663633933646262653336666263313738326465616366666537643662383235
32633534346233303336623565323937356538366532313537363161393030653534323033316535
63646335623535623935383231373438313936333339326263353861623332336239653765636333
31333865326164353937363863353865626531373337396639353339653732356161333733663961
64313031306530663234313630613463383236373331356230616433623562346266356162373034
30363832646538383138393836303331323562633737386132393739373832326432323831616565
64303937653130643766663132306662313639333962323564613935336231323564333236636330
32633937656137343939653964306633326237313736633535393338346330383363383465313937
66613965326630356131333334393865646132633366623335653733393462333166643537393762
65656234636334653363656438633361313039323239353031316333323037636364636438626562
32363834616630373738323064633363356536336336343138373761643935356339623761383138
31633638323934343965346539353232383736663738373130626264383631343463383563373430
35373736313663346662356564636331396464353735643262373362313261306661363136663239
32313036356562333465613366386439323266613437653539313364356631366365633439356261
32316465386266373435383632653564623166656330393637393830336462383561353032376535
34326165626639646364396565363365643131666533353433643132386333613863383066323632
63656566383161303038346138383835303430353161656662313336373536653236356461356664
64386532343061616136343536613064323236613532356237386239373938366636323438373732
62326430646539613363376530626331303636393865353438666434376561666539343737623439
33653962353061666335653663643861636564363761346332326436653339613132393730646535
35353630616637653731396438653635306663626363346565306662623237666531333338333263
63663232633437386434356631373864366234653965336338383965363334623833333437383937
35383864336431656136666630643564316163393037636161303339653233393866616531623437
39643666626166353831616137623530323261373465323436613535313362616131633266373332
62363163316335626461653434643532666630643936666139643634393765366234356633333534
36376330333461326263343164323031366232623264353930336561643637323230336537363236
65353136363932623930376234653732323562663239393337336538383131616564616133613834
3665613863646666663431363062396330366438666431393536

11
inventory/hosts.ini
View file

@ -1,7 +1,6 @@
# ── VISUAL Infrastructure ───────────────────────────────────────────────────── # ── VISUAL Infrastructure ─────────────────────────────────────────────────────
# main 87.249.49.32 — core apps (Traefik, Forgejo, Plane, Vaultwarden) # main 87.249.49.32 — все сервисы (Traefik, Forgejo, Plane, Vaultwarden, Outline, n8n, CI/CD)
# tools 85.193.83.9 — team tools (Outline, Uptime Kuma) # tools 85.193.83.9 — мониторинг (Grafana, Prometheus, Loki, AlertManager, Uptime Kuma)
# mon 188.225.79.34 — monitoring (Grafana, Prometheus, Loki, AlertManager)
[main] [main]
main ansible_host=87.249.49.32 main ansible_host=87.249.49.32
@ -9,20 +8,14 @@ main ansible_host=87.249.49.32
[tools] [tools]
tools ansible_host=85.193.83.9 tools ansible_host=85.193.83.9
[mon]
mon ansible_host=188.225.79.34
# ── Group for roles that run on ALL servers ───────────────────────────────────
[all_servers:children] [all_servers:children]
main main
tools tools
mon
[all_servers:vars] [all_servers:vars]
ansible_python_interpreter=/usr/bin/python3 ansible_python_interpreter=/usr/bin/python3
ansible_user=deploy ansible_user=deploy
# ── Legacy alias (keep for backwards compatibility with old playbooks) ────────
[servers] [servers]
main ansible_host=87.249.49.32 main ansible_host=87.249.49.32

2
roles/backup/defaults/main.yml
View file

@ -5,4 +5,4 @@ backup_user: deploy
# Timeweb S3 offsite backups # Timeweb S3 offsite backups
s3_endpoint: "https://s3.timeweb.cloud" s3_endpoint: "https://s3.timeweb.cloud"
s3_bucket: "visual-backup" s3_bucket: "walava-backup"

37
roles/backup/templates/backup.sh.j2
View file

@ -32,6 +32,12 @@ docker exec plane-db pg_dump -U plane plane \
| gzip > "${WORK_DIR}/data/databases/plane.sql.gz" | gzip > "${WORK_DIR}/data/databases/plane.sql.gz"
log " → databases/plane.sql.gz ($(du -sh "${WORK_DIR}/data/databases/plane.sql.gz" | cut -f1))" log " → databases/plane.sql.gz ($(du -sh "${WORK_DIR}/data/databases/plane.sql.gz" | cut -f1))"
# ── PostgreSQL: Outline ──────────────────────────────────────────────────────
log "Dumping outline-db..."
docker exec outline-db pg_dump -U outline outline \
| gzip > "${WORK_DIR}/data/databases/outline.sql.gz"
log " → databases/outline.sql.gz ($(du -sh "${WORK_DIR}/data/databases/outline.sql.gz" | cut -f1))"
# ── Forgejo data volume (repos, attachments, LFS) ─────────────────────────── # ── Forgejo data volume (repos, attachments, LFS) ───────────────────────────
log "Backing up Forgejo data..." log "Backing up Forgejo data..."
docker run --rm \ docker run --rm \
@ -50,6 +56,24 @@ docker run --rm \
tar czf /backup/uptime-kuma.tar.gz /app/data tar czf /backup/uptime-kuma.tar.gz /app/data
log " → volumes/uptime-kuma.tar.gz ($(du -sh "${WORK_DIR}/data/volumes/uptime-kuma.tar.gz" | cut -f1))" log " → volumes/uptime-kuma.tar.gz ($(du -sh "${WORK_DIR}/data/volumes/uptime-kuma.tar.gz" | cut -f1))"
# ── n8n workflows + credentials ──────────────────────────────────────────────
log "Backing up n8n..."
docker run --rm \
--volumes-from n8n \
-v "${WORK_DIR}/data/volumes:/backup" \
alpine:3 \
tar czf /backup/n8n.tar.gz /home/node/.n8n
log " → volumes/n8n.tar.gz ($(du -sh "${WORK_DIR}/data/volumes/n8n.tar.gz" | cut -f1))"
# ── Plane MinIO (uploaded files / attachments) ───────────────────────────────
log "Backing up Plane MinIO..."
docker run --rm \
--volumes-from plane-minio \
-v "${WORK_DIR}/data/volumes:/backup" \
alpine:3 \
tar czf /backup/plane-minio.tar.gz /data
log " → volumes/plane-minio.tar.gz ($(du -sh "${WORK_DIR}/data/volumes/plane-minio.tar.gz" | cut -f1))"
# ── Add restore instructions ───────────────────────────────────────────────── # ── Add restore instructions ─────────────────────────────────────────────────
cat > "${WORK_DIR}/data/RESTORE.md" << 'RESTORE_EOF' cat > "${WORK_DIR}/data/RESTORE.md" << 'RESTORE_EOF'
# Restore Instructions # Restore Instructions
@ -72,6 +96,9 @@ zcat data/databases/forgejo.sql.gz | docker exec -i forgejo-db psql -U forgejo f
# Plane DB # Plane DB
zcat data/databases/plane.sql.gz | docker exec -i plane-db psql -U plane plane zcat data/databases/plane.sql.gz | docker exec -i plane-db psql -U plane plane
# Outline DB
zcat data/databases/outline.sql.gz | docker exec -i outline-db psql -U outline outline
``` ```
## Step 3 — Restore volume data ## Step 3 — Restore volume data
@ -80,9 +107,17 @@ zcat data/databases/plane.sql.gz | docker exec -i plane-db psql -U plane plane
docker run --rm --volumes-from forgejo -v $(pwd)/data/volumes:/backup \ docker run --rm --volumes-from forgejo -v $(pwd)/data/volumes:/backup \
alpine:3 sh -c "cd / && tar xzf /backup/forgejo.tar.gz" alpine:3 sh -c "cd / && tar xzf /backup/forgejo.tar.gz"
# Uptime Kuma — extracts /app/data/ into the container # Uptime Kuma
docker run --rm --volumes-from uptime-kuma -v $(pwd)/data/volumes:/backup \ docker run --rm --volumes-from uptime-kuma -v $(pwd)/data/volumes:/backup \
alpine:3 sh -c "cd / && tar xzf /backup/uptime-kuma.tar.gz" alpine:3 sh -c "cd / && tar xzf /backup/uptime-kuma.tar.gz"
# n8n
docker run --rm --volumes-from n8n -v $(pwd)/data/volumes:/backup \
alpine:3 sh -c "cd / && tar xzf /backup/n8n.tar.gz"
# Plane MinIO (uploaded files)
docker run --rm --volumes-from plane-minio -v $(pwd)/data/volumes:/backup \
alpine:3 sh -c "cd / && tar xzf /backup/plane-minio.tar.gz"
``` ```
## Step 4 — Restart services ## Step 4 — Restart services

5
roles/services/defaults/main.yml
View file

@ -28,3 +28,8 @@ promtail_image: "grafana/promtail:3.4.3" # https://hub
crowdsec_image: "crowdsecurity/crowdsec:v1.6.8" # https://hub.docker.com/r/crowdsecurity/crowdsec/tags crowdsec_image: "crowdsecurity/crowdsec:v1.6.8" # https://hub.docker.com/r/crowdsecurity/crowdsec/tags
redis_image: "redis:7-alpine" redis_image: "redis:7-alpine"
uptime_kuma_image: "louislam/uptime-kuma:1" # https://hub.docker.com/r/louislam/uptime-kuma/tags uptime_kuma_image: "louislam/uptime-kuma:1" # https://hub.docker.com/r/louislam/uptime-kuma/tags
outline_image: "outlinewiki/outline:0.80.2" # https://hub.docker.com/r/outlinewiki/outline/tags
outline_db_image: "postgres:15-alpine"
outline_redis_image: "redis:7-alpine"
n8n_image: "n8nio/n8n:1.89.2" # https://hub.docker.com/r/n8nio/n8n/tags
outline_mcp_image: "git.{{ domain_base }}/jack/outline-mcp:latest"

120
roles/services/tasks/configs.yml
View file

@ -8,6 +8,15 @@
mode: "0600" mode: "0600"
notify: Restart stack notify: Restart stack
- name: Deploy Outline .env file
ansible.builtin.template:
src: env.outline.j2
dest: "{{ services_root }}/.env.outline"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0600"
notify: Restart stack
- name: Deploy docker-compose.yml - name: Deploy docker-compose.yml
ansible.builtin.template: ansible.builtin.template:
src: docker-compose.yml.j2 src: docker-compose.yml.j2
@ -44,77 +53,19 @@
mode: "0644" mode: "0644"
notify: Restart stack notify: Restart stack
- name: Deploy Prometheus config - name: Configure CORS on walava-outline S3 bucket (required for browser uploads)
ansible.builtin.template: ansible.builtin.shell: |
src: prometheus/prometheus.yml.j2 docker run --rm \
dest: "{{ services_root }}/prometheus/prometheus.yml" -e AWS_ACCESS_KEY_ID={{ s3_access_key }} \
owner: "{{ deploy_user }}" -e AWS_SECRET_ACCESS_KEY={{ s3_secret_key }} \
group: "{{ deploy_group }}" -e AWS_DEFAULT_REGION=ru-1 \
mode: "0644" amazon/aws-cli:latest \
notify: Restart stack --endpoint-url https://s3.timeweb.cloud \
s3api put-bucket-cors \
- name: Deploy Grafana datasource provisioning --bucket walava-outline \
ansible.builtin.template: --cors-configuration '{"CORSRules":[{"AllowedOrigins":["https://{{ domain_wiki }}"],"AllowedMethods":["GET","PUT","POST","DELETE","HEAD"],"AllowedHeaders":["*"],"ExposeHeaders":["ETag"],"MaxAgeSeconds":3000}]}'
src: grafana/provisioning/datasources/prometheus.yml.j2 changed_when: false
dest: "{{ services_root }}/grafana/provisioning/datasources/prometheus.yml" ignore_errors: true
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy Grafana dashboard provisioning config
ansible.builtin.template:
src: grafana/provisioning/dashboards/dashboards.yml.j2
dest: "{{ services_root }}/grafana/provisioning/dashboards/dashboards.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy Node Exporter Full dashboard JSON
ansible.builtin.copy:
src: grafana/dashboards/node-exporter-full.json
dest: "{{ services_root }}/grafana/provisioning/dashboards/json/node-exporter-full.json"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy cAdvisor dashboard JSON
ansible.builtin.copy:
src: grafana/dashboards/cadvisor.json
dest: "{{ services_root }}/grafana/provisioning/dashboards/json/cadvisor.json"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy Prometheus alert rules
ansible.builtin.template:
src: prometheus/rules/alerts.yml.j2
dest: "{{ services_root }}/prometheus/rules/alerts.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy AlertManager config
ansible.builtin.template:
src: prometheus/alertmanager.yml.j2
dest: "{{ services_root }}/prometheus/alertmanager.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy Loki config
ansible.builtin.template:
src: loki/loki.yml.j2
dest: "{{ services_root }}/loki/loki.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy Promtail config - name: Deploy Promtail config
ansible.builtin.template: ansible.builtin.template:
@ -125,15 +76,6 @@
mode: "0644" mode: "0644"
notify: Restart stack notify: Restart stack
- name: Deploy Grafana Loki datasource
ansible.builtin.template:
src: grafana/provisioning/datasources/loki.yml.j2
dest: "{{ services_root }}/grafana/provisioning/datasources/loki.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
notify: Restart stack
- name: Deploy CrowdSec acquisition config - name: Deploy CrowdSec acquisition config
ansible.builtin.template: ansible.builtin.template:
src: crowdsec/acquis.yaml.j2 src: crowdsec/acquis.yaml.j2
@ -143,24 +85,6 @@
mode: "0644" mode: "0644"
notify: Restart stack notify: Restart stack
- name: Deploy Authelia configuration
ansible.builtin.template:
src: authelia/configuration.yml.j2
dest: "{{ services_root }}/authelia/configuration.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0600"
notify: Restart stack
- name: Deploy Authelia users database
ansible.builtin.template:
src: authelia/users.yml.j2
dest: "{{ services_root }}/authelia/users.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0600"
notify: Restart stack
- name: Deploy Traefik logrotate config - name: Deploy Traefik logrotate config
ansible.builtin.template: ansible.builtin.template:
src: logrotate/traefik.j2 src: logrotate/traefik.j2

6
roles/services/tasks/directories.yml
View file

@ -22,12 +22,6 @@
- plane/pgdata - plane/pgdata
- plane/media - plane/media
- act_runner - act_runner
- prometheus
- grafana/provisioning/datasources
- grafana/provisioning/dashboards
- grafana/provisioning/dashboards/json
- prometheus/rules
- loki - loki
- traefik/logs - traefik/logs
- crowdsec - crowdsec
- authelia

30
roles/services/tasks/main.yml
View file

@ -16,29 +16,43 @@
- "{{ plane_redis_image }}" - "{{ plane_redis_image }}"
- "{{ plane_minio_image }}" - "{{ plane_minio_image }}"
- "{{ act_runner_image }}" - "{{ act_runner_image }}"
- "{{ prometheus_image }}"
- "{{ node_exporter_image }}" - "{{ node_exporter_image }}"
- "{{ cadvisor_image }}" - "{{ cadvisor_image }}"
- "{{ grafana_image }}"
- "{{ alertmanager_image }}"
- "{{ loki_image }}"
- "{{ promtail_image }}" - "{{ promtail_image }}"
- "{{ crowdsec_image }}" - "{{ crowdsec_image }}"
- "{{ uptime_kuma_image }}" - "{{ outline_image }}"
- "tecnativa/postfix-relay" - "{{ outline_db_image }}"
- "{{ outline_redis_image }}"
- "{{ n8n_image }}"
register: pull_result register: pull_result
changed_when: "'Status: Downloaded newer image' in pull_result.stdout" changed_when: "'Status: Downloaded newer image' in pull_result.stdout"
retries: 5 retries: 5
delay: 30 delay: 30
until: pull_result.rc == 0 until: pull_result.rc == 0
- name: Allow SMTP relay port from tools server # ── UFW: allow tools Prometheus to scrape exporters on main ──────────────────
- name: Allow tools server to scrape node-exporter
community.general.ufw:
rule: allow
port: "9100"
proto: tcp
src: "{{ ip_tools }}"
- name: Allow tools server to scrape cAdvisor
community.general.ufw:
rule: allow
port: "8080"
proto: tcp
src: "{{ ip_tools }}"
- name: Remove legacy SMTP relay UFW rule (port 1025)
community.general.ufw: community.general.ufw:
rule: allow rule: allow
port: "1025" port: "1025"
proto: tcp proto: tcp
src: "{{ ip_tools }}" src: "{{ ip_tools }}"
comment: "SMTP relay for tools-server Outline" delete: true
failed_when: false
- name: Deploy Docker Compose stack - name: Deploy Docker Compose stack
community.docker.docker_compose_v2: community.docker.docker_compose_v2:

302
roles/services/templates/docker-compose.yml.j2
View file

@ -26,6 +26,12 @@ networks:
monitoring: monitoring:
driver: bridge driver: bridge
internal: true internal: true
outline-internal:
driver: bridge
internal: true
n8n-internal:
driver: bridge
internal: true
volumes: volumes:
forgejo_data: forgejo_data:
forgejo_db_data: forgejo_db_data:
@ -34,11 +40,10 @@ volumes:
plane_minio_data: plane_minio_data:
plane_media: plane_media:
act_runner_data: act_runner_data:
prometheus_data:
grafana_data:
loki_data:
crowdsec_data: crowdsec_data:
uptime_kuma_data: outline_db_data:
outline_redis_data:
n8n_data:
services: services:
@ -372,52 +377,16 @@ services:
- backend - backend
- runner-jobs - runner-jobs
# ── Monitoring Stack ─────────────────────────────────────────────────────── # ── Monitoring exporters (metrics scraped by tools Prometheus over network) ──
prometheus: # Ports exposed: tools server must have UFW rules allowing ip_main:9100/8080
image: {{ prometheus_image }}
container_name: prometheus
restart: unless-stopped
networks:
- monitoring
volumes:
- prometheus_data:/prometheus
- {{ services_root }}/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- {{ services_root }}/prometheus/rules:/etc/prometheus/rules:ro
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--storage.tsdb.retention.time=30d"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:9090/-/healthy"]
interval: 30s
timeout: 5s
retries: 3
alertmanager:
image: {{ alertmanager_image }}
container_name: alertmanager
restart: unless-stopped
networks:
- monitoring
volumes:
- {{ services_root }}/prometheus/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
command:
- "--config.file=/etc/alertmanager/alertmanager.yml"
- "--storage.path=/alertmanager"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:9093/-/healthy"]
interval: 30s
timeout: 5s
retries: 3
node-exporter: node-exporter:
image: {{ node_exporter_image }} image: {{ node_exporter_image }}
container_name: node-exporter container_name: node-exporter
restart: unless-stopped restart: unless-stopped
networks: networks:
- monitoring - monitoring
ports:
- "9100:9100"
pid: host pid: host
volumes: volumes:
- /proc:/host/proc:ro - /proc:/host/proc:ro
@ -434,6 +403,8 @@ services:
restart: unless-stopped restart: unless-stopped
networks: networks:
- monitoring - monitoring
ports:
- "8080:8080"
privileged: true privileged: true
devices: devices:
- /dev/kmsg - /dev/kmsg
@ -444,50 +415,7 @@ services:
- /var/lib/docker:/var/lib/docker:ro - /var/lib/docker:/var/lib/docker:ro
- /dev/disk:/dev/disk:ro - /dev/disk:/dev/disk:ro
grafana: # ── Logging (Promtail pushes to Loki on tools server) ─────────────────────
image: {{ grafana_image }}
container_name: grafana
restart: unless-stopped
security_opt:
- no-new-privileges:true
depends_on:
- prometheus
networks:
- backend
- monitoring
volumes:
- grafana_data:/var/lib/grafana
- {{ services_root }}/grafana/provisioning:/etc/grafana/provisioning:ro
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD}
- GF_USERS_ALLOW_SIGN_UP=false
- GF_SERVER_DOMAIN={{ domain_dashboard }}
- GF_SERVER_ROOT_URL=https://{{ domain_dashboard }}
- GF_AUTH_ANONYMOUS_ENABLED=false
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
interval: 30s
timeout: 5s
retries: 3
# ── Logging Stack ──────────────────────────────────────────────────────────
loki:
image: {{ loki_image }}
container_name: loki
restart: unless-stopped
networks:
- monitoring
volumes:
- loki_data:/loki
- {{ services_root }}/loki/loki.yml:/etc/loki/local-config.yaml:ro
command: -config.file=/etc/loki/local-config.yaml
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3100/ready"]
interval: 30s
timeout: 5s
retries: 3
promtail: promtail:
image: {{ promtail_image }} image: {{ promtail_image }}
container_name: promtail container_name: promtail
@ -523,75 +451,155 @@ services:
# ── Discord Bot ──────────────────────────────────────────────────────────── # ── Discord Bot ────────────────────────────────────────────────────────────
# Infrastructure management bot: /status /logs /restart /deploy /metrics /backup # NOTE: disabled until image is built & pushed to Forgejo registry
# Image is built and pushed by the discord-bot repo CI/CD # discord-bot:
discord-bot: # image: git.{{ domain_base }}/jack/discord-bot:latest
image: git.{{ domain_base }}/jack/discord-bot:latest # container_name: discord-bot
container_name: discord-bot # restart: unless-stopped
restart: unless-stopped # environment:
environment: # DISCORD_TOKEN: "${DISCORD_BOT_TOKEN}"
DISCORD_TOKEN: "${DISCORD_BOT_TOKEN}" # DISCORD_APP_ID: "{{ discord_bot_app_id }}"
DISCORD_APP_ID: "{{ discord_bot_app_id }}" # FORGEJO_TOKEN: "${FORGEJO_RUNNER_TOKEN}"
FORGEJO_TOKEN: "${FORGEJO_RUNNER_TOKEN}" # FORGEJO_URL: "https://{{ domain_git }}"
FORGEJO_URL: "https://{{ domain_git }}" # FORGEJO_REPO: "jack/infra"
FORGEJO_REPO: "jack/infra" # PROMETHEUS_URL: "http://{{ ip_tools }}:9090"
PROMETHEUS_URL: "http://prometheus:9090" # volumes:
volumes: # - /var/run/docker.sock:/var/run/docker.sock:ro
- /var/run/docker.sock:/var/run/docker.sock:ro # networks:
networks: # - proxy
- proxy # Discord API (internet)
- monitoring # Prometheus metrics
# ── Walava Landing ───────────────────────────────────────────────────────── # ── Walava Landing ─────────────────────────────────────────────────────────
# Landing page for walava.io — image built by walava-web repo CI/CD # NOTE: disabled until image is built & pushed to Forgejo registry
walava-web: # walava-web:
image: git.{{ domain_base }}/jack/walava-web:latest # image: git.{{ domain_base }}/jack/walava-web:latest
container_name: walava-web # container_name: walava-web
restart: unless-stopped # restart: unless-stopped
networks: # networks:
- proxy # - proxy
# ── Uptime Kuma ────────────────────────────────────────────────────────────
# Мониторинг доступности сервисов + публичная статус-страница # ── Outline wiki ────────────────────────────────────────────────────────────
# Доступен по адресу: https://{{ domain_status }} outline:
uptime-kuma: image: {{ outline_image }}
image: {{ uptime_kuma_image }} container_name: outline
container_name: uptime-kuma
restart: unless-stopped restart: unless-stopped
security_opt: env_file: .env.outline
- no-new-privileges:true
networks: networks:
- outline-internal
- backend - backend
- proxy # needs internet access for Discord/Telegram notifications - proxy # needs outbound internet for SMTP (Resend) and S3 (Timeweb)
volumes: depends_on:
- uptime_kuma_data:/app/data outline-db:
condition: service_healthy
outline-redis:
condition: service_healthy
healthcheck: healthcheck:
test: ["CMD", "curl", "-sf", "http://localhost:3001/"] test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000/_health"]
interval: 30s interval: 30s
timeout: 5s timeout: 5s
retries: 3 retries: 3
# ── SMTP Relay ─────────────────────────────────────────────────────────────
# Forwards mail from tools-server (85.193.83.9) to Resend SMTP.
# tools-server has outbound SMTP blocked by the VPS provider.
# Listens on 85.193.83.9:1025 (UFW allows only from ip_tools).
smtp-relay:
image: tecnativa/postfix-relay
container_name: smtp-relay
restart: unless-stopped
ports:
- "{{ ip_tools }}:1025:25"
networks:
- proxy
environment:
- MAILNAME={{ domain_base }}
- MAIL_RELAY_HOST=smtp.resend.com
- MAIL_RELAY_PORT=587
- MAIL_RELAY_USER=resend
- MAIL_RELAY_PASS={{ resend_api_key }}
- MAIL_RELAY_MYHOSTNAME=mail.{{ domain_base }}
logging: logging:
driver: json-file driver: json-file
options: options:
max-size: "5m" max-size: "10m"
max-file: "2" max-file: "3"
outline-db:
image: {{ outline_db_image }}
container_name: outline-db
restart: unless-stopped
environment:
POSTGRES_DB: outline
POSTGRES_USER: outline
POSTGRES_PASSWORD: {{ outline_db_password }}
networks:
- outline-internal
volumes:
- outline_db_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U outline"]
interval: 10s
timeout: 5s
retries: 5
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
outline-redis:
image: {{ outline_redis_image }}
container_name: outline-redis
restart: unless-stopped
networks:
- outline-internal
volumes:
- outline_redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
# ── n8n workflow automation ──────────────────────────────────────────────────
n8n:
image: {{ n8n_image }}
container_name: n8n
restart: unless-stopped
networks:
- n8n-internal
- backend
- proxy # needs outbound internet for workflow API calls
volumes:
- n8n_data:/home/node/.n8n
environment:
- N8N_HOST={{ domain_n8n }}
- N8N_PORT=5678
- N8N_PROTOCOL=https
- WEBHOOK_URL=https://{{ domain_n8n }}/
- N8N_ENCRYPTION_KEY={{ n8n_encryption_key }}
- N8N_USER_MANAGEMENT_JWT_SECRET={{ n8n_jwt_secret }}
- GENERIC_TIMEZONE=Europe/Moscow
- TZ=Europe/Moscow
- N8N_METRICS=false
- N8N_LOG_LEVEL=warn
- EXECUTIONS_DATA_PRUNE=true
- EXECUTIONS_DATA_MAX_AGE=336
- N8N_DIAGNOSTICS_ENABLED=false
- N8N_VERSION_NOTIFICATIONS_ENABLED=false
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:5678/healthz"]
interval: 30s
timeout: 5s
retries: 3
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
# ── Outline MCP server ───────────────────────────────────────────────────────
# NOTE: image must be built & pushed to git.walava.io/jack/outline-mcp:latest first
# Disabled until Forgejo registry has the image
# outline-mcp:
# image: {{ outline_mcp_image }}
# container_name: outline-mcp
# restart: unless-stopped
# networks:
# - backend
# environment:
# - OUTLINE_URL=https://{{ domain_wiki }}
# - OUTLINE_API_KEY={{ outline_mcp_api_key }}
# - PORT=8765
# - HOST=0.0.0.0
# - LOG_LEVEL=INFO
# logging:
# driver: json-file
# options:
# max-size: "10m"
# max-file: "3"

41
roles/services/templates/env.outline.j2 Normal file
View file

@ -0,0 +1,41 @@
# Outline env — generated by Ansible
NODE_ENV=production
SECRET_KEY={{ outline_secret_key }}
UTILS_SECRET={{ outline_utils_secret }}
# Database
DATABASE_URL=postgres://outline:{{ outline_db_password }}@outline-db:5432/outline
PGSSLMODE=disable
# Redis
REDIS_URL=redis://outline-redis:6379
# App URL
URL=https://{{ domain_wiki }}
PORT=3000
# S3 file storage (Timeweb Object Storage)
AWS_ACCESS_KEY_ID={{ s3_access_key }}
AWS_SECRET_ACCESS_KEY={{ s3_secret_key }}
AWS_REGION=ru-1
AWS_S3_UPLOAD_BUCKET_NAME=walava-outline
AWS_S3_UPLOAD_BUCKET_URL=https://s3.twcstorage.ru
AWS_S3_FORCE_PATH_STYLE=true
FILE_STORAGE=s3
# Auth
AUTH_PROVIDERS=email
# SMTP via Resend (direct — main server has outbound SMTP)
SMTP_HOST=smtp.resend.com
SMTP_PORT=587
SMTP_USERNAME=resend
SMTP_PASSWORD={{ resend_api_key }}
SMTP_FROM_EMAIL=noreply@{{ domain_base }}
SMTP_FROM_NAME=Visual Wiki
SMTP_SECURE=false
# Optional
DEFAULT_LANGUAGE=en_US
RATE_LIMITER_ENABLED=true
ENABLE_UPDATES=false

2
roles/services/templates/loki/promtail.yml.j2
View file

@ -7,7 +7,7 @@ positions:
filename: /tmp/positions.yaml filename: /tmp/positions.yaml
clients: clients:
- url: http://loki:3100/loki/api/v1/push - url: http://{{ ip_tools }}:3100/loki/api/v1/push
scrape_configs: scrape_configs:
- job_name: docker - job_name: docker

10
roles/services/templates/traefik/dynamic/routes.yml.j2
View file

@ -89,7 +89,6 @@ http:
service: walava-landing service: walava-landing
middlewares: [rate-limit-default] middlewares: [rate-limit-default]
# ── Cross-server: tools ({{ ip_tools }}) ─────────────────────────────────
wiki: wiki:
rule: "Host(`{{ domain_wiki }}`)" rule: "Host(`{{ domain_wiki }}`)"
entrypoints: [websecure] entrypoints: [websecure]
@ -135,28 +134,27 @@ http:
grafana: grafana:
loadBalancer: loadBalancer:
servers: servers:
- url: "http://grafana:3000" - url: "http://{{ ip_tools }}:3000"
uptime-kuma: uptime-kuma:
loadBalancer: loadBalancer:
servers: servers:
- url: "http://uptime-kuma:3001" - url: "http://{{ ip_tools }}:3001"
walava-landing: walava-landing:
loadBalancer: loadBalancer:
servers: servers:
- url: "http://walava-web:80" - url: "http://walava-web:80"
# ── Cross-server services ─────────────────────────────────────────────────
wiki: wiki:
loadBalancer: loadBalancer:
servers: servers:
- url: "http://{{ ip_tools }}:3000" - url: "http://outline:3000"
n8n: n8n:
loadBalancer: loadBalancer:
servers: servers:
- url: "http://{{ ip_tools }}:5678" - url: "http://n8n:5678"
middlewares: middlewares:
# ── Security Headers (applied globally via entrypoint) ───────────────── # ── Security Headers (applied globally via entrypoint) ─────────────────

14
roles/tools/defaults/main.yml
View file

@ -1,7 +1,11 @@
--- ---
tools_root: /opt/tools tools_root: /opt/tools
outline_image: "outlinewiki/outline:0.80.2"
outline_db_image: "postgres:15-alpine" # Image versions (mirrors services role — keep in sync)
outline_redis_image: "redis:7-alpine" prometheus_image: "prom/prometheus:v3.4.0"
n8n_image: "n8nio/n8n:1.89.2" # https://hub.docker.com/r/n8nio/n8n/tags node_exporter_image: "prom/node-exporter:v1.9.1"
outline_mcp_image: "git.{{ domain_base }}/jack/outline-mcp:latest" cadvisor_image: "gcr.io/cadvisor/cadvisor:v0.52.1"
grafana_image: "grafana/grafana:11.6.1"
alertmanager_image: "prom/alertmanager:v0.28.1"
loki_image: "grafana/loki:3.4.3"
uptime_kuma_image: "louislam/uptime-kuma:1"

817
roles/tools/files/grafana/dashboards/cadvisor.json Normal file
View file

@ -0,0 +1,817 @@
{
"__inputs": [
{
"name": "DS_PROMETHEUS",
"label": "Prometheus",
"description": "Prometheus as the datasource is obligatory",
"type": "datasource",
"pluginId": "prometheus",
"pluginName": "Prometheus"
}
],
"__requires": [
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "7.4.5"
},
{
"type": "panel",
"id": "graph",
"name": "Graph",
"version": ""
},
{
"type": "datasource",
"id": "prometheus",
"name": "Prometheus",
"version": "1.0.0"
},
{
"type": "panel",
"id": "table",
"name": "Table",
"version": ""
}
],
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": 14282,
"graphTooltip": 0,
"id": null,
"iteration": 1617715580880,
"links": [],
"panels": [
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 0
},
"id": 8,
"panels": [],
"title": "CPU",
"type": "row"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 24,
"x": 0,
"y": 1
},
"hiddenSeries": false,
"id": 15,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null as zero",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(container_cpu_usage_seconds_total{instance=~\"$host\",name=~\"$container\",name=~\".+\"}[5m])) by (name) *100",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "CPU Usage",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:606",
"format": "percent",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:607",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 8
},
"id": 11,
"panels": [],
"title": "Memory",
"type": "row"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 9
},
"hiddenSeries": false,
"id": 9,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null as zero",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(container_memory_rss{instance=~\"$host\",name=~\"$container\",name=~\".+\"}) by (name)",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Memory Usage",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:606",
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:607",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 9
},
"hiddenSeries": false,
"id": 14,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null as zero",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(container_memory_cache{instance=~\"$host\",name=~\"$container\",name=~\".+\"}) by (name)",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Memory Cached",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:606",
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:607",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 17
},
"id": 2,
"panels": [],
"title": "Network",
"type": "row"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 18
},
"hiddenSeries": false,
"id": 4,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"hideEmpty": false,
"hideZero": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"sideWidth": null,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(container_network_receive_bytes_total{instance=~\"$host\",name=~\"$container\",name=~\".+\"}[5m])) by (name)",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Received Network Traffic",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:674",
"format": "Bps",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:675",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 18
},
"hiddenSeries": false,
"id": 6,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(container_network_transmit_bytes_total{instance=~\"$host\",name=~\"$container\",name=~\".+\"}[5m])) by (name)",
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Sent Network Traffic",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:832",
"format": "Bps",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:833",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 26
},
"id": 19,
"panels": [],
"title": "Misc",
"type": "row"
},
{
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {
"align": null,
"filterable": false
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "id"
},
"properties": [
{
"id": "custom.width",
"value": 260
}
]
},
{
"matcher": {
"id": "byName",
"options": "Running"
},
"properties": [
{
"id": "unit",
"value": "d"
},
{
"id": "decimals",
"value": 1
},
{
"id": "custom.displayMode",
"value": "color-text"
},
{
"id": "color",
"value": {
"fixedColor": "dark-green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 27
},
"id": 17,
"options": {
"showHeader": true,
"sortBy": []
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "(time() - container_start_time_seconds{instance=~\"$host\",name=~\"$container\",name=~\".+\"})/86400",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Containers Info",
"transformations": [
{
"id": "filterFieldsByName",
"options": {
"include": {
"names": [
"container_label_com_docker_compose_project",
"container_label_com_docker_compose_project_working_dir",
"image",
"instance",
"name",
"Value",
"container_label_com_docker_compose_service"
]
}
}
},
{
"id": "organize",
"options": {
"excludeByName": {},
"indexByName": {},
"renameByName": {
"Value": "Running",
"container_label_com_docker_compose_project": "Label",
"container_label_com_docker_compose_project_working_dir": "Working dir",
"container_label_com_docker_compose_service": "Service",
"image": "Registry Image",
"instance": "Instance",
"name": "Name"
}
}
}
],
"type": "table"
}
],
"schemaVersion": 27,
"style": "dark",
"tags": [
"cadvisor",
"docker"
],
"templating": {
"list": [
{
"allValue": ".*",
"current": {},
"datasource": "${DS_PROMETHEUS}",
"definition": "label_values({__name__=~\"container.*\"},instance)",
"description": null,
"error": null,
"hide": 0,
"includeAll": true,
"label": "Host",
"multi": false,
"name": "host",
"options": [],
"query": {
"query": "label_values({__name__=~\"container.*\"},instance)",
"refId": "Prometheus-host-Variable-Query"
},
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 5,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"allValue": ".*",
"current": {},
"datasource": "${DS_PROMETHEUS}",
"definition": "label_values({__name__=~\"container.*\", instance=~\"$host\"},name)",
"description": null,
"error": null,
"hide": 0,
"includeAll": true,
"label": "Container",
"multi": false,
"name": "container",
"options": [],
"query": {
"query": "label_values({__name__=~\"container.*\", instance=~\"$host\"},name)",
"refId": "Prometheus-container-Variable-Query"
},
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
]
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Cadvisor exporter",
"uid": "pMEd7m0Mz",
"version": 1,
"description": "Simple exporter for cadvisor only"
}

15766
roles/tools/files/grafana/dashboards/node-exporter-full.json Normal file
View file

File diff suppressed because it is too large Load diff

153
roles/tools/tasks/main.yml
View file

@ -7,7 +7,29 @@
group: "{{ deploy_group }}" group: "{{ deploy_group }}"
mode: "0750" mode: "0750"
# ── Deploy configs and start stack ──────────────────────────────────────────── - name: Create tools subdirectories
ansible.builtin.file:
path: "{{ tools_root }}/{{ item }}"
state: directory
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0755"
loop:
- prometheus
- prometheus/rules
- grafana/provisioning/datasources
- grafana/provisioning/dashboards
- grafana/provisioning/dashboards/json
- loki
- name: Deploy .env file
ansible.builtin.template:
src: env.j2
dest: "{{ tools_root }}/.env"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0600"
- name: Deploy docker-compose.yml - name: Deploy docker-compose.yml
ansible.builtin.template: ansible.builtin.template:
src: docker-compose.yml.j2 src: docker-compose.yml.j2
@ -16,27 +38,130 @@
group: "{{ deploy_group }}" group: "{{ deploy_group }}"
mode: "0640" mode: "0640"
- name: Deploy .env - name: Deploy Prometheus config
ansible.builtin.template: ansible.builtin.template:
src: env.j2 src: prometheus/prometheus.yml.j2
dest: "{{ tools_root }}/.env" dest: "{{ tools_root }}/prometheus/prometheus.yml"
owner: "{{ deploy_user }}" owner: "{{ deploy_user }}"
group: "{{ deploy_group }}" group: "{{ deploy_group }}"
mode: "0600" mode: "0644"
- name: Pull images - name: Deploy Prometheus alert rules
community.docker.docker_image: ansible.builtin.template:
name: "{{ item }}" src: prometheus/rules/alerts.yml.j2
source: pull dest: "{{ tools_root }}/prometheus/rules/alerts.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy AlertManager config
ansible.builtin.template:
src: prometheus/alertmanager.yml.j2
dest: "{{ tools_root }}/prometheus/alertmanager.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy Loki config
ansible.builtin.template:
src: loki/loki.yml.j2
dest: "{{ tools_root }}/loki/loki.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy Grafana Prometheus datasource
ansible.builtin.template:
src: grafana/provisioning/datasources/prometheus.yml.j2
dest: "{{ tools_root }}/grafana/provisioning/datasources/prometheus.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy Grafana Loki datasource
ansible.builtin.template:
src: grafana/provisioning/datasources/loki.yml.j2
dest: "{{ tools_root }}/grafana/provisioning/datasources/loki.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy Grafana dashboard provisioning config
ansible.builtin.template:
src: grafana/provisioning/dashboards/dashboards.yml.j2
dest: "{{ tools_root }}/grafana/provisioning/dashboards/dashboards.yml"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy Node Exporter Full dashboard JSON
ansible.builtin.copy:
src: grafana/dashboards/node-exporter-full.json
dest: "{{ tools_root }}/grafana/provisioning/dashboards/json/node-exporter-full.json"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Deploy cAdvisor dashboard JSON
ansible.builtin.copy:
src: grafana/dashboards/cadvisor.json
dest: "{{ tools_root }}/grafana/provisioning/dashboards/json/cadvisor.json"
owner: "{{ deploy_user }}"
group: "{{ deploy_group }}"
mode: "0644"
- name: Pull monitoring images
ansible.builtin.command: docker pull {{ item }}
loop: loop:
- "{{ outline_image }}" - "{{ prometheus_image }}"
- "{{ outline_db_image }}" - "{{ alertmanager_image }}"
- "{{ outline_redis_image }}" - "{{ node_exporter_image }}"
- "{{ n8n_image }}" - "{{ cadvisor_image }}"
- "{{ grafana_image }}"
- "{{ loki_image }}"
- "{{ uptime_kuma_image }}"
register: pull_result
changed_when: "'Status: Downloaded newer image' in pull_result.stdout"
retries: 5
delay: 30
until: pull_result.rc == 0
# ── UFW: allow main server to reach monitoring services ───────────────────────
- name: Allow main server to reach Loki (Promtail log push)
community.general.ufw:
rule: allow
port: "3100"
proto: tcp
src: "{{ ip_main }}"
- name: Allow main server to reach Prometheus (discord-bot metrics)
community.general.ufw:
rule: allow
port: "9090"
proto: tcp
src: "{{ ip_main }}"
- name: Allow main Traefik to reach Grafana
community.general.ufw:
rule: allow
port: "3000"
proto: tcp
src: "{{ ip_main }}"
- name: Allow main Traefik to reach Uptime Kuma
community.general.ufw:
rule: allow
port: "3001"
proto: tcp
src: "{{ ip_main }}"
- name: Start tools stack - name: Start tools stack
community.docker.docker_compose_v2: community.docker.docker_compose_v2:
project_src: "{{ tools_root }}" project_src: "{{ tools_root }}"
state: present state: present
pull: missing pull: never
remove_orphans: true remove_orphans: true
retries: 3
delay: 15
register: compose_result
until: compose_result is succeeded

253
roles/tools/templates/docker-compose.yml.j2
View file

@ -1,150 +1,157 @@
# Tools stack — generated by Ansible # Tools stack — generated by Ansible
# Do not edit manually; re-run ansible-playbook playbooks/tools.yml # Do not edit manually; re-run ansible-playbook playbooks/tools.yml
# Monitoring: Prometheus, Grafana, Loki, AlertManager, Uptime Kuma, node-exporter, cAdvisor
networks: networks:
# front — non-internal: needed for Docker port binding to work (expose ports to host) monitoring:
# Docker does not create DNAT rules for containers only on internal networks
front:
driver: bridge driver: bridge
outline-internal:
driver: bridge
internal: true
n8n-internal:
driver: bridge
internal: true
volumes: volumes:
outline_db_data: prometheus_data:
outline_redis_data: grafana_data:
n8n_data: loki_data:
uptime_kuma_data:
services: services:
# ── Outline wiki ──────────────────────────────────────────────────────────── # ── Prometheus ─────────────────────────────────────────────────────────────
outline: prometheus:
image: {{ outline_image }} image: {{ prometheus_image }}
container_name: outline container_name: prometheus
restart: unless-stopped restart: unless-stopped
env_file: .env
networks: networks:
- outline-internal - monitoring
- front # needed for host port binding
ports: ports:
# Exposed only to main Traefik (access controlled by UFW) - "127.0.0.1:9090:9090" # exposed to main via UFW rule for discord-bot
- "{{ ip_tools }}:3000:3000" volumes:
- prometheus_data:/prometheus
- {{ tools_root }}/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- {{ tools_root }}/prometheus/rules:/etc/prometheus/rules:ro
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--storage.tsdb.retention.time=30d"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:9090/-/healthy"]
interval: 30s
timeout: 5s
retries: 3
alertmanager:
image: {{ alertmanager_image }}
container_name: alertmanager
restart: unless-stopped
networks:
- monitoring
volumes:
- {{ tools_root }}/prometheus/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
command:
- "--config.file=/etc/alertmanager/alertmanager.yml"
- "--storage.path=/alertmanager"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:9093/-/healthy"]
interval: 30s
timeout: 5s
retries: 3
# ── Exporters (monitor the tools host itself) ───────────────────────────────
node-exporter:
image: {{ node_exporter_image }}
container_name: node-exporter
restart: unless-stopped
networks:
- monitoring
pid: host
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- "--path.procfs=/host/proc"
- "--path.sysfs=/host/sys"
- "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)"
cadvisor:
image: {{ cadvisor_image }}
container_name: cadvisor
restart: unless-stopped
networks:
- monitoring
privileged: true
devices:
- /dev/kmsg
volumes:
- /:/rootfs:ro
- /var/run:/var/run:ro
- /sys:/sys:ro
- /var/lib/docker:/var/lib/docker:ro
- /dev/disk:/dev/disk:ro
# ── Grafana ─────────────────────────────────────────────────────────────────
grafana:
image: {{ grafana_image }}
container_name: grafana
restart: unless-stopped
security_opt:
- no-new-privileges:true
depends_on: depends_on:
outline-db: - prometheus
condition: service_healthy networks:
outline-redis: - monitoring
condition: service_healthy ports:
- "3000:3000"
volumes:
- grafana_data:/var/lib/grafana
- {{ tools_root }}/grafana/provisioning:/etc/grafana/provisioning:ro
env_file: .env
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_USERS_ALLOW_SIGN_UP=false
- GF_SERVER_DOMAIN={{ domain_dashboard }}
- GF_SERVER_ROOT_URL=https://{{ domain_dashboard }}
- GF_AUTH_ANONYMOUS_ENABLED=false
healthcheck: healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000/_health"] test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
interval: 30s interval: 30s
timeout: 5s timeout: 5s
retries: 3 retries: 3
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
outline-db: # ── Loki ────────────────────────────────────────────────────────────────────
image: {{ outline_db_image }} loki:
container_name: outline-db image: {{ loki_image }}
restart: unless-stopped container_name: loki
environment:
POSTGRES_DB: outline
POSTGRES_USER: outline
POSTGRES_PASSWORD: ${OUTLINE_DB_PASSWORD}
networks:
- outline-internal
volumes:
- outline_db_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U outline"]
interval: 10s
timeout: 5s
retries: 5
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
outline-redis:
image: {{ outline_redis_image }}
container_name: outline-redis
restart: unless-stopped restart: unless-stopped
networks: networks:
- outline-internal - monitoring
volumes:
- outline_redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
# ── Outline MCP server ───────────────────────────────────────────────────────
# MCP server exposing Outline wiki to Claude/AI clients (port 8765, internal only)
outline-mcp:
image: {{ outline_mcp_image }}
container_name: outline-mcp
restart: unless-stopped
networks:
- front # needed for host port binding
ports: ports:
- "127.0.0.1:8765:8765" - "3100:3100" # exposed to main for Promtail log ingestion
environment:
- OUTLINE_URL=https://{{ domain_wiki }}
- OUTLINE_API_KEY={{ outline_mcp_api_key }}
- PORT=8765
- HOST=0.0.0.0
- LOG_LEVEL=INFO
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
# ── n8n workflow automation ──────────────────────────────────────────────────
n8n:
image: {{ n8n_image }}
container_name: n8n
restart: unless-stopped
networks:
- n8n-internal
- front # needed for host port binding
ports:
# Exposed only to main Traefik (access controlled by UFW)
- "{{ ip_tools }}:5678:5678"
volumes: volumes:
- n8n_data:/home/node/.n8n - loki_data:/loki
environment: - {{ tools_root }}/loki/loki.yml:/etc/loki/local-config.yaml:ro
- N8N_HOST={{ domain_n8n }} command: -config.file=/etc/loki/local-config.yaml
- N8N_PORT=5678
- N8N_PROTOCOL=https
- WEBHOOK_URL=https://{{ domain_n8n }}/
- N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
- N8N_USER_MANAGEMENT_JWT_SECRET=${N8N_JWT_SECRET}
- GENERIC_TIMEZONE=Europe/Moscow
- TZ=Europe/Moscow
- N8N_METRICS=false
- N8N_LOG_LEVEL=warn
- EXECUTIONS_DATA_PRUNE=true
- EXECUTIONS_DATA_MAX_AGE=336
healthcheck: healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:5678/healthz"] test: ["CMD", "wget", "-qO-", "http://localhost:3100/ready"]
interval: 30s
timeout: 5s
retries: 3
# ── Uptime Kuma ─────────────────────────────────────────────────────────────
uptime-kuma:
image: {{ uptime_kuma_image }}
container_name: uptime-kuma
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- monitoring
ports:
- "3001:3001"
volumes:
- uptime_kuma_data:/app/data
healthcheck:
test: ["CMD", "curl", "-sf", "http://localhost:3001/"]
interval: 30s interval: 30s
timeout: 5s timeout: 5s
retries: 3 retries: 3
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"

49
roles/tools/templates/env.j2
View file

@ -1,47 +1,2 @@
# Outline env — generated by Ansible # Generated by Ansible — do not edit manually
NODE_ENV=production GF_SECURITY_ADMIN_PASSWORD={{ grafana_admin_password }}
SECRET_KEY={{ outline_secret_key }}
UTILS_SECRET={{ outline_utils_secret }}
# Database
DATABASE_URL=postgres://outline:{{ outline_db_password }}@outline-db:5432/outline
PGSSLMODE=disable
# Redis
REDIS_URL=redis://outline-redis:6379
# App URL
URL=https://{{ domain_wiki }}
PORT=3000
# S3 file storage (Timeweb Object Storage)
AWS_ACCESS_KEY_ID={{ s3_access_key }}
AWS_SECRET_ACCESS_KEY={{ s3_secret_key }}
AWS_REGION=ru-1
AWS_S3_UPLOAD_BUCKET_NAME=visual-outline
AWS_S3_UPLOAD_BUCKET_URL=https://s3.timeweb.cloud
AWS_S3_FORCE_PATH_STYLE=true
AWS_S3_ACL=private
FILE_STORAGE=s3
# Auth
AUTH_PROVIDERS=email
# SMTP via relay on main server (tools-server has outbound SMTP blocked)
SMTP_HOST={{ ip_main }}
SMTP_PORT=1025
SMTP_FROM_EMAIL=noreply@{{ domain_base }}
SMTP_FROM_NAME=Visual Wiki
SMTP_SECURE=false
# Outline DB password (used in docker-compose)
OUTLINE_DB_PASSWORD={{ outline_db_password }}
# Optional
DEFAULT_LANGUAGE=en_US
RATE_LIMITER_ENABLED=true
ENABLE_UPDATES=false
# n8n secrets
N8N_ENCRYPTION_KEY={{ n8n_encryption_key }}
N8N_JWT_SECRET={{ n8n_jwt_secret }}

13
roles/tools/templates/grafana/provisioning/dashboards/dashboards.yml.j2 Normal file
View file

@ -0,0 +1,13 @@
# Generated by Ansible — do not edit manually
apiVersion: 1
providers:
- name: default
orgId: 1
folder: ""
type: file
disableDeletion: false
updateIntervalSeconds: 30
allowUiUpdates: false
options:
path: /etc/grafana/provisioning/dashboards/json

10
roles/tools/templates/grafana/provisioning/datasources/loki.yml.j2 Normal file
View file

@ -0,0 +1,10 @@
# Generated by Ansible — do not edit manually
apiVersion: 1
datasources:
- name: Loki
type: loki
access: proxy
url: http://loki:3100
isDefault: false
editable: false

10
roles/tools/templates/grafana/provisioning/datasources/prometheus.yml.j2 Normal file
View file

@ -0,0 +1,10 @@
# Generated by Ansible — do not edit manually
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
editable: false

36
roles/tools/templates/loki/loki.yml.j2 Normal file
View file

@ -0,0 +1,36 @@
# Generated by Ansible — do not edit manually
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
common:
instance_addr: 127.0.0.1
path_prefix: /loki
storage:
filesystem:
chunks_directory: /loki/chunks
rules_directory: /loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
schema_config:
configs:
- from: 2020-10-24
store: tsdb
object_store: filesystem
schema: v13
index:
prefix: index_
period: 24h
limits_config:
retention_period: 30d
compactor:
working_directory: /loki/retention
delete_request_store: filesystem
retention_enabled: true

38
roles/tools/templates/prometheus/alertmanager.yml.j2 Normal file
View file

@ -0,0 +1,38 @@
# Generated by Ansible — do not edit manually
global:
resolve_timeout: 5m
route:
group_by: [alertname, severity]
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
receiver: all
receivers:
- name: all
telegram_configs:
- bot_token: "{{ alertmanager_telegram_token }}"
chat_id: {{ alertmanager_telegram_chat_id }}
message: |
{{ '{{' }} range .Alerts {{ '}}' }}
{{ '{{' }} if eq .Status "firing" {{ '}}' }}🔴{{ '{{' }} else {{ '}}' }}🟢{{ '{{' }} end {{ '}}' }} *{{ '{{' }} .Labels.alertname {{ '}}' }}*
{{ '{{' }} .Annotations.summary {{ '}}' }}
{{ '{{' }} .Annotations.description {{ '}}' }}
{{ '{{' }} end {{ '}}' }}
parse_mode: Markdown
discord_configs:
- webhook_url: "{{ discord_webhook_alerts }}"
title: >-
{{ '{{' }} if eq (index .Alerts 0).Status "firing" {{ '}}' }}🔴 Alert{{ '{{' }} else {{ '}}' }}🟢 Resolved{{ '{{' }} end {{ '}}' }}
message: |
{{ '{{' }} range .Alerts {{ '}}' }}
**{{ '{{' }} .Labels.alertname {{ '}}' }}**
{{ '{{' }} .Annotations.summary {{ '}}' }}
{{ '{{' }} .Annotations.description {{ '}}' }}
{{ '{{' }} end {{ '}}' }}
inhibit_rules:
- source_matchers: [severity="critical"]
target_matchers: [severity="warning"]
equal: [alertname]

49
roles/tools/templates/prometheus/prometheus.yml.j2 Normal file
View file

@ -0,0 +1,49 @@
# Generated by Ansible — do not edit manually
global:
scrape_interval: 15s
evaluation_interval: 15s
external_labels:
instance: "{{ domain_base }}"
alerting:
alertmanagers:
- static_configs:
- targets: ["alertmanager:9093"]
rule_files:
- /etc/prometheus/rules/*.yml
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ["localhost:9090"]
# tools server metrics
- job_name: node-exporter-tools
static_configs:
- targets: ["node-exporter:9100"]
labels:
host: tools
- job_name: cadvisor-tools
static_configs:
- targets: ["cadvisor:8080"]
labels:
host: tools
- job_name: alertmanager
static_configs:
- targets: ["alertmanager:9093"]
# main server metrics (scraped over network)
- job_name: node-exporter-main
static_configs:
- targets: ["{{ ip_main }}:9100"]
labels:
host: main
- job_name: cadvisor-main
static_configs:
- targets: ["{{ ip_main }}:8080"]
labels:
host: main

86
roles/tools/templates/prometheus/rules/alerts.yml.j2 Normal file
View file

@ -0,0 +1,86 @@
# Generated by Ansible — do not edit manually
groups:
- name: host
rules:
- alert: HighCPULoad
expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
for: 5m
labels:
severity: warning
annotations:
summary: "Высокая нагрузка CPU ({{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%)"
description: "CPU загружен более 85% на протяжении 5 минут."
- alert: HighMemoryUsage
expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 > 85
for: 5m
labels:
severity: warning
annotations:
summary: "Высокое использование RAM ({{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%)"
description: "Использование RAM превысило 85%."
- alert: CriticalMemoryUsage
expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 > 95
for: 2m
labels:
severity: critical
annotations:
summary: "Критическое использование RAM ({{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%)"
description: "RAM заполнена на 95%+. Возможны OOM kills."
- alert: DiskSpaceWarning
expr: (1 - (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|aufs"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay|aufs"})) * 100 > 75
for: 5m
labels:
severity: warning
annotations:
summary: "Заканчивается место на диске ({{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%)"
description: "Диск {{ '{{' }} $labels.mountpoint {{ '}}' }} занят на {{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%."
- alert: DiskSpaceCritical
expr: (1 - (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|aufs"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay|aufs"})) * 100 > 90
for: 2m
labels:
severity: critical
annotations:
summary: "Критически мало места на диске ({{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%)"
description: "Диск {{ '{{' }} $labels.mountpoint {{ '}}' }} занят на {{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%."
- alert: SwapUsageHigh
expr: (1 - (node_memory_SwapFree_bytes / node_memory_SwapTotal_bytes)) * 100 > 50
for: 5m
labels:
severity: warning
annotations:
summary: "Высокое использование swap ({{ '{{' }} $value | printf \"%.0f\" {{ '}}' }}%)"
description: "Swap используется более чем на 50% — RAM под давлением."
- name: containers
rules:
- alert: ContainerDown
expr: absent(container_last_seen{name=~".+"}) or time() - container_last_seen{name=~".+"} > 60
for: 2m
labels:
severity: critical
annotations:
summary: "Контейнер {{ '{{' }} $labels.name {{ '}}' }} недоступен"
description: "Контейнер не отвечает более 2 минут."
- alert: ContainerHighMemory
expr: (container_memory_usage_bytes{name=~".+"} / (container_spec_memory_limit_bytes{name=~".+"} > 0)) * 100 > 90
for: 5m
labels:
severity: warning
annotations:
summary: "Контейнер {{ '{{' }} $labels.name {{ '}}' }} использует 90%+ памяти"
description: "Контейнер близок к mem_limit — возможен OOM kill."
- alert: ContainerRestarting
expr: increase(container_last_seen{name=~".+"}[5m]) == 0 and rate(container_cpu_usage_seconds_total{name=~".+"}[5m]) == 0
for: 0m
labels:
severity: warning
annotations:
summary: "Контейнер {{ '{{' }} $labels.name {{ '}}' }} возможно перезапускается"
description: "Контейнер не активен — проверьте docker ps."

7
terraform/.gitignore vendored Normal file
View file

@ -0,0 +1,7 @@
# Секреты и state — никогда не коммитить
terraform.tfvars
*.tfstate
*.tfstate.backup
.terraform/
.terraform.lock.hcl
crash.log

9
terraform/outputs.tf Normal file
View file

@ -0,0 +1,9 @@
output "main_ip" {
description = "IP main-сервера"
value = "87.249.49.32"
}
output "tools_ip" {
description = "IP tools-сервера"
value = "85.193.83.9"
}

14
terraform/providers.tf Normal file
View file

@ -0,0 +1,14 @@
terraform {
required_version = ">= 1.5"
required_providers {
twc = {
source = "timeweb-cloud/timeweb-cloud"
version = "~> 1.0"
}
}
}
provider "twc" {
token = var.timeweb_token
}

25
terraform/servers.tf Normal file
View file

@ -0,0 +1,25 @@
# ── Серверы ───────────────────────────────────────────────────────────────────
resource "twc_server" "main" {
name = "main"
comment = "Основной: Traefik, Forgejo, Plane, Vaultwarden, Outline, n8n, CI/CD"
os_id = 99
preset_id = 2453
lifecycle {
prevent_destroy = true
ignore_changes = [is_root_password_required]
}
}
resource "twc_server" "tools" {
name = "tools"
comment = "Мониторинг: Grafana, Prometheus, Loki, AlertManager, Uptime Kuma"
os_id = 99
preset_id = 2449
lifecycle {
prevent_destroy = true
ignore_changes = [is_root_password_required]
}
}

27
terraform/storage.tf Normal file
View file

@ -0,0 +1,27 @@
# ── S3 Object Storage ─────────────────────────────────────────────────────────
# Импортировано через:
# terraform import twc_s3_bucket.backup 481333
# terraform import twc_s3_bucket.outline 481335
resource "twc_s3_bucket" "backup" {
name = "walava-backup"
type = "private"
preset_id = 2669
lifecycle {
# name/type — write-once поля, нельзя менять после создания
ignore_changes = [name, type]
prevent_destroy = true
}
}
resource "twc_s3_bucket" "outline" {
name = "walava-outline"
type = "private"
preset_id = 2669
lifecycle {
ignore_changes = [name, type]
prevent_destroy = true
}
}

2
terraform/terraform.tfvars.example Normal file
View file

@ -0,0 +1,2 @@
# Скопируй в terraform.tfvars (он в .gitignore)
timeweb_token = "your-api-token"

5
terraform/variables.tf Normal file
View file

@ -0,0 +1,5 @@
variable "timeweb_token" {
description = "Timeweb Cloud API token"
type = string
sensitive = true
}