sercanto/infra
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
nix-community infrastructure [maintainer=@zowoq]
661 commits 1 branch 0 tags 3.4 MiB
Nix 58.3%
Python 29.1%
HCL 9.2%
Shell 2.9%
sed 0.5%
Find a file
Jörg Thalheim 4e16961bb2 tasks: reformat with black
2022-01-15 13:38:30 +01:00
.git-crypt Add 1 git-crypt collaborator 2021-01-19 09:15:41 +01:00
.github switch to hercules-ci 2021-11-12 10:08:20 +01:00
build01 include correct builder 2022-01-09 15:37:50 +01:00
build02 build02: remove test-cachix service 2021-12-27 21:38:09 -08:00
build03 add raid profile 2021-12-26 07:44:39 +01:00
build04 hercules: add cachix deploy agent secrets 2021-12-24 07:42:09 +01:00
keys import gpg keys with import-keys-hook 2021-09-29 19:09:35 +02:00
nix Merge pull request from nix-community/update/cachix-f5cd1b44c2b3dffd6cc31c56a35c55a8775acf75 2022-01-15 12:19:35 +00:00
roles cachix-deploy: fix service 2022-01-15 13:36:18 +01:00
secrets add build02-root 2021-10-24 01:10:30 +02:00
services matterbridge: fix secrets path 2021-11-01 14:12:14 +01:00
terraform update to Terraform 1.0 2021-08-18 13:05:59 +02:00
users remove worldofpeace () 2021-05-09 18:10:16 +02:00
.envrc switch from nixops to morph 2021-10-03 14:22:24 +02:00
.gitignore update to Terraform 1.0 2021-08-18 13:05:59 +02:00
.sops.yaml hercules: also encrypt for build02 2021-12-26 08:58:06 +01:00
_config.yml configure GitHub pages 2020-05-03 15:11:06 +02:00
ci.nix speed up deployment process a bit 2022-01-15 13:21:35 +01:00
ci.sh switch to hercules-ci 2021-11-12 10:08:20 +01:00
deploy drop morph 2021-10-21 11:24:51 +02:00
deploy_nixos.py deploy: fix printing empty lines 2021-11-01 14:35:19 +01:00
README.md README: update matrix channel address 2021-11-11 17:38:50 +01:00
secrets.nix secrets: only remove suffix "\n" for the buildkite token () 2020-05-01 16:44:05 +00:00
shell.nix drop morph 2021-10-21 11:24:51 +02:00
tasks.py tasks: reformat with black 2022-01-15 13:38:30 +01:00

README.md

nix-community infrastructure

Welcome to the Nix Community infrastructure project. This project holds all the NixOS and Terraform configuration for this organization.

Support

If you hit any issues, ping us on Matrix in the nix-community room (see the admin list below) or create an issue here: New Issue.

Administrators

  • @adisbladis
  • @flokli
  • @grahamc
  • @Mic92
  • @nlewo
  • @ryantm
  • @zimbatm

Services

  • BuildKite agent - on build01
  • GitLab agent - on build01
  • hound - on build01
  • https://hydra.nix-community.org - on build01
  • marvin-mk2 - on build01
  • matterbridge - on build01
  • ryantm-updater bot - on build02

Hosts

build01 build01

This machine is perfect for running heavy builds.

  • Provider: Hetzner
  • CPU: AMD Ryzen 7 1700X Eight-Core Processor
  • RAM: 64GB
  • Drives: 2 x 512 GB SATA SSD

build02

This machine currently just runs r-ryantm/nixpkgs-update.

  • Provider: Hetzner
  • CPU: AMD Ryzen 7 3700X Eight-Core Processor
  • RAM: 64GB DDR4 ECC
  • Drives: 2 x 1 TB NVME in RAID 1

build03

This machine is a replacement for build01.

  • Provider: Hetzner
  • CPU: AMD Ryzen 5 3600 6-Core Processor
  • RAM: 64GB DDR4 ECC
  • Drives: 2 x 512 GB NVME in RAID 1

build04

This machine is meant as an aarch64 builder for our hydra instance running on build03.

  • Provider: Oracle cloud
  • Instance type: Ampere A1 Compute
  • CPU: 4 VCPUs on an Ampere Altra (arm64)
  • RAM: 24GB
  • Drives: 200 GB Block

Cache

All the builds on these machines are pushed to https://nix-community.cachix.org/

Thanks to Cachix for sponsoring our binary cache!

File hierarchy

  • ./build\d+ - build machines
  • ./ci.sh - What is executed by CI
  • ./deploy - Deploy script
  • ./nix - pinned Nix dependencies and overlays
  • ./roles - shared NixOS configuration modules
  • ./secrets - git-crypt encrypted secrets
  • ./services - single instances of NixOS services
  • ./terraform - Setup DNS
  • ./users - NixOS configuration of our admins

Deployment commands:

$ ./deploy

If you want to reboot a machine, use the following command to also deploy secrets afterwards:

$ inv deploy --hosts build02 reboot --hosts build02

Install/Fix system from Hetzner recovery mode

  1. Install kexec image from Hetzner recovery system as described in kexec.nix and boot into it

  2. Format and/or mount all filesystems to /mnt:

$ inv format-disks --hosts buildXX --disks /dev/nvme0n1,/dev/nvme1n1
  1. Setup secrets
$ inv setup-secret --hosts buildXX
  1. Generate configuration and download to the repo
$ nixos-generate-config  --root /tmp
$ scp buildXX.nix-community.org:/tmp/etc/nixos/hardware-configuration.nix buildXX/hardware-configuration.nix
  1. Build and install
$ inv install-nixos --hosts buildXX

Debug VM

You can start a vm from the rescue system in order to debug the boot:

$ nix-shell -p qemu_kvm --run 'qemu-kvm -m 10G -hda /dev/sda -hdb /dev/sdb -curses -cpu host -enable-kvm'