Publié le 29.11.2019 | Jacques Foucry

My previous hosted computer was on scaleway with a huge issue. As you can expect, I crypted my disk. In order to reboot and give the disk key I need a java 7 installed on a Windows computer. With the time It became more and more difficult to make freebds-update.

Another probleme, I host many jails for Exodus Privacy and the 1TBx2 disks became a little bit to much occupied. You know that zfs does not like that :-)

So I decided to change the machine. For a reasonnable price I found Hetzner.

With the BIG help of my friend Ollivier Robert I made a geli installlation. Let me explain how.

First Boot

Form the Hetzner console choose to boot on rescue mode:

Boot on rescue mode (FreeBSD 12 (beta))

First step in rescue mode

Erase existing partitions table

First, swap the previous partition table (look it does not exit, but we’re must sure).

 dd if=/dev/zero of=/dev/ada0 bs=512 count=10
 dd if=/dev/zero of=/dev/ada1 bs=512 count=10

Download and install a minimal FreeBSD image

Download the mfsbsd image into /tmp:

 cd /tmp
 fetch https://mfsbsd.vx.sk/files/images/12/amd64/mfsbsd-se-12.1-RELEASE-amd64.img
mfsbsd-se-12.1-RELEASE-amd64.img                       319 MB   40 MBps    08s

Copy the image into the first disk of the machine.

 dd if=mfsbsd-se-12.1-RELEASE-amd64.img of=/dev/ada0 status=progress
   330774528 bytes (331 MB, 315 MiB) transferred 43.049s, 7684 kB/s
 653376+0 records in
 653376+0 records out
 334528512 bytes transferred in 43.476395 secs (7694486 bytes/sec)

Check the result with gpart and recover the corrupt GPT:

 [root@rescue /tmp]# gpart show ada0
 =>    40  653296  ada0  GPT  (7.3T) [CORRUPT]
       40     472     1  freebsd-boot  (236K)
      512  652824     2  freebsd-ufs  (319M)

 [root@rescue /tmp]# gpart recover ada0
  ada0 recovered
 [root@rescue /tmp]# gpart show ada0
 =>         40  15628053088  ada0  GPT  (7.3T)
            40          472     1  freebsd-boot  (236K)
           512       652824     2  freebsd-ufs  (319M)
        653336  15627399792        - free -  (7.3T)

Create a pseudo-MBR record with fdisk:

 fdisk -a -1 ada0
 ******* Working on device /dev/ada0 *******
 parameters extracted from in-core disklabel are:
 cylinders=15504021 heads=16 sectors/track=63 (1008 blks/cyl)

 Figures below won't work with BIOS for partitions not in cyl 1
 parameters to be used for BIOS calculations are:
 cylinders=15504021 heads=16 sectors/track=63 (1008 blks/cyl)

 Media sector size is 512
 Warning: BIOS sector numbering starts with sector 1
 Information from DOS bootblock is:
 The data for partition 1 is:
 sysid 238 (0xee),(EFI GPT)
     start 1, size 4294967295 (2097151 Meg), flag 0
     beg: cyl 0/ head 0/ sector 2;
     end: cyl 1023/ head 255/ sector 63
 Do you want to change the active partition? [n] y
 Supply a decimal value for "active partition" [1]
 Are you happy with this choice [n] y

 We haven't changed the partition table yet.  This is your last chance.
 parameters extracted from in-core disklabel are:
 cylinders=15504021 heads=16 sectors/track=63 (1008 blks/cyl)

 Figures below won't work with BIOS for partitions not in cyl 1
 parameters to be used for BIOS calculations are:
 cylinders=15504021 heads=16 sectors/track=63 (1008 blks/cyl)

 Information from DOS bootblock is:
 1: sysid 238 (0xee),(EFI GPT)
     start 1, size 4294967295 (2097151 Meg), flag 80 (active)
     beg: cyl 0/ head 0/ sector 2;
     end: cyl 1023/ head 255/ sector 63
 2: <UNUSED>
 3: <UNUSED>
 4: <UNUSED>
 Should we write new partition table? [n] y

Think about removing the host key stored in ~/.ssh/known_hosts with the command ssh-keygen -R <ip-address>

Then is time to reboot the machine with msfBSD:

 [root@rescue /tmp]# reboot

Connect to server in `msfBSD` mode

Once the server is available, you can connect, using root with mfsroot as password:

 ssh root@xxx.xxx.xxx.xxx
 The authenticity of host 'xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx)' can't be established.
 ECDSA key fingerprint is SHA256:ScONL6YYFmtbd7rJY+l2IwTB5JTCy91Bk7qwS39A1Ws.
 No matching host key fingerprint found in DNS.
 Are you sure you want to continue connecting (yes/no)? yes
 Warning: Permanently added 'xxx.xxx.xxx.xxx' (ECDSA) to the list of known hosts.
 Password for root@mfsbsd:
 X11 forwarding request failed on channel 0
 FreeBSD 12.1-RELEASE r354233 GENERIC

 Welcome to mfsBSD SE, the memory based FreeBSD distribution.
 This is a special version intended for full-ZFS install of FreeBSD

 To make a full-ZFS FreeBSD install from this ISO:

 1. Mount the CD device
 ( e.g. mount_cd9660 /dev/cd0 /cdrom )
 2. Run "zfsinstall" with path to release directory and your drive
 ( e.g. zfsinstall -d ada0 -u /cdrom/9.2-RELEASE-amd64 )

 Run zfsinstall with the -h flag for help or without flags for options.
 I recommend creating a GPT swap partition (e.g. -s 2G for a 2GB swap).

 Feel free to email me with any bug reports or feature suggestions.
 Martin Matuska <mm@FreeBSD.org>
 http://mfsbsd.vx.sk/

Let’s take a look to first disk layout:

 gpart show ada0
 =>         40  15628053088  ada0  GPT  (7.3T)
            40          472     1  freebsd-boot  (236K)
           512       656920     2  freebsd-ufs  (321M)
        657432  15627395696        - free -  (7.3T)

Install the minimal system

We will need base.txz and kernel.txz in order to install the system. We can fetch them form the internet. But there is another solution.

We have then on… /dev/ada0.

Just mount the second partition:

 mount /dev/ada0p2 /mnt

And copy the files into /tmp/release:

1
2
3


 mkdir -p /tmp/release
 cp /mnt/12.1-RELEASE-amd64/base.txz /tmp/release/.
 cp /mnt/12.1-RELEASE-amd64/kernel.txz /tmp/release/.

Be sure to umount /mnt, we will need it after.

 umount /mnt

We need to modify the zfsinstall script in order to NOT create dataset for /var and /tmp.

Simply clear the FS_LIST variable at the beginning of the script.

Be really careful with this step, removing zboot/root/var dataset after the installation will be REALLY difficult and it’s presence will avoid any boot. I should redo all the procedure plus cleaning the disks.

It’s time to install the minimalist boot pool. We’re going to use the zfsinstall script which will help us¹.

 zfsinstall -u /tmp/release -d ada0 -d ada1 -m /mnt -s 64G -z 5G -A -4 -c -p zboot

Which means:

-u → URL to find base.txz and kernel.xz files;
-d → geom destination: ada0 and ada1 disks;
-m → mount on /mnt;
-s → swap size (depending of your server RAM);
-z → partition size;
-A → align partition to 4K blocks²;
-4 → use Fletcher4 as default algorithm;
-c → enable lzjb compression;
-p → pool name.

And we have an issue (and the solution):

Notice: two drives selected, automatically choosing mirror mode
Error: /dev/ada0 already contains a partition table.

=>         40  15628053088  ada0  GPT  (7.3T)
           40          472     1  freebsd-boot  (236K)
          512       656920     2  freebsd-ufs  (321M)
       657432  15627395696        - free -  (7.3T)

You may erase the partition table manually with the `destroygeom` command

As recommended just run destroygeom -d ada0 and relaunch the zfsinstall command.

Once the zfsinstall is finish just have a look to the created datasets:

 # zfs list
 NAME             USED  AVAIL  REFER  MOUNTPOINT
 zboot            616K  4.36G    88K  none
 zboot/root       264K  4.36G    88K  /mnt

Some adjustments

We’re going to make some adjustments to the installed zboot environment. We fist need to use chroot /mnt in order to simulate the root system.

The minimal content needed live in /boot/loader.conf and /etc/rc.conf

`/boot/loader.conf`

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


zfs_load="YES"
aesni_load="YES"
geom_mirror_load="YES"
geom_eli_load="YES"
fdescfs_load="YES"
nullfs_load="YES"
pf_load="YES"
pflog_load="YES"
vm.kmem_size="64G"
vfs.zfs.arc_max="28G"
vfs.zfs.arc_min="16G"
vfs.root.mountfrom="zfs:zboot/root"
# http://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061388.html
vfs.zfs.txg.timeout="5"

`/etc/rc.conf`

1
2
3
4
5
6
7


hostname="mynewmachine.example.com
cron_enable="NO"
sshd_enable="YES"
zfs_enable="YES"
local_unbound_enable="YES"
sendmail_enable="NONE"
ifconfig_em0="DHCP"

Now you can do some other customization like add a user, put your ssh key on authorized_keys file, etc.

Think about create a password for root and authorize root login.

Now, cross your fingers and reboot…

 # ssh root@xxx.xxx.xxx.xxx -i id_ed25519
 Password for root@newmachine.example.com:
 X11 forwarding request failed on channel 0
 Last login: Fri Nov 22 17:20:34 2019 from mycomputer.example.com
 FreeBSD 12.1-RELEASE r354233 GENERIC

 Welcome to FreeBSD!

 Release Notes, Errata: https://www.FreeBSD.org/releases/
 Security Advisories:   https://www.FreeBSD.org/security/
 FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
 FreeBSD FAQ:           https://www.FreeBSD.org/faq/
 Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
 FreeBSD Forums:        https://forums.FreeBSD.org/

 Documents installed with the system are in the /usr/local/share/doc/freebsd/
 directory, or can be installed later with:  pkg install en-freebsd-doc
 For other languages, replace "en" with a language code like de or fr.

 Show the version of FreeBSD installed:  freebsd-version ; uname -a
 Please include that output and any error messages when posting questions.
 Introduction to manual pages:  man man
 FreeBSD directory layout:      man hier

 Edit /etc/motd to change this login announcement.

Seems it booted. \o/

Let’s show at the disks:

 # df -h
 Filesystem    Size    Used   Avail Capacity  Mounted on
 zboot/root    4.4G    507M    3.9G    11%    /
 devfs         1.0K    1.0K      0B   100%    /dev

And what about zpool?

 NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
 zboot  4.50G   508M  4.00G        -         -     1%    11%  1.00x  ONLINE  -

Encrypt the disks

With gpart were gonna create GPT partition on each disk:

 gpart add -a 4k -t freebsd-zfs -l tank0 ada0
 gpart add -a 4k -t freebsd-zfs -l tank1 ada1

Then with geli we’re encrypt each slide:

 geli init -s 4096 -l 256 /dev/gpt/tank0
 geli init -s 4096 -l 256 /dev/gpt/tank1

We attach both drives:

 geli attach /dev/gpt/tank0
 geli attach /dev/gpt/tank1

Now where going to encrypt and mirror the swap slice:

 gpart modify -i 2 -l swap0 ada0
 gpart modify -i 2 -l swap1 ada1

I had some trouble with labeling ada0p2. I need to stop the swap (swapof -a) and use the glabel command ( glabel create swap0 /dev/ada0p2 ) in order to have the right label.

Mirroring the swap slices:

 gmirror label swap gpt/swap0 gpt/swap1

The `/etc/fstab` file

The /etc/fstab files must be create in order to use the mirrored swap:

 /dev/mirror/swap.eli    none swap sw 0 0

Filesystems

Before installing the whole system, were gonna enable compression and create filesystems.

Create `tank` zpool

On the mirrored and attached tank{0,1} partition we create a single zpool called tank:

 zpool create -o altroot=/tank -O mountpoint=none tank mirror gpt/tank0.eli gpt/tank1.eli

Enabling ZFS compression

First we’ll enable ZFS compression for all the system (in fact for all the tank pool).

 zfs set compression=lz4 tank

Then create some ZFS datasets with there mount points:

Be very careful with that part. It make me headaches to find what’s going wrong with my installation. Be sure to use tank/root where tank is the name of the zpool and root will become the root (/) of your complete running machine.

1
2
3
4
5


 zfs create -o compression=off tank/root
 zfs set reservation=512m tank
 zfs create -o mountpoint=/tank/root/usr tank/root/usr
 zfs create -o mountpoint=/tank/root/usr/obj tank/root/usr/obj
 zfs create -o mountpoint=/tank/root/usr/local tank/root/usr/local

We use lz4 compression algorithm for all the zpool except for tank/root because compression seems to issues for kernel loading.

I like to have /usr/local/etc in as specific dataset:

 zfs create -o mountpoint=/tank/root/usr/local/etc tank/root/usr/local/etc

Create other datasets

Using dataset give many advantage. So we’re going to create the most common of them:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


 zfs create -o mountpoint=/tank/root/var tank/root/var
 zfs create -o exec=off -o setuid=off tank/root/var/empty
 zfs create -o exec=off -o setuid=off tank/root/var/named
 zfs create -o exec=off -o setuid=off tank/root/var/run

 zfs create -o mountpoint=/tank/root/var/tmp tank/root/var/tmp
 zfs set exec=off tank/root/var/tmp
 zfs set setuid=off  tank/root/var/tmp
 chmod 1777 /tank/root/var/tmp

 zfs create -o mountpoint=/tank/root/tmp tank/root/tmp
 zfs set setuid=off  tank/root/tmp
 chmod 1777 /tank/root/tmp

 zfs create -o mountpoint=/tank/root/usr/home tank/root/usr/home

You certainly manage to use ports in order to compile software by yourself:

1
2
3
4


 zfs create -o mountpoint=/tank/root/usr/ports -o setuid=off tank/root/usr/ports
 zfs create -o mountpoint=/tank/root/usr/ports/distfiles  -o compression=off -o exec=off -o setuid=off tank/root/usr/ports/distfiles
 zfs create -o mountpoint=/tank/root/usr/ports/packages -o compression=off -o exec=off -o setuid=off tank/root/usr/ports/packages
 zfs create -o mountpoint=/tank/root/usr/src -o setuid=off tank/root/usr/src

And if you are planning to use jails with iocage :

 zfs create -o mountpoint=/tank/root/jails tank/root/jails

Installing the system

We still have several step in order to install the system. We need again base.xz and kernel.xz files but we don’t have an archive on the machine anymore. So we need to obtain them. By downloading an image (-memstick.img), a cd image or directly from FTP site.

Downloading the system

I downloaded the 12.1-RELEASE folder from ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.1-RELEASE on /tmp/

 pkg install ncftp
 cd /tmp
 mkdir 12.1-RELEASE
 cd 12.1-RELEASE
 ncftp ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/
 get 12.1-RELEASE/*
 BUILDDATE:                                               9.00 B  152.54 B/s
 MANIFEST:                                                1.02 kB   22.92 kB/s
 REVISION:                                                8.00 B  135.02 B/s
 SRCBRANCH:                                              14.00 B  171.11 B/s
 base-dbg.txz:                                          149.20 MB    1.44 MB/s
 base.txz:                                              153.75 MB    1.27 MB/s
 doc.txz:                                               180.00 B    3.12 kB/s
 kernel-dbg.txz:                                         74.79 MB    1.35 MB/s
 kernel.txz:                                             39.78 MB    1.46 MB/s
 lib32-dbg.txz:                                          14.84 MB    1.39 MB/s
 lib32.txz:                                              58.61 MB    1.58 MB/s
 ports.txz:                                              37.64 MB    1.32 MB/s
 src.txz:                                               158.50 MB    1.25 MB/s
 tests.txz:                                              11.86 MB    1.20 MB/s
 ncftp ...D/releases/amd64/amd64 > quit

    Thank you for using NcFTP Client.
    If you find it useful, please consider making a donation!
    http://www.ncftp.com/ncftp/donate.html

Decompressing and installing the system

by default root use the csh shell. In order to copy/paste the small script used to decompress and install the system, you need to use sh.

1
2
3
4
5


 /bin/sh
 cd /tmp/12.1-RELEASE
 for i in base doc kernel lib32 src ports; do \
  xz -d -c $i.txz | tar -C /tank/root -xf - ; \
 done

If you want to stay with csh, you can use:

1
2
3


foreach foo ( base.txz kernel.txz lib32.txz src.txz ports.txz doc.txz )
 xz -d -c -v $foo | tar -C /tank/root -xf -
end

Configuring some variables in proper places

We need to add variables in order to make the system running.

`loader.conf` file

We’re gonna make zboot/root/boot/loader.conf and tank/root/boot/loaer.conf almost synchronized.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


# fs modules
zfs_load="YES"
geom_mirror_load="YES"
fdescfs_load="YES"
nullfs_load="YES"

# Crypto stuff
geom_eli_load="YES"
crypto_load="YES"
aesni_load="YES"
cryptodev_load="YES"

# pf stuff
pf_load="YES"
pflog_load="YES"

# tuning
vm.kmem_size="96G"

We don’t need vfs.root.mountfrom="zfs:tank/root" since the loader is able to find which dataset to use by is own.

Current recommendations for vm.kmem_size is between 1.5X and 2X the available RAM

`sysctl.conf` file

Some values must be put into zboot/root/etc/sysctl.conf, once again synchronized with tank/root/etc/sysctl.conf:

 ##-- tuning
 kern.maxvnodes=260000

`rc.conf`

The case of rc.conf file is a little bit different. Some input must be the same, but in tank/root/etc/rc.conf we gonna put “high level” information like nginx, iocage, etc.

In fact, the zboot/root/etc/rc.conf will give hostname, network information, starting up sshd, ntp and geli parameters need to boot the second part of machine. Those informations will the same in tank/root/etc/rc.conf:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


#ZFS
zfs_enable="YES"

## Network
sshd_enable="YES"
sshd_flags="-o PermitRootLogin=yes"[^2]
hostname="hostname.example.com"
# or ifconfig_em0="DHCP"
ifconfig_em0="inet a.b.c.d netmask 0xffffff00"
defaultrouter="a.b.c.d"             # Only if you don't use DHCP

# NTP
ntpd_enable="YES"
ntpd_sync_on_start="YES"
# geli
geli_swap_flags="-e aes -l 256 -s 4096 -d"

Once you made those files in zboot/root/boot and zboot/root/etc, you can copy them into tank/root/boot and tank/root/etc.

The final countdown

We have several small steps to do before launching our rocket.

Configuring encrypted swap in `/tank/root/etc/fstab`

Add this this line in your /tank/root/etc/fstab file:

 /dev/mirror/swap.eli   none    swap    sw  0 0

There is an issue (and a solution, see gmirror(8) for more information). By default, you won’t be able to have kernel crash dump on a gmirror device. To solve this issue, we need to use two small scripts:

 echo 'gmirror configure -b prefer swap' >> /tank/root/etc/rc.early
 echo 'gmirror configure -b round-robin swap' >> /tank/root/etc/rc.local

Fixing mount point

We use altroot zfs setting in order to install our system without confusing between, the running one and the new one. We need to change the mount point of all the tank/ datasets:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


cd /
zfs umount -a
zfs set mountpoint=legacy tank
zfs set mountpoint=/jails tank/root/jails
zfs set mountpoint=/tmp tank/root/tmp
zfs set mountpoint=/var tank/root/var
zfs set mountpoint=/var/empty tank/root/var/empty
zfs set mountpoint=/var/named tank/root/var/named
zfs set mountpoint=/var/run tank/root/var/run
zfs set mountpoint=/var/tmp tank/root/var/tmp
zfs set mountpoint=/usr tank/root/usr
zfs set mountpoint=/usr/local tank/root/usr/local
zfs set mountpoint=/usr/local/etc tank/root/usr/local/etc
zfs set mountpoint=/usr/home tank/root/usr/home
zfs set mountpoint=/usr/obj tank/root/usr/obj
zfs set mountpoint=/usr/ports tank/root/usr/ports
zfs set mountpoint=/usr/ports/distfiles tank/root/usr/ports/distfiles
zfs set mountpoint=/usr/ports/packages tank/root/usr/ports/packages
zfs set mountpoint=/usr/src tank/root/usr/src

Then set the boot bootfs property to tank/root:

 zpool set bootfs=tank/root tank

The great jump

It’s now time to reboot and see if all component are OK.

Ready… Set… GO!

 reboot

Once the machine is back online, we should connect on it, always on zboot pool.

Then we need to attach the both geli encrypted partition and use the reboot -r command to finish the boot on the real machine.

May be it’s a good idea to make a little script to do that.

 geli attach ada0p4
 Enter passphrase:
 geli attach ada1p4
 Enter passphrase:
 kenv vfs.root.mountfrom="zfs:tank/root"
 vfs.root.mountfrom="zfs:tank/root"
 reboot -r
 connection closed by remote host

reboot -r means “replace the current / by the value of vfs.root.mountfrom”. Read reboot(8) for more informations

In case of failure

As expected, something went wrong, I was unable to connect through ssh to the machine.

Reboot

I had to connect to hetzner console and ask for a reboot of the server. Once the reboot occurs, I can connect again on the zboot.

Attach the geli partition

I attached the two tank zpool partition:

 geli attach ada0p4
 Enter passphrase:
 geli attach ada1p4
 Enter passphrase:

You can attach all parts of the mirror in one command if you choose the same passphrase: geli attach ada0p4 ada1p4

Then import the zpool and mount the datasets:

 zpool import
 pool: tank
     id: 13605327128547891090
  state: ONLINE
  action: The pool can be imported using its name or numeric identifier.
  config:

    tank            ONLINE
      mirror-0      ONLINE
        ada0p4.eli  ONLINE
        ada1p4.eli  ONLINE

 zpool import tank
 zpool list
 NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
 tank   7.19T  1.89G  7.19T        -         -     0%     0%  1.00x  ONLINE  -
 zboot  4.50G  2.19G  2.31G        -         -     4%    48%  1.00x  ONLINE  -

 zfs list
 NAME                       USED  AVAIL  REFER  MOUNTPOINT
 tank                      1.89G  6.96T    88K  legacy
 tank/jails                  88K  6.96T    88K  /jails
 tank/root                   88K  6.96T    88K  legacy
 tank/tmp                    88K  6.96T    88K  /tmp
 tank/usr                  1.88G  6.96T  1.20G  /usr
 tank/usr/home               88K  6.96T    88K  /usr/home
 tank/usr/local             176K  6.96T    88K  /usr/local
 tank/usr/local/etc          88K  6.96T    88K  /usr/local/etc
 tank/usr/obj                88K  6.96T    88K  /usr/obj
 tank/usr/ports             698M  6.96T   698M  /usr/ports
 tank/usr/ports/distfiles    88K  6.96T    88K  /usr/ports/distfiles
 tank/usr/ports/packages     88K  6.96T    88K  /usr/ports/packages
 tank/var                  2.81M  6.96T  2.46M  /var
 tank/var/empty              88K  6.96T    88K  /var/empty
 tank/var/named              88K  6.96T    88K  /var/named
 tank/var/run                88K  6.96T    88K  /var/run
 tank/var/tmp                88K  6.96T    88K  /var/tmp
 zboot                     2.19G  2.17G    88K  none
 zboot/root                2.19G  2.17G  2.19G  /

The problem should be on tank/root/etc which is not mounted. So we need to mount it on a alternative mountpoint.

1
2
3
4
5


 zfs set mountpoint tank/root /mnt
 cd /mnt
 ls -al
 drwxr-xr-x   2 root  wheel   2 Nov 25 16:06 .
 drwxr-xr-x  20 root  wheel  26 Nov 29 08:21 ..

Something’s wrong. Nothing on this root partition. No /etc/, no /dev, no /boot etc…

I found in /tank a complete hierarchy of installation. So copied it on /mnt:

1
2


 cd /tank/root
 cp -rf * /mnt/.

Then I checked /mnt/etc/rc.conf /mnt/boot/loader.conf and /mnt/etc/sysctl.conf.

And… I copied /root/.ssh into /mnt/root/.

Another `reboot -r`

The geli’s partition are already attached, I just need the run the kenv command:

1
2
3


 kenv vfs.root.mountfrom="zfs:tank/root"
 reboot -r
 Connection closed by remote host

Acknowledgment

I would tho thanks Ollivier Robert who wrote the original documentation about this kind of installation for the patience he show about my silly questions. All my friends who run FreeBSD every days. All the *BSD community.

It’s possible to make all the operation by “the hand” but the shell script do some check for us for example. Be curious, read it! ↩︎
Only if you don’t use calife or sudo, which is a bad idea: “There is 2 kind of sysadmin, those who already made an error being root and those who will do soon” ↩︎