Replacing the overpriced (at least without a useless hardware migration) DistantSun, with a bigger hardisk for less money.

Unfortunately, Yaifo, which was used for DistantSun, doesn't seem to be maintained anymore. The solution is to install an OpenBSD image locally in a VM (PXE booting it), and write it to the server's hard disk using the rescue mode.

1. Fresh install of 5.3 in a VM (30G)
2. Use rtsol on em0 for IPv6 configuration
3. Enable NTP
4. No X
5. Custom label
   • a (a), size 60910476 (leaving 2000000, about 1G, sectors for the swap), /
   • a (b)
   • w
   • x
6. Sets
   1. -x*
   2. -game*
7. Reboot
8. Copy /etc/hostname.em0 to /etc/hostname.sis0
   1. halt the vm
9. Copy the image to something else, reboot the VM with both images)
10. On the Rescue Kimsufi:

rescue-bsd# <in>nc -l 10000 | gunzip -f > /dev/ada0</in>

1. Reboot on a FreeBSD install CD (the OpenBSD one lacks nc and a compressing gzip) and get a live CD session

root@:/root# <in>cat /dev/ada0 | gzip -f -1 | nc IP 10000</in>

I tried Yaifo anyway. There is a version for -CURRENT on GitHub. After some tentative updates to port it to 5.3, the amd64 yaifo.fs image gets built (though only as root if the objects in /usr/src and particularly ssh/umac128.c haven't been built yet), but it doesn't boot: the MBR concludes that there is “No O/S” to be booted. This has been tried with yaifo.fs created, then booted, both in Qemu and VirtualBox VMs…

Fortunately, Manuel Giraud found a workaround, having the disklabel created pseudo-manually rather than with a template. This GitHub fork has all it needs!

64 bytes from white-dwarf.narf.ssji.net (91.121.XX.XX): icmp_seq=91 ttl=243 time=293 ms

The OpenBSD (A6) partition first needs to be resized. As it was created automatically during the install, it is located as the default, partition 3 which fdisk can create automatically.

# <in>fdisk -i wd0</in> # re-initialise MBR, with partition number 3 configured as an OpenBSD MBR partition
Do you wish -o write new MBR? [n] <in>y</in>
# <in>fdisk wd0</in>
Disk: wd0       geometry: 60801/255/63 [976773168 Sectors]
Offset: 0       Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
 0: 00      0   0   0 -      0   0   0 [           0:           0 ] unused
 1: 00      0   0   0 -      0   0   0 [           0:           0 ] unused
 2: 00      0   0   0 -      0   0   0 [           0:           0 ] unused
*3: A6      0   1   2 -  60800 254  63 [          64:   976768001 ] OpenBSD
# <in>reboot</in>

After the reboot, the disklabel can then be extended. We take this opportunity to get disklabel to create the fstab entries for us.

# <in>disklabel -F /tmp/fstab.new -E /dev/wd0c</in>
...
> <in>a</in>
partition: [d]
offset: [62910540]
size: [913857525] <in>146800640</in> # 70[GiB]*2 sectors for /home
FS type: [4.2BSD]
mount point: [none] <in>/home</in>
Rounding offset to bsize (32 sectors): 62910560
Rounding size to bsize (32 sectors): 146800608
> <in>a</in>
partition: [e]
offset: [209711168]
size: [767056897] <in>314572800</in> # 150[GiB]*2 sectors for /srv
FS type: [4.2BSD]
mount point: [none] <in>/srv</in>
> <in>a</in>
partition: [f]
offset: [209711168]
size: [452484097]
FS type: [4.2BSD]
mount point: [none] <in>/data</in>
Rounding size to bsize (64 sectors): 452484096
> <in>p</in>
OpenBSD area: 64-976768065; size: 976768001; free: 21
#                size           offset  fstype [fsize bsize  cpg]
  a:         60910464               64  4.2BSD   2048 16384    1 # /
  b:          2000012         60910528    swap                   # none
  c:        976773168                0  unused                   
  d:        146800608         62910560  4.2BSD   2048 16384    1 # /home
  e:        314572800        209711168  4.2BSD   4096 32768    1 # /srv
  f:        452484096        524283968  4.2BSD   4096 32768    1 # /data
> <in>w</in>
> <in>x</in>

The new partitions then need to be formatted.

# <in>newfs wd0d</in>
/dev/rwd0d: 71680.0MB in 146800608 sectors of 512 bytes
355 cylinder groups of 202.47MB, 12958 blocks, 25984 inodes each
super-block backups (for fsck -b #) at:
 32, 414688, 829344, 1244000, 1658656, 2073312, 2487968, 2902624, 3317280,
 3731936, 4146592, 4561248, 4975904, 5390560, 5805216, 6219872, 6634528,
 7049184, 7463840, 7878496, 8293152, 8707808, 9122464, 9537120, 9951776,
 10366432, 10781088, 11195744, 11610400, 12025056, 12439712, 12854368,
 13269024, 13683680, 14098336, 14512992, 14927648, 15342304, 15756960,
 16171616, 16586272, 17000928, 17415584, 17830240, 18244896, 18659552,
 19074208, 19488864, 19903520, 20318176, 20732832, 21147488, 21562144,
 21976800, 22391456, 22806112, 23220768, 23635424, 24050080, 24464736,
 24879392, 25294048, 25708704, 26123360, 26538016, 26952672, 27367328,
 27781984, 28196640, 28611296, 29025952, 29440608, 29855264, 30269920,
 30684576, 31099232, 31513888, 31928544, 32343200, 32757856, 33172512,
 33587168, 34001824, 34416480, 34831136, 35245792, 35660448, 36075104,
 36489760, 36904416, 37319072, 37733728, 38148384, 38563040, 38977696,
 39392352, 39807008, 40221664, 40636320, 41050976, 41465632, 41880288,
 42294944, 42709600, 43124256, 43538912, 43953568, 44368224, 44782880,
 45197536, 45612192, 46026848, 46441504, 46856160, 47270816, 47685472,
 48100128, 48514784, 48929440, 49344096, 49758752, 50173408, 50588064,
 51002720, 51417376, 51832032, 52246688, 52661344, 53076000, 53490656,
 53905312, 54319968, 54734624, 55149280, 55563936, 55978592, 56393248,
 56807904, 57222560, 57637216, 58051872, 58466528, 58881184, 59295840,
 59710496, 60125152, 60539808, 60954464, 61369120, 61783776, 62198432,
 62613088, 63027744, 63442400, 63857056, 64271712, 64686368, 65101024,
 65515680, 65930336, 66344992, 66759648, 67174304, 67588960, 68003616,
 68418272, 68832928, 69247584, 69662240, 70076896, 70491552, 70906208,
 71320864, 71735520, 72150176, 72564832, 72979488, 73394144, 73808800,
 74223456, 74638112, 75052768, 75467424, 75882080, 76296736, 76711392,
 77126048, 77540704, 77955360, 78370016, 78784672, 79199328, 79613984,
 80028640, 80443296, 80857952, 81272608, 81687264, 82101920, 82516576,
 82931232, 83345888, 83760544, 84175200, 84589856, 85004512, 85419168,
 85833824, 86248480, 86663136, 87077792, 87492448, 87907104, 88321760,
 88736416, 89151072, 89565728, 89980384, 90395040, 90809696, 91224352,
 91639008, 92053664, 92468320, 92882976, 93297632, 93712288, 94126944,
 94541600, 94956256, 95370912, 95785568, 96200224, 96614880, 97029536,
 97444192, 97858848, 98273504, 98688160, 99102816, 99517472, 99932128,
 100346784, 100761440, 101176096, 101590752, 102005408, 102420064, 102834720,
 103249376, 103664032, 104078688, 104493344, 104908000, 105322656, 105737312,
 106151968, 106566624, 106981280, 107395936, 107810592, 108225248, 108639904,
 109054560, 109469216, 109883872, 110298528, 110713184, 111127840, 111542496,
 111957152, 112371808, 112786464, 113201120, 113615776, 114030432, 114445088,
 114859744, 115274400, 115689056, 116103712, 116518368, 116933024, 117347680,
 117762336, 118176992, 118591648, 119006304, 119420960, 119835616, 120250272,
 120664928, 121079584, 121494240, 121908896, 122323552, 122738208, 123152864,
 123567520, 123982176, 124396832, 124811488, 125226144, 125640800, 126055456,
 126470112, 126884768, 127299424, 127714080, 128128736, 128543392, 128958048,
 129372704, 129787360, 130202016, 130616672, 131031328, 131445984, 131860640,
 132275296, 132689952, 133104608, 133519264, 133933920, 134348576, 134763232,
 135177888, 135592544, 136007200, 136421856, 136836512, 137251168, 137665824,
 138080480, 138495136, 138909792, 139324448, 139739104, 140153760, 140568416,
 140983072, 141397728, 141812384, 142227040, 142641696, 143056352, 143471008,
 143885664, 144300320, 144714976, 145129632, 145544288, 145958944, 146373600,
 146788256,
# <in>newfs wd0e</in>
/dev/rwd0e: 153600.0MB in 314572800 sectors of 512 bytes
189 cylinder groups of 814.44MB, 26062 blocks, 52224 inodes each
super-block backups (for fsck -b #) at:
 64, 1668032, 3336000, 5003968, 6671936, 8339904, 10007872, 11675840, 13343808,
 15011776, 16679744, 18347712, 20015680, 21683648, 23351616, 25019584,
 26687552, 28355520, 30023488, 31691456, 33359424, 35027392, 36695360,
 38363328, 40031296, 41699264, 43367232, 45035200, 46703168, 48371136,
 50039104, 51707072, 53375040, 55043008, 56710976, 58378944, 60046912,
 61714880, 63382848, 65050816, 66718784, 68386752, 70054720, 71722688,
 73390656, 75058624, 76726592, 78394560, 80062528, 81730496, 83398464,
 85066432, 86734400, 88402368, 90070336, 91738304, 93406272, 95074240,
 96742208, 98410176, 100078144, 101746112, 103414080, 105082048, 106750016,
 108417984, 110085952, 111753920, 113421888, 115089856, 116757824, 118425792,
 120093760, 121761728, 123429696, 125097664, 126765632, 128433600, 130101568,
 131769536, 133437504, 135105472, 136773440, 138441408, 140109376, 141777344,
 143445312, 145113280, 146781248, 148449216, 150117184, 151785152, 153453120,
 155121088, 156789056, 158457024, 160124992, 161792960, 163460928, 165128896,
 166796864, 168464832, 170132800, 171800768, 173468736, 175136704, 176804672,
 178472640, 180140608, 181808576, 183476544, 185144512, 186812480, 188480448,
 190148416, 191816384, 193484352, 195152320, 196820288, 198488256, 200156224,
 201824192, 203492160, 205160128, 206828096, 208496064, 210164032, 211832000,
 213499968, 215167936, 216835904, 218503872, 220171840, 221839808, 223507776,
 225175744, 226843712, 228511680, 230179648, 231847616, 233515584, 235183552,
 236851520, 238519488, 240187456, 241855424, 243523392, 245191360, 246859328,
 248527296, 250195264, 251863232, 253531200, 255199168, 256867136, 258535104,
 260203072, 261871040, 263539008, 265206976, 266874944, 268542912, 270210880,
 271878848, 273546816, 275214784, 276882752, 278550720, 280218688, 281886656,
 283554624, 285222592, 286890560, 288558528, 290226496, 291894464, 293562432,
 295230400, 296898368, 298566336, 300234304, 301902272, 303570240, 305238208,
 306906176, 308574144, 310242112, 311910080, 313578048,
# <in>newfs wd0f</in>
/dev/rwd0f: 220939.5MB in 452484096 sectors of 512 bytes
272 cylinder groups of 814.44MB, 26062 blocks, 52224 inodes each
super-block backups (for fsck -b #) at:
 64, 1668032, 3336000, 5003968, 6671936, 8339904, 10007872, 11675840, 13343808,
 15011776, 16679744, 18347712, 20015680, 21683648, 23351616, 25019584,
 26687552, 28355520, 30023488, 31691456, 33359424, 35027392, 36695360,
 38363328, 40031296, 41699264, 43367232, 45035200, 46703168, 48371136,
 50039104, 51707072, 53375040, 55043008, 56710976, 58378944, 60046912,
 61714880, 63382848, 65050816, 66718784, 68386752, 70054720, 71722688,
 73390656, 75058624, 76726592, 78394560, 80062528, 81730496, 83398464,
 85066432, 86734400, 88402368, 90070336, 91738304, 93406272, 95074240,
 96742208, 98410176, 100078144, 101746112, 103414080, 105082048, 106750016,
 108417984, 110085952, 111753920, 113421888, 115089856, 116757824, 118425792,
 120093760, 121761728, 123429696, 125097664, 126765632, 128433600, 130101568,
 131769536, 133437504, 135105472, 136773440, 138441408, 140109376, 141777344,
 143445312, 145113280, 146781248, 148449216, 150117184, 151785152, 153453120,
 155121088, 156789056, 158457024, 160124992, 161792960, 163460928, 165128896,
 166796864, 168464832, 170132800, 171800768, 173468736, 175136704, 176804672,
 178472640, 180140608, 181808576, 183476544, 185144512, 186812480, 188480448,
 190148416, 191816384, 193484352, 195152320, 196820288, 198488256, 200156224,
 201824192, 203492160, 205160128, 206828096, 208496064, 210164032, 211832000,
 213499968, 215167936, 216835904, 218503872, 220171840, 221839808, 223507776,
 225175744, 226843712, 228511680, 230179648, 231847616, 233515584, 235183552,
 236851520, 238519488, 240187456, 241855424, 243523392, 245191360, 246859328,
 248527296, 250195264, 251863232, 253531200, 255199168, 256867136, 258535104,
 260203072, 261871040, 263539008, 265206976, 266874944, 268542912, 270210880,
 271878848, 273546816, 275214784, 276882752, 278550720, 280218688, 281886656,
 283554624, 285222592, 286890560, 288558528, 290226496, 291894464, 293562432,
 295230400, 296898368, 298566336, 300234304, 301902272, 303570240, 305238208,
 306906176, 308574144, 310242112, 311910080, 313578048, 315246016, 316913984,
 318581952, 320249920, 321917888, 323585856, 325253824, 326921792, 328589760,
 330257728, 331925696, 333593664, 335261632, 336929600, 338597568, 340265536,
 341933504, 343601472, 345269440, 346937408, 348605376, 350273344, 351941312,
 353609280, 355277248, 356945216, 358613184, 360281152, 361949120, 363617088,
 365285056, 366953024, 368620992, 370288960, 371956928, 373624896, 375292864,
 376960832, 378628800, 380296768, 381964736, 383632704, 385300672, 386968640,
 388636608, 390304576, 391972544, 393640512, 395308480, 396976448, 398644416,
 400312384, 401980352, 403648320, 405316288, 406984256, 408652224, 410320192,
 411988160, 413656128, 415324096, 416992064, 418660032, 420328000, 421995968,
 423663936, 425331904, 426999872, 428667840, 430335808, 432003776, 433671744,
 435339712, 437007680, 438675648, 440343616, 442011584, 443679552, 445347520,
 447015488, 448683456, 450351424, 452019392,

The little contents of /home is moved (su should probably have been called out of ~).

# <in>mount /dev/wd0d /mnt</in>
# <in>mv /home/* /mnt/</in>
# <in>umount /mnt</in>

The system fstab can then be updated.

# <in>cat /tmp/fstab</in>
3c6b2ba4c2cabfad.a / ffs rw 1 1
3c6b2ba4c2cabfad.f /data ffs rw 1 2
3c6b2ba4c2cabfad.d /home ffs rw 1 2
3c6b2ba4c2cabfad.e /srv ffs rw 1 2
3c6b2ba4c2cabfad.b none swap sw
# <in>mkdir /srv /data</in>
# <in>mount /dev/wd0d /mnt</in>
# <in>cp /tmp/fstab /etc/fstab</in>

And we hope/check that everything went fine.

# <in>reboot</in>

This looks about right.

# <in>df -h</in>
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/wd0a     28.6G    442M   26.7G     2%    /
/dev/wd0d     68.9G   16.0K   65.4G     0%    /home
/dev/wd0f      214G    4.0K    203G     0%    /data
/dev/wd0e      149G    4.0K    141G     0%    /srv

# <in>vi /etc/hosts</in>
[replace name and name.my.domain with the actual domain name]
# <in>vi /etc/myname</in>
[idem]

The OVH network has an IPv4 DHCP running and IPv6 support but, oddly, no router advertisements. The IPv6 address has to be set manually. Despite what the original FreeBSD's configuration mentions, the prefix on that network seems to be 56 rather than 64 bits long, which is needed to make the gateway reachable.

cat /etc/hostname.sis0

#dhcp
inet 91.121.146.XXX 255.255.255.0 91.121.146.255 # There are some rumours that OVH's DHCP is sometimes unreliable
!/sbin/route add default 91.121.146.254

#rtsol
inet6 2001:41D0:1:XXXX::1 56
!/sbin/route add -inet6 default -gateway 2001:41D0:1:XXff:ff:ff:ff:ff

up

An issue arises with 5.3's dhclient in that whenever it renews its IPv4 lease, it flushes all routes, including IPv6 ones. This is fixed in 5.4/current: getting the sources and compiling just this program on the 5.3 system provides a working drop-in replacement.

Set up a tight but cooperative firewall (preparing for the kiddy-banning script).

/etc/pf.conf

#       $OpenBSD: pf.conf,v 1.52 2013/02/13 23:11:14 halex Exp $
#
# See pf.conf(5) for syntax and examples.
# Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1
# in /etc/sysctl.conf if packets are to be forwarded between interfaces.
 
# increase default state limit from 10'000 states on busy systems
#set limit states 100000
 
TRACEROUTE="{" 33434 33455:33467 "}"
TCP_IN="{" domain http https smtp svn "}"
 
table <whitelist> persist file "/etc/whitelist"
table <kiddies> persist file "/var/tmp/blockers.list"
 
set skip on lo
 
# filter rules and anchor for ftp-proxy(8)
#anchor "ftp-proxy/*"
#pass in quick inet proto tcp to port ftp divert-to 127.0.0.1 port 8021
 
# anchor for relayd(8)
#anchor "relayd/*"
 
block return    # block stateless traffic
pass            # establish keep-state
 
pass in quick on egress proto tcp from <whitelist> to (egress) port ssh 
block drop  in quick on egress from <kiddies> # let them wast their time
 
block return in log on egress
 
# rules for spamd(8)
#table <spamd-white> persist
#table <nospamd> persist file "/etc/mail/nospamd"
#pass in on egress proto tcp from any to any port smtp \
#    rdr-to 127.0.0.1 port spamd
#pass in on egress proto tcp from <nospamd> to any port smtp
#pass in log on egress proto tcp from <spamd-white> to any port smtp
#pass out log on egress proto tcp to any port smtp
 
# ICMP
pass quick inet proto icmp all
pass quick inet6 proto icmp6 all
pass quick proto udp to port $TRACEROUTE
 
# Incoming SSH, DNS & others
pass in on egress proto tcp from any to (egress) port ssh \
        flags S/SA keep state
pass in on egress proto {tcp, udp} from any to (egress) port domain \
        flags S/SA keep state
pass in on egress proto tcp from any to (egress) port $TCP_IN \
        flags S/SA keep state
 
#block in quick from urpf-failed to any # use with care
 
# By default, do not permit remote connections to X11
block in on ! lo0 proto tcp to port 6000:6010

The svn details must be added to the services registry so pfctl is happy.

/etc/services

...
svn             3690/tcp                        # Subversion
...

The final touch is the Denyhost script, installed as explained in its header, to not let kids play around too long.

For package installation, the Ircam mirror seems to be the closest (according to traceroute).

/etc/pkg.conf

installpath = http://mirrors.ircam.fr/pub/OpenBSD/5.3/packages/amd64/

The usual suspects.

# <in>pkg_add bash vim</in>

We want to be able to receive mail for local users remotely.

white-dwarf.narf.ssji.net

And tell sendmail to use /etc/mail/sendmail.cf

sendmail_flags="-L sm-mta -C/etc/mail/sendmail.cf -bd -q30m"

Then sendmail is restarted.

$ <in>sudo /etc/rc.d/sendmail restart</in>
sendmail(ok)
sendmail(ok)

OpenBSD is transitionning to Nginx. Unfortunately, the chrooted version in base doesn't have some necessary options such as DAV support (for ownCloud). So we need to stick to Apache for now (see this section for Distant Sun).

This is an incoherent work in progress as I think up the least intrusive way to have something flexible from the base configurations. This section is also unfinished and partially non-functional.

$ <in>sudo mkdir -p /srv/www/{cache,conf/nginx,logs,sites/DOMAIN.TLD/www,logs}</in>
$ <in>sudo ln -sf /srv/www/sites/DOMAIN.TLD/www htdocs</in>
$ <in>sudo chgrp -R daemon /srv/www</in>
$ <in>sudo chown -R www /srv/www/{cache,tmp,logs}</in>
$ <in>sed  s^/var/www^/srv/www^ /etc/nginx/nginx.conf | sudo tee /srv/www/conf/nginx/nginx.conf</in>
$ <in>sudo cp /etc/nginx/mime.types /srv/www/conf/nginx/mime.types</in>

nginx_flags="-c /srv/www/conf/nginx/nginx.conf"          # for normal use: ""

$ <in>sudo /etc/rc.d/nginx start</in>
nginx(ok)

With Nginx, PHP is interpreted through the FastCGI Process Manager (FPM). It is packaged separately

$ <in>sudo pkg-add php-fpm</in>
$ <in>sudo mkdir -p /srv/www/conf/php-fpm</in>
$ <in>sudo cp /etc/php-fpm.conf /srv/www/conf/php-fpm</in>
$ <in>sudo /etc/rc.d/php_fpm start</in>
php_fpm(ok)

Nginx needs to be told where to go by uncommenting the relevant section of /etc/nginx/nginx.conf.

$ <in>sudo /etc/rc.d/nginx restart</in>
nginx(ok)

Both the system Nginx and PHP-FPM are chrooted by default, so everything works fine.

~shtrom is the way!

config

http {
  ...
  server {
    ...
        # support PHP in UserDirs [0]
        # [0] http://linuxplayer.org/2013/08/nginx-userdir-with-php
        location ~* ^/~(.+?)(/.*\.php)$
        {
                alias /users/$1$2;
                fastcgi_pass  127.0.0.1:9000;
                fastcgi_param SCRIPT_FILENAME $request_filename;
                include fastcgi_params;
        }
        # support UserDirs [1]
        # [1] http://wiki.nginx.org/UserDir
        location ~ ^/~(.+?)(/.*)?$ {
                alias /users/$1$2;
                autoindex on;
        }
    ...
  }
  ...
}

PHP doesn't work

Using the configuration from DistantSun, the daemon only needs be started at boot.

/etc/rc.conf.local

named_flags="-t /srv/named"

All files in the chroot must belong to group named (as the daemon setuid to that user by default). Thi sis particularly the case for etc/named.conf.

$ <in>sudo /etc/rc.d/named start</in>
named(ok)
$ <in>host -t soa narf.ssji.net</in>
narf.ssji.net has SOA record distant-sun.narf.ssji.net. hostmaster.narf.ssji.net. 2013071701 3600 1800 604800 3600

Firewall, changelist and resolver configuration should also be updated accordingly.

1. Déployer NetBSD sur Kimsufi OVH. Zplay.eu. 30 April 2013.
2. Garry Dolley. How to Resize an OpenBSD Root Partition. scie.nti.st. 4 March 2013.
3. Samiuela L.V. Taufa Secret Sauce, OpenBSD, NGinx and PHP 11 March 2013.
4. Installation d'OpenBSD 5.3 sur kimsufi mKS 2g. Kimsufi Forums. 8 May 2013.