Help me blog more in June

This is a post on my blog.

I both have a blog and enjoy blogging. I think a blog is the perfect way to keep notes for myself with the nice possibility that they might help someone else. I frequently look up my own post on how to take screenshots with imagemagick on this blog.

I wrote a blog post everyday for 6 months, from the 26th September 2016 to the 27th of March 2017 , in the end I added 196 new entries to this blog in 182 days. I can't write an epic detailed post everyday, though I might manage a week or so like that if I have a backlog. Most posts were little more than an image and some text, little more than tweets.

But I still wrote, on a few days I even wrote more than one blog post. Under each blog post I included where in the world I was (borrowed from a characters quirk in Cryptonomicom), the weather and the books I was reading at the time.

These posts cover a range, pictures I was proud of , planning for events , travel and cool projects I worked on . Reviewing these has been a fun experience.

All that said, I haven't published a blog post for 276 days and I think I would like some help getting back into the public writing groove.

I tend to be better at delivering projects when I am held accountable by other people. Think of it as peer help rather than peer pressure.

I think it would be really cool if you too wrote some blog posts. Not for me, not for anyone else, but for you. You today and in the future. Write something now to clear your head, write something today that will help you tomorrow. Writing things down helps you remember them, but when you forget you will get to try searching your own log rather than having to scour the depths of the Internet.

I asked this weekend at campgndd for some peer help so that I would actually write blog posts in June. I suggested we each try to write four this month.

These kind people said they would try and write too to keep my going:

It would be cool if you were to join us. I will look for your response on your blog.

FreeBSD on the NanoPi NEOLTS

The NanoPi NEOLTS is a SBC from FriendlyElec that uses the Allwinner H3 SOC. The NanoPi NEOLTS has a nice selection of hardware including 100Mbit Ethernet, 3 USB Ports and a bunch of exposed GPIO.

FreeBSD on the NanoPi uses GENERICSD image. This image requires a bootloader to be added before it will work. We can prepare a single image to be copied to many SD cards by using a memory disk as an intermediate step.

We need to:

  • Get the latest GENERICSD card image snapshot
  • Install the correct boot loader pkg
  • Create a memory disk
  • Copy the GENERICSD image to memory disk
  • Copy the bootloader to the memory disk
  • Mount the root partition of the sd card image
  • Copy the programs and files we need for the tutorial to the sd card

The latest image is as I write is 13 CURRENT from 20190829:

$ fetch ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/arm/armv7/ISO-IMAGES/13.0/FreeBSD-13.0-CURRENT-arm-armv7-GENERICSD-20190829-r351591.img.xz

We have to decompress the image before we can use it

$ xz -d FreeBSD-13.0-CURRENT-arm-armv7-GENERICSD-20190829-r351591.img.xz

Each u-boot bootloader platform has its own package, currently there are 46 different bootloaders in the FreeBSD ports system. We want the u-boot for the nanopi_neo (our target).

$ pkg search nanopi     
u-boot-nanopi-neo2-2019.07     Cross-build das u-boot for model nanopi-neo2
u-boot-nanopi_a64-2019.07      Cross-build das u-boot for model nanopi_a64
u-boot-nanopi_m1plus-2019.07   Cross-build das u-boot for model nanopi_m1plus
u-boot-nanopi_neo-2019.07      Cross-build das u-boot for model nanopi_neo
u-boot-nanopi_neo_air-2019.07  Cross-build das u-boot for model nanopi_neo_air

# pkg install u-boot-nanopi_neo-2019.07

The u-boot-nanopi_neo package contains the binary bootloader we need in u-boot-sunxi-with-spl.bin

$ pkg info -l u-boot-nanopi_neo-2019.07

u-boot-nanopi_neo-2019.07:
    /usr/local/share/licenses/u-boot-nanopi_neo-2019.07/GPLv2
    /usr/local/share/licenses/u-boot-nanopi_neo-2019.07/LICENSE
    /usr/local/share/licenses/u-boot-nanopi_neo-2019.07/catalog.mk
    /usr/local/share/u-boot/u-boot-nanopi_neo/README
    /usr/local/share/u-boot/u-boot-nanopi_neo/boot.scr
    /usr/local/share/u-boot/u-boot-nanopi_neo/metadata
    /usr/local/share/u-boot/u-boot-nanopi_neo/u-boot-sunxi-with-spl.bin

With the GENERICSD image and the bootloader we need to create the memory disk image we will use for staging. First we need to create a large enough backing file.

$ truncate -s 8G nanopi.img
# mdconfig -f nanopi.img
md0

Now we can dd the GENERICSD image to the memory disk

# dd if=FreeBSD-13.0-CURRENT-arm-armv7-GENERICSD-20190829-r351591.img of=/dev/md0 bs=1m

We need to dd the bootloader to the start of the SD card, i.e. the entire device and not a partition.

# dd if=/usr/local/share/u-boot/u-boot-nanopi_neo/u-boot-sunxi-with-spl.bin of=/dev/da0 bs=1k seek=8 conv=sync

With the memory disk attached we can interact with the image file as if it were a real USB drive or SD card.

$ gpart show md0
=>      63  16777153  md0  MBR  (8.0G)
        63      2016       - free -  (1.0M)
      2079    102312    1  fat32lba  [active]  (50M)
    104391   6187041    2  freebsd  (3.0G)
   6291432  10485784       - free -  (5.0G)

We can mount the root partition of the SD card and modify or add any files we wish:

# mount /dev/md0sa mnt

When we are done changing things we have to disconnect the memory disk:

# sudo mdconfig -d -u md0

Finally we can copy the memory disk to a real sd card using dd :

# sudo dd if=nanopi.img of=/dev/da0 bs=1m

My FreeBSD Development Setup

I do my FreeBSD development using git , tmux , vim and cscope .

I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd

On my fork I have the freebsd/freebsd repo set as an upstream

$ git remote -v
origin  git@github.com:adventureloop/freebsd.git (fetch)
origin  git@github.com:adventureloop/freebsd.git (push)
upstream        https://github.com/freebsd/freebsd.git (fetch)
upstream        https://github.com/freebsd/freebsd.git (push)

See this article for information on setting this up https://help.github.com/en/articles/configuring-a-remote-for-a-fork

I do all work on branches using worktrees, keeping the master branch clean.

Periodically I sync the master branch with the FreeBSD upstream:

$ cd ~/code/freebsd-dev/freebsd-git
$ git checkout master
$ git fetch upstream
$ git merge upstream/master
$ git push

I have a development setup based on Ian Lapore's arm set up documented on the FreeBSD wiki https://wiki.freebsd.org/FreeBSD/arm/crossbuild

I have a freebsd-dev directory in my code directory. It their I keep a copy of FreeBSD in freebsd-git , and obj directory for build output and a projects directory for in progress code.

$ tree -L2
.
├── diffs
│   ├── D15222.diff
│   └── old
├── dstdir
│   ├── boot
│   ├── METALOG
│   └── usr
├── freebsd-git
│   ├── bin
│   ├── sbin
...
│   └── usr.sbin
├── obj
│   └── usr
├── projects
│   ├── atomicstats
│   ├── axp288
│   ├── bugsquash
│   ├── byebyejumbograms
...

I use git worktrees for ongoing projects. git worktrees allow you to have a shallow file system copy on a git branch in a directory.

When starting a new project I do something like:

$ cd ~/code/freebsd-dev/freebsd-git
$ git worktree add thj/newdevelopment ../projects/newdevelopment master
$ cd ../projects/newdevelopment

Once the worktree is set up I launch a tmux session in the projects directory. Each random idea or itch I have, if there is enough there, ends up with a project worktree and a tmux session.

tmux allows me to have many windows in a session, I have a serious tmux problem. Right now I have 11 sessions with 42 windows across them. This is a good indicator of my focus level.

I do FreeBSD development with cscope and vim . With tmux splits I normally have an open file and I use other cscope instances in tmux windows to search for things I need in the tree.

I do testing in a bhyve vm and leave the serial port in a tmux window somewhere. I follow the setup in the FreeBSD handbook and back each vm with a zfs dataset.

I do FreeBSD kernel builds using a command like:

env MAKEOBJDIRPREFIX=/home/tom/code/freebsd-dev/obj make -j 44 buildkernel \
        -DKERNFAST installkernel \
    -DNO_ROOT DESTDIR=/home/tom/code/freebsd-dev/dstdir

I then ship kernels to the test vm with scp. jhb@ has a nicer method using the bhyve-loader , but I am yet to try it.

When changes are maturing I create reviews for them using arcanist, manu@ has a good article on doing this

EuroBSDCon Bucharest Romania

The Wikitravel page for Bucharest has some scary warnings about taxis. I didn't heaer any horror stories from conference goers, but there was a large variation in prices for the same journey.

He held a two day DevSummit before the conference proper. A DevSummit is a chance to talk through technical issues and hash things out face to face. We did some planning for FreeBSD 13 with the idea of setting GGoals for the release.

We tried to match a bit of a hackathon with the DevSummit, but the tutorial schedules meant we couldn't focus the time very well and it was broken up.

EuroBSDCon

Day One :

  • Keynote1: Lightweight virtualization with LightVM and Unikraft
  • Hacking together a FreeBSD presentation streaming box – For as little as possible
    • That was me, I thought it was quite good :D
  • The Evolution of FreeBSD Governance
  • Using Boot Environments at Scale
  • The End of DNS as we know it
  • Keynote2: Some computing and networking historical perspectives
    • Ron's keynote was unreal and it is a massive shame that this sessions wasn't recorded. Ron has a ton of experience with working with network systems since 1976, he shared some stories and anecdotes. The one closest to my heart was pirating away an IMP front pannel and saving it from the scrappers. If you get a chance to see Ron speak you should jump up and down at it.

Day Two :

  • Taking NetBSD kernel bug roast to the next level : Kernel Sanitizers
  • Livepatching FreeBSD kernel
    • This was an interesting study into how many different platforms do live patching. The FreeBSD way to do live patching could be simplified to 'use dtrace fbt probes'. Which is super reductive of all of the work invovled, but it shows the power of the system we have with dtrace.
  • Profiling Packet Processing: Performance & Peculiarities
  • Debugging lessons learned as a newbie fixing NetBSD
    • Maya is a terrifying person. Somehow she manages to hack productivly across the entire range of the stack and across many different architectures. There were many debuggin gems in here, I hope she continues to present on this the information was great.
  • FreeBSD/VPC: a new kernel subsystem for cloud workloads
  • FreeBSD on IBM PowerNV
    • An recount of the porting work Semihalf did to POWER8. Interesting, I hope it is also sumbitted to AsiaBSDCon. There need to be more written account of bringing up on different architectures.

Day Two concluded with announcing the location of the next EuroBSDCon, Lillehammer Norway.

EMF Camp 2018

EMF Camp is a giant hacker camp that occurs in the deep South of England. It managed to attract nearly 2500 people into a field for four days at the end August.

EMF Camp 2018 was the first time I have volunteered to help with the organisation. I volunteered to help out the content team earlier in the year, it wasn't until the week before that we realised lightning talks needed more organisation. Foolishly I stepped up and got a weird split experience between attending the camp and running a tiny slice of it.

It wasn't sooooo awful, I'll probably do it again.

I attended EMF Camp 2014, since then they have really managed to integrate well with the village system used at other camps. The map shows all the spontaneous events that people put together during the camp, the adage 'it is what you make it' really comes out at these events with many participants helping to make it hole.

In our own way the Scottish Consulate contributed too, with our bureaucratic role playing game going beyond the pale and expanding into operation of a phone network (cups and string) and a Hard border from the rest of the camp.