FreeBSD Network Status Week 46 2024

The meta discussion is moving to the end of the report, meta meta discussion will hopefully be rare.

Goings on

Fall 2024 FreeBSD Summit

The Fall 2024 FreeBSD Summit was last week. There were some great presentations which are recorded on the FreeBSD YouTube Channel . I don't know if or when the sessions will be broken up, you can scroll through the recorded live streams to find talks of interest.

BSD Devroom at FOSDEM 2024 

Welcome to the BSD Devroom Call For Participation. The BSD Devroom aims
to provide a dedicated space for presentations covering BSD operating
system family.

Key dates

Proposals can be submitted by October the 30th, 2024
Submission deadline : 1st December 2024 Brussels time
Announcement of selected talks : 15th of December 2024
Conference dates : 1 & 2 February 2025
BSD devroom date : Saturday February 1st, 2025 afternoon (second half)

CFP is here

There are only two weeks left for you to submit something.

FreeBSD 14.2 Code Slush in Effect

From Colin: 

This is a reminder that we have one week remaining in the 14.2-RELEASE
code slush and will start BETA builds on November 1st.  The schedule
for the release cycle is:

 ports quarterly branch:  October 1, 2024
 reminder email:          October 7, 2024
 stable/14 slush:         October 18, 2024
 doc/ tree slush:         October 25, 2024
 releng/14.2 branch:      November 1, 2024
 BETA1 build starts:      November 1, 2024
 BETA2 build starts:      November 8, 2024
 doc/ tree tag:           November 8, 2024
    <<< WE ARE HERE >>>
 BETA3 build starts:      November 15, 2024
 ports package builds:    TBD (Between November 8 and 22)
 RC1 build starts:        November 22, 2024
 RELEASE build starts:    November 29, 2024
 RELEASE announcement:    December 3, 2024
 14.1 EoL:                March 31, 2025
 14.2 EoL:                September 30, 2025
 14.* EoL:                November 30, 2028

https://wiki.freebsd.org/Releng/14.2ISSUES

And an update today: 

I'm pushing the BETA3 build back by 24 hours so I can re-run this week's
current+stable snapshots; the new container-building code broke stuff but
I *think* I've got it fixed now so I want to try again in case I can get
that into 14.2-BETA3.

This also allows a bit more time for people to look at a vfs issue.  I
already have avg, mckusick, and pho looking at it, but if you know
anything about VFS it would be great to have more eyes; see the thread
"vn_alloc_cyclecount is always zero" and commit ab05a1cf321a.

Note: Colin is only sending these announcements to a private list. I asked him last week if I can share them and he said it wasn't a problem. I'll pester him to share these updates to a public list as well.

Struct ifnet is now hidden 

Hi everyone,

Many thanks to everyone involved, either fixing ports, reviewing
changes, or assisting with the API, struct ifnet is now hidden from
drivers and userspace.  This means drivers now use the IfAPI, currently
largely consisting of just getters and setters in place of direct
access, with tools/ifnet/convert_ifapi.sh being a good starting point
for adapting drivers.

In struct ifnet's place is if_t, an opaque pointer.  "Blessed" parts of
the kernel, effectively anything sys/net*, modulo sys/netlink, can
directly access it, as those directories comprise the network stack, so
need direct access, as they manage the interfaces.  Manpage updates
will be coming "soon", I'm hoping in the next few weeks, and will
likely need lots of review and editing.

Again, thanks for everyone's help.  I hope the road forward with this
is not too bumpy.

- Justin

You can find this message on the freebsd-arch mailing list

Removal of devel/kyua 

Hi,

Kyua has been part of base since 13.0, today it means all supported versions.

The tests in /usr/tests usually have parity with Kyua in base, i.e. even if we
consider older unsupported systems then new features from the latest port
offer limited benefits. Anyway, these cases are not supported.

So, in order to avoid double work and user confusion, the devel/kyua port is
being considered for removal.

The motivation of this notification is to collect comments and suggestions in
case if the removal is not a good idea for some reasons.


Best regards,
igoro

igoro@ has been doing a lot of work on testing, particularly in pf. This message went to to three mailing lists ( see it on freebsd-hackers )

Transport

rscheff@ is continuing his work on improving SACK behaviour. For the last while we have been discussing TSO behaviour and loss. Other changes are tidy ups to have we do calculations and other macros.

Network Stack

As mentioned above we have a nice series of commits tidying up the use of ifnet to make it private:

Improvements to sendfile in the Linux compat code.

More tidying in the PCB code:

Netdev

It is not often you get a new media type, this comes with a nice explanation: 

net: if_media for 1000Base-BX BiDi
1000Base-BX uses two wavelengths, commonly 1310nm, 1490nm, 1550nm, or
1590nm, in a Coarse Wavelength Division Multiplexing (CWDM) arrangement
so that a single fiber strand may carry both upstream and downstream.
It is sometimes referred to as BiDi for bi-directional usage of one
fiber.

Optics must be paired such that the RX and TX wavelengths cross over,
with one side often called U(pstream) and the other D(ownstream).

This technology is useful for increasing link density or working around
construction issues, and is also frequently used as a last mile delivery
technology for FTTx.

And a well known trick for the X520 for lab measurements (no, I don't know this trick).

MAN PAGES!

The only way you get a reputation for good documentation is by continuous consistent improvement.

Some nice changes in WiFi this week, iwlwifi device require firmware which has been in tree until now. It is going to move out of our source tree to somewhere else and be made available by a 'fw update' package of some sort.

Firewalls

Fixes and tidying in pf and more probe points.

If you have used FreeBSD as a developer of a sysadmin you will have heard about how amazing dtrace is. A major quality of life improvement for anyone having to use dtrace for analysis on a live system are discrete probe points. They let you get logical events and make it so you don't have to read kernel code to guess at what is going on (ala fdt probes).

ipfilter is moving forward!

Other stuff

This was an interesting one: 

Use the correct idle routine on recent AMD EPYC servers

We have been incorrectly choosing the "hlt" idle method on modern AMD
EPYC servers for C1 idle. This is because AMD also uses the Functional
Fixed Hardware interface. Due to not parsing the table properly for
AMD, and due to a weird quirk where the mwait latency for C1 is
mis-interpreted as the latency for hlt, we wind up choosing hlt for
c1, which has a far higher wake up latency (similar to IO) of roughly
400us on my test system (AMD 7502P).

This patch fixes this by:

- Looking for AMD in addition to Intel in the FFH
 (Note the vendor id of "2" for AMD is not publically documented, but
 AMD has confirmed they are using "2" and has promised to document it.)

- Using mwait on AMD when specified in the table, and when CPUid says
 its supported

- Fixing a weird issue where we copy the contents of cx_ptr for C1 and
 when moving to C2, we do not reinitialize cx_ptr. This leads to
 mwait being selected, and ignoring the specified i/o halt method
 unless we clear mwait before looking at the table for C2.

I'm not sure what the actual impact of this change is, I would guess a reduction in power consumption on most servers (because most servers are at least 40% idle). It might also improve responsiveness, but probably not at a human observable level.

Please Send Feedback

No stream this week, there was a big gap in last nights sleep (hazards of having a young child) and I'm not sure I would make a lot of sense (if I ever do). I have moved the writing time to be around 9am Eastern Time and will aim to stream in this slot next week.

I would love to know if this summary was any help, if it was, or if you think I should cover other thing please let me know (thj@freebsd.org).

If you find a typo or have a correct let me know and I'll thank you at the end here.

You can see all prior posts here. ( rss )

My work on FreeBSD is supported by the FreeBSD Foundation , you can contribute to improving FreeBSD with code, documentation or financially by donating to the FreeBSD Foundation .

Wireshark 'in' operator

The thing I'm working on has gotten to the point of looking at pcaps. I love a good ol' hexdump (shout out to FreeBSD's in kernel hexdump and honourable mention to printf(9) offering bitfield and byte dump function.

More than a hexdump I love a packet capture. When asked I'll say -xx is my favourite tcpdump flag narrowly beating out -ttttt .

Wireshark expressions can get a bit cumbersome if you are trying to look at many things. Luckily you can use the membership ( in ) operator as a filter . As an example this will give you ieee80211 probe requests, probe responses, assoc requests and responses only requiring you find the types somewhere (like a header file /usr/include/net80211/ieee80211.h ): 

#define IEEE80211_FC0_SUBTYPE_MASK              0xf0
#define IEEE80211_FC0_SUBTYPE_SHIFT             4
/* 802.11-2020 Table 9-1-Valid type and subtype combinations */
/* For type 00 Management (IEEE80211_FC0_TYPE_MGT) */
#define IEEE80211_FC0_SUBTYPE_ASSOC_REQ         0x00    /* Association Request */
#define IEEE80211_FC0_SUBTYPE_ASSOC_RESP        0x10    /* Association Response */
#define IEEE80211_FC0_SUBTYPE_REASSOC_REQ       0x20    /* Reassociation Request */
#define IEEE80211_FC0_SUBTYPE_REASSOC_RESP      0x30    /* Reassociation Response */
#define IEEE80211_FC0_SUBTYPE_PROBE_REQ         0x40    /* Probe Request */
#define IEEE80211_FC0_SUBTYPE_PROBE_RESP        0x50    /* Probe Response */
#define IEEE80211_FC0_SUBTYPE_TIMING_ADV        0x60    /* Timing Advertisement */
/* 0111 Reserved                                0x70 */
#define IEEE80211_FC0_SUBTYPE_BEACON            0x80    /* Beacon */
#define IEEE80211_FC0_SUBTYPE_ATIM              0x90    /* ATIM */
#define IEEE80211_FC0_SUBTYPE_DISASSOC          0xa0    /* Disassociation */
#define IEEE80211_FC0_SUBTYPE_AUTH              0xb0    /* Authentication */
#define IEEE80211_FC0_SUBTYPE_DEAUTH            0xc0    /* Deauthentication */
#define IEEE80211_FC0_SUBTYPE_ACTION            0xd0    /* Action */
#define IEEE80211_FC0_SUBTYPE_ACTION_NOACK      0xe0    /* Action No Ack */

The wireshark values are shifted for a 4 bit field already, that gives us a filter like this: 

wlan.fc.type_subtype in {0x0000, 0x0001, 0x0004, 0x0005}

If you are trying to figure out a filter in wireshark you can always right click and pick Apply as Filter->Selected .

My work on FreeBSD is supported by the FreeBSD Foundation , you can contribute to improving FreeBSD with code, documentation or financially by donating to the Foundation .

FreeBSD Network Status Week 45 2024

No stream this week, doing the streams in the morning means that if there is any pressure from elsewhere in my life that makes me later they become quite stressful. They also take a big exertion, running a stream and writing at the same time is hard work, even when no one speaks to me.

I am torn about scheduling, things put off tend to get squeezed out, this was a motivator for doing them at the start of the day. As my mornings are proving busy I'm thinking about moving the writing and stream out to 1400 UK time. This has the benefit of being 9am for the East Coast US for most of the year.

Goings on

This is a lighter week for network stack goings on, I'm still a few reports away from being able to pick out a trend. From the reports I have written this week has the fewest taken commits, but it is within 10% of other light weeks. It is 50% the volume of the busiest weeks.

These numbers are very noisy, I have a habit of taking interesting stuff along with the networking stuff.

Fall 2024 FreeBSD Summit

The summit started yesterday and there are live streams on YouTube, Day One is here , there isn't a link for Day Two right now you can find it on the FreeBSDProject YouTube channel .

update: Day Two

I caught Ian Evans Talk yesterday while I was making supper, it was great to watch and see how they have built out rural internet infrastructure. It was on a project like this that I first got paid to do cool things with the Internet.

BSD Devroom at FOSDEM 2024 

Welcome to the BSD Devroom Call For Participation. The BSD Devroom aims
to provide a dedicated space for presentations covering BSD operating
system family.

Key dates

Proposals can be submitted by October the 30th, 2024
Submission deadline : 1st December 2024 Brussels time
Announcement of selected talks : 15th of December 2024
Conference dates : 1 & 2 February 2025
BSD devroom date : Saturday February 1st, 2025 afternoon (second half)

CFP info is here

Transport

TCP is very complex, this change fixes setting the congestion window when a SYN is lost. I don't remember talking about what the cwnd was being set to before this change. cwnd goes to 1 if a SYN has to be retransmitted, packet loss during establishment can make your connection very painful.

More tidy ups in the SCTP code, I know that is the only comment I manage most weeks. I think the continued work on SCTP is incredible considering how difficult it is to point at a use case.

Netdev

Improvements to the gve(4) driver which is used in GCE. The live lock commit comes with a wonderful diagram: 

gve: Fix TX livelock

Before this change the transmit taskqueue would enqueue itself when it
cannot find space on the NIC ring with the hope that eventually space
would be made. This results in the following livelock that only occurs
after passing ~200Gbps of TCP traffic for many hours:

                            100% CPU
┌───────────┐wait on  ┌──────────┐         ┌───────────┐
│user thread│  cpu    │gve xmit  │wait on  │gve cleanup│
│with mbuf  ├────────►│taskqueue ├────────►│taskqueue  │
│uma lock   │         │          │ NIC ring│           │
└───────────┘         └──────────┘  space  └─────┬─────┘
     ▲                                           │
     │      wait on mbuf uma lock                │
     └───────────────────────────────────────────┘

kbowling@ has made igb(4) work better with misreporting SFP receivers.

Fix build of ice with options RSS.

Bluetooth fixes! I'm not sure if I should be surprised, but I am very surprised.

Firewalls

Tests are steadily improving.

Fixes in the build for pf, including making it work when IPv4 is disabled (we would have IPv6 in that case, no need for pf with no ip stack).

Plumbing to allow ipfilter options to be settable during put from rc.conf.

User Tooling

Ping tests improved, this is good because ping is a core tool which needs attention. There was a lot of churn trying to get the recent 'large ping' change in, more tests more better.

Improvements to tftpd tests, tftpd is an old program that was written to be launched by inetd and this complicated testing. Should we have a tftpd is another question.

Other stuff

Some excellent other commits rolling by, first a change to the vmm interface which enables creation and destruction via a file interface rather than the old sysctl interface. As the commit explains this is a step towards narrowing the privileges required to manage virtual machines.

A previous fix to the bell frequency (yes the pc speaker) was accidentally reverted and has been reinstated. The original commit message is a great read:

Please Send Feedback

I would love to know if this summary was any help, if it was, or if you think I should cover other thing please let me know (thj@freebsd.org).

If you find a typo or have a correct let me know and I'll thank you at the end here.

You can see all prior posts here. ( rss )

My work on FreeBSD is supported by the FreeBSD Foundation , you can contribute to improving FreeBSD with code, documentation or financially by donating to the FreeBSD Foundation .

FreeBSD Network Status Week 44 2024

The report was streamed again this week, I have tried to advertise streaming in all the expected places (discord, irc, BSDNow Telegram). If I missed somewhere let me know.

Goings on

We seem to be back to a much more normal events calendar post-2020. There is a return of a BSD Devroom at FOSDEM and I am hearing about people attending many more events.

It looks like the bugzilla tidy up is calming down, I have it on my todo list to find interesting bugs, but I have gotten bogged down doing other things the last few weeks.

Fall 2024 FreeBSD Summit

This is happening next week on the 7-8th of November. If you still want to go and haven't made any plans time is running out. 

@NetApp San Jose

Program includes:
    - Pawel Dawidek, Fudo Security on "FreeBSD Security Improvements"
    - Dorr Clark, NetScaler on “Using FreeBSD in Products"
    - George Neville-Neil on "OSDB: Turning the Tables on Kernel Data"
    - Dr. Marshall Kirk McKusick on “History of the BSD Daemon”
    - And more!

The summit is open to the public, with a registration fee of US $150.

BSD Devroom at FOSDEM 2024

After a year off there will be a BSD Devroom at FOSDEM 2024, the Devroom needs your talks to be successful. The Devroom talks are shorter than conferences than EuroBSDCon so this is a great opportunity to share work in progress with a diverse BSD crowd. 

Welcome to the BSD Devroom Call For Participation. The BSD Devroom aims
to provide a dedicated space for presentations covering BSD operating
system family.

Key dates

Proposals can be submitted by October the 30th, 2024
Submission deadline : 1st December 2024 Brussels time
Announcement of selected talks : 15th of December 2024
Conference dates : 1 & 2 February 2025
BSD devroom date : Saturday February 1st, 2025 afternoon (second half)

CFP is here

Transport

I think changes in the transport stack go through phases. We have a regular call every two weeks to discuss things, but it shows mostly as tidy ups and commits landing for extremely long running projects. This SACK change is part of an exceedingly long project by rscheff@ to fix SACK functionality and performance.

Network Stack

Improvements to handling of external data on mbufs. Adding external pages to mbufs is an important way to improve performance when interacting with other subsystems. Everything mbufs is a bit of a minefield.

kTLS is supported in GNUtls.

Netdev

ice(4) has been updated to 1.43.2-k through some other updates. ice(4) is a good NIC for high performance interfaces.

if_ffec got some tidy ups.

It is great to see multiple families of Intel network drivers getting consistent attention. Tidy ups in man pages are really important for debugging.

bz@ did a series of commits to update wireless drivers using the LinuxKPI for v6.11 which required disabling the drivers from the build. Feedback on the freebsd-src suggested that this can be skipped over in git-bisect and might not be needed.

Firewalls

User Tooling

More updates following on the recent changes to ndp. More and more stuff is getting netlink support.

Other Interesting Change

I might have gotten carried away taking other commits, there is a lot of cool stuff going on.

br@ landed support for bhyve on riscv, this was developed over the summer (incredibly quickly) on the SPICE simulator and QEMU. Hardware with support for the riscv 'h' (hypervisor) hasn't shipped yet, but there might be a small run of ASICs (200 boards) later this year.

Amazing to have support for a hypervisor that doesn't really exist yet.

emaste@ fixed an IPv4 address in resolver to be valid (8.8.4.4 vs 4.4.4.4) and it started a conversation about example addresses in our documentation and including IPv6. It is a good to acknowledge that we should use documentation addresses as specified by IANA, it does help if examples are directly usable.

nuageinit got support for OpenStack network configuration. As more cloud init support lands FreeBSD cloud images are going to get easier and easier to build.

virtio p9fs implements 9pfs (plan 9 file system) for bhyve making it significantly better to provide file systems to guest in bhyve virtual machines.

Please Send Feedback

I would love to know if this summary was any help, if it was, or if you think I should cover other thing please let me know (thj@freebsd.org).

If you find a typo or have a correct let me know and I'll thank you at the end here.

You can see all prior posts here. ( rss )

My work on FreeBSD is supported by the FreeBSD Foundation , you can contribute to improving FreeBSD with code, documentation or financially by donating to the FreeBSD Foundation .

FreeBSD Network Status Week 43 2024

No stream this week, it would be an odd audio mixture of my keyboard, me coughing and blowing my nose, wondering what the cats are up to (we have a guest doubling the number of cats) and talking about compiler flags. I'll be back on next week if I can match the audio standard. Roughly 1000 UK time on Friday the 1st of November (whichever timezone that is next week).

Goings on

Quiet week on the Network side of things, I've not done this long enough to know if there is a pattern here or not. The big change this week was the import of LLVM 19, newer compilers are stricter which tends to break things. Makes us better through fixing bugs in weird code.

The Fall 2024 FreeBSD Summit is in just over two weeks, John made a call for people to register so the correct amount of food could be bought.

Fall 2024 FreeBSD Summit 

@NetApp San Jose

Program includes:
    - Pawel Dawidek, Fudo Security on "FreeBSD Security Improvements"
    - Dorr Clark, NetScaler on “Using FreeBSD in Products"
    - George Neville-Neil on "OSDB: Turning the Tables on Kernel Data"
    - Dr. Marshall Kirk McKusick on “History of the BSD Daemon”
    - And more!

The summit is open to the public, with a registration fee of US $150.

Performance issues with vnet jails + epair + bridge

Mailing list thread

I mentioned this discussion last week and got some feedback on performance of zones on Illumnos, sounds like there is some ground we can tread to catch up.

FreeBSD 14.2 Code Slush in Effet

The 14.2-RELEASE code slush continues. The schedule looks like a start of December release, but it depends a lot on testing. You can help by testing 14/stable for the next week and branches after. I'm not sure if the schdule is posted publicly so I won't include it here until I know if I should.

Known issues are on this wiki page

Stab week

This was the October Stab Week( announcement ). It doesn't look like there was fallout and the test period was ended on Wednesday ( email ).

Transport

Tidy ups in SCTP and TCP. The FreeBSD SCTP stack is used in a lot of places, on small platforms like Chrome for WebRTC data channel.

Network stack

Locking and VNET improvements in the network stack. Locking around VNETs is quite complex, for a while you could do a lot of damage by running a for loop creating VNETs, but we are improving all the time and crashes like that are less common. That means the remaining issues are harder to find, but overall better.

A small change to export route statistics via a sysctl, a lot of netstat involves looking at live kernel memory and it has all the problems you can imagine.

This was integrated into netstat a little later on:

Netdev

Small improvement in e1000 which should be a no-op.

On server restart with kTLS a long start up pause could be observed under high load. The process for allocating TLS memory was running single threaded and blocking the process start up.

Firewalls

Test with style.

Other stuff

LLVM 19 was imported just after stab week ended. Compiler imports break a ton of things, the second commit here is a good example of the compiler getting stricter and call us on our nonesense.

zlei@ did a run tidying up surplus semicolons (here is one, but there were many):

Updates via vendor imports for unbound (to 1.22.0), libarchive and googletest (to 1.15.2):

Please Send Feedback

I would love to know if this summary was any help, if it was, or if you think I should cover other thing please let me know (thj@freebsd.org).

If you find a typo or have a correct let me know and I'll thank you at the end here.

You can see all prior posts here. ( rss )

My work on FreeBSD is supported by the FreeBSD Foundation , you can contribute to improving FreeBSD with code, documentation or financially by donating to the FreeBSD Foundation .

next