NILO architecture

The NLNet NILO project has expanded requirements over the original NILO concept below. It need not follow the implementation suggestions below; that and others will be decisions to be taken by the implementor. Therefore the technical discussion below is for historic purposes and is not binding on the NLNet NILO project, and should be read in that light.

What is NILO?

NILO is the Network Interface Loader project. It will build on the Etherboot and Netboot efforts. It will, in rough order of desirability:

• Be Open Source
• Use Linux network driver code
• Be written all in C except for assembly language bits for low level operations like interfacing to the BIOS
• Use GCC tools throughout
• Run on 386 or better x86 processors
• Autoprobe devices
• Support BOOTP/DHCP for address resolution and TFTP for file loading
• Try to fit with the GRUB effort.
• Support chaining of network boot loaders by providing entry points to network services for subsequent loaders
• Allow separate testing of the booting code and device drivers
• Perhaps use memory protection to keep the boot code safe from the application and also to detect heap-stack collision.
• Boot Linux initially. Legacy OSes like DOS and Windows may be supported later.
• Support other processor architectures later
• Have a small footprint

How will it be done?

NILO will consist of three portions. The main part of the program will control the BOOTP/DHCP address resolution and the TFTP loading process. The second part comprises the protocol handlers and is interfaced through a jump table or interrupt entry points to the first part. The third part is the device driver.

Why three parts? In order to support many devices the device driver is separate, naturally. To support separate testing of the booting code and device drivers we need a split at the protocol level. This will allow a version of NILO to run as a user process on Linux, interfacing to Linux system calls.

NILO architecture

Additionally, by splitting off the protocol code, it will be possible to load secondary loaders that use these services. This will make it possible to move code for enhancements like menus and similar features into secondary loaders, thus keeping the primary loader small. It will always be possible to do basic netbooting with the primary loader though. In the diagram above, the secondary and following loaders will take over the role of the main control program but the protocol handlers and driver remain active. If the memory for the main control can be reclaimed, all the better.

The Linux network device drivers will be compiled from source. A goal is to not edit the driver source at all. So we have to provide stubs for the functions that the drivers call, or include files for macros. In other words a harness for the device driver needs to be built. I have done some initial experiments with the driver code and the main areas of difficulty seem to be:

• Provide replacements for memory allocation routines. I propose that dynamic memory be allocated from the top of memory since this is available when the machine boots.
• Provide routines to receive the incoming packets. Linux device drivers are interrupt driven and so we will need to interface to asynchronous code.
• Provide support routines for timer ticks, etc.

A lot of code can be recycled from the Etherboot project. And of course, the mknbi utilities from Netboot will be used.

There seems to be some a package called OSKit here that could be used for wrapping the Linux device drivers. An alternative site for the old release until the new release appears.

How will the project be run?

I propose to run the project in a bazaar fashion, following the nomenclature that Eric Raymond used. I will keep the latest snapshot at this site. Each snapshot will be dated. (I do not expect to upload more than once per day.) Anybody is free to take the snapshot and hack on it and send me the changes. In addition, this Web page will contain a LOG and TODO section which can be updated by anybody so that people can see what is going on. I expect that in the beginning things will be very raw and then, depending on the level of hacking interest, useful things will emerge.

Distribution

Latest snapshot: nilo-19981207.tar.gz

How does this datestamping of the distribution work? In the Makefile you will see a macro that uses a small Perl script to provide a 8 digit date code depending on the last update time of README. Therefore after edits have been made to the code, update the README and then run make clean archive which will remove the temp files, then produce the tarball.

Status: There are two main build targets in the Makefile at the moment. One builds nilo, which is a user level program (but needs to run with root privilege) that will go through the bootp and tftp steps, printing status. The server IP address and MAC address are set with environment variables. In the bootptab that MAC address must correspond to the IP address of host nilo is run on or the reply will not be seen. The other builds a bootrom for an arbitrary card. Lots of harness routines are missing. I want to get nilo working first so that people like Vlad can put in DHCP and other goodies. For sure I have made some mistakes in the partitioning of the mainline from the protocol handler (the latter must not rely on global variables or routines as it can be called from a secondary boot), but we can correct things as we go.

LOG

19980429

Started on version that runs as user level program on Linux

19980522

Got the first version which does bootp and tftp

19981109

Ok, I've started hacking NILO again. I'm going to try to get one NIC working. I've chosen the 3c509 as the first guinea pig NIC. I've got the probe routine working. I've hooked up the oskit memory manager routines, allocating from the top 1 MB of memory. Things that need attention: loops_per_sec needs to be set to a roughly correct value so that udelay works well (later). Need to hook up interrupt to handler. Need to figure out how to fill the skb before passing to the xmit routine. Then try to transmit one packet.

19981111

I think I have skbuffs and interrupts figured out. I have the concept down in my mind now. Currently stealing code from Linux kernel for IRQ handling. Also the RTC reprogramming is there too.

19981116

Put in more of the protocol handling code. Ironed out some problems with memory allocation. We need a different IDT for protected mode. Etherboot/32 skirted this issue by disabling interrupts in protected mode, so the RTC was only updated when calling real mode routines, like currticks.

TODO

• Write the harness routines for Linux network device drivers. Need to figure out skbs, interrupt handlers and reprogramming the RTC for 100 Hz.
• irq.o is rather large. We could omit the IRQ handlers that are never available to network cards (later).
• Add code for DHCP.

Issues to think about

• The interface between the main control and the protocol handlers is via a jump table that will eventually be put at the top of 640k. This assumes that the secondary and subsequent loaders use the same subroutine calling protocol, which is the case if they are all compiled with the same compiler. I think this ok for now.
• How do we simulate the jiffies variable in Linux, which is incremented regularly? How regularly and do we have to reprogram the RTC?
• Maybe we can save precious memory by mallocing buffers (except for things like the bootp reply) in extended memory. Then the 32k from 0x98000 can be used mostly for code.

Why a new network booter? Aren't Etherboot and Netboot good enough?

Etherboot suffers from these disadvantages:

• Drivers must be written for each Ethernet chipset. While the effort is not large for an experienced driver programmer, it prevents wider spread of Etherboot. By recycling Linux drivers, we track the Linux effort. Any network device that Linux has a driver for we should be able to support. Additionally NILO may be able to assist the effort of writing Linux drivers.
• Etherboot drivers are polling. This makes it more prone to miss packets in a large network.
• The code is burdened with too many features that really belong in a secondary boot.
• Etherboot requires both GCC and the as86/ld86 tools (and bcc, for the 16 bit version). Although there are now patches to GCC to assemble in 16-bit mode.

Netboot suffers from these disadvantages:

• Not all cards come with Packet Drivers. NDIS drivers are the priority with card manufacturers. Also packet drivers require parts of the package to be written in assembler and this makes it harder to maintain. Packet drivers are also of variable quality.
• Netboot doesn't autoprobe in all cases.
• Not all Packet Drivers are interrupt driven for transmission. This introduces some dead time.
• Netboot has a large footprint due to the need to provide a harness for Packet Drivers.
• Netboot requires as86/ld86 tools.
• Compiling packet drivers, if fixing problems is desired, requires a DOS environment and DOS assemblers.

NILO will have some disadvantages too:

• It will be experimental until it matures. It will not initially support all the frills of Etherboot or Netboot.
• It will not support 16 bit x86 processors (not a big loss).
• It will have a larger footprint than Etherboot (but should still fit in 16kB).