Add new device type and associated devices
Document writer <olivier.nicolas@insa-lyon.fr>
v1.7, 2022-11-24
Architecture Presentation
Where to add information for the new-device-type.
In this document, the color code used is the following: |
Existing Directories Existing Files New device-type directories New device-type files Generated Directories Generated Files Links
Every file not in red under new-device-type directories can be copied and adapted (if necessary) from existing device directories.
We will consider that a new device (new-device) of type new-device-type is connected to the platform. It will have to be installed first, following this process.
Its network configuration being:
-
IP address: 10.0.1.1
-
MAC address: aa:bb:cc:dd:ee:ff
Creation Steps
Here is presented a simplified version of the minimal setup.
reflash ├── devices │ └── new-device-type │ ├── aa-bb-cc-dd-ee-ff | └── scripts ├── images │ └── new-device-type │ └── device.img ├── initramfs │ ├── devices │ │ └── new-device │ └── utils │ └── minimal_dirs │ └── new-device-type └── scripts └── devices
Step 1 - DNS configuration
Update dnsmasq.conf
On server:
-
Add the MAC/IP to /etc/dnsmasq.d/dnsmasq.conf (dhcp-host=aa:bb:cc:dd:ee:ff,new-device,10.0.1.1)
-
Restart dnsmasq service (For example: sudo systemctl restart dnsmasq.service)
Step 2 - New device image(s)
-
Create the new-device-type image(s) and place it/them in the proper directory.
TO BE EDITED to link to dev-Platform-imgCreation-build/* image-creation |
-
Create image(s) with one device of this new-device-type, according to the dev-Platform-imgCreation-build specifications and flash a device of said type with such an image.
-
Add image(s) to
reflash/images └── new-device-type ├── image1.img ├── image2.img └── ...
Step 3 - Configuration
Create its configuration file:
reflash/initramfs/utils └── minimal_dirs └── new-device-type └── initramfs.d └── config
Fill the config file information matching the new device-type.
This device-specific configuration file will contain:
-
the network interface’s name.
-
the name of the type of device, as used throughout the file architecture.
-
the name of the storage device in /dev/
-
the root partition extension name.
The rootpart information is used by set_hostname.sh to know where /etc/hosts and /etc/hostname are located. |
We will want to change this scheme in the future to autodetect the location of those files because it is specific to the image to be flashed. |
Example of a config file:
storage is /dev/sda, boot partion is /dev/sda2.
interface=eth0 type=new-device-type storage=sda rootpart=2
For /dev/mmcblk0 and /dev/mmcblk0p2 this extension (rootpart) will be p2.
Step 4 - Device installation
Physically set the device into the platform.
If not already done, connect the device onto the platform and boot it up.
Step 5 - initramfs generation
Generate its initramfs and retrieve the minimal boot files to initiate our process.
Execute: ~/reflash/initramfs/utils $ ./mk_boot_initramfs.sh 10.0.1.1 This will generate: reflash/initramfs/devices/new-device-type/
This script copies files on the target new-device and executes the following script
(make_initramfs.sh), which might need adjustments for your specific device; (check
the case $type in sections.)
.reflash/initramfs/utils/global/make_initramfs.sh
Details
#!/bin/sh DIR=~/boot_initramfs.d INIT_DIR=$DIR/initramfs.d NEW_INIT_DIR=$DIR/new_initramfs.d BOOT_DIR=$DIR/boot if [ ! -d $BOOT_DIR ]; then mkdir $BOOT_DIR fi sudo apt-get -y update sudo apt-get -y install initramfs-tools gcc sudo apt-get -y autoremove # Check whether bin and lib directories are links or not (example in / : # bin -> usr/bin # and mkdirs accordingly cd $INIT_DIR dirs='bin sbin lib lib32 lib64 libx32 usr/bin' for dir in $dirs do link=$(ls -l /|grep " $dir -> "|rev|cut -d " " -f1|rev) nb=$(echo $link|wc -w) if [ $nb -gt 0 ] then # Is a link bindir=$link rm -rf $link mkdir -p $link ln -s $link $dir else # Is Not a link bindir=bin rm -rf $dir mkdir -p $dir fi done gcc ../ntpclient.c -o $bindir/ntpclient chmod +x $bindir/ntpclient # This needs to be done first: # uncomment the following line # deb http://archive.canonical.com/ubuntu bionic partner # in /etc/apt/sources.list # # # Possibly: # # sudo rm /var/lib/apt/lists/* # sudo apt-get -y update && sudo apt-get -y upgrade # Then, after reboot, this script can be run. #sudo apt-get -y update while read line; do sudo apt-get -y install $line done < "../package_list" #### The busybox version of grep is "incomplete"; it does not have the '-b' option. #### We will use the system's version instead for word in $(busybox --list) do if [ "$word" != "grep" ] then ln -s busybox $bindir/$word fi done #### while read line; do for word in $line; do echo $word if [ $(busybox --list | grep -w $word) ] && [ "$word" != "grep" ] then ln -sf busybox bin/$word else #echo xxxxxxxxxxx BIN=$(whereis $word|cut -d " " -f2) DEST_BIN=$(echo $BIN|cut -c2-) #echo cp $BIN $DEST_BIN cp $BIN $DEST_BIN LIBS=$(ldd $BIN|grep -v linux-vdso.so.1|grep -v "not a dynamic executable"|cut -c2-|rev|cut -d " " -f2|rev|grep -v statically) for lib in $LIBS; do DEST_LIB=$(echo $lib|cut -c2-) DEST_LIB_DIR=$(dirname $DEST_LIB) mkdir -p $DEST_LIB_DIR cp $lib $DEST_LIB_DIR done #echo WAIT #ls -l $BIN |grep grep #/bin/sh #echo WAIT AGAIN #ls -l $DEST_BIN |grep grep #/bin/sh #echo DONE WAITING fi done done < "../bin_list" type=$(grep type config|cut -d'=' -f2) case $type in # jetson-nano) # mkdir $DEST_LIB_DIR/firmware/ # mv tegra21x_xusb_firmware $DEST_LIB_DIR/firmware/ # ;; raspberry-0W|raspberry-3Bplus-wifi) while read line; do firmware=$(echo $line|cut -d "/" -f1) case $firmware in firmware) dir= ;; *) dir=/modules/$(uname -r) ;; esac echo /lib$dir/$line cp -r --parents /lib/$dir/$line . done < "../wifi-file_list" ;; *) esac sudo cp -r /boot/* $BOOT_DIR #sudo chown -R admin:admin $DIR ##### ###cd $BOOT_DIR ###mv initrd.img-$(uname -r) initrd.img-$(uname -r).old ###./overlay-initramfs initrd.img-$(uname -r).old $INIT_DIR initrd.img-$(uname -r) ##### case $type in jetson-nano|tinkerT|nuc7i3bnh|nuc8i7hvk) ############## replaces overlay method; might want to genralize either method" ####### if [ -d $BOOT_DIR/efi ]; then sudo chown -R admin:admin $BOOT_DIR/efi fi rm -rf $NEW_INIT_DIR mkdir $NEW_INIT_DIR ../mkinitramfs.sh -o initramfs.org ../unmkinitramfs.sh initramfs.org $NEW_INIT_DIR/ rm ../mkinitramfs.sh ../unmkinitramfs.sh initramfs.org cp config init udhcp-script.sh $NEW_INIT_DIR/ #Reminder: dirs='bin sbin lib lib32 lib64 libx32 usr/bin' for dir in $dirs do if [ -d $NEW_INIT_DIR/$dir ] then for word in $(ls $INIT_DIR/$dir) do # rm -rf $NEW_INIT_DIR/$dir/$word rsync -a $INIT_DIR/$dir/$word $NEW_INIT_DIR/$dir/ done fi done cp -r $INIT_DIR/usr $NEW_INIT_DIR/ cd $NEW_INIT_DIR rm $BOOT_DIR/initrd.img-$(uname -r) find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > $BOOT_DIR/initrd.img-$(uname -r) ############## End of overlay replacement method ##################################### ;; nuc7i3bnh_|nuc8i7hvk_) cd $BOOT_DIR if [ -d $BOOT_DIR/efi ]; then sudo chown -R admin:admin $BOOT_DIR/efi fi mv initrd.img-$(uname -r) initrd.img-$(uname -r).old echo XYXYXYXYXYXYXYXY ls -l $INIT_DIR/ ls -l $INIT_DIR/bin/ ls -l $INIT_DIR/sbin/ sleep 20 ../overlay-initramfs initrd.img-$(uname -r).old $INIT_DIR initrd.img-$(uname -r) ;; "raspberry-"*) find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > $BOOT_DIR/initramfs ;; *) esac # cleanup cd $BOOT_DIR case $type in jetson-nano) #cleanup rm -r $NEW_INIT_DIR rm initrd ln -s initrd.img-$(uname -r) initrd ;; nuc7i3bnh|nuc8i7hvk) #cleanup rm initrd.img-$(uname -r).old ;; "raspberry-"*) #cleanup mkdir tmp sudo mv * tmp 2> /dev/null cd tmp sudo mv $(cat ../../bootFiles-list) .. 2> /dev/null cd .. if [ $(getconf LONG_BIT) -eq "64" ] then rm kernel* 2> /dev/null cp /boot/kernel8.img . fi sudo rm -rf tmp ../bootFiles-list ;; tinkerT) sudo apt-get -y install u-boot-tools mv initrd.img-$(uname -r) initramfs mkimage -A arm -O linux -T ramdisk -C gzip -n $(uname -r) -d initramfs initrd.img-$(uname -r) sudo rm -f boot.* mv new_boot.txt boot.txt mkimage -C none -T script -d boot.txt boot.scr sudo rm -r lost\+found #cleanup rm initramfs rm -r $NEW_INIT_DIR rm initrd > /dev/null 2>&1 ln -s initrd.img-$(uname -r) initrd ;; *) esac rm -rf $INIT_DIR cd $DIR rm ntpclient.c *_list *.sh overlay-initramfs
In-depth details
Example for already existing device types (for reference) and the new-device-type. The created files can be adapted from existing devices files.
reflash/devices
reflash/devices ├── nuc8i7hvk │ ├── d4-5d-df-xx-xx-xx │ │ ├── init-script.sh -> ../scripts/flash-script.sh │ │ └── reflash.img -> ../../../images/nuc8i7hvk/Debian-5.10.0-9-amd64.img │ └── scripts │ ├── flash-script.sh -> ../../../scripts/devices/flash-script.sh │ ├── resize.sh │ └── set_hostname.sh -> ../../../scripts/devices/set_hostname.sh ├── raspberry-3B │ ├── b8-27-eb-xx-xx-xx │ │ ├── init-script.sh -> ../scripts/flash-script.sh │ │ └── reflash.img -> ../../../images/raspberry-3B/raspbian-buster.img │ ├── b8-27-eb-yy-yy-yy │ │ ├── init-script.sh -> ../scripts/flash-script.sh │ │ └── reflash.img -> ../../../images/raspberry-3B/*raspbian-jessie.img │ └── scripts │ ├── flash-script.sh -> ../../../scripts/devices/flash-script.sh │ ├── resize.sh │ └── set_hostname.sh -> ../../../scripts/devices/set_hostname.sh └── new-device-type ├── aa-bb-cc-dd-ee-ff │ ├── init-script.sh -> ../scripts/flash-script.sh │ └── reflash.img -> ../../../images/new-device/device.img └── scripts ├── flash-script.sh -> ../../../scripts/devices/flash-script.sh ├── resize.sh └── set_hostname.sh -> ../../../scripts/devices/set_hostname.sh
reflash/images
reflash/images ├── nuc8i7hvk │ └── Debian-5.10.0-9-amd64.img ├── raspberries │ ├── raspbian-buster_1.img │ └── raspbian-buster_2.img ├── raspberry-3B │ ├── raspbian-buster.img -> ../raspberries/raspbian-buster_1.img │ └── raspbian-jessie.img └── new-device └── device.img
reflash/initramfs/devices
reflash/initramfs/devices ├── nuc8i7hvk │ └── boot │ ├── initrd.img-5.10.0-9-amd64 │ ├── efi │ │ └── ... │ └── ... ├── raspberry-3B │ └── boot │ ├── initramfs │ └── ... └── new-device-type └── ...
reflash/initramfs/utils
reflash/initramfs/utils ├── minimal_dirs │ ├── __global__ │ │ ├── bin_list │ │ ├── initramfs.d │ │ │ ├── init │ │ │ └── ... │ │ ├── make_initramfs.sh │ │ ├── package_list │ │ └── ... │ ├── nuc8i7hvk │ │ └── initramfs.d │ │ └── config │ ├── raspberry-3B │ │ ├── boot │ │ │ ├── cmdline.txt │ │ │ └── config.txt | | ├── bootFiles-list │ │ └── initramfs.d │ │ └── config │ └── new-device-type │ └── initramfs.d │ └── config └── mk_boot_initramfs.sh
Content of Files
Specific to raspberry pis
reflash/initramfs/utils/minimal_dirs/raspberry-3B/boot/cmdline.txt
ip=dhcp quiet root=/root
reflash/initramfs/utils/minimal_dirs/raspberry-3B/boot/config.txt
initramfs initramfs followkernel gpu_mem=16
The init script
This script may need to be edited if specific tasks need to be performed (check the case $type in sections.) |
reflash/initramfs/utils/minimal_dirs/__global__/initramfs.d/init
#!/bin/busybox sh echo Initramfs Entry Point # Make necessary directories mkdir -p dev etc proc sys mkdir -p mnt/storage mkdir mnt/devices mkdir mnt/images mkdir mnt/scripts mkdir mnt/statusfiles # Mount filesystems mount -t proc none /proc mount -t sysfs none /sys mount -t devtmpfs -o nosuid,mode=0755 udev /dev # Retrieve device-specific configuration information interface=$(cat ./config |grep interface | cut -d'=' -f2) storage=$(cat ./config |grep storage | cut -d'=' -f2) type=$(cat ./config |grep type | cut -d'=' -f2) rootpart=$(cat ./config |grep rootpart | cut -d'=' -f2) # Perform device-specific duties # Vital, in some cases, in order for the system to have time to "fill" /sys/class/net/$interface/address # - "sleep 1" might be enough sleep 3 case $type in nuc7i3bnh|nuc8i7hvk) export blacklist= . /scripts/functions # the important part is /scripts/init-top/udev run_scripts /scripts/init-top ;; raspberry-0W|raspberry-3Bplus-wifi) sleep 1 modprobe brcmfmac sleep 1 mkdir -p /var/run/wpa_supplicant wpa_supplicant -i wlan0 -c wpa_supplicant.conf -B sleep 1 ;; *) esac MAC=$(cat /sys/class/net/$interface/address| sed 's_:_-_g') home=/home/gordon base=$home/reflash # Retrieve ip address through dhcp udhcpc -i $interface -t 5 -q -s ./udhcp-script.sh # Mount server directories through nfs mount -t nfs -o vers=3,nolock 10.0.0.1:$base/devices/ /mnt/devices/ mount -t nfs -o vers=3,nolock 10.0.0.1:$base/images/ /mnt/images/ mount -t nfs -o vers=3,nolock 10.0.0.1:$base/scripts/ /mnt/scripts/ mount -t nfs -o vers=3,nolock 10.0.0.1:$home/statusfiles /mnt/statusfiles # Execute the script specified for the device (link from ~/reflash/devices/$MAC/init-script.sh) /bin/sh /mnt/devices/$type/$MAC/init-script.sh $storage $type $MAC $interface $rootpart # Reboot the system (with its newly installed image) reboot -f
New Device Type scripts
reflash └── devices | └── new-device-type | └── scripts | ├── init-script.sh -> ../../../scripts/devices/flash-script.sh | ├── resize.sh | └── set_hostname.sh -> ../../../scripts/devices/set_hostname.sh └── scripts └── devices ├── flash-script.sh ├── get_disk_info.sh └── set_hostname.sh
flash-script.sh
#!/bin/sh storage=$1 type=$2 MAC=$3 interface=$4 rootpart=$5 my_device=/mnt/devices/$type/$MAC statusfolder=/mnt/statusfiles/$type/$MAC if [ ! -d $statusfolder ] then mkdir -p $statusfolder fi DATE=$(echo $(/bin/ntpclient utc)| sed 's/Time: //g') echo $DATE UTC : "Flashing" $(ls -l $my_device/reflash.img|rev|cut -d "/" -f1|rev) >> $statusfolder/status echo Flashing $(ls -l $my_device/reflash.img|rev|cut -d "/" -f1|rev) dd if=$my_device/reflash.img of=/dev/$storage bs=4M conv=fsync DIR="$(dirname "$(readlink "$0")")" #echo $DIR #echo BASEDIR=$(dirname "$0") #echo #ls -l $BASEDIR/$DIR/ #echo . /mnt/scripts/devices/get_disk_info.sh get_disk_info /dev/$storage echo starting resize /bin/sh $BASEDIR/$DIR/resize.sh $storage $Last_Partition $Last_Partition_Number $type echo resize done echo /bin/sh $BASEDIR/$DIR/set_hostname.sh $storage $type $interface $rootpart /bin/sh $BASEDIR/$DIR/set_hostname.sh $storage $type $interface $rootpart echo set_hostname done sleep 20 DATE=$(echo $(/bin/ntpclient utc)| sed 's/Time: //g') echo $DATE UTC : "Flashing done" >> $statusfolder/status echo All done. Rebooting in 3s sleep 3
get_disk_info.sh
cf Disk Information Retrieval
set_hostname.sh
#!/bin/sh storage=$1 type=$2 interface=$3 rootpart=$4 #IP=$(hostname -I|cut -d " " -f1) IP=$(ip -4 addr show $interface | grep -oP '(?<=inet\s)\d+(\.\d+){3}') mkdir -p mnt/rootfs mount /dev/${storage}${rootpart} mnt/rootfs IP_PART3=$(echo $IP| cut -d "." -f3) IP_PART4=$(echo $IP| cut -d "." -f4) hostname=$type-$IP_PART3-$IP_PART4 echo $hostname > mnt/rootfs/etc/hostname #echo -e "$IP\t$hostname.youpi.citi.insa-lyon.fr" >> mnt/rootfs/etc/hosts echo -e "$IP\t$hostname" >> mnt/rootfs/etc/hosts umount mnt/rootfs
Examples of resize scripts:
This script should be edited for specific tasks will certainly be necessary for each device-type (check the case $type in sections.) |
reflash/scripts/devices/resize.sh
#!/bin/sh storage=$1 Last_Partition=$2 Last_Partition_Number=$3 type=$4 case $type in nuc7i3bnh|nuc8i7hvk|tinkerT|jetson-nano) case $type in nuc7i3bnh) ;; nuc8i7hvk|jetson-nano) sgdisk -e /dev/$storage kpartx -av $Last_Partition ;; tinkerT) sgdisk -z /dev/$storage kpartx -av $Last_Partition ;; esac # Resizing partition to full size of disk END=$(gdisk -l /dev/$storage | grep usable | awk '{print $(NF)}') parted /dev/$storage resizepart $Last_Partition_Number ${END}s # seems needed for the device/partition to be ready to be handled by e2fsck sleep 1 e2fsck -f $Last_Partition mount -t ext4 $Last_Partition /mnt/storage resize2fs $Last_Partition ;; "raspberry-"*) ### NO DISC RESIZE (Handled during 1st system boot) ### ;; esac