Buying No-Name SD Cards from China

2022-05-01T00:00:00+00:00

Introduction</h2>
So I finally decided to upgrade the self-built firewall/gateway for my home internet connection [[1]][1]. I replaced an old Raspberry Pi Model B revision 1 (rpi B1) with a more powerful Pine A64. The transition was seamless, but I found myself in need of a microSD card (the rpi B1 was using a full-size SD card). I resorted to a random class-2 [[2]][2] 4 GB card I had laying around that worked beautifully.
With my internet connection working again I was left wondering: can I do better with as little money as possible? What if I buy some random microSD card(s) on a famous online retail service based in China?
Many articles (such as [[3]][3] and [[4]][4]) test the speed of microSD cards but none deal with no-name brands. Moreover, it turns out that buying cheap from china may be a waste of money due to fake products [[5]][5]. But how bad the situation really is? People usually buy high-capacity drives and end up receiving 8-16 GB cards with a fake firmware that reports a different disk size to the OS. I don't need much space for a standalone router thing. What if I buy small microSD cards directly? Will they be fake nonetheless or will they be the real thing? I found no answer to this question on the web so I decided to run an experiment myself.

The Funny microSD Cards under Test</h2>
On a whim I bought 4 microSD cards for the same price as one 8 GB card from a reputable brand. Those cards have some of the funniest names I've ever seen. Upon arrival I also noticed that they are stamped with the most weird and nonsensical approvals and speed ratings.
</a>
The SD cards under test: Banoi_roox, Miomg, OUIO, ShanDian. </figcaption> </figure>
As a bonus, I added to the test batch two more reasonable cards I acquired while waiting for the shipment to arrive: a 4 GB card labelled 'eb-flash' and an official Raspberry Pi 16 GB microSD card.
The Test Script</h2>
To test the cards I used the f3 - Fight Flash Fraud tools [[6]][6] with a custom script for repeatability. The usage guide [[7]][7] does not provide a reference script for testing cards, so I had to make my own.
#!/bin/sh -e # # sdcard_test.sh - test sdcards for fake/broken flash ###################################################### # WARNING! This script will eat your data. Set those variables correctly DEV="/dev/mmcblk0" MOUNTPOINT="./mnt" # Print some info on the device (will fail if $DEV does not exists) fdisk -l $DEV # Perform a destructive test on the device echo "" f3probe --destructive --time-ops $DEV # Get first partition name, format it, mount it and run read/write tests part=$(fdisk -l $DEV | awk 'END {print $1}') echo "" echo First partition is $part, formatting it to FAT32 umount $part || true mkfs.fat -F32 $part echo "" echo Performing R/W test on $part mount $part $MOUNTPOINT f3write $MOUNTPOINT umount $MOUNTPOINT mount $part $MOUNTPOINT f3read $MOUNTPOINT # Done. Unmount umount $MOUNTPOINT</code></pre> This script is run with sdcard_test.sh > cardname.log</tt> for each of the microSD card tested. As the comments already explain, it does three things:
`Performs a destructive test with f3probe</tt></li> Identifies the first partition on the card and formats it to FAT32</li>`
`Since now we're sure the card is empty, it runs f3write</tt> and f3read</tt> to verify the whole capacity of the device</li> </ol> The umount</tt> - mount</tt> incantations before f3read</tt> are needed to force-flush the filesystem cache. This forces Linux to re-read from the device directly to make sure we are verifying the actual data written.`
</a> Using the test script. </figcaption> </figure> The procedure is thorough and takes 10-15 minutes for each SD card. However, it ensures that I'll also catch broken or failing flash chips. The part taking most of the time is the execution of f3write</tt> and f3read</tt>. These tests are needed because apparently, f3probe</tt> alone is not sufficient to tell for sure if a card is not fake. At first I was confused, so I opened an issue on the f3 repository [[8]][8] that led to a great explanation on this matter by the f3 author himself. Test Results</h2> Enough talking here is a table with the results: SD card</th> Capacity</th> Verdict</th> Write Speed [MB/s]</th> Read Speed [MB/s]</th> </tr> </thead> Banoi_roox</td> 7.44 GB</td> PASS</td> 8.37</td> 17.14</td> </tr> eb-flash</td> 3.69 GB</td> PASS</td> 3.9</td> 18.34</td> </tr> miomg</td> 14.88 GB</td> PASS</td> 9.01</td> 18.47</td> </tr> ouio</td> 7.5 GB</td> FAIL</td> -</td> -</td> </tr> Raspberry Pi</td> 14.75 GB</td> PASS</td> 5.91</td> 18.4</td> </tr> shandian</td> 7.5 GB</td> PASS</td> 4.51</td> 16.21</td> </tr> </tbody> </table> To compare the SD cards against each other, I calculated two figures of merit: the speed score is the product of the read and write speeds in MB/s</li> the overall score which is the speed score times the capacity in GB</li> </ul> The results are summarized in the following table (ordered by overall score). I have removed the OUIO SD card since it did not pass the test. SD card</th> Speed Score</th> Overall Score</th> </tr> </thead> miomg</td> 166.4</td> 2476.3</td> </tr> Raspberry Pi</td> 108.7</td> 1604</td> </tr> Banoi_roox</td> 143.5</td> 1067.4</td> </tr> shandian</td> 73.1</td> 548.3</td> </tr> eb-flash</td> 71.5</td> 263.9</td> </tr> </tbody> </table> All in all, the best card seems to be the miomg one. It is the fastest of the lot and holds 16 GB of data. For the second prize I'd pick the Banoi_roox SD card which, even though is ranked third, it is faster than the stock Raspberry Pi SD. However, I see what the Raspberry Pi Foundation did here. Application responsiveness is tied to the read speed and their SD card heavily favors reading compared to writing. Keep in mind though that the Banoi_roox is almost twice as fast in writing and its read speed is roughly 7% less than the rpi card. I did not test random IOPS while reading and writing because a firewall/gateway is not a desktop computer so I do not really care about application responsiveness. Conclusion</h2> It seems common internet knowledge that fake and/or cheap SD cards from china are bad. By betting on smaller-size cards you can increase your chances to purchase a working card. This is based on the assumption that the higher-capacity cards are just the same as the smaller size ones reporting to the OS a fake capacity. To test this assumption, I bought a few 8-16 GB cards and put them through a test script that reads and writes test patterns on the flash to make sure it's working as expected. From my test results, I witnessed that 3/4 cards were actually working and reported the correct capacity. One of those was even faster than an official Raspberry Pi SD card of the same size. Read/write performance of the other cards was as expected, considering the purchase price (~2-3 $ each). I can now provide an answer to the questions: How bad are cheap small-size microSD cards? Not that bad actually, 25% bad I'd say ;).</li> Can I trust such cards? For niche applications I'd say yes, provided that they are thoroughly tested before use.</li> Will I do it again? Of course. It's worth it and now that I'm confident about the script, testing is not time consuming and requires minimal manual interaction.</li> </ol> P.S. I won't trust those SD cards with my family photos anyway. P.P.S. I received full refund for the broken OUIO card. Always ask for a refund! References</h2> \[1\] [gateway_collection/atlas - electricant][1] [1]: https://github.com/electricant/gateway_collection/tree/master/atlas [2] [Speed Class | SD Association][2] [2]: https://www.sdcard.org/developers/sd-standard-overview/speed-class/ [3] [microSD Card Benchmarks | Raspberry Pi Dramble][3] [3]: https://www.pidramble.com/wiki/benchmarks/microsd-cards [4] [Raspberry Pi microSD card performance comparison - 2019 | Jeff Geerling][4] [4]: https://www.jeffgeerling.com/blog/2019/raspberry-pi-microsd-card-performance-comparison-2019 [5] [Chinese micro SD card. : ExpectationVsReality][5] [5]: https://www.reddit.com/r/ExpectationVsReality/comments/41vl2i/chinese_micro_sd_card/ [6] [f3 - Fight Flash Fraud][6] [6]: https://fight-flash-fraud.readthedocs.io/en/stable/introduction.html [7] [Usage - f3][7] [7]: https://fight-flash-fraud.readthedocs.io/en/stable/usage.html [8] [Fake card not detected by f3probe][8] [8]: https://github.com/AltraMayor/f3/issues/180 </div> Upgrading PostgreSQL Clusters 2022-02-01T00:00:00+00:00 Introduction</h2> I own a Debian PC that is used as Network Attached Storage (NAS) and runs PostgreSQL [[1]][1] with data files stored on a Btrfs RAID1 filesystem [[2]][2]. As expected, with every new Debian release comes a new major version of the PostgreSQL database which is not compatible with the internal data storage format of the previous one. This requires upgrading the on disk data files before using the new database server [[3]][3]. At the time of writing, a new stable release of Debian is available roughly every two years with the oldstable entering end of life (EOL) one year after the latest stable has been released [[4]][4]. It goes without saying that, every time I upgrade Debian on such PC, I have to rediscover once again how to also upgrade PostgreSQL. This guide will hopefully save some time to my future self by offering a step-by-step summary of some guides available online, which are incomplete and/or outdated [[5]][5] [[6]][6] [[7]][7]. Upgrade Process</h2> The following steps assume a fresh installation of PostgreSQL from the official Debian packages. Right after installation, the new version is automatically started alongside the old one. The command: # systemctl | grep postgres</code></pre> will show the two corresponding unit files as postgresql@[version]-[cluster]</tt> both of which should be listed as running. The first step is changing the configuration file of the new server to point to the database directory of the RAID array. To do so, the server must be stopped and the database reinitialized after changing the data_directory</tt> option of /etc/postgresql/[version]/[cluster]/postgresql.conf</tt>. To make any future migration or update process easier, it's best to have a different folder for each postgres version and cluster. Once the configuration file has been changed, it's time to initialize the new database (as the user postgres): # su postgres (postgres)# /usr/lib/postgresql/[new version]/bin/initdb \ -D [mountpoint]/pgsql/[new version]/[cluster] (postgres)# exit # sudo systemctl restart postgresql@[new version]-[cluster]</code></pre> The new server should restart smoothly using [mountpoint]/pgsql/[new version]/[cluster]</tt> as the data directory. If not, double check the config file. If the step above succeeded, it's time to perform the actual migration. To do so, stop all the services using PostgreSQL and the old and new database servers. # sudo systemctl stop postgresql@[old version]-[cluster] # sudo systemctl stop postgresql@[new version]-[cluster] </code></pre> Before migrating, it's wise to perform a check to verify that the source and destination clusters are actually compatible. Only if the check succeeds we'll eventually migrate the data. Please note that the option --link</tt> of pg_upgrade</tt> has not been used on purpose: albeit it makes the migration process faster, if something goes wrong, the cluster will remain in some half-baked state requiring unspecified command-line voodoo to restore it. By not using such option, the old cluster will always be available, the new data directory can be deleted and the migration process can be restarted. NOTE: With all the services and the database server stopped, performing a backup of the old cluster is a clever idea. No seriously, do it now before it's too late. To start the migration run the following commands as the postgres user and double check that they are run from postgres' home directory: # su postgres (postgres)# cd ~ (postgres)# /usr/lib/postgresql/[new version]/bin/pg_upgrade \ --old-datadir "[mountpoint]/pdsql/[old version]/[cluster]" \ --new-datadir "[mountpoint]/pdsql/[new version]/[cluster]" \ --old-bindir "/usr/lib/postgresql/[old version]/bin" \ --new-bindir "/usr/lib/postgresql/[new version]/bin" \ --old-options '-c config_file=/etc/postgresql/[old version]/[cluster]/postgresql.conf' \ --new-options '-c config_file=/etc/postgresql/[new version]/[cluster]/postgresql.conf' \ --check</code></pre> Inspect carefully the output of the command above for any error. If none, the --check</tt> option can be removed to start the actual upgrade: (postgres)# /usr/lib/postgresql/[new version]/bin/pg_upgrade \ --old-datadir "[mountpoint]/pdsql/[old version]/[cluster]" \ --new-datadir "[mountpoint]/pdsql/[new version]/[cluster]" \ --old-bindir "/usr/lib/postgresql/[old version]/bin" \ --new-bindir "/usr/lib/postgresql/[new version]/bin" \ --old-options '-c config_file=/etc/postgresql/[old version]/[cluster]/postgresql.conf' \ --new-options '-c config_file=/etc/postgresql/[new version]/[cluster]/postgresql.conf' \ --jobs [number of cpus]</code></pre> Wait a few minutes hoping for the best and voilà the new data directory should contain a migrated copy of the old cluster. If the commands above are run from ssh, I suggest using screen</tt> or tmux</tt> to be able to reconnect to your session in case ssh hangs for whatever reason. As a final step, swap the old PostgreSQL server with the new one by setting the option port=5232</tt> in /etc/postgresql/[new version]/[cluster]/postgresql.conf</tt>, disable the old server and start the new one. # systemctl disable postgresql@[old version]-[cluster] # systemctl enable --now postgresql@[new version]-[cluster]</code></pre> Since optimizer statistics are not transferred by the pg_upgrade</tt> command, restore them with: # su postgres (postgres)# cd ~ (postgres)# ./analyze_new_cluster.sh</code></pre> Once finished, the services depending on PostgreSQL can be restarted and should work as usual. If not, since the old data directory is still available, the new server can be swapped for the old one to see what went wrong. The following commands will delete the old cluster's data files: # su postgres (postgres)# cd ~ (postgres)# ./delete_old_cluster.sh</code></pre> I usually keep the old data around for a while (until the next migration actually) so that I always have and old copy laying around. Better be safe than sorry. Further Reading</h2> \[1\] [Start PostgreSQL on Debian 8][1] [1]: /articles/postgresql-start-debian/ [2] [A Note About Btrfs RAID1 for my Future Self][2] [2]: /articles/note-btrfs-raid1/ [3] [Upgrading a PostgreSQL Cluster][3] [3]: https://www.postgresql.org/docs/current/upgrading.html [4] [Production Releases - Debian Wiki][4] [4]: https://wiki.debian.org/DebianReleases#Production_Releases [5] [PostgreSQL Migration - Debian Wiki][5] [5]: https://wiki.debian.org/PostgreSql#Migration [6] [Using pg upgrade on Ubuntu/Debian - PostgreSQL Wiki][6] [6]: https://wiki.postgresql.org/wiki/Using_pg_upgrade_on_Ubuntu/Debian [7] [DebianGis/UpdatingPostGIS - Debian Wiki][7] [7]: https://wiki.debian.org/DebianGis/UpdatingPostGIS </div> Post-install Raspberry Pi Tweaks 2021-09-01T00:00:00+00:00 Some notes about the first tweaks I apply on a freshly installed Arch Linux image on a Raspberry Pi. The following notes apply to other Linux distros as well. Maybe the commands and/or package names are different. If, for whatever reason, I'll switch distro (why should I? Arch is the best one) I may also update this guide, pointing to the Arch-specific stuff and offering alternatives. In the meantime, if you stumble upon this page, do your own research (DYOR) before blindly applying those tweaks. Moreover, this list is highly opinionated and DYOR helps de-opinionate it. Feel free to discuss the content by opening an issue on github</a>. Fakehwclock</h2> The Raspberry Pi does not have an hardware real time clock (RTC). Therefore, upon reboot the time is lost and has to be set manually or via the network time protocol (NTP). This process takes time and requires an internet connection available after boot. The utility fake-hwclock [[1]][1] can be used to restore the clock to an approximately accurate value to speed up the NTP clock synchronization procedure. Fake-hwclock is installed the usual way via pacman: # sudo pacman -Syu fake-hwclock</code></pre> By default, the fake-hwclock-save.timer unit is run every 15 minutes and also 15 minutes after boot. The saving interval is quite short and may damage the SD card in the long run. Depending on the time accuracy requirements, by creating an overlay file for the unit with the command: # sudo systemctl edit fake-hwclock-save.timer</code></pre> the interval can be modified. By adding the following lines to the file, the timer is run once per day: [Timer] OnUnitActiveSec=1day AccuracySec=1day</code></pre> The 'AccuracySec' settings is used to save power and avoid waking up the CPU just to save the time. Refer to systemd.timer</a> for more details about the various timer config options. Keep in mind that the clock is automatically saved during shutdown, independently of the last time it was saved. This means that, if the Raspberry Pi is rebooted cleanly the clock will lag a few minutes. In case of a power loss the clock will be one day behind actual time in the worst case. Use the Hardware Random Number Generator</h2> More information is available here: Rng-tools - ArchWiki</a>. This guide is shorter and tailored to the Raspberry Pi. Make sure rng-tools is installed: # sudo pacman -Syu rng-tools</code></pre> Disable haveged since it is no longer needed and it might compete with rngd # sudo systemctl disable --now haveged</code></pre> Check that /etc/conf.d/rngd contains the line RNGD_OPTS="-o /dev/random -r /dev/hwrng"</code></pre> otherwise add it to the file to tell the daemon where to find the hardware random number generator. Rngd can now be enabled and started: # sudo systemctl enable --now rngd</code></pre> You can now check the amount of entropy reported by the kernel: # cat /proc/sys/kernel/random/entropy_avail</code></pre> It should be around 3000 (the more the better). On Raspberry Pis and other SBCs, rngd may exhibit sporadic CPU spikes to 100% which can last quite a long time [[2]][2]. At the time of writing, a fix has already been merged to the repo. Anyway, the issue disappears by editing /etc/conf.d/rngd to include the following option and restarting the daemon: RNGD_OPTS="-x jitter"</code></pre> Apparently, starting from Linux kernel 5.6, both haveged and rng-tools are not needed anymore [[3]][3][[4]][4]. This section is kept for posterity. Mount /var/cache (and whatnot) in memory with zram</h2> ZRAM</a> is a Linux kernel module to create block devices that store the data compressed in RAM, this allows to store more data in the same amount of memory, at the expense of CPU usage. This is ideal, for example, for /var/cache that contains cached data from application programs and does not need to survive a reboot. Many guides already exist [[5]][5][[6]][6], I mixed and merged them to suit my own taste. First of all one needs to load the zram module, telling it how many devices to create. The simplest way to achieve this is to create the file /etc/modules-load.d/zram.conf containing: zram</code></pre> This loads the zram module on boot which, by default, starts a single device. To change this behavior, create /etc/modprobe.d/zram.conf and insert: options zram num_devices=<desired number of zram devices></code></pre> Zram devices are block devices with no actual file system on them, so we need to format the device before we can mount it somewhere. I used udev to format on boot /dev/zram0 as ext4 without journalling. Removing the journal speeds up the file system at the expense of probable data corruption if the power fails. Zram contents are lost anyway if the power is removed so, I'd rather get my performance bonus. Create /etc/udev/rules.d/10-zram.rules containing: KERNEL=="zram0", SUBSYSTEM=="block", ACTION=="add", \ ATTR{disksize}="256M" ATTR{mem_limit}="256M", \ RUN+="/sbin/mkfs.ext4 -O ^has_journal -L $name $env{DEVNAME}"</code></pre> This creates a 256M zram disk formatted as ext4 on /dev/zram0. The device can now be mounted via /etc/fstab as usual: /dev/zram0 /var/cache ext4 defaults,nosuid,nodev,noexec 0 0</code></pre>Enable the Watchdog</h2> NOTE: I'm still checking whether I really need this or not. Or even if it works and keeps my Pi up and running. For the time being I'll keep this section as a dump of unorganized thoughts and references. https://wiki.archlinux.org/title/Arch_User_Repository https://aur.archlinux.org/packages/watchdog/ -> add armv6h to the architectures in PKGBUILD https://raspberrypi.stackexchange.com/questions/92462/installing-watchdog-daemon-on-arch References</h2> \[1\] [fake-hwclock for Arch Linux Arm on Raspberry Pi (using systemd)][1] [1]: https://archplusplus.co.uk/post/40202081414/fake-hwclock-for-arch-linux-arm-on-raspberry-pi [2] [Sporadic excessive CPU usage on RPi4B - Issue #136][2] [2]: https://github.com/nhorman/rng-tools/issues/136 [3] [random: make /dev/random be almost like /dev/urandom][3] [3]: https://github.com/torvalds/linux/commit/30c08efec8884fb106b8e57094baa51bb4c44e32 [4] [Is haveged still useful/relevant? - Issue #57][4] [4]: https://github.com/jirka-h/haveged/issues/57 [5] [zram - Gentoo Wiki][5] [5]: https://wiki.gentoo.org/wiki/Zram [6] [Improving performance - ArchWiki][6] [6]: https://wiki.archlinux.org/title/Improving_performance#zram_or_zswap </div> A Note About Btrfs RAID1 for my Future Self 2021-01-01T00:00:00+00:00 Introduction</h2> This is just a random note I have collected about Btrfs RAID1 [[1]][1] and how to deal with it in my NAS setup (I really have to write about it some day). How Btrfs RAID1 Works</h2> By default RAID1 in Btrfs creates no more than two copies of the data, regardless of the number of disks in the array. This means that, from the data integrity point of view, there is no point in using more than two disks. In RAID1, mode when a third disk is added, only the total capacity is increased. Btrfs distributes the writes in such a way that the array will survive the failure of one disk only. There are different RAID options [[2]][2] that enable more copies of the data and parity with more than two disks. Btrfs RAID1 in my NAS</h2> The motherboard I'm currently using has 4 serial-ATA ports. One of them is used for the OS disk while the other three are left for the data RAID. Of the three remaining ports, two are dedicated to the main hard disks while the third one is left unused. By doing so I can use this spare port to throw in a new hard disk in case one is near its end of life but it is not yet failed. I will eventually remove the failing disk after the new one is in place, leaving its port free for future drive replacements. To add a new device to a Btrfs RAID: # btrfs device add /dev/newdrive /mountpoint</code></pre> To remove the failed disk: # btrfs device remove /dev/broken /mountpoint</code></pre> This command removes the selected disk and automatically rebalances the data within the remaining hard drives. And while I'm at it why not throw in a run of # badblocks -b 4096 -e 1 -wsv /dev/sdX</code></pre> as a new drive burn-in ritual [[3]][3] before formatting and adding the drive to the pool? After going through all the steps above the array is ready. However, I have to buy a new hard drive to have a spare when needed. It is worth having a look at r/DataHoarder wiki</a> for some tips. References</h2> \[1\] [Using Btrfs with Multiple Devices][1] [1]: https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices [2] [Manpage/mkfs.btrfs][2] [2]: https://btrfs.wiki.kernel.org/index.php/Manpage/mkfs.btrfs#PROFILES [3] [New Drive Burn-In Rituals][3] [3]: https://perfectmediaserver.com/hardware/new-drive-burnin/ </div> Netis WF2419I Hacking 2020-08-01T00:00:00+00:00 Introduction</h2> I had an old (2014-ish) and cheap (if I recall correctly it cost me around 15-20 €) Netis WF2419I [[1]][1] router collecting dust in a cabinet at home. Its performance has always been lackluster with the WiFi connection dropping and restarting every few minutes. I bought it to serve as a guest AP for my dorm while I was studying in Padova. It worked dutifully for a few years there but, after I moved, I've never used it any more and switched to a whole different setup (which I hope to describe in another post) with more powerful hardware. This router was almost forgotten, until I read an excellent article by George Hilliard [[2]][2] about hacking Reolink cameras. I had no plans to use this device in any case, given that it is rather old and crappy. So reading the article was enough to spark my interest in trying to reverse engineer and add as much new functionality as possible to this piece of hardware (or, perhaps, should I say "junk"?). It surely won't be missed if something goes wrong in the process. First off I set myself some goals to keep me on track: Open the router and identify chipset, RAM, FLASH size and serial port</li> Dump the flash contents and inspect them</li> Rebuild a bootable image from the extracted contents and flash it</li> Setup a working buildroot image and compile a statically-linked busybox</li> Compile a (hopefully working) custom kernel for the device</li> </ol> As you can see, each goal somewhat depends on the previous one. Moreover, it's my first time that I try something like this and some of those goals are not easy. Murphy's law gets you every time and I have encountered many roadblocks during this journey. In writing, I will not spare any of the details, including the MANY failed attempts. That's why I'm calling this page a diary. At the time of writing I'm quite far from completion and I have somewhat abandoned the project. I may reboot it in the future so, perhaps, I might decide to split this post into a series. All the code and raw notes I have produced are available on GitHub</a>. Opening up the Device & Talking to the Serial Port</h2> Opening the device and finding the serial port has been the easy part. Actually, before rushing to the screwdriver to open the chassis, I searched for the FCC ID of the device on the back of the unit. Every RF device needs to be FCC certified to be sold and imported in the US [[3]][3] and this router is no exception. I found the FCC ID on the lower-right corner of the label: T58WF2419R. By searching for this ID on the official FCC website</a> or on fccid.io</a> I already had access to many low-level details about the router, including internal photos, without the need to open it. Not that opening it was a big deal. The warranty is already expired so no worries. </a> The FCC ID, as seen on the label.</figcaption> </figure> Stop wandering around the FCC website reading test reports! Given my goals, I need access to the flash chip so I have to open the device anyway. This was as easy as just removing the two screws on the bottom side (one is hidden by the mandatory 'warranty void if seal is broken' sticker, about which I couldn't care less) and popping open the case. With the case open, the following components appeared before me onto the sea of green of the board and are highlighted in the picture below: Realtek RTL8196C - the main router chipset</li> Realtek RTL8192CE - WiFi chipset</li> Winbond W25Q32FV - 32Mbit (4MB) SPI FLASH memory</li> ESMT M12L128168A - 2M x 16 Bit x 4 Banks (16MB) SDRAM</li> A few test points, probably for JTAG and UART access</li> </ul> </a> Router board with main components labelled.</figcaption> </figure> The usual first step is to see if a serial port is exposed somewhere on the board in order to have access to a shell on the device. As shown in the picture above, just by looking at the board, I already identified a few suitable testpoints. Just to be sure and also for future use, I looked up on my favourite search engine for the main router chipset datasheet. As expected, somebody (thank you!) released a confidential copy of the document [[4]][4]. I neglected all the other details for the moment and scrolled right-ahead to the pinout, looking for a serial port of some kind. </a> Realtek RTL8196C chipset pinout with serial port pins highlighted. </figcaption> </figure> In the above picture I cut the relevant part of the datasheet, showing the pinout of the chip. In the bottom-right part there are two pins (pin 36 and 37) labelled UART_RX/GPIOA7 and UART_TX/GPIOB0 respectively. Those pins can be configured either as GPIOs (for controlling the router LEDs etc.) or as a serial port. I traced those two pins with a multimeter and they are routed to the unpopulated header labelled J1. Most likely, they are the serial port I'm looking for. Time to solder a pin header and connect a TTL to USB adapter and see who's there. But HEY! Wait! What's the baud rate? A quick check with an oscilloscope revealed it's 38400bps. I know using an oscilloscope for this task it's a bit of an overkill, but I had one on hand so why not using it? By the way, some trial and error works too. </a> Pin header soldered to J1.</figcaption> </figure> After connecting a TTL to serial adapter and issuing $ screen -L /dev/ttyUSB0 38400</code></pre> When the router boots, the following messages appear on my terminal: ============================= <Probe SPI #0 > 1.Manufacture ID=0xef 2.Device ID=0x40 3.Capacity ID=0x16 4.Winbond SPI (4 MByte)! 5.Total sector-counts = 1024(sector=4KB) 6.spi_flash_info[0].device_size = 22 01_8198_SFCR(b8001200) =1f800000 01_8198_SFCR2(b8001204) =bb08000 01_8198_SFCSR(b8001208) =d8350000 ---RealTek(RTL8196C)at 2014.08.29-12:43+0800 version v1.1c [16bit](390MHz) no sys signature at 00010000! no sys signature at 00020000! Jump to image start=0x80500000... decompressing kernel: Uncompressing Linux... done, booting the kernel. done decompressing kernel. RTL8192C-RTL8188C driver version 1.5 (2012-05-04) serial console detected. Disabling virtual terminals. init started: BusyBox v1.00-pre8 (2015.01.23-06:46+0000) multi-call binary BusyBox v1.00-pre8 (2015.01.23-06:46+0000) Built-in shell (msh) Enter 'help' for a list of built-in commands. cp: /etc/tmp/pics: No such file or directory WARNING [factory_args.c:check_and_restore_factory_args:623]: current: [0x2adb521c], the length is zero! 50 4.8d.38.c2.74.9c.0.0.0.0.0.0. save_param load /var/ppp/.pppoe.session len = 0 wlan_check.sh: do nothing...... =================================================================================== dnrd --cache=off & WWW:netiscc, len:9 domain:netis.cc, len:8 WWW2:wwwnetiscc, len:13 domain2:www.netis.cc, len:12 killall: xhcatv: no process killed killall: multi_ppp: no process killed killall: igmpproxy: no process killed killall: udhcpc: no process killed wlanapp_sh:2042:@@@@@@@####;wlan0 MIB_HW_TX_POWER_CCK_A=302f2f2f2f2e2e2e2e2d2d2d2d2c MIB_HW_TX_POWER_CCK_A_new=302f2f2f2f2e2e2e2e2d2d2d2d2c MIB_HW_TX_POWER_HT40_1S_A=302f2f2f2f2e2e2e2e2d2d2d2d2c MIB_HW_TX_POWER_HT40_1S_An=302f2f2f2f2e2e2e2e2d2d2d2d2c MIB_HW_TX_POWER_HT20=ffffffffffffffffffffffffffff MIB_HW_TX_POWER_HT20n=ffffffffffffffffffffffffffff MIB_HW_TX_POWER_DIFF_OFDM=6161616161616161616161616161 MIB_HW_TX_POWER_DIFF_OFDMn=6161616161616161616161616161 SIOCGIFFLAGS: No such device bridge br0 doesn't exist; can't delete it SIOCDELRT: No such process SIOCDELRT: No such process dhcp is dis---------------------------------------------- wlanapp_sh:2042:@@@@@@@####;wlan0 wlan0-va0 wlan0-va1 wlan0-va2 wlan0-va3 wlan0-vxd iapp: not found wlanapp_sh:2334:wlan0:wlan0:wlan0-va3 ip:192.168.1.1 mask:255.255.255.0 gw:192.168.1.254 SIOCDELRT: No such process iwcontrol RegisterPID to (wlan0) Setup Firewall port trigger stop. port trigger init, wan is eth1. port trigger start. doamin=3 * Start NTP daemon /proc/sys/net/ipv4/conf/all/arp_ignore: cannot create /proc/sys/net/ipv4/conf/default/arp_ignore: cannot create iwpriv wlan0 set_mib bcnint=25-- netis(WF2419I)-V2.12.36123,2015.06.30 16:47. time.windows.com: Unknown host ========================END init.sh=============================== 321 348 /bin/ota_init.sh: not found 351 /bin/cdrom_wizard: not found killall: boa: no process killed 365 366 telnetd.sh: not found # killall: boa: no process killed [run_webs_server_by_execv:46:restart_webs.c] Starting Protocol Module: HTTP Server ... OK create_server_socket-357:Bind err!caught SIGTERM, starting shutdown exiting Boa normally (uptime 1 seconds) Starting Protocol Module: HTTP Server ... OK #</code></pre> Good. It looks like the router offers a root command prompt on the serial port. I poked around a bit and tried some commands to gain some info on the hardware and software of the device: # ls bin etc media sbin tmp var dev lib proc sys usr web # mount /dev/mtdblock1 on / type squashfs (ro) none on /proc type proc (rw) tmpfs on /var type tmpfs (rw) # df Filesystem 1k-blocks Used Available Use% Mounted on /dev/mtdblock1 2048 2048 0 100% / tmpfs 6144 176 5968 3% /var # cat /proc/cmdline root=/dev/mtdblock1 console=0 single # cat /proc/cpuinfo system type : Philips Nino processor : 0 cpu model : R3000 V0.0 BogoMIPS : 389.12 wait instruction : yes microsecond timers : no tlb_entries : 32 extra interrupt vector : no hardware watchpoint : no VCED exceptions : not available VCEI exceptions : not available ll emulations : 0 sc emulations : 0 # cat /proc/version Linux version 2.4.18-MIPS-01.00 (root@netcore) (gcc version 3.4.6-1.3.6) #1 Tue Jun 30 16:56:20 CST 2015 # busybox BusyBox v1.00-pre8 (2015.01.23-06:46+0000) multi-call binary Usage: busybox [function] [arguments]... or: [function] [arguments]... BusyBox is a multi-call binary that combines many common Unix utilities into a single executable. Most people will create a link to busybox for each function they wish to use, and BusyBox will act like whatever it was invoked as. Currently defined functions: [, adduser, arping, bunzip2, busybox, bzcat, cat, chmod, cp, cut, date, df, echo, egrep, expr, false, ftpget, grep, head, ifconfig, init, kill, killall, klogd, ln, logger, login, ls, mkdir, mknod, mount, msh, passwd, ping, ps, reboot, rm, route, sh, sleep, syslogd, tail, test, tftp, traceroute, true, umount, vconfig, wc, wget</code></pre> Quite an ancient software stack. Isn't it? Now that I have grabbed some basic information about the system it's time to move on and see what's inside the flash chip. Dumping The Flash Contents</h2> I had already identified the chip by visual inspection while opening the router: it is a Winbond W25Q32FV chip. Time to download its datasheet [[5]][5] to get the pinout and connect it to my beloved Bus Pirate [[6]][6]. The Bus Pirate is a small and useful board to talk to and interact with digital stuff. Out of the box it supports the following interfaces: 1-Wire, 2-Wire, 3-Wire, I2C, SPI, JTAG, UART and LCD. Alternatively, the GPIO pins can be directly controlled and many more protocols can be added using aftermarket firmware images since it is all open source. </a> Winbond W25Q32FV flash chip pinout.</figcaption> </figure> Now that I know the pinout of the chip, I have to figure out how to connect it for flashrom [[7]][7] to read it. I'll try In-System Programming (ISP) [[8]][8] first since it is much quicker and does not require reworking the board every time I have to read or write the flash contents. If this won't work the only solution is to desolder and resolder the flash chip from the board every time. This is time consuming and I'd like to avoid it at all cost. By means of a quick reverse engineering of the traces around the chip, I found out that pin 3 and pin 7 (/WP and /HOLD respectively) which, according to [[7]][7] should be connected to 3.3V on the bus pirate, are already tied to the VCC pin (pin 8). Therefore, only 6 out of the 8 required wires have to be connected. After twenty minutes spent with my trusty soldering iron and a piece of an old ATA cable (the 80-wire version which uses smaller conductors) and I have created a nice little connector ready to be plugged into the Bus Pirate socket. </a> Bus Pirate ISP connection to the flash chip.</figcaption> </figure> To check for a successful connection I issued the following command: $ sudo flashrom -p buspirate_spi:dev=/dev/ttyUSB0,spispeed=2M [sudo] password for pol: flashrom v1.2 on Linux 5.7.0-2-amd64 (x86_64) flashrom is free software, get the source code at https://flashrom.org Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns). Found Winbond flash chip "W25Q32.V" (4096 kB, SPI) on buspirate_spi. No operations were specified.</code></pre> Hurray! The flash chip was recognized correctly. Time to dump its contents. $ sudo flashrom -p buspirate_spi:dev=/dev/ttyUSB0,spispeed=2M -r firmware_orig.bin</code></pre> To see what's inside the image named firmware_orig.bin I extracted it recursively with the following command. At the end of the process binwalk creates a folder named _firmware_orig.bin.extracted containing the files extracted from the various file systems it found. $ binwalk -e firmware_orig.bin DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 4976 0x1370 LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 68320 bytes 207888 0x32C10 LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 2338816 bytes WARNING: Extractor.execute failed to run external extractor 'sasquatch -p 1 -le -d 'squashfs-root' '%e'': [Errno 2] No such file or directory: 'sasquatch', 'sasquatch -p 1 -le -d 'squashfs-root' '%e'' might not be installed correctly WARNING: Extractor.execute failed to run external extractor 'sasquatch -p 1 -be -d 'squashfs-root' '%e'': [Errno 2] No such file or directory: 'sasquatch', 'sasquatch -p 1 -be -d 'squashfs-root' '%e'' might not be installed correctly 851968 0xD0000 Squashfs filesystem, big endian, version 2.0, size: 2076096 bytes, 389 inodes, blocksize: 65536 bytes, created: 2015-06-30 08:57:47</code></pre> Or that was the expectation. What are those warnings and why is my folder squashfs-root empty? Apparently sasquatch is not installed in my system and $ sudo apt search sasquatch</code></pre> does not return any result. Alright it's not the first time I have to compile and install something from source on my system. It's strange however that the Debian repos are missing this useful piece of software. Perhaps I could volunteer as its maintainer? Not today. I have a router to hack now. Sasquatch is downloaded from [[9]][9] and build.sh does all the requisite stuff for building and installing. As usual, the process is never as easy as running a single command. The build failed with a ton of messages like the following: /usr/bin/ld: unsquashfs_xattr.o:/tmp/article/sasquatch/squashfs4.3/squashfs-tools/error.h:34: multiple definition of `verbose'; unsquashfs.o:/tmp/article/sasquatch/squashfs4.3/squashfs-tools/error.h:34: first defined here collect2: error: ld returned 1 exit status make: *** [Makefile:298: sasquatch] Error 1</code></pre> By looking at the issues and pull requests of sasquatch I found pull request #34</a> which should fix the issue. After applying it to my local copy of the code the compilation is successful. I'm now ready to go ahead with the extraction. $ binwalk -eM firmware_orig.bin Scan Time: 2020-08-21 10:55:58 Target File: /tmp/article/firmware_orig.bin MD5 Checksum: f03a657d0fd666ef7b281b178160aa6f Signatures: 391 DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 4976 0x1370 LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 68320 bytes 207888 0x32C10 LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 2338816 bytes 851968 0xD0000 Squashfs filesystem, big endian, version 2.0, size: 2076096 bytes, 389 inodes, blocksize: 65536 bytes, created: 2015-06-30 08:57:47 Scan Time: 2020-08-21 10:55:59 Target File: /tmp/article/_firmware_orig.bin.extracted/1370 MD5 Checksum: b11f1fa953d791774f180b80c290370e Signatures: 391 DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- Scan Time: 2020-08-21 10:55:59 Target File: /tmp/article/_firmware_orig.bin.extracted/32C10 MD5 Checksum: db44ecba377a764a82135b385d6711cd Signatures: 391 DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 674704 0xA4B90 Certificate in DER format (x509 v3), header length: 4, sequence length: 31 1517048 0x1725F8 Certificate in DER format (x509 v3), header length: 4, sequence length: 130 1939552 0x1D9860 Linux kernel version 2.4.18 1947112 0x1DB5E8 Unix path: /usr/lib/libc.so.1</code></pre> Great. The image contains some LZMA compressed data, probably kernel and bootloader, and a squashfs file system [[10]][10]. This time binwalk recursively extracted those files and what I got was in fact a kernel (version 2.4.18 of course), some certificates and the contents of the root file system. I was now able to inspect and modify the image contents as I please. The expert reader (or my future self if he learned something from this endeavour) may already have noticed the first mistake here. If nothing above looks wrong fasten your seat belt we're going deep down a rabbit hole. Rebuilding the Flash Image and First Roadblocks</h2> This is the root folder structure of the extracted image: $ ls -l _firmware_orig.bin.extracted/squashfs-root/ total 0 drwxr-xr-x 2 pol pol 3580 Jun 30 2015 bin drwxrwxrwx 2 pol pol 60 Jun 30 2015 dev drwxrwxrwx 3 pol pol 600 Jun 30 2015 etc drwxr-xr-x 2 pol pol 320 Jun 30 2015 lib lrwxrwxrwx 1 pol pol 10 Sep 11 12:39 media -> /var/media drwxrwxrwx 2 pol pol 40 Jun 30 2015 proc drwxr-xr-x 2 pol pol 180 Jun 30 2015 sbin lrwxrwxrwx 1 pol pol 8 Sep 11 12:39 sys -> /var/sys lrwxrwxrwx 1 pol pol 8 Sep 11 12:39 tmp -> /var/tmp drwxr-xr-x 4 pol pol 80 Jun 30 2015 usr drwxrwxrwx 2 pol pol 40 Jun 30 2015 var drwxr-xr-x 8 pol pol 300 Jun 30 2015 web</code></pre> To test if the image extraction and rebuild processes work correctly, the easiest thing to do is modify the index.htm file of the router configuration webpage and add some test code to it. I copied the directory firmware_orig.bin.extracted and its contents to firmware.extracted_new in order to preserve the original files. Then I modified firmware.extracted_new/squashfs-root/web/index.htm by appending the following lines just before the closing html tag. <script> alert("I have been pwned!"); </script></code></pre> This should display an alert window as soon as the page is loaded. Save and close (:wq) and that's it. Now it's time to rebuild the image and see if it boots. But before attempting to re-create it, I needed more details about the file system structure. $ unsquashfs -s D0000.squashfs Reading a different endian SQUASHFS filesystem on D0000.squashfs Found a valid big endian SQUASHFS 2:0 superblock on D0000.squashfs. Creation or last append time Tue Jun 30 10:57:47 2015 Filesystem size 2076096 bytes (2027.44 Kbytes / 1.98 Mbytes) Block size 65536 Filesystem is not exportable via NFS Inodes are compressed Data is compressed Fragments are compressed Always-use-fragments option is not specified Check data is not present in the filesystem Duplicates are removed Number of fragments 27 Number of inodes 389 umber of uids 1 Number of gids 0 </code></pre>The Lazy Way (Failed)</h3> The first root file system creation attempt I did was using the version of squashfs-tools provided by the Debian packages (version 4.4 at the time of writing). $ mksquashfs squashfs-root D0000.new.squashfs -b 65536 -all-root -noappend $ unsquashfs -s D0000.new.squashfs Found a valid SQUASHFS 4:0 superblock on D0000.new.squashfs. Creation or last append time [... redacted ...] Filesystem size 2788625 bytes (2723.27 Kbytes / 2.66 Mbytes) Compression gzip Block size 65536 Filesystem is exportable via NFS Inodes are compressed Data is compressed Uids/Gids (Id table) are compressed Fragments are compressed Always-use-fragments option is not specified Xattrs are compressed Duplicates are removed Number of fragments 26 Number of inodes 366 Number of ids 1</code></pre> Apart from the different superblock version, we have that the file system size is bigger, the endianness is gone and the number of inodes is lower than the original image. What is going on? I decided to tackle one issue at a time. First of all is the squashfs file system backwards-compatible? Apparently, SquashFS v4.0 moved to a fixed little-endian format (even for big-endian systems). Given the differences, the odds of the new image booting are already quite low. For the sake of science I decided to test the new image anyway. $ cp ../firmware_orig.bin firmware_pwned.bin $ dd if=D0000.new.squashfs of=firmware_pwned.bin bs=1 seek=$((0xD0000)) conv=notrunc $ sudo flashrom -p buspirate_spi:dev=/dev/ttyUSB0,spispeed=2M -w firmware.extracted_new/firmware_pwned.bin </code></pre> And sure enough when I turned on the router, carefully monitoring the serial output, the device was unable to mount the root file system and it was stuck in a bootloop. The Hard Way</h3> Of course it could not be so easy. I needed an older version of squashfs, supporting setting a different endianness. Is there a different/lower version of squashfs-tools in the Debian repos? Of course not. I had to compile it my own. Who cares I'm already used to that. The first thing I did was to download the squashfs sources for version 2.2-r2</a>. And once again the stock sources cannot be compiled with gcc 10. I learned, after trial and error, that this is due to different gcc behaviour and due to some symbols moved to different header files. To get this squashfs-tools version compile on my system I had to apply the following changes: $ diff -u squashfs2.2-r2/squashfs-tools squashfs-tools --- squashfs2.2-r2/squashfs-tools/Makefile 2005-09-01 01:21:14.000000000 +0200 +++ squashfs-tools/Makefile 2020-08-17 14:39:16.792112129 +0200 @@ -1,6 +1,7 @@ INCLUDEDIR = . -CFLAGS := -I$(INCLUDEDIR) -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -g +CFLAGS := -I$(INCLUDEDIR) -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -g -O2 \ + -fcommon -std=gnu89 mksquashfs: mksquashfs.o read_fs.o sort.o $(CC) mksquashfs.o read_fs.o sort.o -lz -o $@ --- squashfs2.2-r2/squashfs-tools/mksquashfs.c 2005-09-09 00:34:28.000000000 +0200 +++ squashfs-tools/mksquashfs.c 2020-08-17 14:42:08.037022333 +0200 @@ -28,6 +28,7 @@ #include <unistd.h> #include <stdio.h> #include <sys/types.h> +#include <sys/sysmacros.h> #include <sys/stat.h> #include <fcntl.h> #include <errno.h></code></pre> Figuring out why the code was not building and what to do to make gcc accept it took me quite a while. However, now that I have a working copy of mksquashfs version 2, I tried again to rebuild the file system: $ squashfs/squashfs2.2-r2/squashfs-tools/mksquashfs squashfs-root D0000.new.squashfs -b 65536 -all-root -noappend -2.0 -be $ unsquashfs -s D0000.new.squashfs Reading a different endian SQUASHFS filesystem on D0000.new.squashfs Found a valid big endian SQUASHFS 2:0 superblock on D0000.new.squashfs. Creation or last append time Wed Sep 23 14:15:42 2020 Filesystem size 2785979 bytes (2720.68 Kbytes / 2.66 Mbytes) Block size 65536 Filesystem is not exportable via NFS Inodes are compressed Data is compressed Fragments are compressed Always-use-fragments option is not specified Check data is not present in the filesystem Duplicates are removed Number of fragments 26 Number of inodes 366 Number of uids 1 Number of gids 0</code></pre> The issue about endianness and superblock version were solved. There was still a different number of inodes and a bigger file system size. But hey, who cares right? I flashed it into the router right-away, repeating the steps in the previous section, but the device is still stuck in a boot loop. What are those missing inodes anyway? I thought that maybe extracting the SquashFS file system manually will yield more information. $ unsquashfs D0000.squashfs Reading a different endian SQUASHFS filesystem on D0000.squashfs read_ids: Bad inode count in super block File system corruption detected FATAL ERROR:failed to read file system tables</code></pre> Now I'm getting why binwalk was using sasquatch to extract the root file system image. Let's do the same $ sasquatch D0000.squashfs SquashFS version [512.0] / inode count [-2063532032] suggests a SquashFS image of a different endianess Reading a different endian SQUASHFS filesystem on D0000.squashfs Parallel unsquashfs: Using 1 processor Trying to decompress using default gzip decompressor... Trying to decompress with lzma... Trying to decompress with lzma-adaptive... Detected lzma-adaptive compression 369 inodes (439 blocks) to write create_inode: could not create character device squashfs-root/dev/ptyp0, because you're not superuser! [... lots of lines as the one above ...] created 237 files created 20 directories created 109 symlinks created 0 devices created 0 fifos</code></pre> This was one of those moments when you realize you were barking up the wrong tree all the time. Facepalm. The missing inodes are the character devices [[11]][11]. Of course nothing will boot! Upon extraction, binwalk did not have the required permissions to create such files. I have to extract the file system as root to get the whole contents of the image: $ sudo sasquatch D0000.squashfs [sudo] password for pol: SquashFS version [512.0] / inode count [-2063532032] suggests a SquashFS image of a different endianess Reading a different endian SQUASHFS filesystem on D0000.squashfs Parallel unsquashfs: Using 1 processor Trying to decompress using default gzip decompressor... Trying to decompress with lzma... Trying to decompress with lzma-adaptive... Detected lzma-adaptive compression 369 inodes (439 blocks) to write [===============================================================|] 439/439 100% created 237 files created 20 directories created 109 symlinks created 23 devices created 0 fifos</code></pre> Now we are talking! Well... not really. I rebuilt the file system once again as I did before, the only difference now is file system size. However, the router is still stuck in a bootloop. A quick check confirmed that the size of the final flash image fits in the flash chip, so the file system size was not the issue. It was a symptom of the issue instead: what was still missing was the correct compression algorithm for mksquashfs. The version 2.2 of mksquashfs does not support lzma compression by default. A working copy of squashfs-lzma [[12]][12] was needed. I was fed up of blindly trying stuff until something works, so I downloaded the GPL code [[13]][13] (thanks Richard Stallman) for my router model and started inspecting the package contents to find some clues about how the flash image is built. GPL Code is Fun</h3> Inside the package I found the directory GPL_Code_RTL/rtl819x-sdk-v1.2/AP/squashfs-tools containing some sources for mksquashfs. After copying it somewhere else to preserve the original files, I had to patch the code inside the new directory in a similar fashion as SquashFS 2.2-r2. $ diff -ru squashfs-tools-orig squashfs-tools/ --- squashfs-tools-orig/Makefile 2013-11-10 14:56:41.000000000 +0100 +++ squashfs-tools/Makefile 2020-08-17 15:37:34.758142333 +0200 @@ -1,7 +1,8 @@ INCLUDEDIR = . LZMAPATH = ./lzma/SRC/7zip/Compress/LZMA_Lib -CFLAGS := -I$(INCLUDEDIR) -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -O2 +CFLAGS := -I$(INCLUDEDIR) -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -O2 \ + -fcommon -std=gnu89 --- squashfs-tools-orig/mksquashfs.c 2013-11-10 14:56:41.000000000 +0100 +++ squashfs-tools/mksquashfs.c 2020-08-17 15:38:32.368332562 +0200 @@ -27,6 +27,7 @@ #include <unistd.h> #include <stdio.h> #include <sys/types.h> +#include <sys/sysmacros.h> #include <sys/stat.h> #include <fcntl.h> #include <errno.h></code></pre> After that, I compiled the sources with make</tt> and I got a fancy new executable to play with: mksquashfs-lzma. It can be used to build the new file system as usual. $ squashfs/squashfs-tools/mksquashfs-lzma squashfs-root D0000.new.squashfs -b 65536 -all-root -noappend -be</code></pre> After flashing the new image, this time the router booted successfully. I pointed my browser to the configuration page (available at http://192.168.1.1 by default) and logged in with the default credentials. The following webpage appeared. </a> Success! A pwned index.htm appears.</figcaption> </figure> Hurray! This meant that I could do whatever I wanted with the root file system! To avoid manually entering a series of commands every time I want to build a new image, I created a small script available on GitHub [[14]][14]. Compiling Software and More Roadblocks</h2> This page is getting long already. Let's recap the ultimate goal: compile a custom kernel for the device. To do so, I need a working compiler for the architecture. This process is streamlined by buildroot: a set of Makefiles and patches that simplifies and automates the process of building a complete and bootable Linux environment for an embedded system [[15]][15]. Buildroot downloads and builds the toolchain and utilities it needs for compiling software for the target architecture, applies custom patches to the sources, compiles the software and assembles the final bootable image. To test the compiler is working, I decided to build a new version of busybox with some added functionality using buildroot. So, the first thing I did was to clone the latest stable buildroot release (2020.08.1 at the time of writing) with $ git clone git://git.buildroot.net/buildroot $ cd buildroot $ git checkout 2020.08.1</code></pre> Then, I tried to build a statically-linked busybox executable with some nice additions to the stock one on the router (such as telnet) using MIPS (big endian) as the target architecture and the stock Generic MIPS32 target architecture variant for buildroot. The command I used are the following: $ make menuconfig /* Select the desided architecture and build options Especially: Target options -> Target Architecture -> MIPS (big endian) -> Target Architecture Variant -> Generic MIPS32 Build options -> libraries -> static only / $ make busybox-menuconfig / Enable telnetd and disable all the stuff we don't need to make the busybox executable small / $ make / have a coffe break /</code></pre> After a few minutes, I found the compiled busybox executable in buildroot/output/target/bin. I copied it as busybox_p (to avoid confusion with the original executable) to the extracted firmware image and created a symlink to it named telnetd. I rebuilt the image as I did many times, flashed it and let the router boot. Sadly this was the result when running the new executable from the serial terminal. # busybox_p Illegal instruction # telnetd Illegal instruction</code></pre> Why can't this stuff just work? Maybe the endianness of the executable is wrong, all in all I just guessed it. In the extracted firmware image directory on my PC I ran: $ file busybox busybox: ELF 32-bit MSB executable, MIPS, MIPS-I version 1 (SYSV), dynamically linked, interpreter /lib/ld-uClibc.so.0, stripped $ file busybox_p busybox_p: ELF 32-bit MSB executable, MIPS, MIPS-I version 1 (SYSV), statically linked, stripped</code></pre> So the endianness was indeed correct. However, according to the chipset datasheet [[4]][4] and some other web resources, the illegal instruction the device was encountering may be due to the core which is a RLX4181 Lexra architecture [[16]][16], a variant of the MIPS32 architecture lacking the lwl, lwr, swl and swr instructions, covered by an US patent. Time to dig into the search engine once again. The following links seemed to point to some useful code: hackpascal/0001-binutils-2.24-add-lexra-support.patch</a></li> Working Realtek SoC RTL8196E 97D 97F in last master - For Developers - OpenWrt Forum</a></li> rlx-router/openwrt-rtl8196c</a></li> rlx-router/linux-rlx-upstream</a></li> shibajee/buildroot-rlx4181</a></li> </ul> The last link contains a complete buildroot repository allegedly supporting the RLX4181 architecture. I added it to my buildroot folder, cleaned it and rebuilt it once again: git remote add https://notabug.org/shibajee/buildroot git remote add -f rlx4181-notabug https://notabug.org/shibajee/buildroot git checkout rlx4181-master make clean make menuconfig / Select the correct architecture and build options */ make -j4</code></pre> After a while the executable files were ready. I ran objdump -D on the output executables, grepping for the instructions which are not accepted by the processor. Those commands did not turn any result. objdump -D root/bin/busybox_p | grep lwl objdump -D root/bin/busybox_p | grep lwr objdump -D root/bin/busybox_p | grep swl objdump -D root/bin/busybox_p | grep swr</code></pre> Now I was really confident that this time everything will have worked as expected so I rebuilt the flash image and flashed onto the chip. Sadly I was greeted again by the darn Illegal instruction message. I was starting to suspect that the error was due to the wrong kernel headers version. The actual kernel header files for the Linux version running on the device are available in the GPL code package. So, within my buildroot folder, I issued: make uclibc-menuconfig</code></pre> To edit the uclibc .config file and set a different directory to fetch the kernel headers. Pointing uclibc to the original (and quite old) kernel headers did not work either. This time I was unable successfully compile buildroot because I got a bunch of undeclared identifiers in many libraries. This is probably due to the kernel headers lacking the symbols required by the new software I was trying to compile. Conclusion</h2> Eventually this has been too much for me. I'm calling it quits here. Porting newer software to the older kernel headers to fix the compilation errors is a tedious job. Another option could be to port a newer kernel to the device, but all the kernel sources available on github seem to compile successfully but do not boot properly. Alternatively, I could try to use an older buildroot version. I have no idea if there is any buildroot version compatible with the Linux kernel version 2.4. This can be easily fixed by searching within the git repository. However, I will have to manually port the binutils patches to correctly compile software for this unorthodox MIPS architecture. And this again is tedious. I think the effort required to go further with this project is becoming too much and it is not worth it any more. Some progress has already been made by the OpenWrt folks albeit on a different processor revision (see links in the previous section) so I may reboot this project in the future. All in all, I learned a lot already and no electronic devices were harmed in the process. So, even though I hit only 3 out of the 5 objectives I set at the start, I'm quite satisfied. It was still a valid learning experience. It's time to try to hack another router, maybe I will be luckier this time (stay tuned). References</h2> \[1\] [Netis WF2419I][1] [1]: http://www.netis-systems.com/Suppory/de_details/id/1/de/181.html [2] [Hacking Reolink Cameras For Fun and Profit][2] [2]: https://www.thirtythreeforty.net/posts/2020/05/hacking-reolink-cameras-for-fun-and-profit/ [3] [FCC Equipment Authorization Procedures][3] [3]: https://www.fcc.gov/general/equipment-authorization-procedures [4] [RTL8196C Datasheet][4] [4]: https://lost-contact.mit.edu/afs/sur5r.net/service/drivers+doc/Realtek/RTL8196C%20Datasheet.pdf [5] [W25Q32FV Datasheet][5] [5]: https://www.winbond.com/resource-files/w25q32fv%20revi%2010202015.pdf [6] [Bus Pirate - Dangerous Prototypes][6] [6]: http://dangerousprototypes.com/docs/Bus_Pirate [7] [Bus Pirate - flashrom][7] [7]: https://www.flashrom.org/Bus_Pirate [8] [ISP - flashrom][8] [8]: https://www.flashrom.org/ISP [9] [GitHub - devttys0/sasquatch][9] [9]: https://github.com/devttys0/sasquatch [10] [SquashFS - Wikipedia][10] [10]: https://en.wikipedia.org/wiki/SquashFS [11] [Device file][11] [11]: https://en.wikipedia.org/wiki/Device_file [12] [Official Squashfs LZMA][12] [12]: https://squashfs-lzma.org/ [13] [Netis GPL Code][13] [13]: http://www.netis-systems.com/Suppory/gpl.html [14] [mkimg.sh on GitHub][14] [14]: https://github.com/electricant/netis-wf2419-router-hacking/blob/master/firmware.extracted_new/mkimg.sh [15] [Buildroot - Wikipedia][15] [15]: https://en.wikipedia.org/wiki/Buildroot [16] [Lexra - LinuxMIPS][16] [16]: https://www.linux-mips.org/wiki/Lexra </div> Internet Radio List 2020-04-01T00:00:00+00:00 Direct streaming links to web radios I like to listen to. A valid source of web radios is RadioBrowser</a>. Last updated on: 16 July 2022. Radio Bob</h3> homepage</a> streaming URL</a> Groove Salad</h3> homepage</a> streaming URL</a> Radio Nula</h3> homepage</a> streaming URL</a> Crucial Velocity Radio</h3> homepage</a> streaming URL</a> Rock Radio Slovenija</h3> homepage</a> streaming URL</a> Radio Marilù</h3> homepage</a> streaming URL</a> Radio 24</h3> homepage</a> streaming URL</a> RAI Radio Classica</h3> homepage</a> streaming URL</a> Linux VMs with qemu 2017-04-01T00:00:00+00:00 Introduction</h2> Sometimes I have to test various network configurations involving linux machines. Sometimes I use my own PC and some other device such as a Raspberry pi this has the advantage that each device physically exists and can be poked at. The main drawback is that the setup is cumbersome and time consuming and there's the risk of screwing something up with the configuration of my main laptop. The ideal would be to have a bunch of virtual machines connected together through some network interfaces, each one doing its thing independently. The VMs have also connected to the internet in order to download the required packages and updates. As an added bonus the test setup should be repeatable and has to be launched easily from the command line. This page contains some notes on how I accomplished such thing. Guest OS installation</h2> I won't go into the details of installing a linux guest OS in qemu as there's plenty of guides on the internet [[1]][1]. Startup Script</h2> Let's get straight into it. This is the setup I use for launching a server with three clients. Each machine is connected to the internet and to each other through a virtual layer 2 switch [[2]][2]. Please note that this script requires root privileges to be effective. #!/bin/bash # Amount of RAM each VM is given RAM_SIZE_MB=512 # Additional qemu options QEMU_OPTIONS="-enable-kvm -vga std" # interface names TAP_SERVER=tapSrv0 TAP1=tap1 TAP2=tap2 BRIDGE=brVrt0 # Preliminary network interface setup (requires root privieges) sudo ip tuntap add $TAP_SERVER mode tap user `whoami` sudo ip link set $TAP_SERVER up sudo ip tuntap add $TAP1 mode tap user `whoami` sudo ip link set $TAP1 up sudo ip tuntap add $TAP2 mode tap user `whoami` sudo ip link set $TAP2 up sudo ip link add $BRIDGE type bridge sudo ip link set $BRIDGE up sudo ip link set $TAP_SERVER master $BRIDGE sudo ip link set $TAP1 master $BRIDGE sudo ip link set $TAP2 master $BRIDGE # Start the testbench qemu-system-i386 $QEMU_OPTIONS -m $RAM_SIZE_MB \ -netdev tap,id=vlan0,ifname=$TAP_SERVER \ -device e1000,netdev=vlan0,mac=52:54:00:00:00:01 \ -netdev user,id=user0 -device e1000,netdev=user0 -hda server.cow & qemu-system-i386 $QEMU_OPTIONS -m $RAM_SIZE_MB \ -netdev tap,id=vlan0,ifname=$TAP1 \ -device e1000,netdev=vlan0,mac=52:54:00:00:00:02 \ -netdev user,id=user0 -device e1000,netdev=user0 -hda client1.cow & qemu-system-i386 $QEMU_OPTIONS -m $RAM_SIZE_MB \ -netdev tap,id=vlan0,ifname=$TAP2 \ -device e1000,netdev=vlan0,mac=52:54:00:00:00:03 \ -netdev user,id=user0 -device e1000,netdev=user0 -hda client2.cow & # Wait for background jobs to terminate wait # Interface teardown sudo ip tuntap del $TAP_SERVER mode tap sudo ip tuntap del $TAP1 mode tap sudo ip tuntap del $TAP2 mode tap sudo ip link del $BRIDGE</code></pre> The most difficult part was getting the interface setup right, both from the host and guest perspective. There's plenty of outdated and misleading guides around the web. The first thing to note is that qemu requires a device instance for each declared netdev. To do so we have to add an id to each netdev (-netdev id=netdev_id,netdev_type</tt>) and then attach a device to it like so -device hw_type,netdev=netdev_id</tt>. As it can be seen this is done twice, the first time for the tap</tt> device and the second for an user</tt> device. Refer to the guide about qemu networking [[3]][3] for details about the role of each device. Then in order to get the network topology we like we have to create and bridge together the tap interfaces at the host side. This has to be done before starting the various virtual machines. The commands have been taken from [[4]][4] and adapted a bit to the situation. Each virtual machine is started as a background job (note the &</tt> at the end) so before tearing down the network we issue a wait</tt> command which halts the script until all the jobs terminate. References</h2> 1. [QEMU - ArchWiki -][1] [1]: https://wiki.archlinux.org/index.php/QEMU "QEMU - ArchWiki -" [LAN switching - Wikipedia -][2] [2]: https://en.wikipedia.org/wiki/LAN_switching "LAN switching - Wikipedia -" </li> [Documentation/Networking - qemu project -][3] [3]: http://wiki.qemu-project.org/Documentation/Networking "Documentation/Networking - qemu project -" </li> [Networking - KVM -][4] [4]: https://www.linux-kvm.org/page/Networking "Networking - KVM -" </li> </ol> </div> LaTeX Templates 2017-03-01T00:00:00+00:00 I was recently looking for some instructions about templates for a presentation written using the Beamer</a> document class in LaTeX. The aim of this page is to collect some instructions and a few bits I learned during the process. I also plan to keep this page updated with stuff about LaTeX in general (hopefully). Guides</h2> First off some useful guides: Beamer Theme Matrix</a> For choosing a base theme and its variant.</li> Better Beamer Themes</a> For some basic instructions on creating a theme.</li> </ul> Boilerplate code</h2> Presentation</h3> Now some boilerplate code for a presentation: \documentclass[a4paper]{beamer} \usetheme{%%yourtheme} \usepackage{mathtools} \usepackage{lmodern} \title{The Title} \author{Author} \date{\today} \begin{document} % %% title frame \begin{frame} \maketitle \end{frame} \begin{frame}{Another frame, with title} Content \end{frame} % %% The last frame is the same as the first one \begin{frame} \maketitle \end{frame} \end{document}</code></pre>Article</h3> Also some code for an article: NOTE: This code requires the IEEEtran document class which can be downloaded from here</a>. On debian-based GNU/linux systems IEEEtran and some other useful packages can be installed with: sudo apt-get install texlive-publishers</tt>. \documentclass[journal, a4paper, 12pt]{IEEEtran} \title{The title} \author{The author} \markboth{Journal of Something}{Author name and paper title} \begin{document} \maketitle \begin{abstract} abstract \end{abstract} \begin{IEEEkeywords} keyword1, keyword2, ... \end{IEEEkeywords} % Sections and subsections as usual \end{document}</code></pre> Refer to http://www.texdoc.net/texmf-dist/doc/latex/IEEEtran/IEEEtran_HOWTO.pdf</a> for more details. Custom University of Padova Theme</h2> During my PhD at the University of Padova (Department of Information Engineering) I often need some theme for a presentation. A good one can be found here</a>. I used this theme as a starting point to create my own variation, adding the department logo and modifying the style a bit. Best results can be obtained using the presentation template above. The theme assumes that the last frame is the same as the first one. Download the theme from my github repo</a> and install it or put the files in the same directory as your presentation. Then use it with \usetheme{PadovaDEI}</tt>. Directory Alias in Bash 2016-10-01T00:00:00+00:00 Frequently I work with long paths from the terminal and typing them is tiresome and time-consuming. So I started to search for a solution to this problem. I thought that if I could create an alias for the path I could just type $cd path_alias</tt> instead of the tiring and error-prone $cd /a/very/long/path/to/write/</tt>. After a trip to the usual search engine this is what I found</a>. The guy that asked the question was trying to make an alias for a path by assigning it to a variable and then cd-ing to such variable, this way: myFold="~/Files/Scripts/Main" cd myFold</code></pre> This fails because by default cd</tt> thinks that myFold</tt> is a folder, not a variable to be expanded. The correct command to run is: cd $myFold</code></pre> But what if we want to get rid of the annoying $</tt>? Enter Shell Options</h2> There is a shell option</a> called cdable_vars</tt>. When it is set "an argument to the cd builtin command that is not a directory is assumed to be the name of a variable whose value is the directory to change to". As a result to make an alias to your favourite directory and make it permanent you could add the following lines to your .bashrc</tt>: shopt -s cdable_vars # enable cdable_vars export myFold="/a/very/long/path/to/write/"</code></pre> After reloading your .bashrc</tt> either by closing and reopening the terminal or by entering the command . ~/.bashrc</tt> no $</tt> is required any more. You can issue cd myFold</tt> and you will be redirected automagically to the directory of choice. Caveat</h2> The only caveat is that subdirectories within the alias do not work. For example the following command will result in: cd myFold/subfolder bash: cd: myFold/subfolder: No such file or directory</code></pre> Whereas cd $myFold/subfolder</tt> works as expected. Start PostgreSQL on Debian 8 2016-06-01T00:00:00+00:00 I run a small NAS computer at home for network file sharing and RSS feed fetching through Tiny Tiny RSS</a>. The hardware is a regular low-power computer running Debian 8. Overall this setup is running fine, requiring moderate amount of maintenance. Recently, after an update, PostgreSQL stopped working and I was unable to access to my beloved RSS feeds. Apparently something was changed in the way Debian starts Postgres via systemd. Now to start and or enable it the version you want to run has to be specified. For example the command to start the service is: systemctl start postgresql@9.4-main</code> Whereas to enable it: systemctl enable postgresql@9.4-main</code> If there is more than one version of the software installed that has to be run, then such version has to be specified after the @</tt> symbol. For more details see this mailing-list entry</a> which states the problem but does not give a complete solution.

Practical Examples</h2>

Upgrading PostgreSQL Clusters

Post-install Raspberry Pi Tweaks

A Note About Btrfs RAID1 for my Future Self

Introduction</h2>
This is just a random note I have collected about Btrfs RAID1 [[1]][1] and how to deal with it in my NAS setup (I really have to write about it some day).</p>

Netis WF2419I Hacking

Internet Radio List

Radio Bob</h3>
homepage</a></p>
streaming URL</a></p>

Groove Salad</h3>
homepage</a></p>
streaming URL</a></p>

Radio Nula</h3>
homepage</a></p>
streaming URL</a></p>

Crucial Velocity Radio</h3>
homepage</a></p>
streaming URL</a></p>

Rock Radio Slovenija</h3>
homepage</a></p>
streaming URL</a></p>

Radio Marilù</h3>
homepage</a></p>
streaming URL</a></p>

Radio 24</h3>
homepage</a></p>
streaming URL</a></p>

RAI Radio Classica</h3>
homepage</a></p>
streaming URL</a></p>

Linux VMs with qemu

Guest OS installation</h2>
I won't go into the details of installing a linux guest OS in qemu as there's plenty of guides on the internet [[1]][1].</p>

LaTeX Templates

Boilerplate code</h2>

Directory Alias in Bash

Start PostgreSQL on Debian 8

Paolo Scaramuzza

Notes on Using AVRDUDE

Guide to Configuring Dynamic DNS with BIND9 Using TSIG Keys

Buying No-Name SD Cards from China

Paolo Scaramuzza

Notes on Using AVRDUDE

Practical Examples</h2>

Debian/Ubuntu/Linux Mint</h4> As root run:</p> # apt-get install wget bison automake autoconf flex gcc libelf-dev</span></span> libusb-dev libusb-1.0-0-dev libftdi-dev libftdi1-dev`</span></span></code></pre>Red Hat/Centos/Fedora</h4> As root run:</p>

Red Hat/Centos/Fedora</h4> As root run:</p>

Compiling and Installing avrdude</h3> First of all we need to prepare a working directory to store the source code, place it where you like:</p>

Guide to Configuring Dynamic DNS with BIND9 Using TSIG Keys

Buying No-Name SD Cards from China

Upgrading PostgreSQL Clusters

Post-install Raspberry Pi Tweaks

A Note About Btrfs RAID1 for my Future Self

Introduction</h2> This is just a random note I have collected about Btrfs RAID1 [[1]][1] and how to deal with it in my NAS setup (I really have to write about it some day).</p>

Netis WF2419I Hacking

Rebuilding the Flash Image and First Roadblocks</h2> This is the root folder structure of the extracted image:</p>

The Lazy Way (Failed)</h3> The first root file system creation attempt I did was using the version of squashfs-tools provided by the Debian packages (version 4.4 at the time of writing).</p>

Internet Radio List

Linux VMs with qemu

Guest OS installation</h2> I won't go into the details of installing a linux guest OS in qemu as there's plenty of guides on the internet [[1]][1].</p>

LaTeX Templates

Guides</h2> First off some useful guides:</p> Beamer Theme Matrix</a> For choosing a base theme and its variant.</li> Better Beamer Themes</a> For some basic instructions on creating a theme.</li> </ul> Boilerplate code</h2> Presentation</h3> Now some boilerplate code for a presentation:</p>

Boilerplate code</h2>

Presentation</h3> Now some boilerplate code for a presentation:</p>

Article</h3> Also some code for an article:</p>

Directory Alias in Bash

Start PostgreSQL on Debian 8

Introduction</h2>
This is just a random note I have collected about Btrfs RAID1 [[1]][1] and how to deal with it in my NAS setup (I really have to write about it some day).</p>

Guest OS installation</h2>
I won't go into the details of installing a linux guest OS in qemu as there's plenty of guides on the internet [[1]][1].</p>