Hacking .img/.sdcard files

We ship around lots of images for use on SD cards, SATA drives, or USB sticks. Some of them are publicly available and listed in our i.MX6 Builds page, but others are used during development and testing of custom boards or customer-specific things.Our convention is to use the name .img or .sdcard for files that comprise a partition table and one or more file-system partitions, and have repeated instructions like these about how you can restore them to SD card.You can also access the internals on a development machine though, and in this post, we'll describe some of the tools we use to create, update, and access the content of these image files without going through the very slow process of placing them on an actual SD card.

Tools and image

The tools used in this post are all documented using man, but it's easier to link to web pages:

• dd
• udisks
• fdisk
• mkfs.ext4
• losetup
• partprobe

We'll use the console image (20150126-nitrogen-3.10.17_1.0.2_ga-trusty-en_US-lxde_armhf.img.gz) from Laci's Cell modem post as an example.

Basics of reading

To access the internals of an SD card image, you'll need to de-compress the image: using gunzip. We'll rename the image in the process to something shorter (trusty).

~/Downloads$ ls -l 20150126-nitrogen-3.10.17_1.0.2_ga-trusty-en_US-lxde_armhf.img.gz
-rw-rw-r-- 1 ericn ericn 784331493 Feb 24 12:09 20150126-nitrogen-3.10.17_1.0.2_ga-trusty-en_US-lxde_armhf.img.gz
~/Downloads$ gunzip 20150126-nitrogen-3.10.17_1.0.2_ga-trusty-en_US-lxde_armhf.img.gz
~/Downloads$ mv 20150126-nitrogen-3.10.17_1.0.2_ga-trusty-en_US-lxde_armhf.img trusty.img
~/Downloads$ file trusty.img
trusty.img: x86 boot sector
~/Downloads$ fdisk -l trusty.img
Disk trusty.img: 3774 MB, 3774873600 bytes
239 heads, 36 sectors/track, 856 cylinders, total 7372800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x6b5130d0
     Device Boot      Start         End      Blocks   Id  System
trusty.img1            2048     7372799     3685376   83  Linux

In English, the file command recognizes the file as containing a DOS-style partition table (which originated on x86) and the fdisk tool shows a single partition.The partition starts at block number 2048 and ends at block number 7372799. Doing the math for you, this starts at an offset of 1MiByte and the total size is 3600 MiBytes.Note that you need to be careful in the use of various utilities because sometimes they refer to Megabytes instead of Mibibytes. For example, the first line of output from fdisk above says 3744 MB and is referring to Megabytes (1 Million bytes). Google if you don't understand the distinction.This image shows our convention of producing images that are 3600 MiB long to account for variations in 4GB cards.

Loopback mounts

Okay, so now we know what's inside the .img file, but don't have access to the internals. This is where the losetup tool comes into play. Since Linux can only mount a block device, we can use a Loopback device to point at offset 1MiB (1048576 bytes) into the file with a size of 3599 MiB (3773825024 bytes):

~/Downloads$ loopdev=`sudo losetup -f`;
~/Downloads$ echo "loop device is $loopdev"
loop device is /dev/loop0
~/Downloads$ sudo losetup -o 1048576 --sizelimit 3773825024 $loopdev trusty.img
~/Downloads$ udisks --mount $loopdev
Mounted /org/freedesktop/UDisks/devices/loop0 at /media/trusty
~/Downloads$ ls /media/trusty/
6x_bootscript      boot  initrd.img  mnt   run   tmp                 var
6x_bootscript.txt  dev   lib         opt   sbin  uEnv-nit6xlite.txt  vmlinuz
6x_upgrade         etc   lost+found  proc  srv   unit_tests          work
bin                home  media       root  sys   usr

Note that the losetup -f command above finds the next available loopback device on your system, and we use that as the name of the device through the variable loopdev.So in a few commands, we now have access to the internals of the .img file. This in itself is useful for examining the content, or even for tweaking something like a boot script. We mounted the filesystem as read-write, so both are possible.We do need to take care in unmounting the filesystem and freeing up the loopback device, but that's also not hard:

~/Downloads$ sudo umount $loopdev
~/Downloads$ sudo losetup -d $loopdev

Basics of creation

Given the tools above, you may already have figured out how to create an image of your own, but we'll walk through a simple example. The process involves three primary steps:

• Create the .img file with the appropriate size using dd
• Create the partition table using fdisk
• Create filesystem(s) using loopback device(s)

~$ dd if=/dev/zero of=my.img bs=1M count=3600
3600+0 records in
3600+0 records out
3774873600 bytes (3.8 GB) copied, 41.9144 s, 90.1 MB/s
~$ fdisk my.img
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x4c145691.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1):
Using default value 1
First sector (2048-7372799, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-7372799, default 7372799):
Using default value 7372799
Command (m for help): p
Disk my.img: 3774 MB, 3774873600 bytes
255 heads, 63 sectors/track, 458 cylinders, total 7372800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4c145691
 Device Boot      Start         End      Blocks   Id  System
my.img1            2048     7372799     3685376   83  Linux
Command (m for help): w
The partition table has been altered!
Syncing disks.
~$ loopdev=`losetup -f`
~$ sudo losetup -o 1048576 --sizelimit 3773825024 $loopdev my.img
~$ sudo mkfs.ext4 -L myrootfs $loopdev
mke2fs 1.42.9 (4-Feb-2014)
Discarding device blocks: done
Filesystem label=myrootfs
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
230608 inodes, 921344 blocks
46067 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=943718400
29 block groups
32768 blocks per group, 32768 fragments per group
7952 inodes per group
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
~$ sudo partprobe
~$ udisks --mount $loopdev
Mounted /org/freedesktop/UDisks/devices/loop0 at /media/myrootfs
~$ ls /media/myrootfs/
lost+found
~$ sudo umount $loopdev
~$ sudo losetup -d $loopdev

Of course, a typical use would actually populate the filesystem after using udisks to mount the loopback device, but this is sufficient to understand the basics.

Use in the wild

We have at least one example of how these tools can be used in our Android KK tree. The mksdimage.sh script uses these tools to create an .img file from an Android build tree.

Other tidbits

Note that the tools above are only some of those available in Linux to manipulate images. Others that are useful in other contexts include:

• parted
• sfdisk
• wic (from the Yocto Project

These others are especially useful when building scripts to automate the process.