QEMU for and on ARM cores

Installing Debian on QEMU’s 64-bit ARM “virt” board

with 14 comments

This post is a 64-bit companion to an earlier post of mine where I described how to get Debian running on QEMU emulating a 32-bit ARM “virt” board. Thanks to commenter snak3xe for reminding me that I’d said I’d write this up…

Why the “virt” board?

For 64-bit ARM QEMU emulates many fewer boards, so “virt” is almost the only choice, unless you specifically know that you want to emulate one of the 64-bit Xilinx boards. “virt” supports supports PCI, virtio, a recent ARM CPU and large amounts of RAM. The only thing it doesn’t have out of the box is graphics.

Prerequisites and assumptions

I’m going to assume you have a Linux host, and a recent version of QEMU (at least QEMU 2.8). I also use libguestfs to extract files from a QEMU disk image, but you could use a different tool for that step if you prefer.

I’m going to document how to set up a guest which directly boots the kernel. It should also be possible to have QEMU boot a UEFI image which then boots the kernel from a disk image, but that’s not something I’ve looked into doing myself. (There may be tutorials elsewhere on the web.)

Getting the installer files

I suggest creating a subdirectory for these and the other files we’re going to create.

wget -O installer-linux
wget -O installer-initrd.gz

Saving them locally as installer-linux and installer-initrd.gz means they won’t be confused with the final kernel and initrd that the installation process produces.

(If we were installing on real hardware we would also need a “device tree” file to tell the kernel the details of the exact hardware it’s running on. QEMU’s “virt” board automatically creates a device tree internally and passes it to the kernel, so we don’t need to provide one.)


First we need to create an empty disk drive to install onto. I picked a 5GB disk but you can make it larger if you like.

qemu-img create -f qcow2 hda.qcow2 5G

(Oops — an earlier version of this blogpost created a “qcow” format image, which will work but is less efficient. If you created a qcow image by mistake, you can convert it to qcow2 with mv hda.qcow2 old-hda.qcow && qemu-img convert -O qcow2 old-hda.qcow hda.qcow2. Don’t try it while the VM is running! You then need to update your QEMU command line to say “format=qcow2” rather than “format=qcow”. You can delete the old-hda.qcow once you’ve checked that the new qcow2 file works.)

Now we can run the installer:

qemu-system-aarch64 -M virt -m 1024 -cpu cortex-a53 \
  -kernel installer-linux \
  -initrd installer-initrd.gz \
  -drive if=none,file=hda.qcow2,format=qcow2,id=hd \
  -device virtio-blk-pci,drive=hd \
  -netdev user,id=mynet \
  -device virtio-net-pci,netdev=mynet \
  -nographic -no-reboot

The installer will display its messages on the text console (via an emulated serial port). Follow its instructions to install Debian to the virtual disk; it’s straightforward, but if you have any difficulty the Debian installation guide may help.

The actual install process will take a few hours as it downloads packages over the network and writes them to disk. It will occasionally stop to ask you questions.

Late in the process, the installer will print the following warning dialog:

   +-----------------| [!] Continue without boot loader |------------------+
   |                                                                       |
   |                       No boot loader installed                        |
   | No boot loader has been installed, either because you chose not to or |
   | because your specific architecture doesn't support a boot loader yet. |
   |                                                                       |
   | You will need to boot manually with the /vmlinuz kernel on partition  |
   | /dev/vda1 and root=/dev/vda2 passed as a kernel argument.             |
   |                                                                       |
   |                              <Continue>                               |
   |                                                                       |

Press continue for now, and we’ll sort this out later.

Eventually the installer will finish by rebooting — this should cause QEMU to exit (since we used the -no-reboot option).

At this point you might like to make a copy of the hard disk image file, to save the tedium of repeating the install later.

Extracting the kernel

The installer warned us that it didn’t know how to arrange to automatically boot the right kernel, so we need to do it manually. For QEMU that means we need to extract the kernel the installer put into the disk image so that we can pass it to QEMU on the command line.

There are various tools you can use for this, but I’m going to recommend libguestfs, because it’s the simplest to use. To check that it works, let’s look at the partitions in our virtual disk image:

$ virt-filesystems -a hda.qcow2 

If this doesn’t work, then you should sort that out first. A couple of common reasons I’ve seen:

  • if you’re on Ubuntu then your kernels in /boot are installed not-world-readable; you can fix this with sudo chmod 644 /boot/vmlinuz*
  • if you’re running Virtualbox on the same host it will interfere with libguestfs’s attempt to run KVM; you can fix that by exiting Virtualbox

Looking at what’s in our disk we can see the kernel and initrd in /boot:

$ virt-ls -a hda.qcow2 /boot/

and we can copy them out to the host filesystem:

virt-copy-out -a hda.qcow2 /boot/vmlinuz-4.9.0-3-arm64 /boot/initrd.img-4.9.0-3-arm64 .

(We want the longer filenames, because vmlinuz and initrd.img are just symlinks and virt-copy-out won’t copy them.)

An important warning about libguestfs, or any other tools for accessing disk images from the host system: do not try to use them while QEMU is running, or you will get disk corruption when both the guest OS inside QEMU and libguestfs try to update the same image.

If you subsequently upgrade the kernel inside the guest, you’ll need to repeat this step to extract the new kernel and initrd, and then update your QEMU command line appropriately.


To run the installed system we need a different command line which boots the installed kernel and initrd, and passes the kernel the command line arguments the installer told us we’d need:

qemu-system-aarch64 -M virt -m 1024 -cpu cortex-a53 \
  -kernel vmlinuz-4.9.0-3-arm64 \
  -initrd initrd.img-4.9.0-3-arm64 \
  -append 'root=/dev/vda2' \
  -drive if=none,file=hda.qcow2,format=qcow2,id=hd \
  -device virtio-blk-pci,drive=hd \
  -netdev user,id=mynet \
  -device virtio-net-pci,netdev=mynet \

This should boot to a login prompt, where you can log in with the user and password you set up during the install.

The installation has an SSH client, so one easy way to get files in and out is to use “scp” from inside the VM to talk to an SSH server outside it. Or you can use libguestfs to write files directly into the disk image (for instance using virt-copy-in) — but make sure you only use libguestfs when the VM is not running, or you will get disk corruption.


Written by pm215

July 24, 2017 at 10:25 am

Posted in linaro, qemu

14 Responses

Subscribe to comments with RSS.

  1. Nice, thank’s ;)


    July 24, 2017 at 4:56 pm

  2. hi,
    is there any chance to run Xorg ?


    October 21, 2017 at 4:48 pm

    • I’ve never needed to, so I’ve never tried. There’s no graphics on the virt board by default, but you can try adding a graphics device (probably virtio graphics) with a suitable -device argument, and then get the guest OS to use it.


      October 21, 2017 at 5:32 pm

      • with: -vga virtio
        I have error:
        qemu-system-arm: Virtio VGA not available


        October 21, 2017 at 7:33 pm

        • You don’t want vga virtio, and you don’t want to try to use the -vga convenience option (you don’t want any kind of VGA device), you want a -device virtio-something option to give you the non-vga virtio graphics device. But you’ll have to investigate for yourself, because I don’t want to do it in comments here one error message at a time, I’m afraid.


          October 21, 2017 at 8:52 pm

  3. Thank you very much! Got working arm64 system.

    Ivan Uskov

    October 28, 2017 at 8:33 pm

  4. Wow that was very very useful thank you! :)


    January 19, 2018 at 1:14 am

  5. Excellent guide! Thank you!

    Balint Szente

    March 20, 2018 at 10:53 am

  6. How do I shut down? Running halt ends with a “system halted” but qemu never returns. :)

    Simon Templar

    March 25, 2018 at 8:17 pm

  7. Hi Peter, thanks for your posts !!

    I was able to get the “virt” machine installed of 32-arm, but when I try doing for 64-arm, I don’t see anything on the terminal whatsoever after I launch the installer-command.

    My host-machine is as below :

    uname -a
    Linux latitude-3480 4.13.0-38-generic #43~16.04.1-Ubuntu SMP Wed Mar 14 17:48:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

    Ajay Garg

    April 20, 2018 at 10:30 am

    • The recipes I give in the blog post work for me. I can’t help any further than that.


      April 20, 2018 at 10:35 am

  8. really nice tutorial. one question though, couldn’t find it anywhere: cortex-a53 is a cpu that has 1-4 cores, but in the default configuration it reverts to 1 core, which is rather slow. maybe i missed the switch, but do you know how to increase the number of cores to use? thanks!


    November 20, 2018 at 10:09 am

    • At the time I wrote this, released versions of QEMU only supported emulation of SMP guests on a single host thread, so emulating 4 guest cores would actually be slower than emulating 1, due to the overhead of switching between them. For newer versions of QEMU, starting with QEMU 2.9, use of multiple threads for ARM guests on x86-64 hosts is supported. You can add more guest CPUs with a command line like “-smp 4” — but note that although that is supported on pre-2.9 QEMU versions it will tend to make it slower rather than faster!


      November 20, 2018 at 10:16 am

      • Thanks for the quick reply. Yeah, I’m actually using qemu 2.11 and parsing `-smp 4` seems to increase performance, rather than diminish it.


        November 20, 2018 at 10:50 am

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s