Create AMI/Image from scratch for EC2/Xen
This blog captures the steps required to create an image from scratch which can be used on Xen virtualization platform using PvGrub boot manager. The later half of the blog also highlights steps which can be used to convert this image into an EC2 AMI and can then be used to boot an EC2 instance.
The instructions below have been tested on Ubuntu 13.10 and builds a Ubuntu 13.10 image/AMI.
1) Create image file and mount it
# creating 1GB image here # image size should be bigger than disk required for AMI # size of root device is chosen when launching instance and not here dd if=/dev/zero of=linux.img bs=1M count=1024 sudo losetup /dev/loop0 linux.img sudo mkfs.ext4 /dev/loop0 sudo mount /dev/loop0 /mnt
2) Install base system
sudo apt-get -y install debootstrap # Installing 64-bit Ubuntu saucy (13.10) here # Modify for your use case sudo debootstrap --arch=amd64 saucy /mnt
3) Chroot into new installed system to configure it
sudo chroot /mnt
4) Configure basic system
mount none /proc -t proc mount none /sys -t sysfs # Adding root mount point cat << EOF > /etc/fstab /dev/xvda1 / ext4 defaults 0 1 EOF # Adding saucy specific apt sources here cat << EOF > /etc/apt/sources.list deb http://us.archive.ubuntu.com/ubuntu/ saucy main deb-src http://us.archive.ubuntu.com/ubuntu/ saucy main deb http://us.archive.ubuntu.com/ubuntu/ saucy-updates main deb-src http://us.archive.ubuntu.com/ubuntu/ saucy-updates main deb http://security.ubuntu.com/ubuntu saucy-security main deb-src http://security.ubuntu.com/ubuntu saucy-security main EOF # setting eth0 for dhcp cat << EOF >> /etc/network/interfaces auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp EOF apt-get update # installing ssh-server for headless setup apt-get -y install openssh-server
5) Setup kernel and system to boot - This was the most tricky part of the setup to learn/find out/get right. PvGrub essentially reads /boot/grub/menu.lst file and find the kernel and initrd information from the file. There is no need to create boot record or boot sectors or anything similar when booting in Xen with PvGrub manager. However, one does have control over what kernel will be running and a kernel needs to be installed.
# install linux kernel # don't install grub on any disk/device when prompted # commands requires manual prompt, haven't scripted next step apt-get -y install linux-image-virtual # remove grub which was installed during kernel install # choose to remove grub from /boot/grub when prompted # commands requires manual prompt, haven't scripted next step apt-get -y purge grub2 grub-pc grub-common # install grub-legacy-ec2 which is NOT a ec2 specific package # this package creates /boot/grub/menu.lst file # this package applies to all PvGrub guests, even outside ec2 apt-get -y install grub-legacy-ec2
6) Do custom configuration
# set up root password passwd # or create users with sudo access
7) unmount filesystems & loop devices
# exit out of chroot /mnt exit sudo umount /mnt/sys sudo umount /mnt/proc # if umount fails, you might have to force it by umount -l # umount will fail if any daemon processes were started in chroot sudo umount /mnt # losetup will fail till daemon process are still running # kill any chroot daemon processes if needed sudo losetup -d /dev/loop0
Your new linux.img file is ready for use. The next set of commands are specific to converting this image file into an AMI which can be used on EC2.
a) Transfer the image to a running EC2 instance
b) Mount a new EBS volume on the EC2 instance. Size of the EBS volume should be greater than size of image file chosen at step (1).
c) Copy image file contents into EBS volume
# assuming that the empty EBS drive is on /dev/xvdf sudo dd if=linux.img of=/dev/xvdf bs=1M
All the steps listed below can also be done in AWS console
d) Create a snapshot of the EC2 instance
ec2addsnap -O AWS_KEY -W AWS_SECRET -d `date +"%Y%m%d%H%M%Z"`
e) Register a new AMI from the snapshot
# Find the latest PvGrub kernel offered by Amazon # hd0 and hd00 are the same thing # choose 32-bit kernel or 64 bit kernel based on image # kernel-id looks like "aki-919dcaf8" (1.04-x86_64) ec2-describe-images -O AWS_KEY -W AWS_SECRET -o amazon --filter "name=pv-grub-*.gz" # registering a 64 bit AMI here ec2reg -O AWS_KEY -W AWS_SECRET --kernel KERNEL_ID -a x86_64 -n `date +"%Y%m%d%H%M%Z"` -b "/dev/sda1=SNAPSHOT_ID"
f) Test your new AMI by starting an instance with it
ec2run -O AWS_KEY -W AWS_SECRET -g VPC_SECURITY_GROUP -t INSTANCE_SIZE -s VPC_SUBNET_ID NEW_AMI_ID