The Workshop: Run Amazon Linux Locally

by Joe Ferguson

Amazon Linux 2

Amazon Linux 2 is the current generation of Linux provided by Amazon Web Services (AWS) specifically for use in its ElasticCloud (EC2) platform. Amazon Linux evolved out of the Red Hat Enterprise Linux and CentOS ecosystems. If you have used either of these distributions you will feel right at home. Amazon Linux also has the added benefit of support directly from AWS as well as security updates and optimized kernel versions.

This month we’re going to focus specifically on running Amazon Linux on your local development machine via Vagrant. While you could spin up resources in EC2 and go to town, maybe you’re not quite ready to jump into the deep end. Maybe you want to test your application in a controlled environment to see how it performs on Amazon Linux. Possibly you’re new to AWS and don’t want to burn through all of your free credits? All great reasons to spend some time to create you own local instance to test your application with. We’re going to follow the official documentation Running Amazon Linux 2 as a Virtual Machine On-Premises - Amazon Elastic Compute Cloud which I also recommend as further reading.

Preparing the Virtual Machine

If you’re following along on your own system, you’ll want to ensure you are running the same version of Virtualbox, 6 and Vagrant, 2.2.4 as I used. I’m also building this virtual machine on MacOS but the commands and instructions will be similar for Windows or Linux.

Traditionally Vagrant boxes are very straight forward however there is more set up involved to run Amazon Linux. First we’ll create a folder to contain our files: ~/Code/ec2-vagrant. From there we’ll create a seedconfig folder to hold files we’ll use to create a seed.iso boot image for our virtual machine to boot from initially. The first file we create inside the seedconfig folder will be named meta-data and contain the following single line:

local-hostname: ec2_vagrant

You might have already guessed this is how we set the hostname of the virtual machine during the boot process. If you need to define custom networking configuration you can add information as well:

local-hostname: ec2_vagrant
network-interfaces: |
  auto eth0
  iface eth0 inet static
  address 192.168.1.10
  network 192.168.1.0
  netmask 255.255.255.0
  broadcast 192.168.1.255
  gateway 192.168.1.254

You may not need to specify a network configuration. In my experience the defaults seemed to work.

The next file we’ll add to the seedconfig folder will be named user-data, this file defines how user accounts will be configured in the virtual machine. I recommend you keep this light and simple as you’re starting out. For our examples we’ll set our username to the default ec2-user with the password secret:

*#cloud-config*
*#vim:syntax=yaml*
users:
*# A user by the name `ec2-user` is created in the image by default.*
    - default
chpasswd:
  list: |
    ec2-user:*secret*
*# In the above line, do not add any spaces after ‘ec2-user:’.*

If you needed another user added you could adjust your user-data file to :

users:
    - default # ec2-user
    - name: extrauser
    groups: sudo
    sudo: [‘ALL=(ALL) NOPASSWD:ALL’]
    plain_text_passwd: hunter2
    lock_passwd: false

This will ensure extrauser will be created with the password hunter2 and additionally we’ll set sudo options so the user can run super user commands without needed to specify their passwords. This is a fine practice for local development however I would caution against this on production system in the event a user’s account is compromised.

Now we should have a file structure as such:

.
+— seedconfig
|   +— meta-data
|   +— user-data

Now we’re ready to create our seed.iso bootable image. On MacOS we’ll run:

hdiutil makehybrid -o seed.iso -hfs -joliet -iso -default-volume-name cidata seedconfig/

On Linux you’ll want to run:

genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data

Windows users will require a little more work. You can download the media creation tool, however it might be easier if you already have the Windows Subsystem for Linux installed you could easily use it to run the Linux command to create seed.iso. When the command has finished we should now have seed.iso and seedconfig/ in our ec2-vagrant folder.

The next step is for us to download the [Amazon Linux VM image](Running Amazon Linux 2 as a Virtual Machine On-Premises - Amazon Elastic Compute Cloud. Since we’re using Virtualbox we’ll follow [Index of /os-images/2.0.20190313/virtualbox and download amzn2-virtualbox-2.0.20190313-x86_64.xfs.gpt.vdi (The .vdi filename may change based on the last time it was updated.)

Building the Virtual Machine

Once we have the VM image downloaded we're going to open up Virtualbox and click on the “New” button to create a new virtual machine. We’ll name the VM Amazon Linux, keep the defaults “Machine Folder”, “Type: Linux”, and “Version: Linux”, and click “Continue”. We’ll leave the memory at the default 1024 MB value and click “Continue”. Now we’re looking at the “Hard Disk” configuration and this is where we derail from the traditional virtual machine setup. Since we already have an Amazon Linux .vdi file, we must use it as our hard disk for the virtual machine. We’ll select the “Use an existing virtual hard disk file” and clicking “Add”, then locating the amzn2-virtualbox-2.0.20190313-x86_64.xfs.gpt.vdi file we downloaded. Make sure the file we added is selected and click the “Choose” button. Now we’ll complete this wizard by clicking on the “Create” button. Now we have a shiny new “Amazon Linux” virtual machine in Virtualbox.

Before we fire up our virtual machine we need to do some additional configuration. Clicking on “Settings” will open the “Amazon Linux - General” window. Click on the “Storage” icon and click on the “Empty” CD image under “Controller: IDE”. On the right of the window we should see under “Attributes” a CD icon to the right of “IDE Secondary Master”. Clicking on this CD icon and selecting “Choose Virtual Optical Disk File” will open a window for us to locate our ec2-vagrant/seedf.iso image. Once you have selected the image you should see the “Storage” window similar to this:

Now we need to ensure in the “System” tab the “Boot Order” list has check marks next to “Optical” and “Hard Disk” and “Optical” is above “Hard Disk” so when our virtual machine boots up it will boot off our seed.iso file and then off our amzn2-virtualbox-2.0.20190313-x86_64.xfs.gpt.vdi hard drive.

The last configuration change we need to make is in the “Network” tab. I often run into networking issues with Virtualbox so I often set my “attached to” network adapter to “Bridged Adapter” and select my MacOS WIFI adapter. You will want to select whichever network device is your primary which may be Wifi or Ethernet. Now we’re ready to click “Ok” and have all of our settings applied.

Before we click on the big green “Start” button to fire up our VM I want to caution you: we’re going to log in, install a few packages, and then shut down the system. This is because any changes we make now, will persist when we create our vagrant box. NOW We’re ready to hit the “Start” button and watch our virtual machine come to life. Once the machine is booted you can login with the username ec2-user and password secret:

Now we’re ready to install the Virtualbox Guest Additions. Run sudo yum install -y linux-headers-$(uname -r) build-essential dkms . Once the installation is complete we can shut down the virtual machine with sudo shutdown -h now.

Packaging the Virtual Machine

Once our virtual machine is in the “Powered Off” state we can go back to our console and package our “Amazon Linux” virtual machine into a Vagrant box by running vagrant package --base Amazon\ Linux. This tells Vagrant to look for a virtual machine in Virtualbox named “Amazon Linux” and package the files into a Vagrant box file.

You can read more about packaging virtual machines into Vagrant :Creating a Base Box - VirtualBox Provider - Vagrant by HashiCorp

You should see output similar to:

$ vagrant package --base Amazon\ Linux
==> Amazon Linux: Exporting VM...
==> Amazon Linux: Compressing package to: ~/Code/ec2-vagrant/package.box

$ mv package.box AmazonLinux-2-20190605.box

The last command will rename the generic package.box to something more descriptive, the name of the operating system and the date it was created: AmazonLinux-2-20190605.box . While you can go ahead and add this box file to Vagrant and begin using it, the better practice would be to share it with Vagrant Cloud) . Once you have created an account you can click on “Dashboard” and “New Vagrant Box”. We’ll name our box “AmazonLinux-2”. Next we’ll create a version, 1.0.0 for our example and click “Create Version”. The next step is to click “Add Provider” and select “Virtualbox” from the list. Because we have not added anything to our image needs to be secret we can leave the default “Upload to Vagrant Cloud” option selected and click “Continue to Upload”

If you need to host a private Vagrant box you will need a paid Vagrant Cloud account. Another option is to share boxes on a restricted intranet only your organization has access to, and share URL as the “External URL” option when adding a provider.

On the “Add Provider File” form we’ll select our AmazonLinux-2-20190605.box file and wait for the upload to complete. Once the upload has completed we can click “Update Provider”. Next we need to click on the “Release” button and release our 1.0.0 version so it will be publicly available. You can see the release I created https://app.vagrantup.com/Svpernova09/boxes/AmazonLinux-2/

Now we have shared our newly created Vagrant box we can take it for a test drive by creating a new folder and initializing it with our Vagrant box information:

mkdir AmazonLinuxVagrant
cd  AmazonLinuxVagrant
vagrant init Svpernova09/AmazonLinux-2
vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'Svpernova09/AmazonLinux-2' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'Svpernova09/AmazonLinux-2'
    default: URL: https://vagrantcloud.com/Svpernova09/AmazonLinux-2
==> default: Adding box 'Svpernova09/AmazonLinux-2' (v1.0.0) for provider: virtualbox
    default: Downloading: https://vagrantcloud.com/Svpernova09/boxes/AmazonLinux-2/versions/1.0.0/providers/virtualbox.box
    default: Download redirected to host: vagrantcloud-files-production.s3.amazonaws.com
    default: Progress: 14% (Rate: 8078k/s, Estimated time remaining: 0:01:18)

Now Vagrant will reach out and download the base box we just uploaded to Vagrant Cloud to our local Vagrant system and create a vagrant virtual machine we can easily reuse without having to download or create the base box again.

Conclusion

Now you can easily create your own local Amazon Linux virtual machines and build your applications to run exactly as they would in ElasticCloud. You’re also free to use the box I built as much as you’d like.

Happy Coding!