I Can Has IPv6?

A few months ago, I got my JNCIA-M certification. I had built a small lab on my Dynamips server using Olive. What is Olive? Here is a quote from JuniperClue:

Olive is also the codename name given to JUNOS software running on an PC rather than a Juniper router. A common misconception is that Olive is some sort of “special software”, but it is actually ordinary JUNOS software running on a PC of similar specifications to a Routing Engine, with no forwarding hardware (or PFE) attached. If you took a Routing Engine out of a Juniper router and booted it in a blade server chassis, it would effectively be an Olive.

To get my lab up and running, I basically mashed together instructions both from the Olive page of the JuniperClue wiki, as well as nemith & sartan’s combined effort on the Internetworkpro.org wiki. I rewrote my version specifically to use FreeBSD as the host OS, as that is what I use on my Dynamips server. So, let’s get started…

Dependancies

First and foremost, it is assumed that your guest OS is up and running, with the ports and packages systems both installed and up to date. Setting up your host OS is beyond the scope of this post.

You will want to have ncurses installed. This will essentially allow you to view the FreeBSD VM’s installation GUI through your CLI. Since we don’t need to compile it in any particular way with any special options, we simply install the binaries using FreeBSD’s packages system.

pkg_add -r ncurses

You also want to ensure you have a tap0 interface installed, configured, and turned up.

ifconfig tap0 create
ifconfig tap0 192.168.0.1 netmask 255.255.255.0 up

Compiling and Installing

As root, go to the qemu-devel port directory.

cd /usr/ports/emulators/qemu-devel

Download the FreeBSD’s adopted patch (big thanks to nox!). Note: Check to make sure the patch version matches the port version! Then, apply the patch, and make config. Note: Do not forget to select the JEMU option! Finally, compile and install.

fetch http://people.freebsd.org/~nox/qemu/qemu-devel-jemu-20080620_1.patch
make fetch
make extract
patch < ./qemu-devel-jemu-20080620_1.patch
make config
make install clean

Installing FreeBSD as a Guest

JUNOS is based on FreeBSD and installing FreeBSD is required when setting up an Olive box.

Download FreeBSD 4.11 mini .iso from the FreeBSD FTP site.

Create a new qcow2 disk image to use for our Olive hard drive:

qemu-img create olive-base.img -f qcow2 4G

This creates a new image 4GB in size, which seems to be enough to install and run Olive just fine.

Now we need to start our modified QEMU and boot from the .iso we downloaded. We’ll bind a NIC to our tap0 interface. This will allow us to scp the jinstall image later on.

qemu -m 256 -hda olive-base.img -cdrom 4.11-RELEASE-i386-miniinst.iso -boot d \
-curses -localtime -net nic,macaddr=00:aa:00:00:01:01,vlan=0,model=i82559er \
-net tap,vlan=0,ifname=tap0,script=no

You will now see QEMU come up in curses mode, which doesn’t require X.

Install FreeBSD as you normally would (just install a standard ‘User’ base, and skip installing any services or ports). When you get to partitioning, use the following values:

ad0s1a    /               512M
ad0s1b    swap            1024M
ad0s1e    /config         12M
ad0s1f    /var            <rest>

After the base is installed, it will ask you if you want to use DHCP to configure your NIC. Select NO, and enter:

Hostname: olive
Domain:
Gateway: 192.168.0.1
IP: 192.168.0.2
Mask: 255.255.255.0

Do this now to allow us to scp your jinstall after we reboot. After the installer completes, your guest OS will reboot. Note: Right after it reboots, close or kill your QEMU process or it will boot into the installer CD again.

Installing JUNOS

Now we are ready to install Olive into our QEMU host. You will need to obtain a jinstall version. Use something less than 8.5 to get it to properly work. (8.5 is based on FreeBSD 6 and not FreeBSD 4.11).

Now we need to boot our guest again but this time boot from the image. Again we will attach one nic to the ‘user’ proccess.

qemu -m 256 -hda olive-base.img -boot c -localtime -curses \
-net nic,vlan=0,macaddr=00:aa:00:00:01:01,model=i82559er \
-net tap,vlan=0,ifname=tap0,script=no

After FreeBSD fully boots, login as root and scp the jinstall package onto your guest.

scp 192.168.0.1:/home/ipv6freely/jinstall-8.3R1.5-domestic-signed.tgz /var/tmp

Modify jinstall

JunOS image after 7.4 version has a binary called checkpic. This binary will fail and the image cannot be installed. Replacing this binary with /usr/bin/true fixes the issue.

cd /var/tmp
mkdir jinst-signed
cd jinst-signed
tar zxvf ../jinstall-8.3R1.5-domestic-signed.tgz

mkdir jinst
cd jinst
tar zxvf ../jinstall-8.3R1.5-domestic.tgz

mkdir pkgtools
cd pkgtools
tar zxvf ../pkgtools.tgz
cd bin
cp /usr/bin/true ./checkpic
cd ..

tar zcvf ../pkgtools.tgz *
cd ..
rm -rf pkgtools

tar zcfv /var/tmp/jinstall-8.3R1.5-domestic-olive.tgz *

Installing Olive

Install your new modified jinstall package.

pkg_add -f /var/tmp/jinstall-8.3R1.5-domestic-olive.tgz

After this finishes issue the halt command and kill your QEMU session window once olive has finished its shutdown process.

First Boot

The jinstall above really just installed a bootstrap enviroment so you need to boot up the guest OS one more time to finish the installation (depending on your version). If you just restarted your guest above you will notice that you will get no output on your screen. This is because a real Juniper router has no VGA out and redirects everything to the serial port. No worries for us since QEMU will redirect the serial port to either stdio or a telnet port.

Start up our guest one more time this time getting rid of the GUI and redirecting the console to a telnet port. Please leave the memory at 256 for this process. After this first boot you can get by with a lot less memory, but we need it now.

qemu -m 256 -hda olive-base.img -boot c -localtime -nographic \
-serial stdio

Now wait while the bootstrap process completes. The virtual olive will reboot itself automatically and nothing is needed. At the end of this process we will be sitting at a login prompt. Login as root and issue the halt command and kill your QEMU proccess, as before.

Now you have a base olive hard drive image. QEMU allows you to use this as a base for other harddrive issues and only writing the changes to your ’slave’ images saving on disk space!

Running Your Olive Router

Creating a router instance

Now we will create a new hard drive image off of your base image above. Repeat for all your routers you want to emulate

qemu-img create -b olive-base.img -f qcow2 r1.img

Now you can start your router with the following

qemu r1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \
-net nic,macaddr=00:aa:00:60:00:01,model=i82559er \
-net socket,listen=:6000

Telnet to localhost port 2001 to connect to your new router and start configuring!

Networking your routers

Socket mode creates a tcp stream between two qemu instances with one a client and the other a server.

qemu R1.img -m 96 -nographic -daemonize -serial telnet::2001,server,nowait -localtime \
-net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er \
-net socket,vlan=1,listen=:6000 

qemu R2.img -m 96 -nographic -daemonize -serial telnet::2002,server,nowait -localtime \
-net nic,vlan=1,macaddr=00:aa:00:60:00:01,model=i82559er \
-net socket,vlan=1,connect=127.0.0.1:6000

You should then be able to telnet to both R1 and R2, and configure them to speak to each other.

Building a Lab

Now that we have Olive working, we should make use of it by building a full lab.

Lab Diagram

Router Configuration File

First, we make a file named router_conf that contains the parameters for each instance of QEMU, including the socket information to ensure our routers are properly connected together. This handy config file was given to me by nemith, and heavily edited to suit my lab:

#
# Custom varibles here
#
PIDDIR=./run
IMGDIR=./images

#QEMU execuable path
QEMU=qemu

#
# List of routers (must exist!)
#
ROUTERS="r1 r2 r3 r4 r5 r6 r7"

#
# r1
#
function run_r1 {
	nice -19 \
	$QEMU \
	$IMGDIR/r1.img \
	-pidfile $PIDDIR/r1.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::5001,server,nowait \
	-net nic,vlan=0,macaddr=00:aa:60:00:01:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
	-net nic,vlan=1,macaddr=00:aa:60:01:01:01,model=i82559er \
	-net socket,vlan=1,listen=:6001 \
	-net nic,vlan=2,macaddr=00:aa:60:02:01:02,model=i82559er \
	-net socket,vlan=2,listen=:6002 \
	-net nic,vlan=3,macaddr=00:aa:60:03:01:03,model=i82559er \
	-net socket,vlan=3,listen=:6003 \
	&
}

function stop_r1 {
	killpidfile $PIDDIR/r1.pid
}

#
# r2
#
function run_r2 {
 	nice -19 \
	$QEMU \
	$IMGDIR/r2.img \
	-pidfile $PIDDIR/r2.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::2002,server,nowait \
    	-net nic,vlan=0,macaddr=00:aa:60:00:02:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
    	-net nic,vlan=1,macaddr=00:aa:60:01:02:01,model=i82559er \
	-net socket,vlan=1,connect=127.0.0.1:6001 \
    	-net nic,vlan=2,macaddr=00:aa:60:04:02:02,model=i82559er \
	-net socket,vlan=2,listen=:6004 \
    	-net nic,vlan=3,macaddr=00:aa:60:05:02:03,model=i82559er \
	-net socket,vlan=3,listen=:6005 \
    	-net nic,vlan=4,macaddr=00:aa:60:03:02:04,model=i82559er \
	-net socket,vlan=4,connect=127.0.0.1:6003 \
	&
}

function stop_r2 {
	killpidfile $PIDDIR/r2.pid
}

#
# r3
#
function run_r3 {
	nice -19 \
	$QEMU \
	$IMGDIR/r3.img \
	-pidfile $PIDDIR/r3.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::2003,server,nowait \
    	-net nic,vlan=0,macaddr=00:aa:60:00:03:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
    	-net nic,vlan=1,macaddr=00:aa:60:02:03:01,model=i82559er \
	-net socket,vlan=1,connect=127.0.0.1:6002 \
    	-net nic,vlan=2,macaddr=00:aa:60:05:03:02,model=i82559er \
	-net socket,vlan=2,connect=127.0.0.1:6005 \
   	-net nic,vlan=3,macaddr=00:aa:60:06:03:03,model=i82559er \
	-net socket,vlan=3,listen=:6006 \
    	-net nic,vlan=4,macaddr=00:aa:60:07:03:04,model=i82559er \
	-net socket,vlan=4,listen=:6007 \
 	&
}

function stop_r3 {
	killpidfile $PIDDIR/r3.pid
}

#
# r4
#
function run_r4 {
	nice -19 \
	$QEMU \
	$IMGDIR/r4.img \
	-pidfile $PIDDIR/r4.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::2004,server,nowait \
    	-net nic,vlan=0,macaddr=00:aa:60:00:04:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
    	-net nic,vlan=1,macaddr=00:aa:60:04:04:01,model=i82559er \
	-net socket,vlan=1,connect=127.0.0.1:6004 \
   	-net nic,vlan=2,macaddr=00:aa:60:07:04:02,model=i82559er \
	-net socket,vlan=2,connect=127.0.0.1:6007 \
    	-net nic,vlan=3,macaddr=00:aa:60:08:04:03,model=i82559er \
	-net socket,vlan=3,listen=:6008 \
	&
}

function stop_r4 {
	killpidfile $PIDDIR/r4.pid
}

#
# r5
#
function run_r5 {
	nice -19 \
	$QEMU \
	$IMGDIR/r5.img \
	-pidfile $PIDDIR/r5.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::2005,server,nowait \
    	-net nic,vlan=0,macaddr=00:aa:60:00:05:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
    	-net nic,vlan=1,macaddr=00:aa:60:09:05:01,model=i82559er \
	-net socket,vlan=1,listen=:6009 \
    	-net nic,vlan=2,macaddr=00:aa:60:10:05:02,model=i82559er \
	-net socket,vlan=2,listen=:6010 \
    	-net nic,vlan=3,macaddr=00:aa:60:08:05:03,model=i82559er \
	-net socket,vlan=3,connect=127.0.0.1:6008 \
    	-net nic,vlan=4,macaddr=00:aa:60:06:05:04,model=i82559er \
	-net socket,vlan=4,connect=127.0.0.1:6006 \
	&
}

function stop_r5 {
	killpidfile $PIDDIR/r5.pid
}

#
# r6
#
function run_r6 {
	nice -19 \
	$QEMU \
	$IMGDIR/r6.img \
	-pidfile $PIDDIR/r6.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::2006,server,nowait \
    	-net nic,vlan=0,macaddr=00:aa:60:00:06:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
    	-net nic,vlan=1,macaddr=00:aa:60:09:06:01,model=i82559er \
	-net socket,vlan=1,connect=127.0.0.1:6009 \
    	-net nic,vlan=2,macaddr=00:aa:60:10:06:02,model=i82559er \
	-net socket,vlan=2,listen=:6011 \
	&
}

function stop_r6 {
	killpidfile $PIDDIR/r6.pid
}

#
# r7
#
function run_r7 {
	nice -19 \
	$QEMU \
	$IMGDIR/r7.img \
	-pidfile $PIDDIR/r7.pid \
	-m 96 -nographic -daemonize -localtime \
	-serial telnet::2007,server,nowait \
	-net nic,vlan=0,macaddr=00:aa:60:00:07:00,model=i82559er \
	-net socket,vlan=0,mcast=239.194.06.1:6000 \
	-net nic,vlan=1,macaddr=00:aa:60:09:07:01,model=i82559er \
	-net socket,vlan=1,connect=127.0.0.1:6011 \
	-net nic,vlan=2,macaddr=00:aa:60:10:07:02,model=i82559er \
	-net socket,vlan=2,connect=127.0.0.1:6010 \
	&
}   

function stop_r7 {
    killpidfile $PIDDIR/r7.pid
}

function stop_r7 {
    killpidfile $PIDDIR/r7.pid
}

Startup Script

We then add a script to read from the configuration file, and start up the routers. In it’s original form, the script ran so fast that it would throw errors and not start all the routers. I’ve added a sleep command into the script to slow it down a bit.

#!/usr/local/bin/bash

source router_conf

function start_router {
        echo "Starting router $1"
        run_$1 || echo "ERROR: No start configuration found for $1"
}

function stop_router {
        echo "Stoping router $1"
        stop_$1 || echo "ERROR: No stop configuration found for $1"
}

function start_all_routers {
        for router in `echo $ROUTERS`; do
                start_router $router
                echo "waiting 2 seconds"
                sleep 2
    done
}

function stop_all_routers {
    for router in `echo $ROUTERS`; do
                stop_router $router
        done
}

function killpidfile {
        if [ -e $1 ]; then
                kill `cat $1`
                rm $1
        else
                echo "Pidfile $1 not found"
        fi
}

function print_help {
        echo "$0 [start|stop|restart] [|all]"
}

case "$1"  in
        'start')
                if [ "$2" == "all" ]; then
                        start_all_routers
                else
                        start_router $2
                fi
                ;;
        'stop')
                if [ "$2" == "all" ]; then
                        stop_all_routers
                else
                        stop_router $2
                fi
                ;;
        'restart')
                if ["$2" == "all" ]; then
                        stop_all_routers
                        start_all_routers
                else
                        stop_router $2
                        start_router $2
                fi
                ;;
        *)
                print_help
                ;;
esac

Now, simply run the startup script, telnet to your routers, and voila!

host# ./labctl.sh start all
Starting router r1
waiting 2 seconds
(qemu) Starting router r2
waiting 2 seconds
(qemu) Starting router r3
waiting 2 seconds
(qemu) Starting router r4
waiting 2 seconds
(qemu) Starting router r5
waiting 2 seconds
(qemu) Starting router r6
waiting 2 seconds
(qemu) Starting router r7
waiting 2 seconds

host# telnet localhost 2001
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
<blah blah blah>
fxp0: Ethernet address 00:aa:60:00:01:00
fxp1: Ethernet address 00:aa:60:01:01:01
fxp2: Ethernet address 00:aa:60:02:01:02
fxp3: Ethernet address 00:aa:60:03:01:03
<more blah blah blah>
R1 (ttyd0)

login: root
Password:

--- JUNOS 8.3R1.5 built 2007-04-13 22:23:02 UTC

root@R1% cli
root@R1> show version
Hostname: R1
Model: olive
JUNOS Base OS boot [8.3R1.5]
JUNOS Base OS Software Suite [8.3R1.5]
JUNOS Kernel Software Suite [8.3R1.5]
JUNOS Crypto Software Suite [8.3R1.5]
JUNOS Packet Forwarding Engine Support (M/T Common) [8.3R1.5]
JUNOS Packet Forwarding Engine Support (M20/M40) [8.3R1.5]
JUNOS Online Documentation [8.3R1.5]
JUNOS Routing Software Suite [8.3R1.5]

root@R1>

Now, get labbing!