COLUG xen talk on developing an appliance in xen ================================================================ http://www.colug.net/ Copyright 2007 Owl River Company info@owlriver.com ================================================================ We have some 'old friend' tools predating mrepo requires: distribution ISOs, install-from-iso.sh ORCsetPXE and the driver file for /etc/ORC/installCDrc-centos_i386_50 and a running vsftpd TBD: we need to add a better failure indicator for a missing 'createrepo' in install-from-iso.sh Mount the ISOs thus: ORCsetPXE centos_i386_50 This leaves a wire install path like: ftp://10.16.1.231/pub/install/ftpinstall Remember at the server to get FTP opened; for the present make it wide open as we are in an isolated network segment /sbin/service iptables stop /sbin/service iptables save /sbin/service ip6tables stop /sbin/service ip6tables save /sbin/chkconfig iptables off /sbin/chkconfig ip6tables off Add an Update mirror: optional, requires: lftp and a mirror: list at: http://www.centos.org/modules/tinycontent/index.php?id=13 eg: http://mirrors.jtlnet.com/centos/5/updates/i386/ ftp://mirrors.kernel.org/centos/5/updates/i386 in file: /root/centos-5-updates.conf mirror -c -e \ ftp://mirrors.kernel.org/centos/5/updates/i386 \ /var/ftp/pub/mirror/centos/5/updates/i396 and keep it updated: lftp -f /root/centos-5-updates.conf Also a running dhcpd is needed, and we add a named as well. Fixup base, updates vs local archive Munge the /etc/yum.repos.d/CentOS-Base.repo to point to the localhost or the domU_0 host IP Do the end user add, passwd, and sudo setup Kill off some evil stuff, which interferes with rpm and yum: /sbin/chkconfig yum-updatesd off /sbin/service yum-updatesd stop Irritants with xen installs and virt-manager ====================================== Silent selinux interactions not caught: non-recovery (silent failure) when virt-manager cannot read or write something same (except noisy) when it cannot get an X display Until we found the fixup as to SELinux permissions, the host dom0 had to be in a selinux permissive or less box, Turn off SELinux enforcement temporarily thus: sudo /usr/sbin/setenforce 0 Correct fix, for running dom0 in full enforcing SELinux mode and the dom0 in full enforcing SELinux mode, is to set the file contexts for the image directory (here we use: /var/xen) correctly: # semanage fcontext -a -t xen_image_t "/var/xen(/.*)?" # restorecon -R /var/xen A tip 'o the hat to: http://wiki.centos.org/TipsAndTricks/Xen Error in doco as to how to start the domU_1 after building it correct is: xm create -c domU_name Nit: NOT right is: xm -c create domU_name even though one expects seeming options to precede operators RFE: inability to inventory available domU's - ls is out for security reasons -- tool should do it safely For the demo ====================================== Retrieve the builder script and turn it loose (we actually have a local copy at the interior FTP server) wget ftp://ftp.owlriver.net/pub/local/trading-shim/shim-builder.sh Tips ====================================== Control sequences for magic keystrokes: ctrl-ctrl-ctrl atl-F2 and such multiple running instances in TUI consie and GUI virt-manager shell TBD: in scripting punch in new yum configs punch in user setup and sudo pull in updates Hints on the client: ====================================== Remote syslogging: at the client: sudo joe /etc/syslog.conf adding: [herrold@dhcp-233 ~]$ grep '@' /etc/syslog.conf *.* @10.16.1.231 sudo /sbin/service syslog restart and on the remote receiving unit (at 10.16.1.231): /etc/sysconfig/syslog SYSLOGD_OPTIONS="-m 0 -r " and restart the syslog here to pick up the option sudo /sbin/service syslog restart Carry over questions ====================================== Q: What are tap devices for? A: on the client (domU) side, there are none: herrold@dhcp-233 ~]$ /sbin/ifconfig eth0 Link encap:Ethernet HWaddr 00:16:3E:14:B8:C3 inet addr:10.16.1.233 Bcast:10.16.1.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe14:b8c3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:87292 errors:0 dropped:0 overruns:0 frame:0 TX packets:37203 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:132533479 (126.3 MiB) TX bytes:5339183 (5.0 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:35 errors:0 dropped:0 overruns:0 frame:0 TX packets:35 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:11638 (11.3 KiB) TX bytes:11638 (11.3 KiB) on the dom0 side, there are several: [herrold@dhcp-231 ~]$ /sbin/ifconfig eth0 Link encap:Ethernet HWaddr 00:19:D1:E8:2C:B5 inet addr:10.16.1.231 Bcast:10.16.1.255 Mask:255.255.255.0 inet6 addr: fe80::219:d1ff:fee8:2cb5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5688302 errors:0 dropped:0 overruns:0 frame:0 TX packets:3175206 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3226846406 (3.0 GiB) TX bytes:1715987907 (1.5 GiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:319339 errors:0 dropped:0 overruns:0 frame:0 TX packets:319339 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1880229587 (1.7 GiB) TX bytes:1880229587 (1.7 GiB) peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:5652110 errors:0 dropped:0 overruns:0 frame:0 TX packets:3495837 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:3305144405 (3.0 GiB) TX bytes:1370412669 (1.2 GiB) Base address:0xecc0 Memory:dfde0000-dfe00000 vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:3175234 errors:0 dropped:0 overruns:0 frame:0 TX packets:5688323 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1715993499 (1.5 GiB) TX bytes:3226847792 (3.0 GiB) vif6.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:37223 errors:0 dropped:0 overruns:0 frame:0 TX packets:87313 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:32 RX bytes:4821189 (4.5 MiB) TX bytes:132534859 (126.3 MiB) vif8.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:1165 errors:0 dropped:0 overruns:0 frame:0 TX packets:6700 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:32 RX bytes:111977 (109.3 KiB) TX bytes:1976089 (1.8 MiB) xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 RX packets:5484 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:267138 (260.8 KiB) TX bytes:0 (0.0 b) Later with several domU running: [root@dhcp-231 xen]# ifconfig | grep encap eth0 Link encap:Ethernet HWaddr 00:19:D1:E8:2C:B5 lo Link encap:Local Loopback peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif6.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif8.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif12.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif13.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif14.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF vif15.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF Q: How is console logging done? A: logins happen through at least these routes: network, local ttyN, and xvcN [herrold@dhcp-233 ~]$ w 09:34:41 up 16:53, 3 users, load average: 0.00, 0.12, 0.36 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT herrold xvc0 - Tue18 14:40m 0.01s 0.01s -bash root tty1 - Tue16 16:46m 0.00s 0.00s -bash herrold pts/0 centos-4.first.l Tue18 0.00s 0.04s 0.00s w ... I misread this -- 'keystroke logging' was the question -- TUI: Adding an EAS (Enteprise Audit Shell) ftp://ftp.owlriver.com/pub/mirror/ORC/eas/ This is the last version of the GPL sources, before they were bought and went closed. see also: http://sourceforge.net/projects/rootsh GUI: As to VNC connections, there are vnc recording proxies, but I don't know how out pull non-visible keystrokes out of that -- the scancodes may be on the wire ================================================================= See also XenSource going closed once bought by Citrix a week ago The Register XenSource calls VMware a cash-hogging automobile Open source explained through car talk, bandits By Austin Modine in San Francisco → More by this author Published Wednesday 8th August 2007 00:46 GMT LinuxWorld XenSource CTO Simon Crosby wishes all the best for the folks at VMware. But he has one simple request; please stop hogging all the money. Open source virtualization has a long way to catch up with VMware's slice of market pie, but Crosby assured a panel at LinuxWorld the industry will be a happier place if open code does keep pace. So why is open source better for everyone? Crosby expounds using that time-tested old chestnut, the automobile metaphor; Xen sells an engine. VMware sells a car. ------------------------------------------------- CNet August 15, 2007 7:29 AM PDT Citrix to buy XenSource for $500 million; open-source company valuations skyrocketing (UPDATED) Posted by Matt Asay Wow. The ink was barely dry on my critique of Tim O'Reilly's position on whether proprietary companies will buy up the open-source companies, and along comes the news that Citrix is buying XenSource. It's a good technology fit, but Citrix would have been one of the last companies I would have accused of a predilection for open source. ------------------------------------------------ http://www.xensource.com/Pages/Download.aspx and all the FOSS versions have already disappeared http://www.xensource.com/download/index_3.0.3.html is a dead link ;( ================================================================= Q: How much memory does DOM0 use? require? Why doesn't the domU memory usage impact dom0? A: In reverse order, the domU _do_ eat away at the dom0 available memory. Each domU virtual machine gets a memory space, but by observation, they flutuate as load changes {demo}. A given dom0 can be dialled up and down with a grub.conf option at initial boot time [herrold@dhcp-231 ~]$ sudo cat /boot/grub/grub.conf ... title CentOS (2.6.18-8.1.8.el5xen) root (hd0,0) kernel /xen.gz-2.6.18-8.1.8.el5 dom0_mem=2048M module /vmlinuz-2.6.18-8.1.8.el5xen ro root=LABEL=/ module /initrd-2.6.18-8.1.8.el5xen.img ... [herrold@dhcp-231 ~]$ free total used free shared buffers cached Mem: 2095872 2087028 8844 0 194724 1670128 -/+ buffers/cache: 222176 1873696 Swap: 8193140 176 8192964 and so the dom0 has that view of available ram, in this box with 4G The domU are configured as to available ram at build time, here with 512 max, and sees: [herrold@dhcp-233 ~]$ free total used free shared buffers cached Mem: 512172 292616 219556 0 44000 176692 -/+ buffers/cache: 71924 440248 Swap: 622584 4 622580 Q: How is locking handled for multiple access to the vm's? A: It is handled in the dom0 start up code --- trying to start a second instance results in: [root@dhcp-231 xen]# xm create russlocal Using config file "./russlocal". Going to boot CentOS (2.6.18-8.el5xen) kernel: /vmlinuz-2.6.18-8.el5xen initrd: /initrd-2.6.18-8.el5xen.img Error: VM name 'russlocal' already in use by domain 8 [root@dhcp-231 xen]# This is probably detected with the UUID; we can test this speculation. Q: Can /etc/xen be moved? A. JAW: (turns out it is not needed since the xend's transfer the config files during migration) A: as to whether the path is hard coded, there is clearly that default -- there is an option to pull config files from other file paths, with the '-f' option to: xm create and also with a --path option for locations of configuration scripts. Q: How does migration work? A. Don't know yet -- who will present? Limits and nitpicking ================================================================= With 7 running domU, and a dom0, in 4G or ram, and dom0 locked to a max of 2G, I get; libvir: Xen Daemon error : POST operation failed: (xend.err 'Error creating domain: I need 512000 KiB, but dom0_min_mem is 262144 and shrinking to 262144 KiB would leave only 225400 KiB free.') Failed to create domain russ7 so eating away the dom0 max happens, and setting a limit is perhaps not a good idea to get max creation -- re-test without the mem restriction Poor (and non-existent from unhandled exceptions) error messages by virt-manager Prep by starting 6 domU's: cd /etc/xen/ for i in `seq 1 6`; do xm create russ${i} ; done Then try to use virt-manager to start another; this failure: libvir: Xen Daemon error : POST operation failed: (xend.err "Error creating domain: (22, 'Invalid argument')") Failed to create domain russ7 MAY have arisen from a prior failure existing: -rwxrwxr-x 1 root root 2097152000 Aug 22 10:50 russ7 -rwxrwxr-x 1 root root 2097152000 Aug 22 10:58 russlocal [root@dhcp-231 xen]# rm russ7 rm: remove regular file `russ7'? y or perhaps not -- on the retry: 2000+0 records in 2000+0 records out 2097152000 bytes (2.1 GB) copied, 34.6545 seconds, 60.5 MB/s libvir: Xen Daemon error : POST operation failed: (xend.err "Error creating domain: (22, 'Invalid argument')") Failed to create domain russ7 But, we can still start one more domU, so it is not a ram constraint or resources issue. Who knows why it failed? (starting russlocal) worked fine. Cheatsheet for first part of the demo ================================================================= ### locally as root: # useradd herrold cd ~herrold ( echo "cd /pub/local/COLUG/" ; echo "mget *" ; echo "quit" ) | \ lftp ftp.trading-shim.com chown herrold.herrold ~herrold/* # # point to a local archive to speed things up mkdir /etc/yum.repos.d/attic \cp /etc/yum.repos.d/* /etc/yum.repos.d/attic # \cp ~herrold/CentOS-Base.repo-localhost \ /etc/yum.repos.d/CentOS-Base.repo sed -i -e 's/localhost/10.16.1.231/g' /etc/yum.repos.d/CentOS-Base.repo sed -i -e 's/i386/x86_64/g' /etc/yum.repos.d/CentOS-Base.repo sed -i -e 's/gpgcheck=1/gpgcheck=0/' /etc/yum.repos.d/CentOS-Base.repo # # as we are running locally, it is inexpensive to do the following # demonstrate fast yum upgrades from a local mirror yum -y clean all yum -y upgrade # passwd herrold # yum -y install joe joe /etc/sudoers # ### log out and in as herrold: # chmod 755 *.sh ./shim-builder.sh Presentation part 2 is at: /pub/local/trading-shim/tools/README on server ftp://ftp.trading-shim.com/ ----------------------------------------------------------------- http://wiki.xensource.com/xenwiki/HVM_Compatible_Processors link: NOT Compatible Processors * Intel® 2 Core(TM) Duo processor E4300 (source) * Intel® Pentium® D processor 9x5 (D945) --------------------- centos-4:~/temp/xen-COLUG-Aug2007.txt $Id: xen-COLUG-Aug2007.txt,v 1.4 2007/08/28 19:55:57 root Exp root $