Flock 2015 – Report Day0 and Day1

Day 0:

I almost missed my flight at the san jose airport, My flight was supposed to take off by 2:55. There was a big queue at the airport around 2:30 pm.  I cleared TSA around 2:40 pm. It was tight!

Met  Major Haydon(major.io) at the Chicago midway (MDW) airport. Major was a keynote speaker at flock. He works for rackspace. The flight from chicago landed about 12:45 am in the morning at Rochester. We took a cab to the hotel. The hotel is 10 minutes from the airport.  My roommate aditya was jetlagged and was already sleeping. Managed to slip into  the bed without waking him up!

Day 1:

FPL (fedora project leader) Matt Miller did his usual state of the union in the morning. As the name implies it is about the current state of fedora project. More details about the state of union can be seen at http://fedoramagazine.org/state-fedora-2015-edition/

Koschei – Continuous integration for fedora

The next talk i attended was Koschei – Continuous integration for fedora packages by mikolaj. koschei is a CI  system that schedules koji builds and make sure that fedora packages are sane all the time. https://github.com/msimacek/koschei. koschei uses koji scratch builds. The scheduler of koschei is intelligent enough to schedule these builds when koji slaves are not busy.

The main motivation of koschei is to find FTBFS early and inform maintainers. The production instance of koschei is here https://apps.fedoraproject.org/koschei. More details about this presentation is here https://github.com/msimacek/koschei/blob/master/doc/pp/koschei.tex

Koji 2.0

Mike McLean (https://fedoraproject.org/wiki/User:Mikem) wrote the first line of code when dinosaurs were still alive :). Now he has plans to clean up koji under koji 2.0 project. koji is used to build packages for fedoraproject, it has many roles (rpm building, compose etc).

Mike wants to use python 2.6 for koji-2.0. It will also has support for python 3.0 using python-six. Most of the audience including me suggested that python 2.7 as it is the latest stable in 2.x release. Mike explained that he wants to support koji-client on RHEL6 which comes with python 2.6.

Luke Macken in the audience went even further and suggested that the server side of koji  should drop support for python 2.x and entirely written using python 3.0.

some of the koji 2.0 proposed features include,

•  build namespaces (re-building same nvr again and again)
• json-rpc
• content generators (https://fedoraproject.org/wiki/koji/ContentGenerators)
  •  other type of build process to feed into koji
  • robust metadata import
•  Following build types are proposed
  • rpm builds
  • maven builds
  • windows builds
  • image builds
  • + ???

Mike’s email about koji2.0 to koji-devel mailing list is available here https://lists.fedorahosted.org/pipermail/koji-devel/2015-June/000000.html

His talk sides are available here https://mikem.fedorapeople.org/Talks/flock-2015-koji-2.0/

Reproducible builds using koji

Reproducing koji builds was scheduled at 2:30 pm, It was the 3rd talk of the day.

In this talk Mike Mclean talked about debian’s reproducible build project
https://wiki.debian.org/ReproducibleBuilds. Builds are not (binary) reproducible because of following reasons,

•  Timestamps embedded into binaries during build time
• Usage pseudo-random numbers to generate code data
• Umask/uid
• uname, hostname, username
• locale
• Timezone

Mike was really appreciative of the debian reproducible build project. The debian team is actively upstreaming their patches. Some one in the audience noted that debian still allows builds created on developer workstations to be uploaded and deployed to repositories. So by having reproducible builds the binaries are easily verifiable.

Then Mike went on to talk about what it would take to make builds reproducible on koji He talked about using task-id/repo-id to preserve the state of repo’s and recreating them at a later point.  He also noted that the rpm metadata included in the rpm package makes it impossible to reproduce.

One of the audience suggested that the metadata could be  moved out of the
package in future to enable reproducible builds. Mike noted that he is very busy with  koji-2.0 work and does not want to spend more time on making builds  reproducible. However he was open to helping out someone who is willing to take on this challenge. Any takers?

Super privileged containers

The last talk of the day i attended was about ‘super privileged containers’ by Dan Walsh. Dan, showed lot of funny gif’s about selinux and docker before starting the presentation.His presentation is available here https://dwalsh.fedorapeople.org/Presentations/SPC/

RedHat’s atomic host doesn’t support yum install. Redhat customers often want some utlity to be included in the atomic host and Redhat wants atomic host to be minimal as possible. As you one can see these two goals are competing with each other. The current rule to include an utlity in the atomic host is to prove that it won’t work in a container.

Customers want to ship an application that will manage a host or manage other containers. Enter Super privileged container aka SPC.
A super privileged container must have the following

• It should  be a privileged container
• will enable all capabilities (CAP_*) in the container
• disable selinux separation (it will lie in the container)
• disable user namespace;
• disable mounting read only file systems;
• Allow creation of linux devices.
• Specific namespaces like network, ipc and pic should be disabled
  respectively using (‘–net=host;–net=ipc,–pid=host)
• SPC should mount /run into /run of the container and let container process to communicate with system dbus, systemd, or even docker daemon (docker run -v /run:/run)
• The entire host file system should be shared inside the container using
  docker run -v /:/host -e HOST=/host.

To do all these, you have to run a big docker command:
"/usr/bin/docker run -t -i --rm --privileged -v /:/host -v /run:/run -v
/etc/localtime:/etc/localtime --net=host --ipc=host --pid=host -e HOST=/host
-e NAME=fedora-spc -e IMAGE=fedora fedora /bin/sh"

As you can see this is a big command, redhat has introduced a ‘rheltools’ container image with project atomic. This tools image includes strace,gdb,sosreport and other tools The atomic command now allows users to run containers in SPC mode.

'atomic run --spc rheltools /bin/sh'

The big docker command now is encapsulated into a small atomic command.

Atomic command wraps os-tree as well,
– atomic host upgrade
– atomic host rollback
– atomic host status

Today there is not a good way to tell your users how to run the container you created. Some container may need special privileges for example ntpd needs –cap_add SYS_TIME; Without SYS_TIME ntpd container will break; To solve this problem redhat has introduced container image labels. Redhat added labels patch to docker which allows developer to create labels during container build time.'LABEL RUN docker -d -n ntpd --cap_add SYS_TIME IMAGE'

Now, ‘atomic run ntpd’ will automatically read this image json metadata and run the container appropriately.

Dan also distributed his container coloring book at the talk. If you want a pdf version of it please see http://bit.ly/1KuB1c6 (pdf). If you haven’t see his selinux coloring book checkout http://bit.ly/1K4Kueu. These books are designed by mairin duffy.

After the conference, we had a game night where we played board games until 11 pm

Note to self: Resizing byobu to terminal size (tmux)

Often byobu reattaches to old session where the tmux windows are smaller than the terminal size

To fit the tmux window to terminal size run

Ctrl+a :attach -d

Reference: https://superuser.com/questions/905954/tmux-detaches-from-one-side-when-force-to-adjust-redimension

Contributing to the openstack project

I have been busy at work  for the past few weeks trying to get a big picture of how openstack works and how all these projects fit together. Especially my focus has been on the bare metal project ironic. I was reading the code and trying to get the dev environment working. In the process i noticed that the python-ironicclient wasn’t working properly as advertised. Immediately i filed a bug and posted a patch. To my surprise, this patch turned out to be my first contribution to the project.

Here is my first ever review request to the project https://review.openstack.org/#/c/131307 . So far it has been fun to read the code. and learn about new python stuff like  ‘tox‘, ‘pbr‘, ‘eventlet and greenthreads‘. Thanks to Yahoo and the management team for letting me contribute. Thanks to ruby,deva, jroll, and all other reviewers for your excellent support! and reviews. I hope i can do more in the future.

Looking for Linux kernel engineers at Yahoo

About the Role

Yahoo’s Linux Operating System team is looking for a Systems Engineer with kernel development experience. This job involves building, patching, performance evaluation and tuning of the Linux kernel. The candidate will be involved in hardware validation of Linux releases and participation in upstream and vendor communities. The overall goal is to support infrastructure needs by providing a secure and scalable operating system for cloud and non-cloud properties to build on.

Please apply for the position here http://j.rfer.us/YHODt-BAD (referral link). Feel free to contact me regarding the position at sagarun@fedoraproject.org

Conference Report – Flock 2014

This years Flock conference was held at Prague, czech republic. This was my first trip to the Europe. I needed a Visa for this trip unlike my American friends. I got the visa in the last minute from the Consulate of czech republic, Los Angeles. The Visa officer needed an insurance of minimum 50000 Euro with Medical reparation and Evacuation converge.  My company Yahoo was able to get that sorted in time.  I attended lot of talks and workshops at Flock. I took notes on some of the sessions i attended. Here is my conference report

Status of COPR build service – by Miroslav suchy

 

https://fedorahosted.org/copr/wiki/UserDocs
https://copr.fedoraproject.org/

COPR is an automatic build system to build rpms. COPR allows users to select
Arch and system (target) , accepts src.rpm from the user and generates binary
rpms in the backend and creates repo as well.

Unlike koji COPR doesn’t need a ‘fas’ account to build rpms. Technically any one
can build rpms on COPR.

Due to public nature of COPR it uses Virtual Machines to build rpms. A virtual machine
is setup and mock is used inside the VM to build the rpm.

COPR currently runs on openstack. There are 1381 projects, 25k builds, 250 G of data,
and 1 TB/month data transfered in COPR as we speak. Koji/OBS was evaluated to use in
COPR but the decided against it for some reasons. OBS signing daemon might be used
with COPR to sign rpms in future.

* Mock is kind of slow, there is a GSOC project to speed it up using LVM snapshots *
* Radek Holy is working on docker for rpm builds *

It is important to note that redhat software collections are built on COPR. There
is a jenkins plugin available for COPR which lets users to trigger COPR builds
from jenkins. There is a copr-cli available to builds.

ARM architecture support, package signing are in future TODO.

Here is the video of the talk

 

 

UEFI – The great satan and you – by Adam

 

I am a fan of UEFI. I have been closely following UEFI development and support in Linux for a while. If you do not know about UEFI, Adam Williamson has an impressive write up about it at https://www.happyassassin.net/2014/01/25/uefi-boot-how-does-that-actually-work-then/

Adam started with what is UEFI and then moved on to explain how older BIOS works. Adam’s talk focused on Desktop machines

How BIOS work?

– Boots the 1st sector of disk
– Chainloading
– The bootloader is sneaked in between MBR and the partition

UEFI

– Defines an EFI executable format.
– EFI executable is copied into FAT filesystem and the firmware can read it
– UEFI boot manager is used to change the boot order and EFI variables.
– There is a fallback path if the EFI executable is not found on the specified path
– Supports BIOS mode named CSM. CSM is going away soon.

Adam proposed following tests to tell if your machine is UEFI?

– Machine is Windows 8 pre-installed. Then it is must have UEFI in it.
– The “firmware” has mouse support then it is UEFI (BIOS can’t do that sh*t)

Adam showed some screenshots of crazy UEFI firmware UI implementations that makes identifying it more difficult for the user.

While multibooting adam asked the users to install both OS in same mode. Mixing BIOS (CSM) and UEFI is discouraged and unsupported in Fedora.

Adam then proposed following special commands to write a USB stick with EFI support

– dd: use dd on usb sticks
– livecd-iso-to-disk: pass –efi –format -reset-mbr
– liveusb-creator: well..it might work
– DO NOT use Unetbootin

Adam then revealed that, Peter Jones and Matthew Garret lobbied Microsoft to enable option to disable secure boot. They even have weekly calls. The engagement with microsoft has been very professional. Microsoft takes UEFI signing seriously.

I asked peter about completely removing microsoft key from the firmware. He said it is a “bad” idea because ROM based firmwares won’t load and they are signed by the Microsoft key. He also mentioned that there is a complex workaround to this problem. The workaround is generating the hash of the firmware and adding it to the shim whitelist.

Here is the video of the talk

 

Fast OS Deployment with Anaconda – By Arun S A G (me)

 

I presented and showed a demo on how to deploy operating systems fast on bare metal
machines. The entire talk was well received by the anaconda team.

The demo showed installations of a  Fedora 21 (pre release) VM which took 2 minutes
to complete.  Most of the audience were pleasantly surprised.

• There were some interesting thoughts and area for improvements came out of this talk
•  RedHat developer proposed me to make use of the cloud kickstart file which has very minimal set of package
• Peter Jones suggested that anaconda can/should be modified to produce tarballs as one of the build targets (anaconda right now supports iso targets)
• Most of the installation time was spent on generating ramdisk. So peter suggested we should pre-generate the ramdisk and include it in the tarball.
• rpmdb cache needs to be removed from the tarball.
• Adam williamson asked me to share some sample kick-start files from work so that we can well test different use cases before releasing anaconda.
• Automating the biosboot partition during the installation process was discussed

Here is the video of my talk

Overall it was a wonderful conference.  Thanks Yahoo and RedHat for sponsoring my travel and accommodation. It was good to see lot of volunteers again and i had a good time in Prague, Czech republic. I am looking forward to Flock 2015

I am going to Flock 2014

I will be speaking at flock. The topic is ‘Fast OS deployment with anaconda’. See you all there!

http://flock2014.sched.org/

Flock 2013 – Fedora at Yahoo

This is kind of a late post. I spoke at Flock. It was about “Fedora At Yahoo!” – How we use Fedora in desktops and laptops at Yahoo!

Here is the presentation http://sagarun.fedorapeople.org/misc/FedoraAtYahoo.pdf

Surgical Eyes

I have a pair of them now. On April 1 (yes, fools day ) i got my lasik laser surgery done.  I am writing my experience down here without my spectacles 😉  hope it is useful to some of you who want to do the surgery.

 

(wikipedia: https://en.wikipedia.org/wiki/File:Augenoperation_1195.jpg )

 

Preparation

 

I took a leave of absence from work for a week before the surgery. I consulted a doctor in Aravind Eye Hospital ,Tirunelveli a small city near my home town and decided to get it done there.

I had my pre surgery check up on March 30 (saturday), They took  cornea topology scan  on both eyes and analyzed my cornea’s thickness. I had around 580 micron thick cornea which is good enough to do a laser surgery. The doctor suggested me to do  zyoptix laser instead of ordinary laser. Zyoptix uses wavefront guided laser technology and claims to fix refractive errors which cannot be fixed by other lasers. I asked the doctor about complications, she mentioned about halos and starbursts but didn’t mention about scary things like ectasia but it was mentioned in the agreement that i signed before the surgery. The hospital doesn’t have a “bladeless” surgical option ( i.e there is no femtosecond laser)  but the doctor is experienced so i decided to trust her hands with microkeratome. The hospital does refractive surgery on Wednesday’s and Friday’s, i asked them to prepone it to Monday (April 1) they were able to accommodate my request.

I had myopia and Astigmatism on both eyes. Myopia was around +5 and astigmatism was around 1.5

 

Surgery

 

Surgery’s are done early in the morning so i was asked to come to the hospital 7’o clock in the morning. We reached the hospital little late thanks to my dad’s driving 🙂 I was asked to have breakfast before the surgery. After the breakfast i was asked to sign an agreement which basically says no one is responsible if my eyes are screwed. I was asked to remove my shoes and asked to enter the waiting room (room in front of the surgery room)  Then i was asked to wear a long gown  and a cap to cover my hair.  I was also given anesthetic eyedrops before i entered into the surgery room. In the surgery room three scary people were waiting for me with masks over their lower face, covering their mouths and noses. I was asked to lay down on a moving surgical bed, top of the bed had  a curve  where i could just fit my head. Now one of those scary people covered my face with a sheet. The sheet had a opening that just came right on top of my right eye. It also had a tape like thing , they used to it hold my eyelids so i don’t close them during the surgery. Another nurse used a clamp to enlarge my eye. This was bit uncomfortable.

The doctor,  moved the bed back into the laser equipment. The equipment had very bright lights focused at my eyes, it was uncomfortable for me to be under those lights.  I would normally use profanity at people who don’t dim their main beam headlights while driving. But these lights were 3-4 times worse than those. The setup had a green and red color light accompanied by those very bright main beam headlights. You could imagine a big alien spaceship on top of your head flashing lights at you. I was asked to look at the red color light to reduce the discomfort. In the mean time the doctor was cleaning my eye and a big equipment was kept on top my eye holding it (microkeratome) . The doctor asked me not to move my eye. (Hello, how can you not roll your eyes when you have a big a** light focusing straight at you )  I heard a wheel rotating sound  and could feel something being cut. In my opinion this is the scary part of the surgery. The microkeratome is like your lawn mower but it cuts and creates a flap on the eye. See this video on how it works. I was informed that the lights would go off and asked  not to panic. By the time the lights were back on i was asked to focus on the red light again. After the lights were back on i could smell something burning The smell was from the laser shaping my cornea tissues. Once the shaping was done my eye was cleaned with a really cool liquid. The flap was lifted back up was placed in its place. They took a two minute interval before moving on to the second eye.

While doing the second eye i was little bit tensed (because now i know what they gonna do to my eye). More over the doctor was asking about another patient during the interval in a yelling voice (WTF? ) The nurse messed up the clamp so they had to remove the clamp and put it back on and the doctor made a sound that indicated something was messed up (more likely subconjunctival hemorrhage). IMO a doc should not make this kind of  sound while the patient is conscious and in surgery. I was shit scared still didn’t open my mouth, as it might add up to the tension. The same procedure was done to the left eye. Once everything was done i was asked to get up and go to the inspection room. The doctors assistant helped me remove the gown and cap. The doctor came back into the inspection room and inspected my eye (flap) again. Everything was normal. The whole procedure took 10 minutes and it was painless. I was asked to wear a protective glass to protect my eye from dust and accidental rubbing. I stayed back in the hospital on the day of surgery. The next morning my eye was examined and my visual  acuity was 20/20 . I could read the last line on the board with some help. I was asked to come back for examination after a week for further evaluation. After a week i could read the last line without any help.

After the surgery,  first three days are critical because the flap sticks there just using osmotic pressure. It can easily dislocate. I was asked to not to look at TV/Computer for five days. I was given with a steroid (Gatilox GM) (2 weeks) and a lubricant (Systane ultra)  for dry eyes.

 

Side Effects

 

I had surgery induced dry eyes and subconjunctival hemorrhage.  Dry eye problem was severe for the first 2 weeks. I even had head aches because of that. I kept using the lubricant to keep my eyes wet. My eyes became tired in the evenings. I am writing this post three weeks after the surgery, the dry eye problem is almost gone now eyes are not getting tired as earlier. But i have problems with night vision. I have glare problems with Fluorescent lights. I also see halos. These are indications that the flap is not completely closed yet. Night driving is difficult too. I will post more information here as i get better with night vision.

So Is lasik worth the risk? I am still not sure. You will have to evaluate the risks and current conditions before doing the surgery.  Lasik has a long healing period unlike what they advertise. For  5% of people the lasik surgery can result in ugly complications. As long as you are not in the 5% everything is fine. Now that’s the risk you will have to take 🙂

Fedora 18 on MacBook Pro

OSX crashed on me while i was trying to update firmware. Instead of recovering it, i decided to install Fedora 18 on my MacBook.

System Info

lspci

lsusb

cpuinfo

Installation

I had made couple of Fedora 18 x86_64 DVD’s for the Fedora freemedia program. I grabbed one of them and started the installation to find myself dropping into a prompt. Apparently it is one of known issues with the Fedora 18 CD/DVD. After typing in ‘(cd0,apple3)/EFI/BOOT/grub.cfg’ the installation started. There was no problem with the installation.

Tweaks

I am used to natural scrolling, enabled it using the following setting

$ cat /etc/X11/xorg.conf.d/99-wt-natural-scroll.conf
Section “InputClass”
Identifier “Natural Scrolling”
Option “ZAxisMapping” “5 4”
EndSection

I like my function keys, following sets the fn keys as primary in the apple keyboard

$ cat /etc/modprobe.d/hid_apple.conf
options hid_apple fnmode=2

Above option also can be passed using kernel command line. You will have to edit the grub.cfg for that /boot/efi/EFI/fedora/grub.cfg

Wireless

By default Fedora 18 didn’t detect my broadcom wireless network card (Broadcom Corporation BCM4331 802.11a/b/g/n (rev 02) to be exact) It needed a proprietary firmware. I downloaded b43-firware module from “russian fedora” http://pkgs.org/fedora-18/russian-fedora-nonfree-i386/b43-firmware-5.100.138-1.fc17.noarch.rpm.html and installed it. After the reboot things started working.

The proprietary driver doesn’t seem to support power management, it may cause problems during suspend/resume operation. So i instructed the power manager to unload the module while suspending the machine

$ cat /etc/pm/config.d/defaults
SUSPEND_MODULES=”b43″

Other bugs

GNOME 3 Desktop often doesn’t get show up after unlocking the screen https://bugzilla.redhat.com/show_bug.cgi?id=878736 If it happens press CTRL+ALT+F2 , login via tty and run ‘DISPLAY=:0 gnome-shell –replace’

The new kernel kernel-3.8.2-206.fc18.x86_64 panics while booting on MacBook Pro https://bugzilla.redhat.com/show_bug.cgi?id=922175

python-requests package update

The newest version of python-requests now supports python 3. The python 3 version of requests is available as python3-requests package. Thanks Rex Dieter for the patch https://bugzilla.redhat.com/show_bug.cgi?id=807525 . I have also decided to maintain python-requests for EL6, i have pushed the latest version 0.11.1 to EL6.

Though it violates the updates policy https://fedoraproject.org/wiki/Updates_Policy, i have updated python-requests from 0.6.6 to 0.10.8 in Fedora 16 as python-requests is a fast moving young project.

All the packages should land in updates-testing in a few days. Please visit https://admin.fedoraproject.org/updates/python-requests to learn more about the updates.