Is thinking that because something is working well that it will continue to do so.
Is thinking that because something is working well that it will continue to do so.
coprs and buildsystems and a question I was asked today.
Work continues on coprs but today someone asked me if I knew of a simpler buildsystem that would let them spin up a builder, stuff a bunch of packages at it, and return a pile of results.
I started thinking about HOW that would work and realized I have all the pieces – but not a lot of motivation to put it together at the moment.
So I thought I’d tell folks how I would do it and maybe someone will:
1. write a playbook to spin up an instance in one of the public clouds using ansible and it’s ec2 module:
2. then provision the system with mock, a mockbuilder user and mockchain.
3. tell the user the ip the instance has
4. user submits jobs to it with:
mockremote.py -r chroot -b ip –destdir=/somewhere/ -c –recurse pkg1 pkg2 pkg3
5. user comes back whenever it feels like it
6. user runs a terminate playbook like:
which shuts down the instance they had spun up.
I can see wrapping that whole thing in a single script or just letting the user do it a bit at a time. Either way – trivial to setup your own personal, temporary and pristine builder.
coprs and fudcon
I’m still working on recaps from fudcon but I wanted to mention this to anyone who asked:
If you asked for access to coprs and you haven’t received a reply from me that is b/c I forgot that you asked. Please drop me an email and I can add you.
We’ve gone through about 45 individual builds of a goodly number of pkgs thanks to the help of the testing volunteers. We’ve found a bunch of bugs and fixed many of them.
Here’s what’s changed:
1. we’re working on cli for copr – more info in this thread: https://lists.fedorahosted.org/pipermail/copr-devel/2013-January/000216.html
2. we’ve got some mockups from the UI team which should help mold where things can go
3. better logs and results for multiple builds
4. I took a first cut of an outline for setting up copr: http://git.fedorahosted.org/cgit/copr.git/tree/copr-setup.txt
fudcon tips and the flu
Hi everyone, it is time for my annual plea to everyone going to fudcon to wash their hands many many times, try not to cough or sneeze on anyone and to do their best to not touch me at all.
It is winter and we’re in the middle of a massive flu and norovirus outbreak here in the US.
If at all possible I would like to come back from fudcon without bringing any sort of pestilence with me.
This is just a reminder that if you see me incased in a plastic ball don’t be alarmed just smile and wave.
My call for testers was successful. We have 21 people now who have access to coprs and a number of folks have started testing builds out. We’ve found some bugs and some feature enhancements needed. Also discussion has started on the copr devel mailing list and we’ve gotten some patches contributed to help us make a cli come to pass.
If anyone else wants to take a look at the code and help out, you can see it all here in copr cgit.
thanks to all the folks who are helping test and develop the copr code.
call for serious testers
Right now, we have coprs working. It’s not fancy or beautiful and I am POSITIVE many bugs are lurking there (and some aren’t lurking at all just sitting there). So, we need people who will seriously undertake testing. I don’t want people who want to build things and don’t care about reporting issues. I also don’t want people who are going to be filled with destructive criticism.
If you want to test it out and cite issues please email me: skvidal at fedoraproject.org
you must already have an active fas account and some srpms you want to build.
I only need maybe 10 or 20 people at the moment. Not looking to overdo it
jenkins and ansible and ‘the cloud’
Working with Pingou on this I merged the playbooks into a single playbook. We were setting up jenkins and wanted a way of putting all the systems together quickly and easily. All it does is spin up a master instance, provision it, spin up 2 workers (one in el6 the other f17) provision them and end:
It could easily be done in ec2, euca, openstack or pretty much any system. I know there are more ways to skin this cat but I thought this was simple enough to follow along and others might find it interesting.
This is the playbook:
all of our ansible playbooks and inventory files are available here:
copr – testers needed soon
Bohuslav and I have been diligently working on the copr buildsystem code in the last month or so. We originally targetted the middle of november but that wasn’t to be. However, right now we have working rough-around-the-edges code. It has some completed successful builds so we know that part works.
A little background:
copr is intended to provide a safe place for packages to be built that are not packages that are not or cannot be built in fedora. Think of it as a place where you can scratch and chain build whatever you want (up to the obvious limits of legality . It takes your packages, builds them against themselves and puts the repo up available for download.
Why not just do this in koji?
How does copr do things?
You submit your build request (name of the copr, chroots to build against, repos to use to build and the list of src.rpms to build) into the frontend. The backend takes this request, creates a builder in our private cloud, builds the pkgs, pulls back the results onto a webserver, provides you with a link to the results and destroys the builder.
What’s the status right now?
The backend code works, but there are a lot of rough edges to sand down. The frontend code works, but there is a bunch of ui work needed there (right now Ryan Lerch is creating some mockups for how things should be). The code is available from the link at the top. Right now everything is in two branches a front end branch (bkabrda-workspace) and a backend branch (skvidal-backend). We’ll be merging them into master in short order.
We’ll be sending out a call for folks to help test in the not-so-distant future. The whole goal is to provide a place for people to build packages they would like to have available but maybe cannot have in fedora for one reason or another.
Things which cannot be in a copr:
The fedora buildsys mailing list (firstname.lastname@example.org) is where we will discuss things going forward but I just wanted to let people know what we’ve been working on.
FAD: Two-Factor Auth setup
This week we held a Fedora Activity Day in Raleigh at RH HQ to get two-factor auth setup for Fedora Infrastructure. We had a pretty good plan and we knew the pieces we needed to put together – it was just a matter of doing it. We got our goals accomplished but that’s not what I wanted to talk about.
We ended up with 9 people there. I was originally a bit concerned that was too many people, that we would end talking more than accomplishing things and that would suck, especially if we had failed to get things implemented. I was keeping track of what everyone was doing, how they were helping. What I noticed was that everyone contributed in some way. There wasn’t anyone on the sidelines. At one point we had 2 people working on the package reviews of the pkgs we needed to get into fedora (totpcgi and pam_url). We had a person writing the cgi to let us use both yubikeys and google auth, a person working on the provisioning interface to get people setup using google auth, a person working on the puppet config, a person setting up the certs/pki needed to let pam_url connect securely. We had a person setting up cloud instances for us to use to test/blow things up and we had a couple of people writing/rewriting their yubikeys and auth secrets in order to test and retest and reretest.
The FAD just removed all friction (as someone else put it to me yesterday). It meant that instead of waiting a few days or more to solve the problems we only waited 20minutes. Like often said about mediation – good facilities are sometimes all that is required to get things done.
It was great having everyone there and able to work it was great being able to ONLY focus on this one thing. I think we will have this again in the future to help accomplish tasks which are just too involved to bite off a little at a time or will take years to get done at that rate.
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!