Older blog entries for skvidal (starting at number 452)

coprs and buildsystems and a question I was asked today.

Work continues on coprs but today someone asked me if I knew of a simpler buildsystem that would let them spin up a builder, stuff a bunch of packages at it, and return a pile of results.

I said well… ansible and mockremote could do it.

I started thinking about HOW that would work and realized I have all the pieces – but not a lot of motivation to put it together at the moment.

So I thought I’d tell folks how I would do it and maybe someone will:

 

1. write a playbook to spin up an instance in one of the public clouds using ansible and it’s ec2 module:

http://skvidal.fedorapeople.org/misc/ansible-euca-transient.yml

2. then provision the system with mock, a mockbuilder user and mockchain.

3. tell the user the ip the instance has

4. user submits jobs to it with:

mockremote.py -r chroot -b ip –destdir=/somewhere/ -c –recurse pkg1 pkg2 pkg3

5. user comes back whenever it feels like it

6. user runs a terminate playbook like:

http://infrastructure.fedoraproject.org/infra/ansible/files/copr/provision/terminatepb.yml

which shuts down the instance they had spun up.

I can see wrapping that whole thing in a single script or just letting the user do it a bit at a time. Either way – trivial to setup your own personal, temporary and pristine builder.

 

Syndicated 2013-02-01 21:38:17 from journal/notes

coprs and fudcon

I’m still working on recaps from fudcon but I wanted to mention this to anyone who asked:

 

If you asked for access to coprs and you haven’t received a reply from me that is b/c I forgot that you asked. Please drop me an email and I can add you.

 

Thanks

Syndicated 2013-01-22 16:53:16 from journal/notes

copr status

We’ve gone through about 45 individual builds of a goodly number of pkgs thanks to the help of the testing volunteers. We’ve found a bunch of bugs and fixed many of them.

Here’s what’s changed:

1. we’re working on cli for copr – more info in this thread: https://lists.fedorahosted.org/pipermail/copr-devel/2013-January/000216.html

2. we’ve got some mockups from the UI team which should help mold where things can go

3. better logs and results for multiple builds

4. I took a first cut of an outline for setting up copr: http://git.fedorahosted.org/cgit/copr.git/tree/copr-setup.txt

 

Syndicated 2013-01-15 20:12:03 from journal/notes

fudcon tips and the flu

Hi everyone, it is time for my annual plea to everyone going to fudcon to wash their hands many many times, try not to cough or sneeze on anyone and to do their best to not touch me at all.

It is winter and we’re in the middle of a massive flu and norovirus outbreak here in the US.

http://www.nytimes.com/2013/01/10/health/flu-widespread-leading-a-range-of-winters-ills.html?hp&_r=0

If at all possible I would like to come back from fudcon without bringing any sort of pestilence with me.

This is just a reminder that if you see me incased in a plastic ball don’t be alarmed just smile and wave.

 

 

 

Syndicated 2013-01-10 15:15:01 from journal/notes

coprs status

My call for testers was successful. We have 21 people now who have access to coprs and a number of folks have started testing builds out. We’ve found some bugs and some feature enhancements needed. Also discussion has started on the copr devel mailing list and we’ve gotten some patches contributed to help us make a cli come to pass.

If anyone else wants to take a look at the code and help out, you can see it all here in copr cgit.

thanks to all the folks who are helping test and develop the copr code.

 

 

Syndicated 2013-01-08 18:30:00 from journal/notes

call for serious testers

Right now, we have coprs working. It’s not fancy or beautiful and I am POSITIVE many bugs are lurking there (and some aren’t lurking at all just sitting there). So, we need people who will seriously undertake testing. I don’t want people who want to build things and don’t care about reporting issues. I also don’t want people who are going to be filled with destructive criticism.

If you want to test it out and cite issues please email me: skvidal at fedoraproject.org

you must already have an active fas account and some srpms you want to build.

I only need maybe 10 or 20 people at the moment. Not looking to overdo it

Thanks

 

Syndicated 2013-01-02 20:09:02 from journal/notes

jenkins and ansible and ‘the cloud’

Working with Pingou on this I merged the playbooks into a single playbook. We were setting up jenkins and wanted a way of putting all the systems together quickly and easily. All it does is spin up a master instance, provision it, spin up 2 workers (one in el6 the other f17) provision them and end:

It could easily be done in ec2, euca, openstack or pretty much any system. I know there are more ways to skin this cat but I thought this was simple enough to follow along and others might find it interesting.

This is the playbook:

http://infrastructure.fedoraproject.org/infra/ansible/playbooks/groups/jenkins-cloud.yml

 

all of our ansible playbooks and inventory files are available here:

http://infrastructure.fedoraproject.org/infra/ansible/

 

Syndicated 2012-12-19 06:15:18 from journal/notes

copr – testers needed soon

Bohuslav and I have been diligently working on the copr buildsystem code in the last month or so. We originally targetted the middle of november but that wasn’t to be. However, right now we have working rough-around-the-edges code. It has some completed successful builds so we know that part works.

A little background:

copr is intended to provide a safe place for packages to be built that are not packages that are not or cannot be built in fedora. Think of it as a place where you can scratch and chain build whatever you want (up to the obvious limits of legality . It takes your packages, builds them against themselves and puts the repo up available for download.

Why not just do this in koji?

1. koji is just for fedora builds
2. koji does not allow external, arbitrary repositories for build deps
3. koji’s builders are not destroyed and cleaned after each build
4. koji just isn’t designed to do what copr does and grafting it on top of koji is a dangerous proposition to the safety and security of fedora builds.

How does copr do things?

You submit your build request (name of the copr, chroots to build against, repos to use to build and the list of src.rpms to build) into the frontend. The backend takes this request, creates a builder in our private cloud, builds the pkgs, pulls back the results onto a webserver, provides you with a link to the results and destroys the builder.

What’s the status right now?

The backend code works, but there are a lot of rough edges to sand down. The frontend code works, but there is a bunch of ui work needed there (right now Ryan Lerch is creating some mockups for how things should be). The code is available from the link at the top. Right now everything is in two branches a front end branch (bkabrda-workspace) and a backend branch (skvidal-backend). We’ll be merging them into master in short order.

We’ll be sending out a call for folks to help test in the not-so-distant future. The whole goal is to provide a place for people to build packages they would like to have available but maybe cannot have in fedora for one reason or another.

Examples:

• alternative configuration/build options for an existing package
• not very easy to package in fedora (bundling, etc) but perfectly good software otherwise
• test builds
• your idea here
• builds based on other repos that are not included in fedora

Things which cannot be in a copr:

• software which are not legal to include in fedora

 

The fedora buildsys mailing list (buildsys@lists.fedoraproject.org) is where we will discuss things  going forward but I just wanted to let people know what we’ve been working on.

 

Syndicated 2012-12-10 15:45:15 from journal/notes

FAD: Two-Factor Auth setup

This week we held a Fedora Activity Day in Raleigh at RH HQ to get two-factor auth setup for Fedora Infrastructure. We had a pretty good plan and we knew the pieces we needed to put together – it was just a matter of doing it. We got our goals accomplished but that’s not what I wanted to talk about.

We ended up with 9 people there. I was originally a bit concerned that was too many people, that we would end talking more than accomplishing things and that would suck, especially if we had failed to get things implemented. I was keeping track of what everyone was doing, how they were helping. What I noticed was that everyone contributed in some way. There wasn’t anyone on the sidelines. At one point we had 2 people working on the package reviews of the pkgs we needed to get into fedora (totpcgi and pam_url). We had a person writing the cgi to let us use both yubikeys and google auth, a person working on the provisioning interface to get people setup using google auth, a person working on the puppet config, a person setting up the certs/pki needed to let pam_url connect securely. We had a person setting up cloud instances for us to use to test/blow things up and we had a couple of people writing/rewriting their yubikeys and auth secrets in order to test and retest and reretest.

 

The FAD just removed all friction (as someone else put it to me yesterday). It meant that instead of waiting a few days or more to solve the problems we only waited 20minutes. Like often said about mediation – good facilities are sometimes all that is required to get things done.

It was great having everyone there and able to work it was great being able to ONLY focus on this one thing. I think we will have this again in the future to help accomplish tasks which are just too involved to bite off a little at a time or will take years to get done at that rate.

 

Syndicated 2012-11-30 14:38:38 from journal/notes

ansible presentation from @jp_mens

Great presentation slidedeck:

 

https://speakerdeck.com/jpmens/ansible-an-introduction

 

introducing ansible.

 

Syndicated 2012-11-08 16:29:44 from journal/notes