Older blog entries for katzj (starting at number 142)

Announcing ami-creator

I’ve been having to build some new CentOS images to be used with EC2 for work recently. I went into it thinking that it shouldn’t be too big of a deal. I know that some work had been going on in this area and Fedora 14 is now available on EC2, so I figured I could convince the same toolchain to work.

Unfortunately, I was pretty disappointed with my options.

  • Do some building by hand on an actual instance, then do the bundling and upload off of the running instance.
  • Some of the ThinCrust stuff initially looked promising, but it seems like it’s largely unmaintained these days and the ec2 conversion bits didn’t really work at this point. I was able to get my initial images this way, but mostly by having a wrapper shell script of doom that made me sad.
  • There’s always the rPath tools, but I wanted to stick to something more native and fully open source
  • The new kid on the block is apparently BoxGrinder but I found it to be a lot over-complicated and not that robust. I’m sorry, but generating your own format that you then transform into a kickstart config and even run through appliance-creator via exec from your ruby tool just felt wrong. No offense, but just felt like a lot more than I wanted to deal with

So, I sat down and spent an evening hacking and have the beginnings of a working ami-creator.
It’s pretty straight-forward and uses all of the python-imgcreate stuff that’s used to build Fedora live images. Your input is a kickstart config and out the other side pops an image that you can bundle and upload to EC2.

Thus far, I’ve tested it to build CentOS 5 and Fedora 14 images. I’m sure there are some bugs but at this point, it’s worth getting it out for more people to play with. Hopefully it’s something that’s a lot simpler and more accessible for people to build images and I think it will also fit in a lot better with having Fedora release engineering building the EC2 images in Fedora 15 if they want.

One of the big outstanding pieces that I still want to add is the necessary bits to be able to (optionally) go ahead and upload and register as an AMI with your EC2 account.  But release early, release often.

Comments, etc appreciated in all the normal ways.

Comments

Syndicated 2010-12-14 02:30:54 from Jeremy's Thoughts

I’ve moved my bike blogging

I’ve been wanting to play with tumblr, so I’ve set up a new blog for my bike blogging to try it out. Check it out for exciting race reports, some video and probably some other random thoughts on cyclocross as I begin my inaugural season of cyclocross racing.

There might be some other reorganization and moving around here as well in the future when I have a little bit of spare time. Which, since I’m racing cross, might not be for a few months ;-)

Comments

Syndicated 2010-10-15 02:49:50 from Jeremy's Thoughts

Stop Using the Word “Cloud“

The more I see it, the more I want to just completely see the usage of the word “cloud” go away. While it’s somewhat of a cliche to say so, it’s a term that has a very hazy and non-concrete meaning. So whenever you start to use it, you immediately end up in the “well, what is a cloud” discussion. And thus, I have a set of suggestions for those places where you might have wanted to use the word “cloud” to instead use something which actually has meaning.

  • If you’re using cloud to refer to EC2, use EC2 instead. It’s concrete and it means very real things about your deployment and scaling models as well as how you’re managing your infrastructure.
  • If you’re using cloud to refer to some service which runs over the Internet, either refer to the service or just say the Internet. You don’t store your mail “in the cloud”, you host it with Google apps. You don’t backup “to the cloud”, you have your backups stored over the Internet with Mozy or Carbonite.
  • If you’re using cloud to refer to the idea of some hosted application platform, just say the platform. You don’t run your python app “in the cloud”, you run it on AppEngine (or something else).
  • If you’re using cloud to mean that you are using virtualization and have some management stack on top of it, then please just say you’re running in a virtualized environment.
  • If you’re using cloud to refer to having your server infrastructure hosted in a virtualized environment by someone else, again, just say you’re running in a virtualized environment.
  • If you’re using cloud to refer to a “visible mass of little drops of water or frozen crystals suspended in the atmosphere”, then congratulations, you can continue to use the word cloud. And thanks to Wikipedia for the definition

Following this simple idea will let you avoid the otherwise impossible to avoid discussion of the semantics of the word “cloud” and what you happen to mean about it and how you might be wrong and … This then means you’ll be that much closer to achieving whatever goal you hoped to achieve as you’ll spend less time talking and more time doing. And as an added benefit, you’ll avoid getting grumpy emails from me about the fact that you’ve used such a terribly over-used and under-meaninged term.

Comments

Syndicated 2010-01-21 05:11:00 from Jeremy's Thoughts

Time to get back into a training routine

As a road rider and racer, my cycling season tends to wind down about this time. If I were to start racing cyclocross, I’d extend it out, but for now, I’m staying out of that. The past two years, I’ve marked the end of my season with racing at the Jamestown Classic down in Rhode Island. This year, a combination of the fact that I really kind of needed to work the day and also that my fitness wasn’t really where it should have been for a race led to me skipping it. Now I’m a little bummed that I did, but c’est la vie. I’ve spent most of the past six weeks generally riding just for fun and without any real training goals in mind, although I have been watching the power numbers on my shiny new powertap out of curiosity.

Looking back on the season, it was one that was both successful on some fronts and utterly not on others. I did a good job of keeping up a good base training routine through the winter but then ended up doing little in the way of racing over the course of the spring and summer. First it was waiting for the new bike, then it was being busy, then the weather sucked, then I got hit by a car, then travel, and then the season was over. Even though I didn’t race much, I felt like I was a lot better prepared for the races I did do and that my fitness was higher as a result of the base training through last winter.

So I think it’s now time to start easing myself back into a bit more of a routine in preparation for the winter of base training. I picked up a new trainer to replace the freebie I had been using that’s significantly quieter. Last winter, I was able to do trainer time in the evenings, but with my current schedule that seems unlikely so I’m going to start getting up a little earlier to get time in before the ride into work.

Set it up last night and had the first ride on it was this morning and it’s pretty nice — quieter than the old one and seems a bit smoother as well. I’ve got a pretty good set up to start with to be able to watch DVDs or online video. I’m then streaming the audio to my iPhone with AirPhones so that I don’t have to have a long headphone cable or worry about turning up the speakers really loud. Today was watching some TV via Hulu and then a Spinervals DVD. For the latter, though, I need some better music. What do other people listen to as a good upbeat playlist for time on the trainer or even general race warmup, etc?

Comments

Syndicated 2009-10-18 23:02:34 from Jeremy's Thoughts

EC2 and Fedora: Still stuck at Fedora 8

Amazon’s EC2 service is great for being able to roll out new servers quickly and easily. It’s also really nice because we don’t ever have to worry about physical hardware and can just spin up more instances as we need them for experimenting or whatever.

Unfortunately, they’re still stuck in the dark ages with the newest AMIs available for Fedora being Fedora 8 based. With Fedora 12 around the corner, that’s two years old — something of an eternity in the pace of distribution development. I’d love to help out and build newer images, but while anyone can publish an AMI and make it public, you can’t publish newer kernel images, which really would be needed to use the newer system.

So, if you’re reading this at Amazon or know of someone I can talk with to try to move this forward, please let me know (katzj AT fedoraproject DOT org). I’d really strongly prefer to continue with Fedora and RHEL based images for our systems as opposed to starting to spin up Ubuntu images for the obvious reasons of familiarity.

Comments

Syndicated 2009-10-16 19:46:24 from Jeremy's Thoughts

Why do all deployment systems suck?

At HubSpot, we have a pretty wide array of different things being used for the webapps running behind the scenes. This isn’t surprising. There’a also some home-grown scripts (in python, as that’s the scripting language of choice… something I’m not complaining about) to take care of deploying the various webapps. It works, but I really want to get it doing a bit more so that it’s more useful and also get the different scripts doing a bit more sharing of code so that we can improve one place and get the benefits for everything.

Given that this seemed like a pretty typical problem, I figured I’d take a look and see what open source projects exist out there to see if any of them were suitable or could be at least close to a good fit for what we need and want. Unfortunately, I was kind of disappointed…

  • Capistrano seems to be the big player in this arena. It was originally written for Rails and still very very strongly shows that heritage. This isn’t necessarily bad, but it makes it a lot harder to get to work if you’re not doing something that’s rails-like. There are some people who have gotten some things working with Java app deployments for tomcat, but they all feel a bit hacky. The other downside for me/us is that Capistrano is very much Ruby-based, both in how its own deployment language looks as well as some of the “how it depends on things working” aspects. Also, the fact that it’s written in Ruby and thus a little bit more difficult for us to hack on if/when we run into problems is a point against. So it’s probably a non-starter for now, or at least a pretty difficult sell
  • Fabric is written in python and seems to be following in the footsteps of Capistrano. Right now, it’s far far simpler. This is in some ways good but some of the pieces that we’d want (eg, scm integration) aren’t there and so I’d have to write them. And I’m not sure if the Fabric devs are really interested in expanding in that way; haven’t sent email yet, but planning to tomorrow to feel it out.
  • Config Management + Binary deployment is the approach taken in Fedora Infrastructure for app deployment and it seems to be working pretty well there. It might be something to get to eventually, but that’s going to be a longer term thing and I’m not actually convinced that it’s really the best approach. For Fedora it grew out of only a couple of things which could be considered “webapps” and a lot of system config that has turned much later into more webapps. It also pre-supposes a bit more homogenous of an environment than we use at HubSpot from the work I did there
  • Func is something that a few people have been working on that I keep wanting to find a use for but it seems a little less well suited to doing a lot of java app building/deployment given that it’s more https/xml-rpc based than shell based.
  • Roll your own is what we’re doing now and what it seems like is pretty common. I don’t necessarily like this, but it’s certainly the path of least resistance

So, what am I missing? Is there some great tool out there that I haven’t come found that you’re using for Java (and more) webapp deployments? Bonus points if its python-based and pretty extensible.

Comments

Syndicated 2009-10-07 02:50:18 from Jeremy's Thoughts

10 Year NCSSM Reunion: Has it really been that long?

I spent the weekend at my ten year high school reunion. A lot of people that I’ve mentioned this to have been somewhat surprised or responded along the lines of “I wouldn’t go to my high school reunion”. Which I can understand. But Science and Math is different than your typical high school. As a two-year residential magnet school, you end up getting to know people pretty well — it’s basically like going to college a couple of years early, only with a little bit more supervision. And so it’s the same sort of bond and friendships that a lot of people end up getting in college.

Kara and I flew down on Friday morning and then met up with a friend for lunch. We then spent some time just kind of relaxing before meeting up with a group of my friends from high school for dinner. It’s a good group although we don’t get together anywere near as frequently as we used to… for the first few years after high school, we made it a point to get together around New Year’s or at some point over the holidays. This has fallen off as we’ve all moved increasingly far apart distance-wise and have increasingly busy lives. It was good to get together and catch up, though, and especially in a smaller group environment.

Later in the evening was the sort of opening party/reception type thing and my class had a surprisingly good turn-out. The event was held at a bar in Durham (Tyler’s) which was fine. It was a little weird seeing people smoking indoors as that’s something kind of foreign after living in Massachusetts for almost six years. This probably contributed to the scratchy throat feeling I had the rest of the weekend. As I walked around, I was able to see and catch up with a ton more people; both that I had been friends with through high school or people I barely talked with. It was really nice as pretty much everyone is doing something that is kind of interesting now and it was cool to talk with them. None of the usual crap that people complain about with high school.

Saturday was the main day of reunion stuff but my group of friends started earlier with breakfast at Elmo’s on 9th St. As usual, Elmo’s was good and we got there just in time to beat the majority of the rush. Then, it was down to campus and we watched a presentation about some plans to expand the school. I’m not really sure how I feel about it. On the one hand, it looks like a fair bit of work is going into making sure that things are nice and really trying to turn what had been a hospital hastily converted to a residential high school into a very nice residential high school. On the other, more people in the same sized space could be a bad thing. A day later and I’m still not sure where my opinion falls on the expansion. The rest of the day was filled with catching up with more people, walking around campus to see what had changed (both a lot and very little at the same time), our slide show (always fun to watch again) and the like.

Kara and I then skipped the evening’s festivities so that we could grab dinner with my parents as we haven’t seen them in a while and it’s unclear exactly when we’re going to be down again. It was a low-key evening and then rather than attend the late night party, we just watched Valkyrie in the hotel room.

This morning we slept in a bit more and then headed to campus for what had been billed as the faculty brunch. The location stated for it had been a little bit odd, and when we got there, we learned why — it was actually in PFM (the cafeteria). Also, the number of faculty present overall was pretty low at least that we saw.

We skipped out a little, though, so that we could grab lunch at the NC Farmer’s Market Restaurant with another group of friends. This was, as always, tasty and it was good to get to catch up with another group of friends. Then we met up with my sister and her husband and now we’re sitting at the airport waiting on our flight back to Boston. (Well, we were… home now. But I’m a day later posting it)

It was a fun weekend and it was great to catch up with a whole bunch of people. There were some people who I would have liked to have seen who couldn’t make it, but I got to see a lot of people that I hadn’t seen since our 5 year reunion or even longer for some of them. And while most are on Facebook, some aren’t. The event was better organized and a better set up than we had for the five year reunion, but there were a few things which bothered me and can hopefully be better for the next one…

  • The cost and especially the way the costs were organized was unclear. As it turns out, very few things were only for those who paid. This may be partially due to complaints over how much some things were, but it still wasn’t at all clear what you were paying for up-front.
  • Having the full schedule sooner would be nice — I think the final version was published a week or less ago. That’s just kind of poor for people trying to decide if they want to come or not and especially how to schedule their travel
  • Faculty presence was low. I vaguely remember complaining about this five years ago. I know it’s hard to get the faculty to come out, but some of it is communication with former faculty who have retired. They can’t come if they don’t know what’s going on.
  • Little interaction with current students. This is one that seems easily fixable but it’d be really nice to get a chance to have more current students around and talk with them and the like. Get the real story about what’s going on at the school; I didn’t trust the administration ten years ago, I don’t trust them now :-) Things like lunch yesterday or brunch today would have been perfect opportunities for them to be around, eating and join some alums. It would also be great for the current students to get to talk to people who have been there, done that, and are maybe doing something cool or interesting now. I would have liked it as a student and I would have liked it now as an alum.

Even with those suggested improvements, it was a fun weekend and I figure I’ll probably be back for the fifteen in five years and look forward to seeing everyone again then. Even if it does make me feel old to see current students wandering around.

Comments

Syndicated 2009-10-06 02:19:42 from Jeremy's Thoughts

Commuting Pedal Failures

One thing that’s quite nice about the new gig is that the office is in Kendall Square. Much, much, much better location-wise than Westford. It means that my commute is just about seven miles which is quite nice to do via bike. Also, if the weather’s bad or I feel lazy, I can take the bus to Alewife from right outside my house and then take the train in.

Unfortunately, I’ve now had two weird pedal failures in the past week. Last Thursday, I was leaving the office and clipped in. As I got about a block away, I noticed my foot moving weirdly on the pedal. As I pulled over to check it out, it became clear that the cleat was stuck in the pedal. After some investigation later, I realized that I lost one of the two screws holding the cleat into the plate in the shoe. It looks like the plate where the screw went in is actually pretty stripped. And in getting the cleat and shoe disengaged from the pedal, I essentially had to take the pedal apart so I decided to switch the pedals out for the plain SPDs instead of the slightly fancier SPDs that were on there.

Today, I was riding home and realized about halfway home that one of the pedals was coming unscrewed from the crank. I made it home without incident and re-installed the pedal without any noticeable problem, but I’m going to be keeping an eye on it over the next few days. Hopefully the crank isn’t stripped — it looked okay, but at this point, I’m a little cautious of it.

Maybe I should look at building a new commuter bike sooner rather than later :-) Although I really would like to get the Redline to last another year to year and a half.

Comments

Syndicated 2009-09-15 02:34:31 from Jeremy's Thoughts

Review: Amazon Kindle 2

I had asked via twitter a while ago for opinions on the Kindle as I had been thinking off and on about getting one for a while. Responses were very positive with really one exception: the DRM aspect. And it’s something that bothered me, but I decided that the convenience benefit was enough to outweigh that for me at this point. Especially when I considered that I often buy books, read them, put them on a bookshelf for six months to a year and then drop them off at a book drop. Re-reading isn’t my usual, instead buying new books is :-) Especially as the primary purpose for the Kindle is more for pleasure reading as opposed to more technical stuff

Since I knew I’d end up reading a lot while I was on vacation, I figured I’d take advantage of Amazon’s good return policies and try it out. If I liked it, great. If I didn’t, I could certainly hit a bookstore to get “regular” books and return the Kindle when I got back home.

As lots of people have said, the packaging of the device is very good — Apple-like is really the best way I have to describe it. They really want you to buy into a full experience. I turned it on and had already set up a few “free sample” books to be on the device. And reading on it has turned out to be a great experience. The page turning was a little bit distracting at first with a little bit of a flicker, but after half an hour or so of reading on it, I don’t notice it any more than I notice turning the page of a regular book. One thing that I didn’t find at first (because I didn’t read the documentation) is that you can change the font size — this was pretty handy as I can definitely handle smaller than the default — I’m currently using the smallest and am pretty happy with it.

One of the obvious questions with the Kindle is battery life. With the wireless off, it was incredible (I didn’t apparently need to charge during most of my 10 day trip with it sporadically on). Leaving the wireless on I need to remember to charge it every three or four days. But the charge is pretty quick. Unfortunately, it uses microUSB instead of miniUSB, so it’s another cable to have to be carried bringing me to three (miniUSB, microUSB, iPhone).

The selection of books seems to be pretty good. Not everything that I’ve looked for, but the vast majority. Pricing is okay, usually a little cheaper than the price for the dead-tree version and the over-the-air delivery is wonderful. Especially when sitting in an airport between flights.

In terms of reading on it, the screen is incredibly nice. I had my doubts, but they’re entirely gone. Reading in bed? Check (just be sure there’s a light). Reading outside beside the pool? Check. On a plane? On the couch? … And so on. The screen is incredibly readable and I don’t end up with any of the eye strain problems that I sometimes get after staring at a laptop screen or a monitor for long periods of time. How well it worked in sunlight is one thing that really surprised me. Obviously it’s supposed to be one of the strengths of the screen, but I’m wary of marketing-speak :-) But I really had no problem sitting and reading it even in direct sunlight.

Downsides? There are a couple. The biggest is the DRM stuff. It’d be nice to be able to share books (eg, if Kara decided she wanted to get a Kindle also so that we could both read the same books). It’d also be nice to have some sort of library functionality, although the samples are a big help in deciding whether it’s worth paying for a book.

The other real downside is the case that I got — the Patagonia neoprene case. It’s okay in that it protects it pretty well; it’s kind of crappy to use while reading, though and I’ve taken to taking the Kindle out of the case when I’m reading from it. Other case suggestions would be appreciated since I’m certainly going to keep the Kindle at this point.

Comments

Syndicated 2009-09-14 02:40:21 from Jeremy's Thoughts

Garmin 705 Tip: Sending Routes Between Devices

While I think I’m pretty good with the capabilities of the Garmin Edge 705, I learned a new trick with it last weekend. We were planning to do a century up to Cape Ann and Gloucester and I had mapped out a route with Bikely and loaded it onto my GPS. I sent out the link to everyone else who was planning to come, but Charles had not pre-loaded it onto his. Instead, he said that you can actually transfer routes, courses, etc between devices wirelessly!

It’s actually pretty simple — you hit the menu button, go to Settings, then “ANT+” and then Send on the device doing the sending and Receive on the one doing the receiving. And then, voila, you’ve sent the route between Garmins.

Very very cool, and something I’ll definitely have to keep in mind for future rides to new places as more and more of the team start getting the 605 and 705.

Comments

Syndicated 2009-09-07 18:51:56 from Jeremy's Thoughts

133 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!