OpenStack at DreamHost

So I guess this is old news now, but DreamHost is really into OpenStack :-)

(In fact, during recess, DreamHost asked if I would pass a note to OpenStack. I didn't look inside the note, but we can all guess what it said...)

I was hired specifically to work on cloud stuff here at DreamHost, and we've got a new team that's super-excited about this -- they're starting to gear up for increased contributions and community engagement, gettin' themselves some cloud. We've now got our own Launchpad team, we're working on a handful of blueprints, chatting it up on mail lists and IRC meetings -- you get the picture :-) Exciting times.

For official blog posts and other news items that highlight DreamHost's interest and involvement in OpenStack, check these out:

• DreamHost Pledges Support to OpenStack Project
• DreamHost and OpenStack sitting in a tree...
• DreamHost uses Dell’s Crowbar as a lever to raise automation (more on Crowbar here) 
• Web Host DreamHost Contributing Code for Ceph File System to OpenStack
• Ceph and OpenStack at OSCON 2011

I'll be writing more about our OpenStack work later, but wanted to get a quick cloud-shout-out done before too much time passed...

P.S. We're hiring Python rock-stars!

P.P.S. Did I mention that DH is an AWESOME place to work? We got an award for that, two years in a row :-)

Syndicated 2011-12-07 02:21:00 (Updated 2011-12-07 02:49:46) from Duncan McGreggor

Occupy's Declaration of Independence

Illustration by Peter Whitley

There's one thing I would really, really love to see under the tree this year -- under everyone's tree: Occupy the World.

For the first time in history, it seems that there might be enough momentum, enough communication, enough strength of individual convictions, and enough mass support to be able to have a world-wide, non-violent, revolution.

Taking the US as an example in this beautiful hope: imagine 2 or 3 million people showing up on the lawns of the US law-making machinery in Washington, D.C., issuing their declaration of independence and simply stating a fact: "Things are now going to change, we will not leave until we have the government that we want."

Far from mob rule, the 99 is intelligent, lucid, and a collection of THE overwhelming majority... regardless of old political parties. An enormous amount of discussion, insightful inspection, exploration of alternatives has been researched, written about, and promoted over the past 5 to 10 years, co-culminating in what we see around us today as the Occupy movement. I have the utmost faith in these thinkers (by which I mean everyone from Gar Alperovitz to my second cousins working in Detroit, MI automotive plants) and their (our!) ability to produce a new constitution that provides for the 99 fairly. Such a new system would stand in stark contrast to that of today's system: one that caters to policies driven by enormous, "legal" bribes or banking systems that continuously steal from the customers and shareholders, running off with the loot.

The 99 is saying it, and has been saying it for a while: "It's time for a change."

They've taken things further by showing an undeniable presence at the scenes of crimes (financial and governmental institutions). I say let's take the last logical step, and fix the problem: let's put a new system of government in place. Let's have radical, peaceful change. Let's do it with poise, grace, and while keeping the benefit of the entire planet foremost in our minds. Let's do it world-wide, and not stop until the job is done. Let's have a revolution.

Let's have the revolution.

Syndicated 2011-11-21 20:37:00 (Updated 2011-11-21 22:08:49) from Duncan McGreggor

Two Months at BlueLibris

Back for a Quick Write

Well, I've been at BlueLibris for two months and a day, now... and each one of those days I've wanted to blog about something that's happened. On occasion, I've G+'ed about something, but in general I've just been too busy to take a breath, much less use that breath to write some informal prose.

I've taken a few photos during this time, and put them up on flickr in this set; smaller versions are peppered throughout this post :-)

The First Day

Half Moon Bay, about 30 minutes from work.

I was born in California, but left when I was 8 years old. I returned after high school and lived for 2 years (1990-1992) in Northern California to attend language school. As a consultant and later an employee of various companies, I made regular trips out to the Valley and Bay Area, so was no stranger when I arrived in August this year. However, it was quite a shock to be actually working and living here!

After I left Canonical, I headed out from Colorado and took a few weeks off to go on a meditation retreat up in the Siskiyou Mountains of Oregon. This area was deeply reminiscent of the Colorado mountains, of which I am so fond. Such environs are my natural habitat... so imagine my shock when I was hurled 50 years into the future, deposited in something like a night scene from Bladerunner. 'Cause that's exactly how it felt :-) Not bad... just very, very different. Even though I'd been to the Valley a bunch of times in the past 10 years or so, it never really hit home... until it was my home.

Note the executive dress: shorts and flip-flops.

In the morning, the bright sun washed away visions of flying cars and robots. I was greeted by palm trees and traffic. After an hour commute from the East Bay to Menlo Park, what I had only seen in Google Maps was now staring me in the face: my new office building :-)

I had arrived early at 9:00am, so the place was locked up tight. I took that opportunity to meet some of the neighbors, walk around a bit, and then read up on the interesting facilities we were renting for our offices.

The text describes the facility's power.

I'd seen the solar panels of the building in the Google Maps satellite view, and had just assumed that this was a common accoutrement of California architecture. Turns out, our entire complex is soloar powered -- 100%! Our laptops, servers, monitors, microwave, fridge, and air condition all run on the Sun :-) This info was covered in great detail by one of the boards posted out in front of our small office complex.

Eventually, Soroush showed up, I introduced myself, and we made our way inside. The interior was even cooler than the exterior :-) There's a large open space where most people sit, a dark cave for the software engineers, an unused loft upstairs that looks down over the open space, and a cute little kitchen area adjacent to (and part of) the loft.

The Intervening Weeks

Within a day or two, I'd purchased some bean bags, nerf guns, nerf balls, action figures, and die cast cars. It was starting to feel like a start-up :-) (And the nerf gun fights reamain a regular brain-break for many of us!). A few days after that, Tim Hyland bought some sling-shot monkeys... I think we use these even more than the nerf products!

Playtime aside, the work has been very grueling. We all work late hours (but have a good time hanging out while we do so), wear many hats, and have more responsibilities than we care to enumerate. Equally, however, we're all deeply invested in the company, believe in the services we're providing to the health and other industries, and most of all are committed to the success of the company.

On that note, most of us were involved in ventures during the dot boom/bomb, and accrued wisdom from that area on how to conduct ourselves responsibly, even in the midst of all the excitement our innovations are generating.

I look forward to the time (soon!) when I'll be able to post more about these :-) For now, though, I can be very general and give a brief overview of our tech.

Our Technology

Matt playing along with a thumbs-up :-)

I worked on several device projects for Canonical, but never were we actually building our own hardware like we do at BlueLibris. This has been a fascinating introduction to a part of the tech world with which I am starting to gain some familiarity: circuitry design, component integration, plastics work -- the list goes on and on. Soroush Salehian and Matt Maibach lead these efforts, with Ram Fish weighing in regularly with his extensive EE background. These are the guys iterating on the hardware for our real-time, 3G health monitoring device.

As events are generated on the device, they get pushed up to "the cloud" where additional processing and heuristics are performed. We're working with lots of maths, machine learning, custom algorithms, etc., all in a distributed computing environment. We're using nosql solutions for long-term storage, and a Python-based framework for web management tools. We're in the process of converting the original application into a JavaScript-driven suite of applications. We're also giving the Python an overhaul in anticipation of significantly increased load and general usage over the next year (we've got some exciting partnerships and customers -- both existing and forth-coming). Henry Messenger and myself are working on these bits right now. To get the device talking to the cloud and collecting all the right data, we have the talented Kent Ryhorchuk hacking like a madman and consuming the APIs we're providing him.

There are other software development efforts in the works that I can't really talk about... but given the space, I'm sure you can imagine the possibilities :-)

Future Posts

I've run out of time for now, but I do want to mention that with BlueLibris, there is a tantalizing connection to where I see technology going in the next 10-20 years (ultra large-scale systems), and I'd like to say more about that soon. Furthermore, we should have some nice news-worthy items that will get their own press releases, but I'll be making community-friendly updates here on that as well. I'm also taking steps to make sure that we contribute back to the open source community, and will have posts relating to those efforts as well.

So watch this space, and look for more developments soon... :-)

Syndicated 2011-10-04 18:41:00 (Updated 2011-10-04 19:58:53) from Duncan McGreggor

Two Months at BlueLibris

Back for a Quick Write

Well, I've been at BlueLibris for two months and a day, now... and each one of those days I've wanted to blog about something that's happened. On occasion, I've G+'ed about something, but in general I've just been too busy to take a breath, much less use that breath to write some informal prose.

I've taken a few photos during this time, and put them up on flickr in this set; smaller versions are peppered throughout this post :-)

The First Day

Half Moon Bay, about 30 minutes from work.

I was born in California, but left when I was 8 years old. I returned after high school and lived for 2 years (1990-1992) in Northern California to attend language school. As a consultant and later an employee of various companies, I made regular trips out to the Valley and Bay Area, so was no stranger when I arrived in August this year. However, it was quite a shock to be actually working and living here!

After I left Canonical, I headed out from Colorado and took a few weeks off to go on a meditation retreat up in the Siskiyou Mountains or Oregon. This area was deeply reminiscent of the Colorado mountains, of which I am so fond. Such environs are my natural habitat... so imagine my shock when I was hurled 50 years into the future, deposited in something like a night scene from Bladerunner. 'Cause that's exactly how it felt :-) Not bad... just very, very different. Even though I'd been to the Valley a bunch of times in the past 10 years or so, it never really hit home... until it was my home.

Note the executive dress: shorts and flip-flops.

In the morning, the bright sun washed away visions of flying cars and robots. I was greeted by palm trees and traffic. After an hour commute from the East Bay to Menlo Park, what I had only seen in Google Maps was now staring me in the face: my new office building :-)

At 9:00am, I had arrived early, so the place will locked up tight. I took that opportunity to meet some of the neighbors, walk around a bit, and then read up on the interested facilities we were renting for our offices.

The text describes the facility's power.

I'd seen the solar panels of the building in the Google Maps satellite view, and had just assumed that this was a common accoutrement of California architecture. Turns out, our entire complex is soloar powered -- 100%! Our laptops, servers, monitors, microwave, fridge, and air condition all run on the Sun :-)

This was covered in great detail by one of the boards posted out in front of our small office complex. Eventually, Soroush showed up, I introduced myself, and we made our way inside.

The inside was even cooler than the outside :-) There's a large open space where most people sit, a dark cave for the software engineers, an unused loft upstairs that looks down over the open space, and a cute little kitchen area upstairs.

The Intervening Weeks

Within a day or two, I'd purchased some bean bags, nerf guns, nerf balls, action figures, and die cast cars. It was starting to feel like a start-up :-)(And the nerf gun fights are a regular brain-break for many of us!). A few days after that, Tim Hyland bought some sling-shot monkeys... I think we use these even more than the nerf products!

Playtime aside, the work has been very grueling. We all work late hours (but have a good time hanging out while we do so!), wear many hats, and have more responsibilities than we care to enumerate. Equally, however, we're all deeply invested in the company, believe in the services we're providing to the health and other industries, and most of all are committed to the success of the company. Most of us were involved in ventures during the dot boom/bomb, and accrued wisdom from that area on how to conduct ourselves responsibly, even in the midst of all the excitement our innovations are generating.

I look forward to the time (soon!) when I'll be able to post more about these :-) For now, though, I can be very general and give a brief overview of our tech.

Our Technology

Matt playing along with a thumbs-up :-)

I worked on several device projects for Canonical, but never were we actually building our own hardware like we do at BlueLibris. This has been a fascinating introduction to a part of the tech world with which I am starting to gain some familiarity: circuitry design, component integration, plastics work -- the list goes on and on. Soroush Salehian and Matt Maibach lead these efforts, with Ram Fish weighing in regularly with his extensive EE background. These are the guys iterating on the hardware for our real-time, 3G health monitoring device.

As events are generated on the device, they get pushed up to "the cloud" where additional processing and heuristics are performed. We're working with lots of maths, machine learning, custom algorithms, etc., all in a distributed computing environment. We're using nosql solutions for long-term storage, and a Python-based framework for web management tools. We're in the process of converting the original application into a JavaScript-driven suite of applications. We're also giving the Python an overhaul in anticipation of significantly increased load and general usage over the next year (we've got some exciting partnerships and customers -- both existing and forth-coming). Henry Messenger and myself are working on these bits right now. To get the device talking to the cloud and collecting all the right data, we have the talented Kent Ryhorchuk hacking like a madman and consuming the APIs we're providing him.

There are other software development efforts in the works that I can't really talk about... but given the space, I'm sure you can imagine the possibilities :-)

Syndicated 2011-10-04 17:55:00 (Updated 2011-10-04 17:55:57) from Duncan McGreggor

Physical Beings with Digital Lives

There's a lot that one could say about that title. In fact, it could be the title of a high-volume collaborative blog... That aside, here's the context for this post: books. Books and Reality. And data.

This post got so long that I now need to add a list of sections here, just to make it more accessible. My apologies :-/

Mini Table of Contents

• Books
• Books in the Sky
• Yeah, I Know: Go Social
• Human Data History
• Reality Merges
• Conclusion

Books

I have tons of books. Actual, physical books. Walls of them. Some I use all the time (reference). Some I read once a year (good books that support multiple reads). Others I've only read once, perhaps as far back as high school (when I started collecting). My bookshelves are like a random associative memory array: reading each title or the act of pulling one from the shelf brings back a flood of memories, relived experiences, sometimes actual sense perceptions. It's a powerfully visceral activity.

But that's just my books. When I'm at friends' homes or offices, I cannot keep my eyes off their bookshelves. It's an irresistible compulsion. I linger and browse, often past any semblance of socially acceptable time limits. My conversational replies experience a rapid exponential die-off -- in duration, gaps, and semantic value -- culminating in grunts and finally silence. (My favorite offices to visit so far? friend mathematicians/maths professors!)

Books in the Sky

Oddly, I love books in digital format. I never really got hung up on the bit about not having the paper entity in my hands (though I have turned a digital book reader over, expecting the next page... though that was deep in the plot of a Greg Egan novel!). Traveling as much as I do, I'm in heaven with ebooks. I feel like Superman, carrying around a library with me everywhere I go.

But when I passed a bookshelf the other day on my way out the door and fell under the spell of a book-memory flashback, I realized what was going away as I transitioned to virtual books. And the painful question arose: How am I going to nurture future layers of book-mulch and text-humus with this new æthereal, cloud-bound library I'm building?

How can I share with others, my stacks of books? How will I browse friends' books in their offices, asking about author X and title Y? How will we borrow from each other? What can be done to add this and related missing richness back into our lives once we adopt the virtual versions?

Light-emitting walls that can display titles from your Amazon account? Virtual over-lays visible with wearable/immersive computing accessories? Whatever we end up with, a gimmick isn't going to cut it. It will need to reflect the same depth of history that stacks of physical books have come to represent to us and the collective human psyche since we first started gathers works of the written word.

Yeah, I Know: Go Social

I'm hung up on books here, I admit it. But the same goes equally well for much of what we experience in online social media as well. Everyone's trying to make a buck on people chatting, playing games, reading, etc. Business as usual.

But the problem is that everyone coming up with their own little solution, one piece at a time. Google, Facebook, LinkedIn, Last.fm, etc. "We socialize X." Wow. Good for you. Now, for every activity or group I'm interested in, I've got some tiny little corner of the internet that I need to pay attention to.

Right.

Maybe I'm just an online social idiot, but this isn't working for me. Too many places and pieces. The physical analogy would be me spending all day on the road, hitting all the social hot spots in the Colorado Front Range. Ain't gonna happen. Ever.

Human Data History

The social data scene is a big stick up my butt. I really don't like it there and I'd love to get rid of it. It's poorly engineered, primitive state makes me grumpy. I don't own a thousand hammers [1]; I don't want a thousand of anything that all do basically the same thing [2]. Most of us probably don't own hundreds of houses, either (for ourselves, that is). We keep most of our stuff in the same location or two.

Speaking of houses, let's talk about settlement. How did we choose where to set up camp, towns, etc.? Trade routes, availability of resources (direct physical presence or presence by virtue of trade routes). Are we doing that now on the internet? Are we looking at the analog to fertile valleys, productive rivers, and protected harbors? Whose priorities do we have in mind? As we set up virtual presences, are we in locations that benefit businesses? Or ourselves?

If we choose the latter, the businesses will come because the people are there. If we do it the other way around, we'll be looking for new virtual homes if the businesses close shop or change the rules too much.

Coming back to data (but on the same anthropological note), historically we've had distinct divisions of our data:

• the secure location of our huts/houses/castles
• what we presented about ourselves in adornment/fashion (public data)
• what we could carry with us in bags/crates/vehicles

Because of their prevalence in our history, any attempt at realizing personal data in a virtual environment would do well to reflect on these. We're naturally already predisposed to such approaches; such divisions are things that anyone can intuitively grasp.

The problem is that we're all used to a single platform: our mutually agreed-upon reality. There's no such thing online yet. And if there were, who would own/run it? Monopolies are eventually overthrown. We hate them. So how do we get around this?

Reality Merges

This very naturally led to thoughts on digital lives in general. And this is more than just a question of usability or human-computer interaction. Rather, this is a question that borders on the metaphysical: how do we solve the problem of syncing divergent realities? Reality-reality interaction.

The problem shouldn't be minimalized by analogy: this isn't a "simple" matter of ensuring that the data in my address book on Google is the same as what's on my iPhone. My self-perception, many reminders of self-reflection, etc., take place as a result of various interactions I have with my surroundings: both real[3] and virtual. No problem. Except that the things that remind me are also things that others can see and interact with as well. Often, they will have associations that spark a neural cascade for them too.

I've had many conversations take place around objects in a shared environment where the name of the object was never mentioned, it was implicitly understood. When there's no shared object (or concept), we have to name the object, define it, share some basic associations, make sure that we're talking about the same thing, etc. Thats all prelude. Only with that done, can we have genuine communication take place around the given concept.

Now rinse and repeat for everything you want to talk about that revolves around or is at least related to something that exists virtually for you and isn't part of your shared, physical environment.

I can't imagine many useful general solutions to this. In fact, I can only imagine one (given our biological wiring): use what we know (in our bones) and overlay or augment our visual reality with another.

With augmented shared realities, there's no platform. You just need hardware that runs it and senses that can perceive it. Just like reality. At that point, we can start sharing what we want, allowing access to data about ourselves and what we like by dumping it into a shared perceptual space, regardless of the original data source. Merged.

Obviously, we're not there yet. We're going to need crazy improvements in mobile technology, storage, computer vision, etc. But once the technology catches up, I think we'll see some powerful needs being filled. And we might start coming out of the internet dark ages...

Conclusion

None of this is new; Pick up any number of books by Charlie Stross[4], and there's all sorts of fun to be had by exploring his ideas. But the point of this post wasn't to be new. While we're all busy enjoying the latest fad in social media, I think it's important we think about where the progressive succession of fads is taking us. At each point, there's a natural next step (more accurately, set of possible next steps). Let's look more than just one in front of us and let's not forget what our biology has made us. We may not be able to engineer truly wise decisions about our future, or even make our lives better/more efficient. But it would be nice if we could at least not make things worse :-)

Notes

[1] I think I have three, actually.

[2] This is one of the reasons I'm a big fan of http://ping.fm.

[3] Here I mean "real" in the "conventional" (shared) reality sense of Mahdyamikas. Ultimate reality... well, that's a topic for an entirely different sort of post...

[4] Check out his Amazon page: http://www.amazon.com/Charles-Stross/e/B001H6IW0Q/. Accelerando is probably his most praised book (and likely my favorite), but the ideas touched on in this blog post are explored in other works of his, most notably Glasshouse and Halting State.

Syndicated 2011-06-23 23:44:00 (Updated 2011-06-23 23:57:03) from Duncan McGreggor