Older blog entries for grey (starting at number 19)

OK, fucked up and posted this as an article instead of in my diary, never noticed that option before, now I know why. Anyway, here is the write up:

http://advogato.org/article/860.html

15 Sep 2005 (updated 15 Sep 2005 at 20:16 UTC) »
Some assembly required, glue not included.
TXT/SMS directions to cell phone. Initial spec/draft.

Section 1: Intro/premise/justification:

The other day before driving to meet a friend & go to a concert, I wanted to look up driving directions to his house and then the venue. Rather than print these out, I thought I would just send them as a text message to my cell phone. I had remembered that mapquest had a "send to phone" option before, and though what the hey. Unfortunately, when I clicked on that link this time - it isn't just a simple text message, it requires a Get It Now/java applet type download and a $4/mo subscription more or less. They also had a send to PDA, which requires avantgo so again - additional software is required, as well as syncing steps and the like. I can understand that, they're using traditional approaches to getting content to these mobile devices, and in the case of mapquests' send to phone thing - they have a business and want to make money. But I think they're also missing a bigger picture.

Not everyone has a PDA, but a LOT of people (even kids) have a cell phone now. However, not all cell phones support these java app things, moreover - while a map is NICE to have, it's by no means necessary. Just about any modern cell phone (or heck at least any of the FREE with subscription type cell phones, let's not go into 5+ year old analogue phones) support a TXT/SMS message, and the cost per message is either very cheap, or included with your basic plan just about now. In other words the txt/sms message is ubiquitous - has a huge footprint, and doesn't require additional software to be loaded. If you can send a text message with a simple smtp client, or heck a 12-key phone pad, that seems to be the lowest common denominator/biggest footprint of any portable device (again, way more cell phones than PDA's).

If I had a pencil and paper I would just jot down the turn-by-turn, they already give me the turn-by-turn (plus a lot of other stuff I don't care about like mileage between each) which I just don't want to have to transcribe by hand or print. Most people look up directions before they leave. Even though Sprint & Garmin recently announced a $9/mo subscription service to do on-the-fly mobile GPS Navigation, which is very cool - that's a lot more than people need. Moreover, I don't have Sprint, my cell phone is company paid for I don't have a choice in the carrier (they use Verizon and Nextel depending on your job responsibilities, I got a verizon phone). I really don't want to have to worry about what cell phone provider I am using for something this basic, I don't want to have to use some sync software or Java app that may cost me money or require me to install things I may not be able to.

I also want to stop wasting paper - I HAVE to carry a cell phone. Heck, txt/sms messages can even be sent to most modern pagers, the footprint/cost for these things is HUGE as an information delivery platform.

But where are the tools to do this? Well, they're all there already, they just need some glue. Last night I spent about a minute glueing by hand, but as I did it, it occurred to me that this glue should be easily automated programmatically, so that instead of me spending a minute, a program could do it in microseconds.

One more note: while researching this more, I came across GoogleSMS http://www.google.com/sms/demo.html. This is closer, it's free and let's you interact from your cell phone directly. This is great, but still not quite what I'm hoping for, here's why:

As I said earlier, I want to do this before my trip, when I have access not as I'm driving or on the road. Ok, that's minor (arguably it's actually better to add the feature to do this remotely as GoogleSMS already has, since it's way more useful that way for anyone) and not really a complaint, just a different usage. But, I'd also like to send it to a text pager, googleSMS can't work that way currently because it requires interaction, my pager can't do that. Remember what I said about wanting to do this to use the maximum footprint? Well, text pagers fall into that realm for me.

Section 2: What I did in the present (i.e. the human GLUE)

1. I went to google maps maps.google.com and put in my from/to crap.
2. I clicked the search button.
3. I copied the direction text.
4. I pasted the text and sent an email to my phone#@vtext.com (actually I used www.vtext.com's form, but you could use any simple smtp engine/client).
5. Checked txt message on phone.

Wow, done took me less than a minute, really simple - why the hell hasn't someone written a little glue to do this automatically? OK, some have done a lot more than this, but no one has delivered quite what I'm hoping for. If you know of someone who HAS done this already, let me know please, and I'll save my breath. There are also some obstacles... I'll present those next as well as what tools are already out there to help.

Section 3: Obstacles/Requirements

1. Phone # to carrier smtp address lookup parser.

Ok, the first one isn't actually related to the direction part - it's how to get a message to your cell phone with just a phone number as entry. Not everyone is going to know that phone#@cellphonecarrier-smtphost.com type address for their phone, or pager. You will have to match up the NPA (area code) NXX (next three digits) to a carrier and thus email address format appropriately.

1a. Sanity Check

It's probably a good idea to implement a sanity check to make sure input is alphanumeric, maybe allow for it to include - and (). Country codes could get evil, but let's just worry about North America for starters. Don't implement a sanity check and let it barf if you're making a first run of it sure.

2. TXT/SMS message character limitations lookup.

Each cell phone/text pager has a limit on the number of alphanumeric characters that can be sent per TXT/SMS message. Verizon is 160 characters, some carriers are 110 characters. It would be best to know this in advance, and if the number of characters in your message is greater than that carrier-max-variable number.

3. Message splitter.

If the number of characters in your message is greater than that carrier-max-variable number, then split the message and send two, or three, or however many is divisible by that #-of-characters-variable. If you don't do this, your message will get truncated and be missing the end, which is arguably the most important point. 'Course for a PoC initial run of things, maybe skip this step, but definitely leave it in for rev2.

Note: Parts 1-3 go together as a foundation, but could be useful for lotsa other things too (most TXT/SMS aware clients do this automatically, I'm not sure where, as I'd imagine that #1 and #2 would best be implemented at a carrier gateway level. However, most/any/all? cell phones I have used with TXT/SMS functionality have done #3 in-client [i.e. even your cheap-o cell phone splits the message, does it just guess @ the maxchar variable, or use the native carrier setting as a default, or does it have #1 or at least #2 already stored? Maybe this sort of information is what OTA updates are useful for]).

4. To/From get direction parser.

You need to feed your start/stop info to the directions engine (e.g. maps.google.com, mapquest.com, yahoo maps, etc.). Google maps 1:1 field for To and From is really nice in this way. Ideally take that. Push it out.

5. Direction output text scraper.

Once you get back the directions output, just grab the pertinent direction text and copy it a buffer or variable or something for use at the end.

5a. Returned directions sanity check

If you get one of the "this address doesn't exist/needs clarification" then you need to go back to #3 and get the user to re-input it. For an initial go at this, you could forget this error checking step - don't leave that in for any production system, but who needs sanity checking for PoC? (famous last words?)

5b. Direction cleaner/trimmer/substituter extraneous crap reducer/substituter

If you wanted to get really fancy you could trim out the extraneous things that are returned in the output (like mileage between direction instructions) and shorten/substitute key terms (e.g. LEFT becomes L, RIGHT becomes R, EXIT becomes X or something). It's probably easiest to first implement substitutions for key terms, and then implement ripping out crap most people don't care about like mileage. This is totally optional - and for a quick get-it-off the ground demo implementation should be skipped, especially if you've implemented #3 already. #3 will take care of extra crap by sending more messages if you go over. But if you get this feature implemented, then when #3 gets called you might be able to save TXT/SMS wastage which could in some cases translated to small amounts of money being saved (like pennies per message sent). I would strongly suggest implementing this, but do it as a rev2. I don't want to make this too complicated, but this is the one feature I'll leave in the original spec that is completely optional, by spec v2 or v3, it should be mandatory (but all other specs should hopefully be done by then in some rough form), if you're trimming optional things, this goes first - trim the sanity checks last, and of course the unsubbed #'s are not optional.

5. Send the message.

By now, you've done #4, so you can take the output retrieved in #5, as well as all the stuff in #1-3 and put it all together and send it, yay!

If this weren't flat html written in my spare time I would probably want to flow chart this out, but the process should be pretty intuitive, and in some cases I did state what goes first/comes next so you're smart you can figure it out I'm sure. Or spend a couple minutes drawing the flow out on paper just to get it clear.

Section 4: Things that do this work for you already

After all, as I sit here I'm thinking that all the tools for this exist already, I don't want to write them from scratch nor do I think that needs to be done, I just want them to be glued them together. Particularly as google maps has an API, this should hopefully be easy right? Well, a little more research reveals that google maps currently doesn't have an API for their directions engine, but there may still be ways to get at it (or not, if that changes or has changed).

So what is already at our disposal?

#1 http://www.telcodata.us/telcodata/telco let's you do NPA/NXX lookups to carrier. 'Course to use this you may need to do something like #4 & #5 again, I've seen other sites feed to their engine already e.g. http://freesms.1888usa.com/revcell.htm so this might not be too hard, and any licensing concerns might already be waved because like I said, others are doing that already. Definitely with all of this check on terms of use, don't go building this and advertising to the world unless you're in line with them, but that should be obvious.

#2. http://freesms.1888usa.com/ Has a list of the carriers individual email addresses as well as character limits for each.

#4/5 - well there's a lot of direction services already, mapquest, yahoo maps, google maps - pick your poison.

Tools to glue: dunno, curl, scripting language? Your choice.

Frustrations (opportunities?) & Future

As discussed previously, Google Maps has an API - but currently not directly for their directions portion. D'oh! Also, Google SMS let's you do a lot of this already, directly from your phone, but as far as I've looked I see no way to interface with GoogleSMS directly from a browser. I'm sure somewhere in the bowels of Google are a couple of people who could put this together themselves, such that you look up directions on google maps, and then you don't just have links for javascript:_print() javascript:_email() and link-to-this page, but you could have a javascript:_sendSMS() option as well. I don't know if doing a proof of concept is even fruitful, but I've told this idea to three people in the past couple days, and all the responses have been positive - just someone needs to do it, so I thought I would write up a spec as I saw it needed, even if you wanted to try to cobble it together yourself. I guess this way I can forward this write up as a feature request to the google maps group and see what they have to say. But if anyone wants to try to do this with some spare time hopefully I didn't leave much out. Again, most of this has been done already by others.

If you wanted to get fancier, you could start looking into sending a picture of a map as well, but y'know that's going to start cutting out the number of potential clients who can take advantage of the data. I've been looking at a simplest implementation, highest target footprint/audience approach. You could add on all sorts of other things later. I've heard great ideas about hooking into the integrated GPS of more recent cell phones and getting positional information based on that - again awesome stuff, and the Sprint+Garmin things no doubt do that, but that's heavy weight and costs money currently or is tied to a carrier. If you wanted to go ahead and glue some of these pieces together yourself (#1-#3 being most important first) then you could stick whatever you want in the data, be it directions, pictures of maps, positional/GPS distance-to-interestingpoint stuff (so far that's what most of the uses of the google maps API have been, be it to find hotornot people, gas prices by locale, or the opposite side of the earth). All clever stuff, some quite useful - but all higher layers and bigger projects. I just thought here's one useful, practical and simple starting point.

Anyway, I welcome comments (though I should probably look to another hosted blog that actually has comments by default now that I say that) so umm, you can email christian.c.david-!AT!-gmail.com If you know of someone/place that has already done this le tme know too! Remember I JUST want txt/sms done from a browser to a txt/sms aware device, even if it's one-way such as a pager, and ideally with all the map directions stuff already for this particular application - googleSMS doesn't help me with all of that criteria, not yet at least.

Jose hit me with the stick. Which I had never heard of before, but I guess I am just supposed to answer the questions in bold and pass it along. That said here goes.

Stickify:

You're stuck inside Fahrenheit 451, which book do you want to be?

Geeze, having to memorize an entire book? I'd probably go with the Tao Te Ching. First off it's short, which would make the chore a lot easier to do. Also, it can be deep and give one lots to ponder without sounding haughty. Then again, I could sound like a Taoist sage just by reciting passages from it even if I ended up being a doofus in day to day conversations. I've got some other reasons too, but damn it sure would be easy to remember compared to some possible choices.

Have you ever had a crush on a fictional character?

Most definitely, in fact too many to list comprehensively. Heck, I still do! I'm guessing here the intention is for fictional literary characters. Admittedly most of the fictional characters I've had crushes on have been animated or in comics, or heck even live action fictional characters. Still, too many to list. I will say that crushes on real people have been more complex, if not more fantasy fulfilling. :) Herm, just to mention a few even if they're not literary that come to mind:

Nausicaa (from kaze no tani no nausika)
Queen Emeraldas (from Captain Harlock)
Kusanagi Motoko (from Ghost in the Shell comics/films/etc.)
Aeon Flux
Matilda (from Leon)

The last book you bought is:

Well, I bought them for my wife at her request - a book of love poems, and also a book of Japanese poetry. Sadly, she took the love poems with her on her journey to Hawaii I think so I don't know the precise title, but she did leave the Japanese poetry book behind which is: _One Hundred Poems from the Japanese_ by Kenneth Rexroth.

The last book you read:

Ok, not counting the myriad books I read to my son on a nightly basis (he's five, but there are some damn good ones mixed in there mind you), but last week I finished:

_The Ethical Slut: A Guide to Infinite Sexual Possibilities_ by Dossie Easton, Catherine A. Liszt.
I can say that was one of the few books I've ever read that I legitimately felt I could say was "eye opening."

What are you currently reading?

Roughly ordered from closest to finishing to furthest from finishing (I tend to read a lot of books simultaneously in a queue which isn't rigidly ordered even if I'm placing order to them here):

_Verbal Judo - The Gentle Art of Persuasion_ by George J. Thompson, Ph.D. (way less dry than Non Violent Communication)

_Divorce Busting_ by Michele Weinger-Davis (not a book most people probably want to be in a situation to merit reading, but I have been, so there. It's so-so, actually _The Monogamy Myth - A Personal Handbook for Recovering from Affairs_ by Peggy Vaughan was the best in this category I've read so far and I've been reading a lot in this category of late)

_Non Violent Communication - A Language of Life_ by Marshall B. Rosenberg, Ph.D.

_The C Programming Language_ 2nd Edition by Brian W. Kernighan & Dennis M. Ritchie (K&R)

_The C Answer Book_ 2nd Edition by Clovis L. Tondo & Scott E. Gimpel (just to go along with the above, these last two I am going through slowly, but plan to pick up the pace)

Five books you would take to a deserted island:

1. _1999 Military Field Manual on Survival, Evasion, and Recovery Army, Marine Corps, Navy, Air Force Multiservice Manual_ by Department of Defense http://www.equipped.com/multiservice_ser_manual_1999.pdf After all, I would want to stay alive, this looks like it has some good pointers, it's free to boot! Yeesh, so call my a pragmatist.

2. _The Elements of Computing Systems: Building a Modern Computer From First Principles_ By Noam Nisan and Shimon Schocken http://www1.idc.ac.il/tecs/ Jose pointed this out to me and I was actually just telling jsyn and some others about this. Though I don't own a copy for myself yet, I thought "Perfect! If I ever ended up on a deserted island hopefully I could use what's contained in this one book to build a computer from the ground up!" Ok, so computing on a deserted island isn't a high priority, but I really dig computers, and want to know them in a practical sense well enough that I could do something like that if needed. Such a book holds that promise, though perhaps the intended place for practicing the techniques described within is different than what I'm currently proposing. Maybe I'm just using this to demonstrate I'm not a pragmatist too. ;)

3. One of my bagua texts to keep in practice with my current favourite martial art. Possibly: _Classical Baguazhang, Volume I - BAGUAZHANG LIANXI FA (Baguazhang Practice Method)_ by Jiang Rongqiao, translated by Joseph Crandall.

4. The OED (Oxford English Dictionary). I love etymology, dictionaries, linguistics and the like, but moreover this lexicon is frigging HUGE, over many volumes, so it could serve me for years or decades before getting through it all, thus staving off boredom if I'm stranded for a really long time. If worse came to worse, it could also serve as building materials, or good kindling for fires. I wonder if I could live off the paper? Or maybe it would help with staying sanitary.

5. Man, this last slot needs some really good piece of fiction I think, something to spark the imagination, some piece of escapism, maybe with a bit of good outlook on life. Ideally pretty darn long too. Sadly I don't know what to put here in particular, maybe the series of _A Wrinkle in Time_? I haven't read those since gradeschool, but they were pretty neat, regardless I think I'd actually want some substantial body of fiction which I _hadn't_ yet read, so maybe I can't answer this one that well. As a matter of fact, I think almost all 5 of these texts I haven't read. I don't really like to reread books. Life is short enough without going over old ground, when there's limitless new things to explore.

Who are you going to pass this stick to (3 persons)? And Why?

Dan "danimal" Weeks - because he actually noticed I'd been sticked before I even posted a response of my own. For that he gets to indulge in the pleasure himself. ;)

Nick Rucka - a long time friend, he doesn't have a blog, but he reads more books than anyone I know (and probably watches more films too, I have no idea how he crams everything into his life and still has time to create so much). Maybe this would give him an incentive to start a blog. Plus, I'd like to hear his highlights given how wicked smart and literary he is. Mind you, he's a kickass film maker too, I hope we'll be seeing more from him soon in that respect. Of course, if he does setup a blog now I'll have to update this with a link.

Chris Abad - I had a lot of fun hanging out with aempirei at CanSecWest this year, and he's always got an interesting take on things. I'm sure that carries over into books he might read as well.

current stick path as it got to me: Barrie unleashed the stick on 3/7/05 - Amanda - scooterdeb - Brian - Karma Police - Evelio - Ivy - Suzanne - Jeff - Patricia Lockwood - Frank - Amy - Steve - Ginger - Scopylaw - AI - jose - me ...

I realized I want to find some positive music to listen to . While some of the stuff I've been playing helps as a cathartic to release negative emotions, I really want something new and upbeat to help assist in making my mood positive, not just getting out bad feelings. I realize that my focus has been more of the cynic, be it politics, economics, music, looking at computer vulnerabilities, and even how to train martially and look for weaknesses (though to the martial training credit, it is more balanced). If I want to have a more optomistic outlook, I should look for the positive in all of my experiences. I always consider myself to be more pragmatic, and that my perspective has been a way to identify problems so that I can work to resolve them, but I see how often it has been exaggerated to the point of feeling helpless (esp. in politics or economics, things outside of direct control). I do good things in all of those fields, but I plan to change and focus differently so that I feel better about those contributions and that I'm making them.

Anyway, with positive music, I'm not too sure where to start. Cocteau Twins I always liked, but maybe there's some good electronic music or even tribal stuff that would be powerfully positive, I like Infected Mushroom and Magic Firesheep as upbeat interesting electronic music, but I have never found much on CD. Maybe it would be good to even break away from electronics a bit. I'll see what I have in my collection, and maybe go to a CD store soon as well and see if anyone could recommend something along those lines. I have enjoyed revisiting things that I had left neglected for a while, like music and friends, but at the same time I want new good things in my life, not just going back to the old, especially when I am working to break habits and build new fulfilling experiences.

On that note, the near daily walks have been great. Not only for my health, though I've lost a lot of weight and am having to wear a belt and down 4 notches already! But sharing the time has been great too. My wife and I shared a walk two nights ago, and last night I went just with my son again. Even if I'm not with a friend or family, being in nature is its own bit of sharing, be it the trees, the animals, or the moon. Man, I love living near the ocean!

On the drive to the concert on Friday I talked with Kevin a lot about surfing, and though he is very advanced such as finding remote beaches, big breaks, using short boards, even dealing with aggro locals/groups, etc.; he says he wasn't doing much surfing while in school due to his schedule. Now that he's done with school Kevin is going to a gym and getting in shape and said that he'd love to go out with me sometime, especially since he'll be getting his feet wet all over again. He gave me some good pointers too, told me to rent a board or buy a used one and go to Cowell's for a while until I see whether it's something that I want to do more and then it becomes more economical to buy a board. He also talked about the buddy system that he has employed, though he said if I'm just at Cowell's, there are usually enough people around to help out, and that it's a great spot for beginners. I'm going to find out the rental shops hours, I've already got a wetsuit and booties. I'll call Jeff too and see if we could meet up sometime since he's local and has been surfing for about a year now.

I don't know if I need to live right next to the ocean, but I love being near moving water, and at least a short drive away would be OK. The biggest thing is just timing I think. The weekends in Santa Cruz get really crowded with out of towners, especially as the weather has improved, and early morning or evening I don't know about rental stuff. Those seem like ideal times however, I will look to find a used board, or borrow someone's to not worry about that hassle and just start doing it. Especially early morning, I see some surfers go down to the lane right at dawn on my early morning walks. They look so peaceful and chill, surfing as the sun rises. It's calming just watching them. What a way that would be to start the day! And it would be before the family is up, so I wouldn't be missing any time with them, I think that would be my ideal time to surf - but it's hard to know until I start doing it and seeing what works.

As far as the mornings, I have to admit I enjoy making the family breakfast, it starts the morning off on a much calmer cooperative note. Well, honestly I had been making our son breakfast most mornings (he starts most days asking for waffles!), and even my wife a couple times a week (or well, at least some tea or picking up a breakfast burrito or something after dropping of my daughter). But with my daughter it has been a great change. She's been extremely appreciative, accepting, even excited, though that depends more on what I make, if it's a favourite like pancakes or mangos. We both end up in a better mood, and we are both a lot more conversational on the drives to school. We are talking about a lot more than just school now: her interests, her friends, what events are going on in her life. I really enjoy the bonding, it's only getting better and easier. For so long it felt awkward to connect, the distance was palpable. I really feel like when I had to act more as a father and enforce discipline instead of just being friendly, about a year or two into my wife and my relationship is when things changed between my daughter and me as we got along great at the start.

Focusing on the present, now my daughter is meeting my efforts to change and be more connected and supportive in kind, and I really see a difference not only in how she and I interact, but in how she is interacting with others. She is much calmer, happier and less jumpy or critical. She gives me comments and perspective that have blown me away with her insight now that we're not just talking about the superficial like toys or tv shows. I regret that for many years I had acted so distant, but now she is showing me that it's never too late to get on track and has been really warm and receptive to my efforts to change, and even empathetic when I tell her I'm having problems, she actually expresses her care and compassion. She hasn't just quiet and tight lipped so much as in the past if I said I was tired or had a rough day. As she says to herself, and I like to reinforce, she is the best. ;)

I think it would be nice to make more dinners come to think of it, it's something we used to do a lot - but in recent years, being so busy with a young child, moving hectically and erratic work hours - we ate out a lot more. Don't get me wrong, there are some fantastic restaurants around Santa Cruz, and I think my enjoyment of them had contributed to my waistline growing. But the occasions where I (or even better, with my wife or as a family) make dinner, it is also really calming. Last week my wife got the ingredients to make Matzo Ball soup. Something she, and well I, had never done before. It took a while, but was a lot of fun - our son even helped. The end result, filled with chicken stock I didn't partake in, but merely the process of creating something as simple as a meal was very rewarding, and it was really enjoyable to take a breather with the family and cooperate like that. The 2 hour preparation process itself to me was far more meaningful than sitting down to eat for 20 minutes after it was done.

Man, where am I finding all the time to write this stuff? Well, work has been really supportive of giving me time since I've asked. I appreciate that so much, everyone has really been considerate of me and offered to help. And it feels good to ask for what I need and get it. I realize that now that I am asking for the time, I don't feel so overworked. Granted, there are still a lot of projects to get done, and I've been a bit more distracted than usual, but the understanding and support has been fantastic! Going out to lunch with some coworkers was never too regular, and to be honest I actually like to have a breather to myself, but it has happened a couple times in the past two weeks, and some people are just being there for me without asking details and understanding that I don't want to complicate things - that respect alone has been appreciated.

Breaking my focus on being "right" and instead just flowing, has been a challenge. Especially since my habits in the past have been so introspective. This is most pronounced in martial arts class, where I am getting over the reaction of stopping or pausing when I screw up, and instead am starting to jump back into the mix, or take a breather for a moment and come back rather than wear myself out. These same changes are going on throughout my efforts in life, and it's really different for me to distinguish between getting caught up reacting to flowing with the changes, taking where I am given something, giving where I am asked of something.

I do not have it figured out yet, but I'm working on it, it is really hard some times, but the challenge is worth it, and the more I change my habits, the easier it seems to get. Slowly. It finally feels like I'm on the path I've wanted to be on for a long time, instead of just sitting in park, talking or thinking about where I wanted to be, now I'm working towards my goals and really committing to my family and work and what's important to me, committing to myself. It's a little bit of a "who moved my cheese" but part of it is that as I move, it's all relative, and some of the things I thought I wanted I don't even understand why I did, or regret that I did because they were so trivial to what has always been important to me and is more clarified now, even as the opportunity might now and then present itself.

I feel a LOT right now, pain, pleasure, a million emotions and feelings, it's a tornado or a crashing wave, but in some ways it's so much better to be feeling the ups and down than plodding along distant. I see this as different from my other stumbling blocks in my past, where the pain felt all consuming, now hopelessness is balanced with hope, things are shifting, changing, mistakes are being made, but I am aware of them and not repeating them endlessly. I am not perfect, I'm not out of the woods, but I'm beginning to appreciate the journey.

11 Apr 2005 (updated 11 Apr 2005 at 20:26 UTC) »

Monday. New Order's song "Blue Monday" I will dig up later today, the title is apropos.

I am having a really really hard time coming up with anything positive that I did today, so all the more reason to do it since this can be part of what I accomplished postively - just wrote. Yesterday went to the beach with the kids, had a lot of fun. I was feeling really tense in my shoulders and did some stretching exercises. It was nice to walk on the rocks and feel the sand and seaweed beneath my feet along with the feel of the large stones/rocks I walked along.

Not to focus on the negative but I slipped on a rock, fell flat on my ass and banged up my tailbone rather good. This morning it feels a little better, but unfortunately a wave crashed on me right afterwards and trashed my new cell phone from work.

I'm getting my ass kicked in the game of life, literally right now with that event. I'm working on patience, right now. And fortitude, that I won't give up on my hardships but rather carry on and work to resolve them.

Leaether Strip albums have been great to hear the same agony and agression I'm feeling, but let's me get through some of it non destructively. I am working to come out of this intact, I do not want to live a life filled with resentment and anger, so using outlets to express those non-destructively is helpful. I think I will get a punching bag. I think it's time to get some creative juices flowing again, write some music, paint, program, whatever.

Let's see, Sunday morning. I made some waffles for the kids, and did some more cleaning around the house. My plans for today are to sort through clothes and toys to set aside for donation pick up on Tuesday. A lot is done already actually, but I will try to rebag/rebox some of them better. I sure hope it doesn't rain on Tuesday so that everything will be dry for pickup. Doing the preparations should help keep my busy and organized, even if the pick up becomes problematic.

Yesterday I went to Big Brothers Big Sisters bowling for kids sake event with the family. My wife is on their board and helped organize the event, although my company had about five lanes all together. It was a lot of fun, the whole family (even my 13 year old daughter) seemed to have a good time. We also went to a City of Santa Cruz planning event, where they had various exhibits discussing the General Plan and were soliciting ideas. Lots of tough problems, and some good proposals for solutions. I wrote down a few of my own. I'm hoping that they take to heart more diverse building styles (encorporating living&working space), as well as some of the traffic proposals seriously. Even something as simple as enforcing no left turn on Mission St. intersections unless there's a turning lane could really help with rush hour congestion. I also hope that they take seriously the suggestions to keep away WalMart and other big box corporate stores. Though I'm sure half the city of SC would be up in arms if they tried, at least that's been the case in the past when they've made such attempts.

In the evening I went for a walk with my son and our neighbour's dog through Lighthouse Field and down to watch the surfers in The Lane for a bit. The dog's leash broke about halfway through and I didn't even notice for a while because he was carrying the end in his mouth. It wasn't until the second person commented, "What a cute dog! Taking himself for a walk," that I noticed what had actually happened. Talk about well behaved, I had a good laugh over it. Of course -after- I noticed, then he stopped carrying his leash in his mouth, and later even jumped in a big muddy pound that appeared from the recent rains. I felt bad about returning him dirty and with a broken leash, but our neighbours seemed OK (apparently it wasn't their main leash anyway). I'll think of something to make it up to them.

Kept it mellow for the evening as I was still pretty tired from a23 the night before. That was fun BTW, great to see Kevin and catch up. He's really a top notch guy. I felt honoured to be seen with him by the time we got to the club and Da5id Din, a fellow from Battery Cage and others all came up to him to see how he'd been and what he was up to. He and I have both been away from the scene for a while, but the contributions he made, obviously left a lasting impression. Dinner was great too, I didn't catch the name but we went to an Ethiopian joint in SF. The okra dish as well as their variation on samosa (sambosa?) were great, the sambosa had a very dark spicy sauce that was awesome, reminiscent of a mole' almost.

What else? Oh right, the mellow evening. I read to my son and got him asleep a bit later than usual ~10pm, he had a long day but I think a late day donut treat kept him hyped up (even after our long walk and lots of running). My daughter was at a friend until late, but happy when she got home, I had done some laundry (which is her chore) and she was totally cooperative when I asked her to fold clothes I think because she appreciated the helping hand.

Anyway, once the kids were asleep I finished reading one of Michele Davis' books on marriage. The Sex Starved Marriage for those curious, as it was the only one I could find at the bookshop by her. The framework in her books I think is similar regardless, and just puts it in different contexts. While the context of this I think has applied at different times to my marriage, right now it's a little different I think. Still, I guess I'll see if any of the suggestions help out, but the one of the best things for me were her examples of other couples' problems and reading ones that were similar as well as what techniques helped them out positively.

Anyway, I want to keep this exercise in blogging focused mostly on the positive, and not detail too many personal things beyond myself. But that said, the last thing I did was write a list of things that I have really appreciated from my wife in the past few days. That was good to focus on the positive in our relationship, just like I'm working here to focus on the positive in my life.

Oh, and undeadly turned 1 year old yesterday! I feel a little bad since I have been slacking a bit on it the past week or two with everything else that's been going on. But I told everyone on the editors list that I would be busy, so they can pick up the slack (of course they always can). Jose left a really nice comment too. I really appreciate his lending an ear, as well as his links and useful suggestions, it's helped me a lot in keeping motivated on the site, as well as provided some kind of consistency maybe? Anyway, since he aggregates this blog and might end up reading it (and all the lengthy babbling). Thanks Jose!

Outside of general accomplishments at work, not much positive to reflect on yesterday. Speaking of work, I just got another raise, and a favourable review. It's a good company, just not the most fulfilling for my career path, but it's nice to have their stability when everything else is not that right now.

Anyhoot, I did get work to replace my cell phone, and spent part of the evening moving contacts over, testing a bluetooth headset my wife got for free from some Plantronics dinner a month back. That's positive! Those who know my opinions on cell phones, just go along that that was positive. Oh, and I had a great lunch in Monterey at veg2u (weird name, but vegan/raw foods - very wholesome)

Tonight I'll be heading up to see Assemblage 23 with my friend Kevin whom I haven't had a chance to spend much time with in a couple years. Looking forward to catching up, and eating some good vegan cuisine, whereever that happens to be (Kevin is also vegan).

Been a few days, slow start to a new activity. I sorted through all my CD's yesterday and dug up the Leaether Strip disc with the quote I tried to paraphrase yesterday, here it is (from the liner notes of Underneath the Laughter):

"This album is the most demanding thing I've ever done yet. Like it or not. If you don't like this album, well I did it for my own pleasure anyway."

So, that's the intention I had with these entries a bit, just substitute 'diary' for 'album.'

Cleaning up was nice, also spent a lot of time with my son this weekend. Spending two days straight with him without breaks made me marvel at how talkative he is, energizer bunny not just in activities at the park and out and about but verbally too. He's firing on all cylinders. It's a real pleasure to spend more time with my family. I'm grateful that work and my schedule are being permissive of that.

Took my car in to get serviced this morning, it was nice of them to pick me up and drop me off - definitely great service, especially since the whole thing was free (the dealership gives free oil changes, car washes, basic services for the first 60,000 miles).

Met up with my wife afterwards to look at some houses and have lunch. While I was out of the office for about 3 hours, it was nice to spend some time with her without constant arguing. Bought some roses, smiled a lot at each other and talked a lot. Oh, and had a great meal. We also looked at a few homes. Of course, now we're both a bit torn about homes, but we've got options at least. And since lunch I've been thinking about it more and feeling calmer about that decision.

(got a call in the middle of writing) In fact, she went to see the planning department, about the two homes we're eyeing. Turns out one that I thought would be a better fit at first would be a pain to fix in the ways we needed. The other, which needs a lot more work, and it's harder for me to see the potential in however, is actually much better suited for getting permits and the like for the types of improvements we need. I think we're going to make an offer on it tonight. I am excited! It is encouraging that we're on the same page about something important and current, with neither one compromising for the other. I hope I have more successes to report in that light soon.

I was planning to go out tonight to a local elektro/ebm/goth club but I'm not sure I'll manage to stay awake at this rate with the time change. However, on Friday - Assemblage 23 is playing up in SF. It's been a long time since I've gone to a concert, and I'm looking forward to the experience. Maybe see some friends I haven't seen in years, and maybe make some new ones.

Also I've just been getting lots of organizing, cleaning and project prep done. Feels good to dig through old crap & discard a lot of it. I've also ordered a new hdd and some components for the home computer, time to do an overhaul of that, with my daughter I think she'll get a kick out of it.

And speaking of my daughter, I signed her up for a shadow at a local private school she's been interested in. Also did some hunting for a sewing class as she's expressed an interest in owning a sewing machine for a while. She started going to a knitting club at her school, but kept asking for a sewing machine anyway. After trying unsuccessfully at some sewing (and vacuum?) shops, I called a fabric store locally which has a four week class on Thursday nights. Sounds perfect, and the woman on the phone even offered to let my daughter borrow her spare sewing machine. I hope she'll get a kick out of that. Also gives her a chance to see if she likes it before we shell out a lot for equipment and fabric, not to mention try to find space for that.

Filling in some blanks from last week: had my first experience with acupuncture thanks to my sifu, Charles, it was really nice. The kwoon has been immensely supportive. Offering me good advice, and better than that - good practice. Be it sparring more to release tensions or healing techniques to do the same, I really appreciate it.

I've been going through quite a bit of turmoil in my personal life, namely with my family. Initially with my 13 year old daughter (I guess that's expected to be a challenge ;), but it seems to be rippling out to my relationship with my wife as well. It feels like my life is crumbling around me a bit, and the counselor, trusted friends and part of me has been telling me to do things for myself right now. That alone is really hard for me since I want to focus on repairing the damage with others. Others have warned that if I do not focus on myself, then at the end of this I will be an empty shell, worse off for healing. Whereas, if I do do things for myself, I will improve as a person regardless of the outcome, I'm up for that! Giving others something positive to look at instead of an empty husk I guess will help me in the long run too, but I really can't worry about my actions evoking a desired reaction from others. It's tough letting go, emptying my cup as they say - but I'm working on it, and looking forward to seeing what comes in to fill it.

At any rate, this morning the point was hammered home a bit, last night after a long talk with my wife, some arguing, some making up, very little sleep and a really nice lovemaking session - I woke up still feeling pretty miserable. It made me realize that even with things nice between me and my wife or whatever, my emotions might still be in the pits, that a lot of the resolution and healing needs to come from within me, not from the actions of others. This whole thing is all new to me, and I feel like I'm learning a lot about myself, but it's also really hard and I am unsure of what I am doing a lot of times.

So, why am I writing this? I've been writing a hand written journal of all my negative thoughts to get them out - I guess it's OK, it's definitely good to get them out, but in the end rather than a healing catharsis response I had been hoping for (though I guess not expected) I pretty much just feel drained. So, instead I thought I would write down the things I am doing for myself, to help me keep track and feel like I am making progress. I plan to use this as a tool to focus on the positive things, maybe it will make me feel better.

Why here? Well, I don't carry the paper journal with me, and if I'm going to focus on the positive, no reason to hide it. My dad always used to tell me things like, "let your light shine" or "toot your own horn" and I have rarely done so. Not that a blog is exactly the best place, but why the heck not. While I'm not doing this for others, perhaps some will see something positive in themselves reflected in my documented experiences that they can related to. While thinking about this I was reminded of Claus Larsen's liner notes (can't remember the album, now I'll have to dig it up and edit this with the album title and precise quote, plus I would enjoy listening to some LeaetherStrip right now), but he said something like:

"This is the hardest thing I've ever done in my life. If you like it, great - but I did it for myself."

That's kind of how I feel right now, and why I don't mind writing about the positive here even if others might see. I don't really know how many others might read this, and right now I'm not too concerned about how many might - it's just a space for me to get things out. I won't apologize if I ramble on, or if the things I talk about are relatively insignificant, maybe they are to me too - but right now it feels like I don't necessarily have many profoundly positive experiences to talk about, so I will just mention what little things that do come to mind.

With that introduction, I'll begin.

----------------------------------

This morning, after dropping off my daughter at school and before going into work, I stopped at Rio Del Mar beach and parked the car. I had just planned to eat my breakfast watching the waves, but couldn't get up the appetite, my stomach was in knots, so I got out and started walking. While walking I came up with the idea that I should document the positive things I'm doing, which is what I'm doing now. :) So there's one. Walking in the sand I was tempted to take off my shoes, but just as the thought entered my mind I walked over a bunch of broken seashells, and thought twice, another time I'll do that. I walked probably a mile or so down the beach before turning around, and as I did, I noticed my footprints and began to step back on them, marveling a little at the ease which with the steps fell into place back where they were, without having to put much intention behind following my path back. After a while of this, and enjoying the waves and air with the sun to my back, a dog walked up to me looked and looked straight into my eyes. I put my hand out to let him smell me, then he smiled and turned to his side and let me pet him for a bit. He dashed off moments later, and as his owner came up and remarked, "He doesn't usually let anyone touch him." I wished them a good day and started walking again, smiling that this dog had seen something in me to come say hi, that I had not asked for at all, then felt some tears welling up as I affirmed to myself that I am a good person deserving love and respect. They didn't come out, not because I held them back, but I quickly noticed that I was no longer following my footsteps back. I snapped out of it and felt that at least metaphorically, I should stop following the same steps back. I didn't want to return to the car along the same path that I left, I wanted to return having walked a new way back, without the same dismay, feeling uplifted. With that I took a deep breath and continued on along a new path back, closer to the ocean. I even saw a barefoot footprint here and there! Next time I'll keep a towel in my trunk so that I can dust my feet off before hopping in the car. I saw a surfer suiting up and was reminded that I should follow my desire to surf myself, and in the front yard of one of the multi-million dollar homes along the beach I heard a child crying, turned to look and saw a young girl maybe 5 or 6 barely older than my son, with a jump rope, bawling like it was the end of the world. It reminded me that even those with all the material benefits one could hope to obtain, that happiness is still found from within and that family life is full of its ups and downs regardless of the family. I returned to my car feeling refreshed, looked for a leaetherstrip CD in my trunk to no avail, and ate a banana, orange and some green tea while watching the waves and listening to the crashing sounds. It took only an hour out of my day but I left in a much better state of mind, feeling calmer and more positive and eager to write here.

There's even more I want to write, about some midnight oceanside walks I've made in the past few days with the full moon, and standing i-chuan exercises I've done that left me feeling buzzing with qi in a way that I've rarely experienced and am so excited to feel after years of wanting to get to that level of skill, it's incredible to be feeling these things with just some time spent working at it. But another time - for now let me just say that the moonlight reflected in the ocean is a gorgeous sight to behold, and that there is a LOT more to internal martial arts practices than I think many believe is even possible, it's amazing to see entire new dimensions of experience open up as I continue to train, and I expect it to be the same with my other endeavours, as difficult as they may feel now.

11 Mar 2005 (updated 11 Mar 2005 at 21:02 UTC) »
TWO PRACTICAL METHODS FOR STAYING RESISTANT TO THE EFFECTS OF HASH COLLISIONS (that are already in use)

(modified somewhat from my undeadly comment to the OpenSSH 4.0 release and Damien mentioning he foobed the md5 on the release notes, wanted this to get out to a wider potential reading audience, I and I don't really have a suitable personal blog so this will do)

IANACBIMPOOF (I am not a cryptographer, but I may play one on fora)

http://cryptography.hyperlink.cz/md5/MD5_collisions.pdf is worth taking a look at. I realize it's recent (March 5th), but gives an example of finding a full md5 collision in 8 hours on a notebook, and they're predicting that time will go down once Wang et al release their actual speed up method (perhaps the prediction of 2 minutes is overstating it, but you never know). That said, getting a collision on meaningful substitutes (e.g. a backdoored OpenSSH) might be another challenge, but I doubt it's going to be too much harder if speed keeps increasing.

At any rate, I was wondering - why provide just one type of hash (e.g. just md5) if you are releasing something? Why not also provide a sha-1, or even several different hash types? As we witness more hashes fall victim to improved collision attacks (and there will _always_ be collisions anyways because that's the nature of a frigging hash), it's understandable that any individual (md5, sha-1, crc32, whatever) hash will have possible meaningful collisions. However, finding a meaningful collision for _several_ different hashes simultaneously, I would posit is probably very unlikely.

It could make a damn fine fun challenge to break or open up a new science problem. I can just envision future math assignments where the teacher is telling their students to find the Lowest Common Denominator for crc32, and md5 values.

Maybe I'm wrong about that in some cases, as I know sha-1 is based off of previous work from md5, so maybe any sha-1 collision also results in an md5 collision. But I highly doubt that, since such a property would undoubtedly have been noticed and mentioned by the researchers breaking this stuff. Or at least one would hope such an obvious check would have been noticed in such research, I haven't seen mention of it. If anyone has examples let me know, by no means have I read every paper on the subject. Regardless, it wouldn't be hard to check, just take an example of a two different files wherein they both have the same md5, and then see if they both have the same sha-1 (or crc32, or ripe-md or whatever-the-hell-hash you want to have as well). Does anyone have two files that have an md5 collision I could test against? Would be simple enough to perform the test if I had the files, but IANAC.... stuff above.

Also, it should be noted (and Jose thankfully reminded me of this at RSAConf when we were discussing hashes briefly) that the OpenBSD ports system already provides several different hashes on distfiles. Just check a /usr/ports/blah/blah/distinfo yourself and you'll see something like this:

$ more distinfo
MD5 (nmap-3.81.tgz) = 9b32f74e2f6999e4f7668a24f2a1ea85
RMD160 (nmap-3.81.tgz) = d57533f1bf614541dd0cdfcf0f14b257d26a28c9
SHA1 (nmap-3.81.tgz) = 9d1ce1ab3e097ce5d61078fd4bc713f9b701fa1c
SIZE (nmap-3.81.tgz) = 1846196

So, since OpenBSD does it in the ports system already, maybe we as an entire security community should look to add it to our release methodologies as well? (See update below on this - while several different hash values are provided, only one is currently checked in the OpenBSD ports system - checking more than one must be done manually).

Put another way - given the properties of hashes, any one hash is likely to fail, but many hashes all failing together in the same way at the same time is very unlikely.

Another proposal for trying to skirt the problems with hashes, rather than just invent a new hash that hasn't been "broken" is to do what bittorrent does. Take _multiple_ hashes over parts of the file rather than just one hash of the whole file. Again, it becomes highly unlikely that one could generate a different file that would be split myriad times and have the same crc's for each split piece.

As an example, a recent torrent of the hitbSecConf2004 vids I leeched had over 7000 pieces - and afaik, in the .torrent each file sub-piece has its own hash value listed. The .torrent files aren't plaintext so I can't verify this easily, I'm basing this understanding upon what I've seen written up about bittorrent. Assuming my understanding is roughly accurate however, in the case of bittorrent in the example provided instead of finding one collision, the attacker needs to generate over 7000, one for each file sub-piece. Even if using something not cryptographically sound or very resource intensive with easily found collisions such as crc32, that becomes a tougher problem.

Of course, this is speculative, maybe it's not that hard. If an attacker is smart enough to put the meaningful change in only a small number of sections, then possibly he would only need to create one CRC collision for that sub-piece, or several for the several sections, and the rest he could leave untouched, and they'll all generate the same hash. I don't know for sure but it's a thought, and moreover already an implementation that I think will prove itself to be rather robust against hash collision attacks that keep improving, even as bittorrent's chosen hash itself will undoubtedly fall prey to smarter researchers over time.

So one problem, two possible ways of dealing with it that can, and are already in use today. In other words - other people should start using these techniques NOW to afford protections, rather than sitting around waiting for some silver-bullet sha-1 replacing hash to be approved, which undoubtedly will also crumble over time. I mean, that's not to say that right now there aren't already other hashes we could be using for which there aren't such attacks for - and by all means we could start using those right now to as a preventative. But given the properties of hashes, it's probably just a matter of time and researcher attention before other algorithms fall victim to more efficient collision generation techniques. So rather than put all our eggs in one basket, or foresake hashes which are still useful the majority of the time, we can just get creative with how we use our existing tools.

Updates and Corrections!

Jolan informs me that even though the OpenBSD ports tree records several different hashes in distinfo, it only checks one. So in order to really make this work for OpenBSD's ports system in the manner in which I'm discussing, the user needs to manually check against the other hash types. Currently that means that the disk would need to be read for each distinct hash as well, so that can obviously be time consuming. In my personal experience, the biggest bottleneck in generating hashes is disk reads - so if one were to check a file with several hash types, it would be wise to design a system in which data is only read from the disk once, even if that data is being fed to several algorithms.

Jolan asks: "got code?"

Chris Palmer correctly points out:"CRC" is not a synonym for "hash", and certainly not "cryptographic hash"

I had some more time sooner than I thought and removed the abusive CRC references Chris mentioned, thanks! I'm not as concerned about the distinction for cryptographically strong hashes actually - just as long as people use hashes that aren't all suceptible to the same collision weaknesses at the same time.

10 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!