the 26th alliance
Thought i should talk about the initiative that we launched on jan 26th...
Powered by Wufoo
Follow 26th on twitter, or stay tuned to the 26th project on github.
the 26th alliance
Thought i should talk about the initiative that we launched on jan 26th...
Xmas Hols Advice for juniors in collegeThe following is a edited version of mail conversation I had with a junior from SVCE asking for advice, thought it would make a decent blog post as well.
Erlang, distribution & bspawner
Sriram Krishnan was enquiring about distributed programming tools and all things distributed. No doubt, the first thing that came into my mind was "erlang", and taking into consideration that he was inviting feedback & his recent work on cacheman and on server /app performace i thought it would be a good time to talk about bspawner - my own pet project among other things erly.
Load Balancing & Introducing bspawner
Erlang nodes can communicate over the network if they find each other. You can send a message to any process (differ from OS process) through its Pid – even if the process is on another node.
Although you don't have to know where processes are, it is still up to the erlang programmer to decide which node initially spawns a task on which node as there is no inherent way to make use of multiple nodes , even though they recognize each other & can message-pass between them.
bspawner is a project i've open sourced - that attempts to load-balance the task of spawning across multiple nodes . The steps involved can be isolated into a couple of distinct problems.
This project in its essence, deals with the first part and the implementation of the message passing begins with the "messenger.erl" sample program and modifiedto handle the intended message-passing, load-balancing and node-information
Cheers to the feedback, suggestions , comments, and further changes inspired by the growing involvement of the erlang community & encouragement from #erlang in particular.
However, since the project is still in its early stages - and although it was a fantastic learning experience - I learned that Erlang already had inbuilt load-balancing module that I quote:
How cool can it get! Since it's even build in a master-slave basis - when one worker goes down, restart strategies can be configured courtesy of Erlang's OTP supervisor and behaviours. this basically allows you to horizontally scale and have distribute processing among box's. I'm even wondering to test this setup at hover.in on weekends by maybe doing some bizarrely wild clustered processing job like ...
pool can be used to run a set of Erlang nodes as a pool of computational processors. It is organized as a master and a set of slave nodes and includes the following features:
- The slave nodes send regular reports to the master about their current load.
- Queries can be sent to the master to determine which node will have the least load.
Testing & Test suites
I'm not much into the whole test suite setup as yet, but there a good test suite that is widely adopted would be
Distributed Monitoring & Debugging
Regarding utilities for debugging & profiling ,etc apart from user-contributed packages, and there are loads of them on process-one, trapexit , jungerl, google code ,etc - I thought I would list few of the interesting utilties . ( see more on the left side of the documentation at erlang.org under Tool Applications )
Behaviours are templates or formalizations of common design patterns. The three inbuilt behaviours are gen_server (client-server paradigms), gen_event (event-driven paradigms) and gen_fsm ( finite state machine paradigms). In addition you can create your own behaviours and have modules implement them. (Emacs in erlang mode, even gives nice skeletons for all behaviours, common design patterns)
You can also have stateless or stateful process's which can run indefintely, or be blocking in nature or exit after a timeout. Which really opens up a whole lot of possibilities, but again - since the key is in message passing - regardless of the process's being on your local node or half way across the globe.
To think about ....
Updates on Identities on the web
Some thoughts on identities and their related data , online accounts and related news over the past few weeks.
Developing developer platforms Part-1
It's no secret that I've always been an ardent admirer of companies that invest in impressive internal tools, developer resources, and internal things that make your developers go mmmh! Perhaps thats the secret to how they manage to release public api's and platforms for developers quickly in their product life cycle than companies with more mature products,but spend more time and effort trying to open up API's even for internal use in a plug-n-play, easy to lookup&hack fashion. Facebook & YUI will always be remembered for the part they've played in this very paradigm.
So, I thought it was time to kickstart my own soul searching, API intimidating, developer friendly environment right here at hover tech.. . What follows is a of the platform for the developers ( thats emp#1 srivatsan, and upcoming recruit thyaga- both theoretical CS & functional programming fanatics ) who one day will i hope influence the lives of several thousands of users, just as my own programming rockstar idols have done in the past.
Firstly, I jotted down a number of things that would seem fairly familiar in an environment where you track api changes, features, test them in real-time ,and discuss/update them. Things that came to mind in the first version of db_api were
it doesnt matter what language you write in, if you can write the future
I often hear about an upcoming platform going to revolutionize the next decade. Now I have no qualms with this , and infact biased towards big-thinkers . But i just think that on the contrary this is an inherent potential that any platform / technology can claim. How big was it designed to be when it was visualized ? Were they thnking big enough when the users were thought of ? I think these are interesting questions. Because for all practical reasons ,it is you who writes the future. Literally in your code , in your packaging, and its scaling before it goes out there for better or for worse. Who would you blame for X's success or failure? seems reasonable enough to claim X to please stand up. But looking deeper I think it's only fair to say that you may have limited yourself to build something just big enough for a million people to notice that it's doing well. Might sound cruel? but thats the way the way consumer web products space works.
Vision of a startup
What would you think of a 3 month old startup for sale on ebay ? TechCrunch would put you in "the deadpool". The founders could call it an exit. But the obvious reason in the difference in reactions is that the founders may have wanted just to hack a product in a couple of months, when they started off and get bought over . On the other hand, i completely understand why the founders over at Yahoo! were still not satisfied with 65$bn dollars. I can see why Zoho had'nt diluted much when they had numerous bids,much to the misery of the biggest players and investors in the world. Because that's not the future they want to write, and it simply is'nt time yet, they are'nt there yet, ... and miles to go before they sleep.
Writing the future
If you think you can "write the future", please let me know . I think i can too and maybe we could start a forum for us ;) . Maybe we could go back and think of all the things we are'nt doing. Maybe we could think of all the things that show you that there are so many things that we still don't know. But at least we know what we don't know , as cliched or odd as it sounds. We know that we suck at some things. And these are the kind of things that are needed to get you doing the things you want to do .
I call it a "reverse engineering process". It's a fantastic feeling when you can identify what you want in life. "What you want" ,not the how or the if's. But the long-term dream. Although that question itself more often has more to do with what you don't want . That's what incidently provoked my entrepreneurial ventures . I wanted to be responsible for my own failure /success by taking things in my own hand. And fortunately or unfortunately - recognition can only come in two scenarios. I can't remember what the second one is , but the first one is - as Paul graham said once -is a startup . You don't need to be a chemist to realise that the implications of your actions is best seen in a isolated environment. So that definitely convinced me that joining a 100,000 services company would be conflicting to my interest (no offense they are amazing wealth & welfare creators ) . Agreed that youngsters and executives from today's indian middle class can afford to take the risk. But they don't. I have nothing against people who don't want to write their future. But ignorance is something that something i love to fix. And I have no problem admitting to it- because i think it's a human right violation to not find out how ignorant you are. And i love finding out how ignorant i am. Just love it! Because then it's upto you to decide what ignorance you want to rectify , before moving onto the next ignorance to rectify,...and so on! And the earlier you start , the better. They should have dropped you in a pool,the day you were born . Hows that for a claim to evolve for survival. ( jk! )
Will every moment in your life bring you in a position where you are an expert. Nope, so like i said, the sooner you start the better, and believe me - everyone in the industry is watched. And people only just want you to keep doing what you have been doing, but only with fancier names. Now depending on your appetite for risk , you could either take the road that puts you in the situation at par of your natural growth-path - in a rat race just without recognition. OR purposely take yourself through the road less traveled ,where the solution is not known, but what is eminent is that you will not know what to do, and you'll have to find out and get through it & bultly put - be a creator of something out of nothing. And my definition of getting through it is simple. I dont care if that dream is getting into being a Google or getting into the payroll of the indian post office's . can it be done ? if it can be - then great . Take solace in the fact and accept that there will situations of embarrassing ignorance, and a cost to innovation which you will learn the hard way. But if nobody has done it - then that's a couple of patents right there ! You might not be able to build a real estate search engine for the moon yet, but you could improvise on the fact that you've come up with it . Depending on how many years the concept is realistically away "crossing the chasm" - you could probably patent, or discuss with your peers. Get recognition,possibly mocked at, but keep doing it until you are recognized by your peers as someone willing to try everything atleast twice, and keep innovating within the community. And if you can't even get to your peers who are looking out the slightest glimpse of innovation & market erach - then how the hell do you plan on reaching your consumers half way across the world!?.
Which bring me to the what you want to do again . Do you want to build years of expertise in a technology before launching your own company in a totally different technology ? ( Remember everyone in the industry is being watched - whether you notice it or not and the people who invest in you or your company - predominantly want you to continue doing what you have been doing). Tricky, tricky,tricky....
Does the language/medium/timer matter ?
On the other hand a hacker or an artist would know that it really does'nt matter what technology/medium you're working on. I'd love to could pick up ice-sculpting as a career if i thought that's the future i wanted to see myself in . What would linus torvalds, be doing if he was a kid today or if windows was 'nt closed source, would he have still hacked linux ? or build a great product that revolutionizes the next 50 years ? would the hexa hackers of yesteryears be working on hacking pixels on 384's or would they be building a holographic 3d second life ? would paul graham have manifested his ideas of "move to the web" in the form of a haptic interface to yahoo stores / amazon / ebay if he was hacking today?
I think that great innovators can innovate independent of their times, and the languages they use ,and were'nt in any more a gifted environment in their days , as the one we are in today, or anytime in future. This btw, i reiterate is my view on all those flame wars and rants on programming languages . I think it could be best explained in this great little xkcd strip that shows how different editors and talk about how old school they are, from the more prominent ones ,to and end up saying how "real hackers" write in assembly language or else straight away magnetic tape or conjure voltage from the sky and binary. hilarious, and it's the first thought that comes to my mind when i see a language /flame war or rant.
Reverse Engineering the future
The fact of the matter is that we dont look to reverse engineer our future, and that my friend is the only way to write the future. How much are you going to keep studying before you think you 're ready to change the world ? I may not know the answer to that, but i'm happy to question the same. To question what the optimal qualifications, or the geography is to do what you want to do in life. In all the articles ive read on startup founders, and innovators - i'm surprised that i've never read of anyone mentioning "reverse engineering" your future. If you wanted to see yourself as the official advisor to the queen of england on quantum computing , or lead the indian ultimate frisbee team, or the ambassador to bermuda in secondlife or trade shoes with peter norvig.
An example of a "reverse engineering your Future" log
eg : Final Destination : ?
ideally , before I ...Final destination, ... i'd like to .... "leanr the intricacies of an IPO ", "learn to do a 'ballmer'"
ideally , before ...
ideally , before I ...0 , i'd like to "build an awesome product" and Y
ideally , before I ...Y , i'd like to "build a rockstar team" ...
ideally , before ...
ideally , before ...
Ideally , before i get/do N, i'd like to do get/do( N -1 )
Start off with a N = 5 goal , or a N = 2 goal . If i were to be scobleized next week , what would i have to liked to do this week, how many twitter message would i have to ping him with , etc ? If you wanted to get an grueling feature X done , what unknown thing would i have to read in the last 10 minutes before sleeping for the next month.
And since the final destination keeps escalating as you progress through the N things, based on your ability to execute what you want to do . The truth is that trying to start off with the Final Destination from day1 may not be quantifiable, but what is quantifiable is documenting if doing something is likely to get you closer to the eventual goal. Be it something trivial or absolutely drop-dead common sense. Thats the reason why It really struck me as to why people havent thought of how their personal identity on the internet would be jeopardized in step X of your reverse engineering lifecycle because of something that sounds silly today. That's also another reason why it's pretty darn easy for one risk taker- to find another hign-risk -high-gain motivated person. Perhaps thats how companies flock in the ecosystem themselves. Ofcourse, it all makes sense. How many people woudl want to let go of opportunities that they dont take today ,when they think about it 10 years down the line - you'd give anything to go back and do it right ? The thought of all the things i could do in that each day, in one week , or even one year keeps me up several nights. Honestly, i'm particularly happy with the way things have gone over the last 6 months with hover tech as well. It only has if anything only strengthened my conviction that you really can write the future if you want to.
The question for you/me really is and was always supposed to be , what kind of future do want to write...
Bhasker V Kode
Blogged with Flock
pipesCamp, unconference for yahoo! pipes enthusiasts [del.icio.us]
pipesCamp, is an unconference for yahoo! pipes enthusiasts, and will be held in Chennai, India for the first time on december 23, 2007 .
First pipesCamp unconferenceyahoo! pipes is one the most truly innovative applications that have provided a platform for application developers and end-users alike. And ever since their launch, several reviews later, it has grown well beyond being just a "An interactive feed aggregator and manipulator" . Be it a toolkit on Wall Street , a utility on a box or iphone hacks.
Squeak, Erlang and Startups @ chennai.rb meet
"The AJAX Experience" sites are cool. But Why Is It In Flash?
"Hopefully someday - good friend of mine" Mike Arrington ; ) started this awesome series on techcrunch . I'm so glad this came up - i've been itching to tell it to as many people as possible ...
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!