Older blog entries for mcr (starting at number 75)

2 Mar 2011 (updated 17 Mar 2011 at 18:14 UTC) »

Deploying Django applications with Capistrano

Yesterday, I cooked up a deploy.rb so that Capistrano can deploy a Django application. While there is a Python app called http://docs.fabfile.org/0.9.0/ from what I could tell, it was very general to running commands on multiple servers, and not really specific to checking out a web framework and deploying it to one or more servers.

First, my deploy.rb, and then my notes about how I used it. I have changed only one or two things from my real code. My application is called "clientportal" and the host running it is called "clientportal.isp.example.net". On the server, it runs as a user called "clientportal".

This code does not yet invoke the Django database migrations, which it ought to, and I'll do another blog post once I figure out that part.

set :application, "clientportal"
set :me, "#{ENV['LOGNAME']}"
set :repository,  "git+ssh://#{me}@code.credil.org/git/path/to/repo/clientportal"

set :scm, :git
set :user, :clientportal

set :ssh_options, { :forward_agent => true }
set :use_sudo, false
set :git_enable_submodules, true
set :deploy_to, "/home/#{user}/#{application}"

role :web, "clientportal.isp.example.net"     # Your HTTP server, Apache/etc
role :app, "clientportal.isp.example.net"

# This is where Rails migrations will run
role :db,  "clientdb.isp.example.net", :primary => true

namespace :deploy do
  task :start do ; end
  task :stop do ; end

  # this overrides a rails specific thing.
  task :finalize_update do ; end
  task :migrate         do ; end

  task :restart, :roles => :app, :except => { :no_release => true } do
    # something to restart django.
    run "sudo /usr/sbin/apache2ctl graceful"

  end
  task :update_database_yml, :roles => [:app,:web] do
    db_config = "/home/#{user}/settings.py"
    run "cp #{db_config}   #{release_path}/settings.py"
    run "ln -f -s #{release_path} /home/clientportal/clientportal/clientportal"
    puts "Ran update database settings"
  end

end

after "deploy:update_code", "deploy:update_database_yml"

Some details. First, I put my settings.py file into my /home/clientportal directory. I do not check this file into my repo, because it always specific to the installation (it's different on your laptop than on the devel server or the production server). Also see my:

Like http://blog.perplexedlabs.com/2010/02/08/deployment-using-capistrano-and-webistrano-via-rails-and-phusion-passenger/ I had to adjust my django.wsgi file as well. I wound up with:

import site
site.addsitedir('/usr/local/pythonenv/CLIENTPORTAL/lib/python2.5/site-packages')

import os, sys

sys.path.append('/home/clientportal/clientportal')
sys.path.append('/home/clientportal/clientportal/current')
os.environ['DJANGO_SETTINGS_MODULE'] = 'clientportal.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

The important changes were to the path that was added. It used to add $HOME/clientportal and $HOME to the path, but now it is one directory deeper, and you will notice above in the update_database_yml task that it creates a symlink in $HOME/clientportal with the name "clientportal" that is essentially the same as "current".

This is necessary because the settings are loaded as "clientportal.settings", and python basically turns the . into a / when looking for the file. I could have just changed the name of the settings file, but we had other modules that were loaded using the clientportal. namespace.

Note that the server already had it's apache configured to do what was needed. I would normally package these config files up into a .deb file, but I haven't done that yet for this project, it being my first django project.

I am not sure if I actually have to restart apache. I added that for good luck, and and I added:

clientportal ALL=NOPASSWD: /usr/sbin/apache2ctl graceful
to sudoers.

My apache config looks like:

<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        ServerName clientportal.isp.example.net
        ServerAlias portal1.isp.example.net
        ServerAlias portal.example.net

        DocumentRoot /home/clientportal/clientportal/current
        <Directory "/home/clientportal/clientportal/current">
                Options Indexes FollowSymLinks
                Options -MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog /var/log/apache2/error.log

        Alias /media/ /home/clientportal/clientportal/current/media/
        WSGIScriptAlias / /home/clientportal/clientportal/current/wsgi/django.wsgi
        <Directory /home/clientportal/clientportal/current/apache/>
                Order allow,deny
                Allow from all
        </Directory>
...

Some other links I found, but I didn't use much: http://groups.google.com/group/django-developers/browse_thread/thread/f34e59275e04f9c5?pli=1 http://gnuvince.wordpress.com/2008/01/10/deploying-django/

Syndicated 2011-03-02 12:06:00 (Updated 2011-03-17 18:14:24) from Michael's musings

8 Feb 2011 (updated 2 Mar 2011 at 17:14 UTC) »

To disable gnome-settings daemon from messing with your background

If you are like me, and do not use much of gnome, when you start a gnome application, it may start gnome-settings-daemon, which will mess with your background.

gconftool-2 --set /apps/gnome_settings_daemon/plugins/background/active --type bool False

Thank you to: http://ubuntuforums.org/archive/index.php/t-874816.html

Syndicated 2011-02-08 11:03:00 (Updated 2011-03-02 17:14:36) from Michael's musings

3 Jan 2011 (updated 8 Feb 2011 at 16:09 UTC) »

How can Research in Motion survive?

There are lots and lots of opinions in the media about whether or not RIM and it's flagship product, the Blackberry can survive. For instance, a quick google gives me: http://gigaom.com/apple/poking-holes-in-rims-anti-apple-rhetoric/

http://www.cbc.ca/technology/story/2010/07/14/rim-blackberry-6-annual-meeting.html

http://www.thestar.com/business/companies/rim/article/834957--rim-could-face-tough-questions-today

A friend and I discussed this over lunch the other day. We care only because as far as the Government of Canada's IT policy is concerned, they see RIM as a golden goose who can do nothing wrong. RIM is an endorsement for the government of canada's "IT policy" --- it's a sign of success: those of in the industry see it as more of a four-leaf clover. Something that was happened in Canada by chance, not by design.

Assuming that we wanted RIM to survive, what would we do as CEO?

We have a lot of challenges, and we think that many of them are internal, not external. The major one is the huge amounts of Not-Invented-Here (NIH). The second major issue is the degree to which RIM is the centre of the Blackberry world, with data from most models of phones going through expensive data centres for reprocessing. When the Blackberry first appeared, it did not have Internet connectivity, but rather worked through various proxy servers and HTML-reformatters. We were surprised that more people didn't ask questions when CIBC sued some former employes, and subpeoned emails from their BlackBerry.. Why was the subpeaona served to RIM? http://www.robhyndman.com/2005/01/06/cibc-sues-six-former-employees/ http://www.heydary.com/publications/blackberry-pin-monitoring.html

Today, it does better, but few developers really want to write apps for the Blackberry JavaMobileEdition. Android is seriously taking a lot of market.

What advantages does RIM have? Sure some patents about some user interface things (mostly physical stuff).

What it does have is a brand name, and Blackberry devices are considered serious status symbols. For instance,a salesperson I was meeting with explained to me that, "of course", his company offered to get him the latest BlackBerry, but he found the keyboard too small for his aging eyesight, and preferred to keep his 2 year old unit.

Along with this brand name is a pretty good email system with pretty good integration into Microsoft Exchange. This is something that Google and Apple does not have as well done. (Yes, Google has most of it, but they don't have the same level of trust from the right places, and many Microsoft IT fanboys hate Google, just on principle)

The problem that I see is that Blackberry has started to go after the consumer market, and with this, they are diluting the BlackBerry brand name. Used to be only big companies could get email integration, and only the important people had BlackBerrys. Now every second person on the bus has one... and they aren't even the cool people. The cool people have their own iPhone or Android. If a cool person has a BlackBerry, it is because their company made them take one because it integrated, but said person has their own phone for their real use.

So, my advice to RIM is as follows:

1) break up the company. Spin off BlackBerry hardware as private company. Have them make handsets under the BlackBerry name. Sell them for premium dollars. For about 18 months (one hardware design cycle), they can make BlackBerry OS units, but by mid-2012 they have to shipping Android as the base OS.

2) port all of BlackBerry's custom software to Android in native mode. (i.e. not Java). I do not think much of the core software on the Blackberry is written in Java Mobile (I could be wrong, it could all be JavaME now, but it wasn't a few years ago). Porting to native is probably easier, and may even make it easier for them offer some unique features.

Native mode code often requires a rooted phone to work well. In this case, it is a feature, not a bug: the target audience is not end users in some sense, but rather, carriers. If you can get BlackBerry Email on just ANY smartphone, then the carrier does not get a chance to charge more for this.

Many carriers have a BlackBerry plan which is different than just Internet, because they know that BlackBerry's can't run torrent clients.

3) create the RIM cloud, and go into competition with GMAIL, Rackspace's Mailtrust, etc. Offer strong integration into corporate email, and offer various DRM-ish controls on what can be done with email that is accessed via the native apps. (it's all pretend security of course, but many people seem to insist on drinking coolaid...)

What is the result?

  1. RIM is no longer competing on price at the low end. Rather they are using the existing low-end smartphone makes to drive corporate/carrier business to them.
    1. RIM's hardware spin-off is still making high-end, high-margin handsets for executives. This helps to return much of their status symbol. The new handsets should be easily distinguished from the old low-cost ones they used to make. (New colour? Breath-mint dispenser? ...)

      Since they are offering the same integrated apps on other vendor's phones, it means that the peons who need to be "integrated" no longer need BlackBerry handsets, and so nobody will confuse them with important people.

      1. RIM's communication cloud can expand into areas they have no yet been into. This is where the money is in the future. How about if executives can now approve expense reports/authorize-purposes from their units via digital signatures?

Will RIM do this? Unlikely. RIM will be Canada's Polaroid.

Syndicated 2011-01-03 17:36:00 (Updated 2011-02-08 16:09:49) from Michael's musings

14 Dec 2010 (updated 3 Jan 2011 at 23:10 UTC) »

WIND Holiday Miracle

<pre>

Michael Richardson: what can you tell me about the Holiday Miracle Plan?
WIND Mobile: Connection established.
WIND Mobile: Initiating Call, please hang in there!.
WIND Mobile: Connecting...
WIND Mobile: Hey, Welcome to Live Help! A WIND Specialist will be with you soon.
Michael Richardson: I hear that there is a $40 plan, which is unlimited calls and unlimited data.
Tiffany: Hi there. Thanks for joining the conversation with WIND!
Tiffany: I see you would like some information on the Holiday Mircale plan
Michael Richardson: yes.
Michael Richardson: I couldn't find anything about it on the web site.
Michael Richardson: I have the 100minutes plus unlimited data.
Tiffany: It's not listed on our website at the present time
Tiffany: It includes: *        Unlimited Canada-wide calling

Unlimited US Long Distance

Unlimited Canada/US Text messaging

Unlimited Canada/US MMS (picture messaging)

Unlimited Global Text Messaging

Caller ID

Unlimited WIND to WIND calling

Call Waiting, Call Forwarding, 3-Way calling

Voicemail

Infinite BlackBerry or Infinite Mobile (depending on the device)

Michael Richardson: I really need more minutes, but actually can live with less data. Tiffany: for $40 Michael Richardson: sounds perfect. How do I switch to it? Michael Richardson: is the the $40 a special rate, or a regular rate? Michael Richardson: will I find myself paying more in 6 months? Tiffany: It's a special rate Michael Richardson: what is the regular rate?

Tiffany: It's more than a $95 value Michael Richardson: I understand it's a good deal. Michael Richardson: How long does the deal last? Tiffany: The last day is Dec 26 Michael Richardson: what will I pay when the promotion runs out? Tiffany: It will go back to regular price Michael Richardson: what is the regular price? $95? Tiffany: more than $95 Michael Richardson: Please listen carefully. Michael Richardson: Is Wind going to give me this deal for $40/month forever? Michael Richardson: Or, like your %50 off on the 100 minute plan, only for 6 months? Michael Richardson: At the end of six months, what will I pay for the same thing? Tiffany: It's forever but you would have to make sure your account is active Tiffany: on a monthly basis Michael Richardson: I see. Thank you! Michael Richardson: How do I sign up for it?

Syndicated 2010-12-14 14:49:00 (Updated 2011-01-03 23:10:25) from Michael's musings

Ruby has problems with getaddrinfo(3)

Today I was trying to deploy some rails code to a host with both an RFC1918 and IPv6 address. The RFC1918 address is only valid within my CREDIL office, while the IPv6 is globally unique. When I'm in the office, IPv4 or IPv6 is fine, when I'm not, then it has to be IPv6.

With other applications when I do this, I sometimes get a delay as it tries the RFC1918 address, fails and tries IPv6 instead. SSH works great like this.

Ruby 1.8 (at least) fails: the RFC1918 address does not connect, and then ruby gives up. This makes Capistrano fail.

Capistrano uses Net::SSH, which calls TCPSocket.open. This is implemented in C code in the ruby interpreter. My reading of ruby-1.9.2-p0/ext/socket/ipsocket.c suggests that it might be okay in ruby 1.9, but I didn't look at the 1.8 code yet, and I haven't tried ruby 1.9.p

The following code exercises the problem, but you need to have an address which is both IPv6 and IPv4. It also does not seem to be consistent: it seems to depend on the network activity a bit. In theory, /etc/gai.conf can change the order that is returned, but I suspect that ruby is not using the system getaddrinfo(3).

% cat socktest.rb require 'socket' require 'timeout' factory = TCPSocket n = factory.open("sakura.gatineau.credil.org", 22) puts n.readlines n.close

Syndicated 2010-11-23 04:10:00 from Michael's musings

Ruby has problems with getaddrinfo(3)

Today I was trying to deploy some rails code to a host with both an RFC1918 and IPv6 address. The RFC1918 address is only valid within my CREDIL office, while the IPv6 is globally unique. When I'm in the office, IPv4 or IPv6 is fine, when I'm not, then it has to be IPv6.

With other applications when I do this, I sometimes get a delay as it tries the RFC1918 address, fails and tries IPv6 instead. SSH works great like this.

Ruby 1.8 (at least) fails: the RFC1918 address does not connect, and then ruby gives up. This makes Capistrano fail.

Capistrano uses Net::SSH, which calls TCPSocket.open. This is implemented in C code in the ruby interpreter. My reading of ruby-1.9.2-p0/ext/socket/ipsocket.c suggests that it might be okay in ruby 1.9, but I didn't look at the 1.8 code yet, and I haven't tried ruby 1.9.p

The following code exercises the problem, but you need to have an address which is both IPv6 and IPv4. It also does not seem to be consistent: it seems to depend on the network activity a bit. In theory, /etc/gai.conf can change the order that is returned, but I suspect that ruby is not using the system getaddrinfo(3).

% cat socktest.rb require 'socket' require 'timeout' factory = TCPSocket n = factory.open("sakura.gatineau.credil.org", 22) puts n.readlines n.close

Syndicated 2010-11-22 23:10:00 from Michael's musings

26 Sep 2010 (updated 23 Nov 2010 at 04:12 UTC) »

Mountain Orchard needed help, hit with frost

Today we went to Mountain Orchard. Alas, they had two problems: a very nice summer meant that the fruit was ready earlier, and frost meant that it was going to go bad on the tree, so they picked it all two weeks ago.

We could get windfall from the ground (half price) or apples in a bin, no picking from trees, alas. (I gotta note that the windfall on the ground, was often very good quality, and we came home with 30lb of windfall, and 10lb from the bins... not to mention the apple cider donuts...)

This is a place where social networks can benefit them: if we'd known 15 days ago, we might even have taken a day off work to visit, but we thought it would generally too early in the season. Facebook used to have groups, like the "Ottawa Network", which could have gotten the word out.

Of course this would cost them a fee, but it would have been worth it, as they could have sold more fruit at the regular price, and well, gotten people to pick it rather than hire people to do the work...

Syndicated 2010-09-26 18:12:00 (Updated 2010-11-23 04:12:35) from Michael's musings

14 Sep 2010 (updated 26 Sep 2010 at 23:11 UTC) »

Reset password for MSSQL

If you are using RHEV Manager, you'll have a whole headache of Windows 2008 and 2008 SQL Server crap. After an upgrade, we realized that we did not have the SQL server account password used. Redhat told us:

To recovery of sql server sa password should be easy by using the local windows administrator credentials to login and set the sa password.

sqlcmd -S .\sqlexpress -drhevm -Q "ALTER LOGIN sa WITH PASSWORD = 'new_password'"

Syndicated 2010-09-14 12:02:00 (Updated 2010-09-26 23:11:41) from Michael's musings

IPv6 with mcast UserModeLinux backends

I am doing some work with IPv6. (see http://bluerose.sandelman.ca/projects/show/unstrung )

I have a test network shown at: http://bluerose.sandelman.ca/repositories/changes/unstrung/doc/network-1.png

In automated testing I would normally use the daemon mode, with uml_netjig. In casual use, I was using the mcast backend, because it has fewer moving parts.... but my network interfaces kept remaining in state "tentative" and I could not send packets.

What was the problem, I debugging for awhile through the IPv6 code, and finally thought that it had something to do with the UserModeLinux network interface never providing low-level LINK "UP" signal, and so it never did Duplicate Address Discovery, and remove the tentative mark.

5: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 fe80::1000:ff:fedc:bcff/64 scope link tentative
       valid_lft forever preferred_lft forever
6: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 fe80::1000:ff:fe64:6423/64 scope link tentative
       valid_lft forever preferred_lft forever

Note that it says "tentative".

NO! DAD was occuring just fine, but it FAILS. Why? Because it thinks it has a duplicate... finally I noticed

eth0: duplicate address detected!
eth1: duplicate address detected!

Why is this? It's because the mcast interface gets a copy of the packets that are output. I.e. it hears itself. DAD should work even when that happens, I think.

I need to look at whether the mcast interface should be fixed (remember it's own packets and ignore them? Drop packets that originate from it's own MAC address?), or should DAD be fixed?

Syndicated 2010-09-08 19:45:00 from Michael's musings

8 Sep 2010 (updated 14 Sep 2010 at 16:09 UTC) »

How to run XEN on Ubuntu Lucid

http://www.chrisk.de/blog/2008/12/how-to-run-xen-in-ubuntu-intrepid-without-compiling-a-kernel-by-yourself/

documents how to install the Debian DOM0 kernel on Ubuntu LTS. This is an update.

aptitude install libuuid-perl
wget http://ftp.de.debian.org/debian/pool/main/l/linux-2.6/linux-image-2.6.32-5-xen-amd64_2.6.32-21_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/l/linux-2.6/linux-modules-2.6.26-2-xen-amd64_2.6.26-25_amd64.deb
wget http://ftp.de.debian.org/debian/pool/main/l/linux-2.6/linux-base_2.6.32-21_all.deb
dpkg -i linux-image-2.6.32-5-xen-amd64_2.6.32-21_amd64.deb linux-modules-2.6.26-2-xen-amd64_2.6.26-25_amd64.deb linux-base_2.6.32-21_all.deb

Syndicated 2010-09-08 15:47:00 (Updated 2010-09-14 16:09:37) from Michael's musings

66 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!