Older blog entries for dalke (starting at number 27)

My apologies in advance for the length. OTOH, I only post every couple months so it evens out.

I don't see anything wrong with our current counter terrorist policy. Rather, I don't see any good way to improve it, and I fear what's likely to happen will end up making things worse. Our policy is (was):

1. make no concessions to terrorists and strike no deals
2. bring terrorists to justice for their crimes
3. isolate and apply pressure on states that sponsor terrorism to force them to change their behavior
4. bolster the counterterrorism capabilities of those countries that work with th e U.S. and require assistance

Bush wants a "war against terrorism." What does that mean? Here are the 44 groups the US said in year 2000 are terrorists:

• Abu Nidal organization (ANO)
• Abu Sayyaf Group
• Armed Islamic Group
• Aum Supreme Truth
• Basque Fatherland and Liberty
• Al-Gama'a al-Islamiyya
• HAMAS (Islamic Resistance Movement)
• Harakat ul-Mujahidin
• Hizballah
• Islamic Movement of Uzbekistan
• Japanese Red Army
• Al-Jihad
• Kach and Kahane Chai
• Kurdistan Workers' Party
• Liberation Tigers of Tamil Eelam
• Mujahedin-e Khalq Organization
• National Liberation Army
• The Palestine Islamic Jihad
• Palestine Liberation Front
• Popular Front for the Liberation of Palestine
• Popular Front for the Liberation of Palestine-General Command
• al-Qaida
• Revolutionary Armed Forces of Colombia
• Revolutionary Organization 17 November
• Revolutionary People's Liberation Party/Front
• Revolutionary People's Struggle
• Sendero Luminoso
• Tupac Amaru Revolutionary Movement
• Alex Boncayao Brigade
• Army for the Liberation of Rwanda
• Continuity Irish Republican Army
• First of October Antifascist Resistance Group
• Irish Republican Army
• Jaish-e-Mohammed
• Lashkar-e-Tayyiba
• Loyalist Volunteer Force
• New People's Army
• Orange Volunteers
• People Against Gangsterism and Drugs
• Real IRA
• Red Hand Defenders
• Revolutionary United Front
• United Self-Defense Forces/Group of Colombia

The geographic areas listed under the "external aid" section: Iraq, Libya, Syria, "Islamic extremists in the Middle East and South Asia," Sudan, Lebanon, Nicaragua, Cuba, "reside in South America", Iran, Saudi Arabia, Western Europe, North America, "other Gulf and Islamic states", "Pakistanis and Kashmiris", "Central and South Asia", Europe, Asia, "Laurent Kabila regime in the Democratic Republic of the Congo", "sympathizers in the United States", Balkans, "Afghan Arabs and the Taliban", "Pakistani community in the Persian Gulf and United Kingdom", "Islamic NGOs", "Pakistani and Kashmiri businessmen", "Islamic extremists in the Middle East", "President Charles Taylor of Liberia", Gambia, and Burkina Faso. (elsewhere includes North Korea).

Take a look at the list of countries. Yep, people in the US help fund what the US calls terrorist organizations. Indeed, on May 16, 2001, Bush designated the Real IRA, 32 County Sovereignty Movement, and the Irish Republican Prisoners Welfare Association as "foreign terrorist organisations." Before then it was completely legal for US citizens to give them money.

It is believed that American supporters send hundreds of thousands of dollars to the [Real IRA] each year and that these funds are used to purchase explosives and weapons

    - Foreign Policy Association

Think the US is going to attack US citizens who fund terrorist organizations in other countries? Think the US will go after terrorist organizations in South Africa? In Ireland?

Hmm, just scanned the Anti-Terrorism and Effective Death Penalty Act. (Went into law in 1996.) Specifically, "SEC. 324. FINDINGS":

1. international terrorism is among the most serious transnational threats faced by the United States and its allies, far eclipsing the dangers posed by population growth or pollution;
2. the President should continue to make efforts to counter international terrorism a national security priority;
3. because the United Nations has been an inadequate forum for the discussion of cooperative, multilateral responses to the threat of i nternational terrorism, the President should undertake immediate efforts to develop effective multilateral responses to international terrorism as a complement to national counter terrorist efforts;
4. the President should use all necessary means, including covert action and military force, to disrupt, dismantle, and destroy international infrastructure used by international terrorists, including overseas terrorist training facilities and safe havens;
5. the Congress deplores decisions to ease, evade, or end international sanctions on state sponsors of terrorism, including the recent decision by the United Nations Sanctions Committee to allow airline flights to and from Libya despite Libya's noncompliance with United Nations resolutions; and
6. the President should continue to undertake efforts to increase the international isolation of state sponsors of international terrorism, including efforts to strengthen international sanctions, and should oppose any future initiatives to ease sanctions on Libya or other state sponsors of terrorism.

Who has power to say that something is a terrorist organization? SEC. 219. DESIGNATION OF FOREIGN TERRORIST ORGANIZATIONS says it's the Secretary of State, but the designation can be overriden by an Act of Congress.

Under Clinton, the designation was a very laborious and difficult process.

So pretty much Bush can declare any foreign group to be a terrorist group and, using some of the $40 ,000,000,000 requested from Congress, attack said group in any fashion whatsover.

What's a terrorist? Go to http://www4.law.cornell.edu/uscode/unframed/8/1182.html and search for "''Terrorist activity'' defined".

Interesting - looks like carjackers are terrorists:

(I) The highjacking or sabotage of any conveyance (including an aircraft, vessel, or vehicle).

Scanning down further:

(V) The use of any - [...] (b) explosive or firearm (other than for mere personal monetary gain), with intent to endanger, directly or indirectly, the safety of one or more individuals or to cause substantial damage to property.

If matters in the US are going the way I think they are, then we need to be very, very careful not to do things that, were they turned about, could be construed in turn as terrorism. Yes, perhaps we are justified, but it wouldn't be American.

One American ideal - treat others as equals. ("Ideal" doesn't always mean reality, sadly ):

As the sole superpower, it's too easy to ignore treaties, policies, and agreements we don't like. If this is indeed a foreign terrorist attack on the US we have to wonder what was the cause for this. Perhaps by not following international rules we end up making things worse.

We've got a justice system which can extradite and prosecute criminals. (Again, assuming foreign terrorists.) If whoever was involve hasn't been extradited, round up the evidence showing who did it and present the result to the world. (I'm reminded of Adlai Stevenson at the U.N. during the Cuban Missile Crisis.) That puts a lot of pressure on the host country.

But could they receive a fair trial in the US? I don't think so. Unlike, say, the trials of the terrorists involved in the American embassy bombings in Kenya or Tanzania in 1998, it will be hard to find unbiased jurors.

(BTW, in that case, two men were found guilty of terrorism and of killing 213 people. Their sentence? Life emprisonment. The jurors said they didn't want to turn the two men into martyrs. Yes, at least one was a follower of Osama bin Laden.)

If there's no chance of getting a fair trial in the US, the only solution I can think of is to prosecute that person in international court, as this would likely be called a "Crime Against Humanity."

But the US doesn't like international courts. (BTW, I'm not all gung ho for a single, world-wide system myself, except for limited circumstances. This counts.)

That idea assumes everyone prosecuted is a foreign national. Question: If a US citizen is prosecuted, is it at all possible to get a fair trial? (The news is estimating 50+ people involved and some having been in the US or some time, so it's possible.) Who could be on the jury but wouldn't have heard or been affected somehow by this disgusting event?

Tell me how I'm wrong. Assuming this was done by foreign terrorists, how can we use our military might (or covert operations, as allowed by law) to get these people? The only way I see would be to use them as police, not as a military.

This all assumed things weren't state sponsored, so everything can be treated as a crime - a heinous crime, but a crime. If it was state sponsored, then it's an act of war, and different rules apply. I really hope it's state sponsored.

Sorry, only states can legitimize crimes through war. In that, all states (or at least those who signed the Geneva Convention) are agreed.

BTW, what ever happened to diversity in Congress? Who can forget Jeannet te Rankin's voting against the US involvement in WWI (1 of 56) and against US involvement in WWII (1 of 1).

In the true spirit of democracy, once war had been declared Rankin promoted Liberty Bonds, which were sold to support the war effort, and she voted for the draft. However, she voted against the Espionage Act, which targeted foreign residents of the United States and suppressed dissent. [...] In 1985 a bronze statue of Rankin was placed in the U.S. Capitol

Regarding gary's comment on The ultimate hacker's car. I program so I can forget about things - I can forget how something is implemented because it just works the right way. I don't want an infinitely adjustable car, just one I like. Knowing how to tweak the "camber, castor and toe angles" simply does not interest me. I don't have a customized .emacs file. I haven't changed my app skins. Only thing window manager setting I've changed is to make the background black and have a "focus follows mouse" - in order to keep things more like the 8 years I spent using SGI's 4DWM.

Watched the Discover channel on cable a few weeks ago. (Don't have cable for fear of being a deeper thrall to the TV Fates. :) Couple of years ago I watched it when a show started on about severe storms. "Cool" I thought, expecting to learn more about how storms are formed. Nope. Sorry. It was a dramatic reenactment of a family who lived through a tornado. No science.

Then, two weeks ago I watched a show about storms on other planets, with ... dramatic reenactments of: "what would happen if we had dust storms like those on Mars?" "Storms like Jupiter's Great Red Spot?" "Flying (in a 727 I think) above a supercell in Jupiter's atmosphere?" Experience a coronal mass ejection 10,000 times greater than any ever experienced?"

Bleh.

Going to be in Cambridge, UK in a couple weeks, visiting EBI. Then to BOSC and ISMB. Bought luggage yesterday because not only is my only other luggage carry on size, but the handle broke just before my last set of trips.

Moths everywhere. Open a door and in come a slew. They love cooking themselves on my halogen lamp. I now know the aroma of toasted bug.

Been looking around for text search engines for the biopython.org project. I want to be able to do fast text searches of multi-gigabyte data sets, like GenBank.

So far what I have is the parser generator, which can identify the semantically important regions of the various formats. These would pull out things like "author", "organism", "accession number" and "description", possibly with the help of some Python code or with XSLT.

People would like to be able to search that data. Some of the searches are easy to implement, like a search for the identifier "100K_RAT." Others are a bit more complicated, like searching for the author named "Smith", as compared to "Smith, D. C." which would be the full text of an author field. Finally, people would like boolean and phrase search capabilities, like searching "'hoof and mouth disease' or 'foot and mouth disease'" in the description field. Some of the fields may allow stemming and some may not, although stemming is not a requirement.

All this data is record oriented and I have full access to the original data files (which may be compressed, so I wouldn't want to require access to the data to do the search).

The data is not in a form which can be handled directly by an indexing engine. For example, every record may have the word "GENE" because that's part of the format definition, but I don't want a search for "gene" to return all records, or ignore all records because "gene" got put on a stop list. ("Gene V" is the name of a specific gene.)

Instead, I can convert the input into a proper form and either call an API to say "here's the text for a new 'description' field" or convert the text into appropriate XML.

I would like to be able to update the database so modified forms of a record replace old entries. (There is a guaranteed unique key for each record.) This is needed for GenBank records which distribute a delta file every day but only do full releases every few months.

Finally, it needs to work on Linux, Solaris and IRIX and hopefully MS Windows and Mac (at least X). Python interface a plus, but I've done plenty of interoperability with command-line programs before, and with calling C APIs using Python.

Oh, and did I mention that fast is good? Ideally, simple lookups for some fields, like record identifier or aliases, should be blisteringly fast, while more complex keyword searches should be a fraction of a second. The system I have now, which only does exact match word searches is built on Sleepycat's BerkelyDB and does the lookup part quite quickly.

I know I'm not asking for all that much, am I? :)

I tried looking around for existing software which does this. I've used Glimpse before, but that was when I didn't worry about being able to search for given subfields. (The price of a couple thousand dollars is okay with me, so long as I can test it out before committing.)

The only program which seems close is Zebra, or rather a Z39.50 system in general. But I know almost nothing about those systems and it doesn't look like all that many people use that specific program.

A completely different one would be eXist which allows searches of XML fields using mySQL. However, I would like not to have a database system running because that makes for a more complicated setup. Plus, it is written in Java, which would put an undo requirement for a bioPYTHON project.

Any thoughts?

Have I ever mentioned how much I hate computers?

Now, don't get me wrong. I love programming. But I hate computers.

I replaced the hard drive in my computer about 1.5 months ago. It dual boots between Windows 98 and Mandrake. Why am I using Windows? Because I couldn't get the ppp connection working under Mandrake - it would stop anywhere between 1 minute and 30 minutes of use. The phone line would still be in use, but just sitting there making noise.

So I replaced the hard drive and upgraded Mandrake from 6.5 to 7.2. Thanks to the help of a friend, it went pretty smoothly, although I exlaimed "I hate computers" a few dozen times during that process.

Last Friday evening, Norton Antivirus kicked in and say the master boot record had changed and would I like to fix it. Admittedly, I didn't read the text too closely where it said "this could be caused by upgrading the OS" so I hit the "Repair" button. This replaced the MBR with the old and now invalid information NAV had saved somewhere. Which meant I couldn't boot my machine.

Did I mention I hate computers?

Please realize that I didn't and still don't have a good idea of what is on the master boot record. All I thought it did was wipe the boot loader, so I spent about 4 hours on Saturday trying to reinstall grub.

It wouldn't work. It wouldn't identify the partition but would say it was uknown and of type "0x83". This is strange because 0x83 is the ext2fs partition type identifier. Perhaps it's something to do with the "stage 1.5" loader, so tried figuring that out. Nope. Pulled up the latest CVS to see if that fixed it. Nope.

Started going through the code. Frustrating code. Global variables everywhere makes it hard to follow the thread of meaning. Various #ifdefs for stage 1.5 vs. stage 2 code. Mixed use of "grub_printf" and "printf". Using a statically embedded function as an iterator for a variable in the surrounding scope. Overuse of the for() statement so all of the checks for the end of loop are inside the for() so to have an empty body.

Anyway, I managed to track the problem down to a seven line conditional check for an if statement. Everything was find except for the last part, which was a superblock check.

It failed because the partition entry no longer pointed to an ext2 file system. This is a good thing. However, it didn't tell me why it failed and only implied that something was wrong with the filesystem type identifier.

After I figured out that the partition table was bad, I spent the rest of Saturday getting the machine back into shape. Managed to dig up some tools (like rescuept) which reconstruct the partition table information.

Even getting that going was nasty. Had to figure out how to make a bootable floppy. Used a Red Hat 7 distribution to get into a bare-bones Linux mode then figure out how "mknod" worked so I could talk to the different partitions.

But I don't know anything about partition tables. I managed to get the settings in fdisk to match the output of rescuept. (Used another program to double check, "s" something.)

Amazingly enough, that seemed to work.

EXCEPT!

Under Win98 I could no longer put my laptop into suspend or hibernate mode. No problem under Linux. The only thing that changed was the partition table. I had assigned the FAT partition for my D: drive an identifer of 0xB, but couldn't tell if 0xC was really the right one. (There are suggestions that early FAT32 needed special BIOS support, so I conjectured that perhaps a different identifier was used when that changed.) Changed to 0xC. Still no suspend. Conjectured that there was a setting changed in the registry.

There I was able to pull out my secret weapon, which was that I hadn't made all that many changes to Windows since I copied the files from my old drive to the new one, so I was able to export and diff the two settings. There were a couple of differences, but changing them didn't fix things. Replacing the new registry with the old also didn't. Doing a recursive diff on the two Windows' didn't highlight anything.

Windows running from the old disk can hibernate. So there isn't some PROM setting which needs to be changed. I backed up the /dev/hda1 partition and copied over the old installation. Should be identical code. But still Windows doesn't hibernate from that drive.

A friend of mine works for Microsoft. Called him up hoping he could divine why two almost identical configurations would allow one to standby and the other not to do so (even though it did the week before). His divining agents were no help. And he can't get his laptop to hibernate at all under Windows Me.

Did I mention I don't like computers?

While rereading the Linux man page for "fdisk" I see the comment "you should always use an OS-specific partition table program." So I played around with the Windows fdisk, which is a pale shadow of the Linux one.

Then I noticed a small comment on my worksheet. (Paper doesn't get a corrupted partition table!) It had the statement "in extended partition." Apparently there is a difference between "primary" and "extended" partitions and what I had used as primary partitions 2, 3 and 4 really should be extended partitions 5, 6 and 7. While Linux and Windows understood them just fine as primary partitions, I guessed that perhaps Windows acts differently when there's more than one primary partition.

Rebuilt the partition table to use an extended partition. When I used primary partitions I needed to specify the start and end clusters. When using the extended table I noticed that the start clusters were always correct. Figured this was a good thing.

Somewhere in here my FAT16 and EXT2 partitions became corrupted. Don't know if it was when I created the extended partition or playing around with the Windows fdisk or I forgot to unmount nicely or what it was. But I semi-ruined them. Remember that I said I backed up the FAT16 partition? It was onto the ext2 one, and now that file is in never never land. Also got the only time I've seen where fsck stops and asks for manual intervention.

Computers. Hate. Blech.

Managed to mount and save my home directory. It's about 3GB of which over 1GB are files from various bioinformatics databases. Easy to drop a GB on a 20GB drive. I must admit that nice feature of computers.

So now I'm in the process of reinstalling everything back onto my new hard drive. Didn't loose that much. I was very cautious about my email, so had an extra archive of that. Worst problem will be the final details of my Q4 finances, but even there I still have the paper copies.

Still, I've been working at this for four days, and learned a slew of things I didn't care about and would rather forget. Everything I did needed to be double checked because if I, oh, swapped an if= with an of= I would wipe everything. (I am able to borrow 10GB of space on someone else's machines, so I do have a second backup.)

I hate computers.

How to improve things?

First would be if Norton used the word "Restore" rather than "Repair" in that button I pressed which sent me on this nasty journey. I would really enjoy it if Norton has a "undo last repairs" option since I figured out there would be a problem before I rebooted. I could also have booted into Windows from a grub'ed boot disk to run that command.

Huh. When anything other than Norton touches the MBR, NAV kicks in to say there might be some virus activity. That would have warned me that something possibly back was taking place. I wonder how NAV recognizes self and if an anti-antivirus program could take advantage of that.

Second would be if there was no distinction between "primary" and "extended" partition. I bet it was some sort of hack to get around the 2GB limit in early hardware.

Third would be more verbose reporting information in grub, to say why it couldn't recognize a disk with a correct partition number.

Fourth would be better reporting in Windows to say why things failed - that is, an equivalent to the system log under unix. Hmm, I bet there is, but I just don't know how to find it.

There's probably more, but that's enough for now.

Don Norman is right. Computers are still in their primitive infancy. Will they mature in my lifetime or will there continue to be levels upon levels of needless complexity?

Don't come back telling my why things have to be done this way. Think of it as a challenge. People use software to achieve goals in their domains. How can you reduce (eliminate!) the need for knowledge unrelated to those goals?

Bear in mind too that I've been a programmer since '83 and a professional programmer since '95. If I have this much problem with computers, no wonder I'm not the only one who hates them. But as a programmer, they're the only game in town.

How about a Python solution to the Scheme problem? Of course, I made a more general solution allowing any number of "f"s :)

class f:
    def __init__(self, s):
        self.s = s
    def __add__(self, other):
        return Merge(self, other)
    def __getitem__(self, i):
        return self.s[i]

 class Merge:
    def __init__(self, left, right):
        self.left = left
        self.right = right
    def __getitem__(self, i):
        return self.left[i] + self.right[i]
    def __len__(self):
        return min(len(self.left), len(self.right))
    def __str__(self):
        s = ""
        for i in range(len(self)):
            s = s + self[i]
        return s
    __repr__ = __str__
    def __add__(self, other):
        return Merge(self, other)

 >>> f("Js nte ceeWnae")+f("utAohrShm anbe")
Just Another Scheme Wannabee
>>> f("123") + f("ABC") + f("abc") + f("!@#")
1Aa!2Bb@3Cc#
>>>

This uses Python 1.5.2. With Python 2.0 I would probably do

class f(UserString.UserString):
    def __add__(self, other):
        return Merge(self, other)

An uglier version of __str__ using 2.0 tricks is

def __str__(self):
    return "".join(x[0]+x[1] for x in \
                    zip(self.left, self.right))

I could probably cut the line count in half if I got rid of the generality, so no fair doing a direct line count comparison between this and the Scheme solutions!

jdybnis asked:
Does anybody have any others [points to regexp package]?

There's pcre at pcre.org. Python 2.0 includes the sre regexp library. There are a lot more. A quick web search also found Regex++ and the ORO library for Java which is part of Jakarta. Then there's my bastardized regular expression converter to the mxTextTools state machine as part of Martel. (Bastardized since it doesn't do full backtracking.)

I wrote an article for Dr. Dobb's Journal. It's in the January 2000 edition, which I understand is starting to arrive in the mail. Soon, yes, you too will be able to own a copy of words written by yours truely! As an added bonus, this is their 25th year anniversary edition.

deekayen:

You can't have safe sex when you're having sex with someone infected with HIV or AIDS. There are pores in latex condoms. Sure, you can't see them... they're small... but HIV is still 1/100 the size of the pores in the condom and can easily slip right through.

This statement is completely incorrect. Here are some STD virus sizes from The Big Picture Book of Viruses:

• Retroviridae (HIV) are spherical; 80-100 nm in diameter
• Herpesviruses (herpes and cytomegalovirus) are spherical; 120-200 nm in diameter
• Papillomavirus (from http://www.uct.ac.za/depts/mmi/stannard/papillo.html) are about 550 nm in diameter.
• Hepadnaviridae (Hepatitis B) are filamentous; 40-48 nm in diameter
• Flaviviridae (Hepatitis C) are spheroidal; enveloped; 40-60 nm in diameter
• Poxviridae (Molluscum contagiosum) are ovoid, or brick- shaped; 140-260 nm in diameter; 220-450 nm long

So Hepatitis B and C are both smaller than HIV, and the ratio of smallest HIV to largest Molluscum contagiosum is still less than a factor of 4. (The volume is about a factor of 100, but that's for the extreme case and you're talking about pore size.)

Second, what is the pore size of a condom and how important is that in transmission? I found claims like yours which state it as fact but don't back it up. On the other hand, sites like http://www.fda.gov/cdrh/ost/reports/fy98/INFECTION.HTM and http://hivinsite.ucsf.edu/topics/condoms/2098.32ca.html point out the results of tests on the permeability of condoms and find them effective barriers. Yes, the pore size is larger (seems to be about 1000nm if I read the reports correctly) but pore size isn't the only consideration on transmittability. These are back up with theoretical, laboratory and epidemiological studies.

So you are incorrect both on the relative sizes of the viruses and the importance of pore size on transmissability of viruses through condoms. Unless you have evidence to back yourself up?

Update

*Chagrin* Oops, I misread the "1/100" and thought deekayen said the HIV virus was 1/100th the size of other viruses. He did not say that. He said they were 1/100th the size of the pores in condoms. The numbers I found suggest it's more towards 1/10th than 1/100th, but not enough to be seriously wrong on that account.

Still, there have been tests upon tests which show that condoms are an effective means to prevent STD infection.

My apologies for the misinterpretation.

Today's my birthday (okay, it's tomorrow in most of the world, but not according to my watch :). I am now 30 years old. Oh my.

As my present to the world, I released Martel- 0.2 today and put up my poster from BOSC.

The short description is that it's a parser generator written in Python and designed for stateful *regular* grammers, as compared to context free grammers where you can lexically tokenize with very little knowledge of where you are in the file. This is very handy when parsing files designed to be read by humans or hand-written parsers (that is, written by people with no knowledge of recusive descent).

I use a modified version of a subset of the Perl5 regular expression language to describe the file format. I take the expression and convert it to a table for mxTextTools, which parses the string and returns a taglist describing the file as a tree. (?P<named>groups) are used to label interesting nodes in the tree.

The tree is traversed in prefix order and sent to the callback object as events. I reuse the SAX API for the event names and meanings, so I can leverage off of a lot of existing code and ideas, like DOM.

I'm pretty proud of it. It took a lot of thought and work, and the result is quite nice, and I haven't come across anything like it elsewhere. (Pointers anyone?)

Oh, I see I forgot to mention the license. It's the old Python license "with the serial numbers scratched off", which is basically BSD but without the adversiting clause. Enjoy!