Older blog entries for async (starting at number 92)

ladypine :

Thanks for the link to the essay by Hofstadter. It is an exceptional piece of satire.

Fixing OS X's GUI:
i have cured all my cmd-tab annoyances in mac OS X! it was easy: all it took was installing VNC. now the OS X desktop runs in an 800x600 window-of-goodness on my X desktop (a second box). the music applications seem to work alright over VNC, although i probably should change the hub from the 10baseT to the 100baseT one i have sitting in my desk drawer.

ahhh peace at last.

BitTorrent & RHN:
i started to pull the ISO's (430kB/s and 20 minutes left thank you very much) for redhat 9 via bittorrent (hooray). and i figured i'd go ahead and buy it from rh too.

i started off getting a boxed set, but then figured i could sign-up for the rh network, since i didn't really want the box or paper manuals, etc. it seemed nice until i read the TOS which said basically, RH can terminate the agreement whenever they feel like, but i have to submit in writing 60days before the next billing cycle (yearly) in order to make them stop charging me.

this seems designed only to take advantage of people by making it hard for them to unsubscribe and easy for them to just forget for another year. it would be nice if they had 2 options, one for reoccuring charges yearly and another for a single year with no renewal. they should also make it as convenient to unscubscribe as subscribe.

i'm sure reoccuring charges and snail-mail only unsubscription make their accounting easier and their renewal rate higher, but they should act with respect towards their customers and without the appearance of trying to screw you over. it's good for business as well as the soul.

so a box set it is.

27 Mar 2003 (updated 27 Mar 2003 at 09:35 UTC) »
Mac OS X & Proprietary Softwares:

i've been playing around with mac os 10.2 lately. it's a nice os, though it's baroque in places (natuerlich). and while drawin is open (and i read through the scheduler sources for fun), the above-ground parts are not.

i also got a music app as well (which cost nearly 1/2 of what the new dual g4 did...(replacements are being worked on and yes i've donated work and money to them)). it's interesting that in paying for non-free software, instead of getting good, personal support, open channels of communication with the developers, and information about what plans are happening for future products, you get no source, bad support, no flexibility, and no information.

in fact you get to plug-in a usb key thingy (DONGLE!) and punch in a bunch of numbers and then register and punch in a bunch more numbers, to show that you have what it takes to own their software. (although, there are certainly versions available without these features). but i'm sure i don't have to work hard to convince anyone here of the evils (or at least inconveniences) of proprietary software.

i also tried figuring out how to hook the cmd-tab (alt-tab) key so that i could write an application to fix the stupid way OS X does window switching (alt-tab switches between applications, alt-` switches between multiple windows of a single application (they don't show up on alt-tab)). there are a few apps out there that sort of fix this behaviour but not to my liking.

first, i looked for a documented method. it doesn't appear that any exists though. then i figured i'd see what Dock.app does (well first i had to figure out that Dock.app was the one hooking alt-tab). interesting library calls included CPSRegisterForKey, CGSSetHotKey, and CGSSetHotKeyEnabled. i looked for documentation for those, but apparently they are private and undocumented.

when this happens in normal situations the next step of course is to fetch the source and look at the library's implementation.

but, noooo.

i couldn't find a disassembler (no objdump(1) installed) so i broke out gdb, set (sat?) a few breakpoints, and started single stepping through the CPSRegisterForKey code to try to figure out the parameters and how they're used. eventually, i figure out it's 'OSError CPSRegisterForKey(ProcessSerNum * psn, SInt32 specialkeycodeEnum)' ( in this case the keycode is 1 iirc (up to 5)). well dandy. i finally get that returning 0 (success!), (you have to kill the dock), and then tried the standard way of doing Carbon/Cocoa hotkeys. but no luck...it seems the specialkeys are filtered earlier on.

the next step is (of course) to figure out how to use CGSSetHotKey and friends.

what a spectacularly unmitigated useless waste of time.

Kernel Hackings:
so i've tried to make a new scheduling policy for linux. i've called it SCHED_USERFIFO. the intent is basically to allow a process to ask for X amount of processor time out of every Y ticks. (the user part is in the hope that the administrator can set rlimits on the percentage of time requested and allow non-priviledged users to use this policy). (it's basically constraint scheduling as seen in mach).

this is mainly for stuff like multimedia (eg. audio production stuff) where you want to have programs run when they need to for as much time as they need to, but not bring the box to it's needs if there's an error.

it works just like SCHED_FIFO as long as the process doesn't take more than the amount of time it asked for. if it does try to take more time, it is allowed to be preempted until the period is over.

i hacked this up quickly, and tested it with the program below. i was able to get a process in a while (1) loop to take up >99.4% of processor while still being able to kill it from the console (although trying to do anything interactive was excruciatingly slow during this time). when i allowed the process to take up 100% of the period, the system hung (with respect to other processes) as expected. (tested in vmware, btw).

(i'm pleased with myself; hopefully this sort of does what i say it does too).

    here is the test program, note the two new fields in sched_param:

  param.period_reserved
  param.period_length

these are specified in system ticks.  below i give the program
49 out of every 50 ticks.



------------- testprogram.c ----------------
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
//#include <sched.h>
// import these directly 
#include <sys/time.h>
#include <sys/poll.h>


#define SCHED_OTHER		0
#define SCHED_FIFO		1
#define SCHED_RR		2
#define SCHED_USERFIFO          3

struct sched_param {
  int sched_priority;
  /* user fifo stuff */
  unsigned long period_length; 
  unsigned long period_reserved;
};


/* return -1 on failure */
int
set_realtime(void)
{
  int r;
  struct sched_param param;
  pid_t pid;
  
  pid = getpid();

  param.sched_priority = 50;
  param.period_length = 50;
  param.period_reserved = 49;
  return sched_setscheduler(pid, SCHED_USERFIFO, &param);

}

void
waste_time(void)
{
  int  i = 0;
  while (i<40000000L) { 
    printf("unf\n");
    i++;
  }
}

int
main(int argc, char ** argv)
{
  int i = 0; 
  if (set_realtime() < 0) { 
    fprintf(stderr, "couldn't set scheduler to SCHED_USERFIFO\n");
    return -1; 
  }
  waste_time();
  return 0;
}

---------------------------------------------

here is the kernel patch against 2.4.20+lowlatency.  it probably won't
work on systems that use APIC's, it doesn't do any sanity checking,
and it doesn't handle jiffy rollover.

--- pristine/linux-2.4.20/kernel/sched.c	2003-03-17 23:24:02.000000000 -0500
+++ linux/kernel/sched.c	2003-03-17 22:11:13.000000000 -0500
@@ -188,7 +188,42 @@
 	 * runqueue (taking priorities within processes
 	 * into account).
 	 */
+
+	if (p->policy == SCHED_USERFIFO) { 
+	  
+	  /*
+	   *   check if we are in the right time period
+	   *
+	   *   XXX if it burns though it's entire period and
+	   *       into the next ? 
+	   *     
+	   */
+	  if (jiffies >= p->t_period_end) { 
+	    /* no, start over from now */
+	    p->t_period_start = jiffies;
+	    p->t_period_end = p->t_period_length + p->t_period_start;
+	    p->t_period_remain = p->t_period_reserved; 
+	  }
+
+	  /*
+	   *  is there any remaining time ? 
+	   *  
+	   */
+
+	  if (p->t_period_remain > 0) { 
+	    weight = 1000 + p->rt_priority;	    
+	  }  else { 
+	    /* redundent, for clarity */
+	    weight = -1; 
+	  }
+	  goto out;
+	}
+
+	/* normal RT case */
+
 	weight = 1000 + p->rt_priority;
+
+
 out:
 	return weight;
 }
@@ -939,8 +974,8 @@
 	else {
 		retval = -EINVAL;
 		if (policy != SCHED_FIFO && policy != SCHED_RR &&
-				policy != SCHED_OTHER)
-			goto out_unlock;
+		    policy != SCHED_OTHER && policy != SCHED_USERFIFO)
+		  goto out_unlock;
 	}
 	
 	/*
@@ -964,6 +999,15 @@
 	retval = 0;
 	p->policy = policy;
 	p->rt_priority = lp.sched_priority;
+	
+	if (policy == SCHED_USERFIFO) { 
+	  /* FIXME: no checks, doesn't handle jiffy rollover */
+	  p->t_period_length = lp.period_length; 
+	  p->t_period_reserved = lp.period_reserved;
+	  p->t_period_start = jiffies;
+	  p->t_period_end = p->t_period_length + p->t_period_start;
+	  p->t_period_remain = p->t_period_reserved; 
+	}
 
 	current->need_resched = 1;
 
--- pristine/linux-2.4.20/include/linux/sched.h	2003-03-17 23:24:02.000000000 -0500
+++ linux/include/linux/sched.h	2003-03-17 21:55:04.000000000 -0500
@@ -119,7 +119,7 @@
 #define SCHED_OTHER		0
 #define SCHED_FIFO		1
 #define SCHED_RR		2
-
+#define SCHED_USERFIFO          3
 /*
  * This is an additional bit set when we want to
  * yield the CPU for one re-schedule..
@@ -127,7 +127,10 @@
 #define SCHED_YIELD		0x10
 
 struct sched_param {
-	int sched_priority;
+  int sched_priority;
+  /* user fifo stuff */
+  unsigned long period_length; 
+  unsigned long period_reserved;
 };
 
 struct completion;
@@ -419,6 +422,15 @@
 
 /* journalling filesystem info */
 	void *journal_info;
+
+
+  /* our extra stuff */ 
+
+  unsigned long t_period_end; 
+  unsigned long t_period_start; 
+  unsigned long t_period_remain;
+  unsigned long t_period_length;
+  unsigned long t_period_reserved;
 };
 
 /*
--- pristine/linux-2.4.20/kernel/timer.c	2002-11-28 18:53:15.000000000 -0500
+++ linux/kernel/timer.c	2003-03-17 22:42:52.000000000 -0500
@@ -610,6 +610,15 @@
 				p->need_resched = 1;
 			}
 		}
+		if (p->policy == SCHED_USERFIFO) { 
+
+		  if (p->t_period_remain == 0) { 
+		    p->need_resched = 1;
+		  } else { 
+		    p->t_period_remain--;
+		  }
+		}
+
 		if (p->nice > 0)
 			kstat.per_cpu_nice[cpu] += user_tick;
 		else

-------------------- end -------------------------------
pphaneuf:

I guess it's time for a little rant...

async: what are the advantages of CORBA exactly? An RPC interface (which, in my opinion, has more to do with "sex appeal" or "cool factor" than actual advantages)? Dealing with language issues (initially caused by the RPC interface itself)? Constructing a message can be done in many languages without raising any "issues".

the benefits for me include:

  • not having to define a message protocol for complex types
  • not having to marshall and unmarshall arguments
  • compiler support for argument/msg type checking
  • not having to define a separate client/server-side wrapper to keep track of the other's state
  • providing life-cycle semantics
  • existing implementations for most all languages i use
  • standard method of defining messages/objects/functions (IDL)
  • support for dynamic instantiation of clientside objects (in python for example, i can load an idl file, and it creates the the objects described there in)
  • support for introspection (cf above)
  • discpline resulting from having the interfaces defined

by no means do i mean to say "rpc is better than messaging". as i stated, i think that dbus is useful for those cases in which the use doesn't fit the semantics of rpc.

messaging is a more general framework than rpc. when i want to have the properties i mention above it is easy to use corba as a solution. when i don't need those properties, i would not use corba as a solution.

misc: corba has async functions (basically the same as sending a message), and i mainly use corba so that my front-end can control my back-end in certain applications without having them in the same process (or even same machine).

corba is far from perfect. it's a huge standard: it's complex, over-featured, and designed by committee. but i find it helpful, especially since it currently exists and has fairly mature support (well sorta).

DBus, Corba, and ORBit, oh my:

(these are my thoughts after doing a bit of reading, but i probably still have misunderstandings and misconceptions. feel free to disabuse me of them.)

i saw the stuff about gnome and dbus recently. after some reading, it looks like dbus can be a win for everyone involved. my main concern is that orbit will be marginalized and/or abandoned. this i think would be a mistake.

dbus appears to fulfill a niche for IPC that lies between SysV primitives and CORBA in terms of features and complexity. It provides a lightweight framework (in terms of footprint and semantic complexity) that can be used for system-wide purposes (not just on the desktop). It is a message model that doesn't impose CORBA's rpc semantics (which is a plus for those applications in which the model is overkill or doesn't fit well). It deals with naming and activation issues which must be formalized compared to just using sockets and the like.

On the other hand, it doesn't provide an RPC layer or deal with the myriad of language issues that CORBA does.

This is to say that dbus doesn't replace CORBA. I'm not a huge fan of the Bonobo framework (i don't use it much), but i think ORBit (the CORBA orb Bonobo uses) is important (i do use it). It solves a problem that would require a large layer on top of dbus to solve...essentially reinventing CORBA without the benefits.

So my main worry is that people will throw away (or let bitrot) a useful component rather than working on it and letting it continue evolving in its niche. Reading the messages from those working on it, this doesn't appear to be happening, but i would like to actively throw my hat in and say that i would like to see ORBit remain around (regardless of GNOME).

The C connection

I wonder how much the fact that doing CORBA in C completely sucks plays into the desire to have something like DBus. jamesh's work on CORBA bindings for python make it a complete joy to work with: it's like magic.

C on the other hand is a maze of twisty structs all alike and numerous pitfalls. it seems too hard to do right (though not impossible).

While python is great for some applications, there are certain cases where a compiled app would be more suitable. Clearly, the way CORBA is handled in C is not magic, but i'm not sure what the alternative is. C++ has a nice binding, but i have issues with C++ (viz its schizophrenic approach to objects and references, weak type system, etc).

but perhaps i should look into it more.

Innovation:
garym: Dave Winer says [...]. Are Advogadorians going to sit back and let that go by without a challenge? [...]

sure, i'll sit back and let it go, because

  1. what he thinks doesn't affect me
  2. i don't care
  3. whether it is true or not that no 'cutting edge' work happens in freesoftware doesn't matter.

whether or not it's true is basically a matter of ego and bragging rights: 'my development model is better than your development model', 'your development model will mean the end of civilization as we know it.' blah blah blah.

and in the end novelty buys you what?

also, i think livejournal is opensource.

zhaoway:
Life of an Independent Software Writer

atai, I'd really like to know why he's in such an unfortunate situation? 'Cause an independent software writer is the position I'm working to achieve. I'd really like to know why.

that would be because he hasn't come to terms with the reality that redhat, et. al. aren't going to hire him just because he whines that he needs a job on various mailling lists and the fact that to get income you sometimes have to do something that someone else wants that you may not.

22 Feb 2003 (updated 22 Feb 2003 at 13:42 UTC) »
pphaneuf, dto: letting the recipient queue their own messages might nullify some of the key advantages of an async message passing system (from here on amp).

in a single cpu, multiprocess, shared-memory AMPS (SCMSMAMP :D) (ie one where the kernel isn't involved in the exchange (eg ULIPC/URPC of Bershad, et.al.)), one of the main advantages is that you don't (necessarily) need a context switch to pass a message. you could implement a similar setup to the one you describe (ie selective queueing) by having each process attach a callback to their queue which would be executed in the sender's context and determined whether it would queue the message (empty space in the queue), would not bother queueing the message, or would queue and context switch after the queue. (someone write a paper: Active Queues... sort of the inverse of Active Messages). mostly i think this is kind of unnecessary since you can waste a bit of memory and make the queues large with respect to the message passing frequency so you hopefully never overwrite your queue.

generally in my applications (and probably in dto's), i use SCMSMAMP's to communicate between two threads (or two sets of threads which are synchronous with respect to the members of their set) so that the queue itself is a lock-less circular buffer with little operational overhead. there is also the assumption that under normal operation no thread will be starved. the general case would probably deserve more attention.

pun-tasm: this is an async message, and advo is an AMPS.

dto: i've implemented a sort of similar system (doesn't make use of runtime plugin's though)...remind me to put it up somewhere. it might be interesting to look at some of the trade-offs.

15 Jan 2003 (updated 15 Jan 2003 at 17:25 UTC) »
Eldred Copyright Unextension Act:

i read the opinions issues by the US SC.

the majority opinion was that the petitioners were unpersuasive in arguing that the framers hadn't meant for extension to be within the purview of congress. it is difficult to argue against history (that copyright had been retroactively extended under the framers' tenure). Furthermore, the SC did not think it was their place to second guess congress' wisdom in attempting to execute the clause.

the dissents were based off of analogy from the SC's opinions in patent law. the reasoning was that both the patent and copyright clause provide for a limited monopoly, and the SC had in the past ruled that attempts to extend a patent's life through various means were illegal, thus retroactive copyright extension should be illegal as well.

the thing that persuaded me in the majority's opinion was the argument that copyright wasn't a monopoly on the idea contained in the work. that is to say that copyrights don't limit fundementally taking an idea contained in a work and restating or deriving other works based on it in the same way that patents protect ideas. copyrights only protect the mechanical execution.

so i think it was a fair decision (that the SC shouldn't intervene). a problem does exist. and if people want to fix it they just have to pursue a legeslative solution rather than a judicial one.

i think we (i) should work more to encourage giving works of art (software, fine arts, etc) to the community (ie skipping content companies, and putting less restrictive licenses on their reproduction). as a society we should treat it as part of being a good person: that you should contribute to society because it is the right thing to do, because it adds something of value for all of us.

83 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!