Edge- Versus Level-Triggered Events
A good while ago, I declared my preference for edge-triggered events (for I/O), because in my mind, it most closely paralleled the actual workings of the hardware (IRQs being triggered when new data arrives on network interfaces or when a read/write request to disk is completed). I also figured that since it was possible to emulate level-triggered efficiently on top of edge- (that's what happens when you use close-to-the-metal "abstractions"), it probably was the more flexible choice.
I then changed my mind, when I realized that what is very often needed is level-triggered, someone has to remember what file descriptor still has work to do. I figured that it might as well be the kernel, since it is in the best position to do that safely, simply and efficiently. Otherwise, you end up either having to remember myself (which isn't very safe if your framework is only providing the event delivery mechanism, a buggy user could easily "get lost"), or to re-arm the file descriptor (more system calls, less efficient). On the other hand, there were also some other relatively common usage where edge-triggered was preferable (specifically, when transferring data from one file descriptor to another, where you do not want to re-arm the source until you managed to write the data to the sink first.
But recently, I changed my mind again, and about a certain number of things. Many know me to be one who dislikes threads, but it's not exactly true, I have a dislike of how it's haphazardly used as magic pixie dust by hordes of people who are apparently utterly confused by event-driven state machines. Now that we're seeing more and more multi-core systems, I feel something has to be done about it. And it turns out that level-triggered events are a bit of a pain to handle with multiple threads: a thread going into epoll_wait could get a notification that a file descriptor is readable while another thread is in the process of dealing with it. Adding a monitor to prevent re-entrance would just make it busy-wait instead, as it wouldn't sleep. Edge-triggered events deal with this neatly, and I think this combines with the existing cases where it was preferable to make them actually the better choice.
Now, I want to have a select()-based reference implementation, for portability, and it turns out it's kind of tricky to have multiple thread service a common set of file descriptors... I have some ideas, but that'll be for next time.