People are seeing very nice improvements with my block + elevator patch, that is good news. I suppose I should split it up a bit and get around to shipping it to Linus... I added more parts to it today, a much better version of the tq_disk I did earlier. Now we don't use task queues to handle plugging, but rather tasklets (or softirq's). This made it easier and more clean to implement firing of individual devices when in need of a buffer/page/kio. This means that now firing all devices is not the main use, it can still do that however. We need that for memory pressure reponse, etc.
I'm seeing an odd XFS crash when really pushing it. I'm hoping the XFS guys will fix this soon.