Some more sketching of voronoi diagrams. I really thought I had an idea of how to choose the correct line when calculating the bisector of two line segments. Seems i was wrong again... I probably have to pass the intersection point to the bisector calculation and then just choose the nearest segment.
To tired to think, I'll go watch some tv.
