Recent blog entries for olecom

User-Agent: jed (x86_64-pc-linux-glibc-debian)

Value of syntax highlighting for programming in text editors is hard to overestimate. I recall one my buddy, a keen school olympic competitor on programming, who said, that move from Borland Pascal 5 to 6 and 7 was a great thing because of highlighting.

Many syntax and logic errors just popped up right before one's eyes. Monotony strains. OTOH, Reading the book isn't like programming. Programming requires great thinking efforts and no distractions. Those LISP coders also track those parenthesis and think this is a cool thing.

So, i didn't see something quite comprehensive even in the great Emacs. Unreadable regexs, reinventing parsing, small efficiency wrt CPU burning. Not saying about *no* highlighting of regexs themselves, non stupid highlight of `sh` scripts, `sed`, etc.

Maybe in 70x, where all "modern UNIX/Linux userspace software" stuck, there was no computation and brain power, money and programmer's imagination to implement that in a rational way.

The Right Thing(R) IMHO is to have any parser `sh`, `sed`, `awk`, `gcc` etc. (yea, stupid XML!) implementing interface for highlighting of the input. Isn't that those parsers know and do all their syntax handling?

Yet more than decade x86 clones in terminal emulators are waiting infinite microseconds between user's key presses...


#comments
--
sed 'sed && sh + olecom = love' << ''
-o--=O`C
 #oo'L O
<___=E M
5 Apr 2008 (updated 7 Apr 2008 at 21:29 UTC) »

Newsgroups: gmane.comp.hardware.open-hardware.ethervideo
From: Oleg Verych <!gmane?olecom.ENOMSG@flower.upol.cz>
Subject: Back to text and graphic terminals.
Organization: Palacky University in Olomouc, experimental
physics department.
Mail-Copies-To: never
User-Agent: slrn + jed (x86_64-pc-linux-glibc-debian)
Keywords: text-mode, graphics, video, ethernet

Hallo. I'd like to ask about a graphical device set for simple
PC usage (tell me, please, if this place is wrong).

My PC is a laptop 95% of the time for nearly 3 years.
Activity:

pure text mode (Linux):
 * gmane news reading, all those mailing lists one place (slrn)
 * programming (emacs)
 * web browsing (lynx)
 * watching video
http://article.gmane.org/gmane.comp.video.mplayer.devel/46227

graphic mode (MS Windows mostly, because X sucks):
 * pictures
 * pdf (while partially i can screen it by `pdf2text` + some
formatting)
 * djvu (so many books appeared)
 * web browser (with lot more useless traffic, flash  and
pictures)

In the text-mode i realized, that it can be much more
interesting and productive:

 * more colors
 * more char. attributes
 * character "shaders" (hardware and software)
 * hardware windowing system with enough memory

Framebuffer text console in Linux is known to be very slow and unpleasant. Text terminal emulator in Linux and in X11 as well are just deep in 70s, but with transparency. Particular in Linux kernel vt102-like emulator wasn't developed or optimized for quite some time.

I believe, i have latency problems (music playing) when i page or output text in ssh remote console (i.e. network can send so much text quickly to process byte by byte in kernel's terminal emulator, that kernel gets unresponsive).

Software compatibility and legacy aside, proprietary UNIX zoo is dead, open source is yet to see something like Turbo Vision kind of look and feel.

So, i'd like to see a device (or set) with features:

* text terminal-like interface in text mode, i.e. stream of characters to the graphics hardware, but improved and with windowing system support

* framebuffer (graphics: dynamic video, static pictures) is switched on only on demand (saving power, less EMI noise).

* text/graphic processor/uC + small OS (text windowing, buffering, smart to do stupid job, etc.) is inside of display.

* interface to PC is Ethernet (or equivalent).

Thanks for you attention.


#comments
-- 
sed 'sed && sh + olecom = love' << ''
-o--=O`C
 #oo'L O
<___=E M

A view: linux RAW net, raw sockets.
A fast communication for small devices without overhead of
IP, TCP, UDP.
http://kerneltrap.org/node/15896 ______
1 Apr 2008 (updated 2 Apr 2008 at 22:15 UTC) »

sed 'sed, shell and Oleg'  <<  ''
the most selfish and cool sed script

sed 'sed, sh and Verych Oleg' << '' the most selfish and cool sed scrip2
sed 'sed + olecom = love' << '' the most selfish and cool sed scrip3
sed 'selfish and cool sed script by Oleg' << '' -- -o--=O`C #oo'L O <___=E M

More about text processing in open source programming:
http://kernelnewbies.org/olecom
It includes heavy
`shell` and `sed` stuff.

About stupid ttys and ergonomic user interface later.
_____

3 Feb 2008 (updated 6 Feb 2008 at 05:20 UTC) »

,--
| [1] ftp://flower.upol.cz/dts/Sed0000/hacks/strip-c.sh
`--

Viewing this in the browser window can scare anybody. That's why noone so far read whole links in prev. post. But let's put away the fact, that open source browser doesn't support syntax highlighting of the most basic thing -- shell language and UNIX tools, available for 20-30 years.


,--
| http://kerneltrap.org/Linux/Decoding_Oops
`--

Here is the main point of the kernel developmnet: understanding, knowledge, ways of debugging. Starting point is assembly.

Text-processing assembly language is sed. Userspace assembly is shell (mainly POSIX, but not ksh or bash). Both are in very-very upsetting state.


,--
|[3] http://lxr.linux.no/linux/scripts/cleanfile 3492 bytes
|[4] http://lxr.linux.no/linux/scripts/cleanpatch 5132 bytes
|
|I've came up with this one:
|
|ftp://flower.upol.cz/clean-whitespace/clean-whitespace.sh
`--
Looking on 1024 bytes of this one, it is clear how syntax highlighting is important in understanding and developing of BRE+sed+shell tools.

So, what is the progress?

First, shell language is very complicated in terms of parsing. I don't know if any academia compliler theory professors have actually looked at this one. Kenneth Almquist, who wrote ash, was from EDU domain, but he have done a practical implementation.

I must edit this post, to say: "What an utter crap!" An Introduction to ANTLR: A parser toolkit for problems large and small

Recursive nesting of subshells in ``,$(), quoting are main features. Even using them is not such trivial thing at all. But i saw trivial scripting isn't that easy for some kernel developers(script in 47128D76.5050407@garzik.org).


#!/bin/sh -e

#
# usage: mkmsg [branch] [output text file]
#

test "$1" && BRANCH=$1 || { echo "Empty BRANCH,
exiting.";
exit 1; }

  if test  "$2"
then exec >"$2"
else : "Print to stdout"
fi

# should work much better than 2 fork()s
REPO=${PWD##*/}

echo "Please pull from '$BRANCH' branch of
master.kernel.org:/pub/scm/linux/kernel/git/jgarzik/$REPO.git
$BRANCH

to receive the following updates:
"
git diff -M --stat --summary master..$BRANCH
echo
git log --no-merges master..$BRANCH | git shortlog
git diff master..$BRANCH

So, with this one it is even possible to use script interactively and/or in the pipe chain to mail stdout directly, as addition to better, i.e. efficient, usage of the system and the kernel...

Second. While trying to implement highlighting i've discovered, that kernel doesn't apply background attribute to the tab symbol. I.e. tabs are always black, no matter what background color is there. This led me to more deep understanding of the tty processing, mainly tabstops (theory can be found in UNIX Power Tools).

Problem can be solved by converting tabs to spaces. But knowing, what tabstops are, it's not such trivial thing to do. In clean-whitespace.sh i've used `expand` UNIX tool. Now i wanted to do it in sed.

Andreas Schwab is making very interesting, mostly one-line comments in LKML (i even checked old archives of 90s). Once he pointed out, that i wrongly explained "\{\}" BRE construction. While i knew what it means, i really didn't used them at all. But after that occasion i sat down and did read+exercise. This helped me to do my version of `expand`.

Funny, that this[0] turned to be even more efficient, that one from recognised sed guru Greg Ubben[1].

[0] ftp://flower.upol.cz/dts/Sed0000/hacks/expand.sed.sh

4 ("3 basic" + "1 speed optimization") op-lines, 2 numbers, no loops

[1] http://sed.sourceforge.net/local/scripts/expand.sed.html

6 (5 + 1) op-lines, 2 numbers, loop

@(#)14apr89/31aug01 expand.sed by Greg Ubben

means, that in more than 10 years there was no better solution, heh. Also coloring is a bit broken: openning "\(" and closing "\)" do not match.

I hope, i have no errors there. Checking is left to the reader.

Thus, progress on such basic thing is kind of unexpected. After understanding of achievement, i was depressed. More depression followed, when i've discovered original ash/atty sources.


-- comming next --
shell, tty, text user interface
-- 
-o--=O`C
 #oo'L O
<___=E M
3 Feb 2008 (updated 3 Feb 2008 at 06:44 UTC) »
http://lwn.net/Articles/265739/

About first one.

Yes, C is cool. In the kernel development it is even cooler. Or is it?

If source writing was done with line-based parsing in mind (sed), then crap like [0] wouldn't been existed.

[0] http://lxr.linux.no/linux/scripts/unifdef.c

Also i want to ask, what is more cool, that CPP/GCC triks or [1] C/C++ comments remover?

[1] ftp://flower.upol.cz/dts/Sed0000/hacks/strip-c.sh

So far i didn't find mis-parsing problems. But C and especially C \/ C++ comment soup can have many stupidly hard things to do, that i was fail to envision. This script, BTW, runs faster than most comprehensive one on sed.sf.net: remccoms3

== This leads to question why text parsing is so doomed? ==

Why i type those html tags _here_, why syntax parsers of compiler cannot be used and are not used to high quality and usability syntax highlighting, etc., etc.?

A bit about some issues i wrote here http://article.gmane.org/gmane.linux.usb.general/632. Yet modern academia students, that are doing hard-core(?) kernel programming, like to fix kernel, rather than stupid, useless zoo of syntax highlighters [2].

[2] http://kerneltrap.org/mailarchive/linux-kernel/2008/1/24/603419

As skills go, it's far more useful than "how to trim the trailing whitespace" and the rest of checkpatch.pl-inspired crap that got so popular lately... http://kerneltrap.org/Linux/Decoding_Oops

Yea, Al, whitespace and syntax LKML-guys appeared quite recently. They doing stupid, but yet useful work, work from the wrong end.

I myself, as an amateur newbie, was deeply impressed by appearance of the stuff like[3,4],

[3] http://lxr.linux.no/linux/scripts/cleanfile 3492 bytes
[4] http://lxr.linux.no/linux/scripts/cleanpatch 5132 bytes

I've came up with this one:

ftp://flower.upol.cz/clean-whitespace/clean-whitespace.sh 1024 bytes

Which isn't quite same funcionality, but this meant to be same result[5].

[5] http://mid.gmane.org/20070607230657.GV7266@flower.upol.cz (use subject link to navigate in thread)

Yes, whitespaces were my PITA right from the first patch. Crappy Linux Makefile(s) + GNU Emacs with smart whitespace damage cleanup ==> big diff just after opening the file. I didn't understand, why there is such damage, because before any programming work i've studied The Tool - A Text Editor. It turned out not only technical problem (text editing), but maintaining one: whitespaces in diffs's context are important. Thus, cleanup in one file may break quite many patches elsewhere in people's trees.


-- coming next --
tabstops, `expand` (from clean-whitespace) and more on text
parsing
-- 
-o--=O`C
 #oo'L O
<___=E M

From summer 2006 to summer 2007 i was reading and sometimes participating in LKML. I was quoted once on LWN[0], like that:

One remarked that "Coloring isn't useful. If it was, it would be implemented ~16 years ago."

[0] http://lwn.net/Articles/253726/

That was very silly topic(more on that later).


But i'm going to share some real stuff. Main topics will be:


* userspace sucks * parsing, compilers, editors (that are suck) * building (which is suck) a program from the source code * and more (like HTML, XML, CSS, browsers, OSes suck)

I'm doing this writing in Firefox/win32. I tried lynx once, without success, so i forced to do spell checking in gmail by copy/paste. Also i cannot use jed and more accessible text editing.

For some reason every new paragraph's first line here in pre tag is shifted right. Crap, i thought pre should have no formatting, except damaged CSS.

Anyway, let's try first real posting, and see what will happen.

Almost an year ago I've posted my first entry here. Now I decided to patch and update it (comments are welcome).

Also, after more than year, there are some news about my GIMP's usability bug. Dark desktop theme and GIMP toolbox' buttons.

Debian GNU+Linux updated X Window system, so ati driver (as noticed by it's author) is trying to damage (burn) video chip on my laptop.

Blackbox's (window manager) author, after many years after 0.65 version started to develop and released 0.70 version, that has completely new look of window and menu decorations, that I dislike(hate!). But this let me saw Xfce. And after a week of stall (I've managed to kill Xfce in testing and broke testing at all ;), I'm going to check sid's 4.3.90.1-2 version.

Having such a great opportunity, I want to meet freetype here: "Hi ! How are you doing ? Good luck !!!".

5 Jun 2005 (updated 2 Jun 2006 at 19:57 UTC) »
My first post is my shame.
I must patch it with (hopefully) correct English and reply to myself. 

> I don't know if somebody will read this... I still wonder and waiting for advice.

> Is there something like was proposed on <http://www.levien.com/fps.html>? Yes. It seems to me yes, there is, and name for that is Gnome Bounties <http://www.gnome.org/bounties/>.

Anyway I don't know why on modern notebook PC all desktop is even slower than I felt it on win16-win95-xfree3+rh62.

Discovering LISP myself with help of Emacs, Paul Graham, and others, then seeing traces of it in modern python-based projects (f.e Twisted, with it's banana protocol), I still wonder why all is that, as it is ? And "XML optimisation" isn't a way <http://www.gnomefiles.org/app.php?soft_id=1397>. IMHO HTML, XHTML and other brain-damaging "technologies" with it's XML are broken by design[1].

Why world doesn't have any usable computer and OS with it ? Only home-wifes with ready-to-use-get-the-facts toasters and technicians with 1-2GHz FSB, Gigabit ethernet, 21" screen oscilloscopes[2] ?

I can't buy one even on cost of new excellent car ! I'm deeply disappointed...[3]

[1] IMHO PDF, as published by Adobe, is current best solution for every type of documents -- online or offline. PDF is not craft of html,css,gif,flash or whatever ! And Xpdf software on linux-gnu is an excellent proof of good style of layout (comparing to Adobe's software and hance PDF spec.), vs. having Gecko/KHTML/gtkhtml... Dot-coms crashed, but consequences remain: web standardisation is a crap, home-grew "specialists" in web-design-programming with flash ads. with dot.com's like business models are everywhere.

[2] Originally IBM opened architecture of IBM PC. So why many of modern hardware is being closed by manufacturers, that use this IBM's concept ?

[3] IMHO. There're a lot of good ides, but very few of them have correct implementation and wide usage...

> I'm _still_ pretty sick of GTK2/Pango rendering speed. > Maybe I'm wrong and need to do something with > preferences on my Debian GNU+Linux system. > .. by design...

> Thanks.

And thank You, all !

[I've finished physics department in one of the post-soviet countries. Now I really want to have chance to study CS. But it seems like technical progress had stopped in 80's on USSR(well, something is developing, but inside of black, military closed boxes), computer science and application progress (in software) had stopped on time even prior to dot-coms.]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!