jnewbigin is currently certified at Apprentice level.

Name: John Newbigin
Member since: 2002-04-29 09:06:58
Last Login: 2008-06-27 08:38:21

FOAF RDF Share This

Homepage: http://uranus.it.swin.edu.au/~jn/

Notes:

I work at Swinburne University of Technology in Melbourne, Australia. I completed my Bachelor of Applied Science in Computer Science and Software Engineering in 1999. My job is split between external software contracts and internal Unix (Linux) administration.

I like to think of myself as a tool smith. I would rather write a tool to do something than do it manually.

I also can’t stand broken software, so that is why I like Open Source. I can easily fix any bugs or add features that I want.

I have written a number of programs that relate to Linux. You can get a list from my home page.

Projects

Recent blog entries by jnewbigin

Syndication: RSS 2.0

How to run a 64 bit kernel on 32 bit CentOS 4

Imagine that you have CentOS-4 i686 installed. Your CPU can run x86_64 and you decide you would like to use the 64 bit kernel.

The reason I wanted to try this was that I have a server with 6 Gig of ram and it was suffereing from lowmem starvation wich in turn was triggering the OOM killer.

Is this supported by Red Hat/CentOS/Anyone?
No. This is just my own experiment.

They said you can’t do this!
I am trying to prove that (wrong it seems).

Why would you do this anyway?
The 64 bit linux kernel can run 32 bit linux applications.
Even though 32 bit applications are still restricted to 3Gig of RAM each, the kernel will be free from the 1Gig limit which applies to the 32bit kernel.

The main problem I found was running 32bit programs from initrd. I don’t know why this is a problem but I do know how to fix it.

This process seems to work for me. It is just an experiment and should not break your 32bit setup. You should however do a backup before you start!

The process
Step 1: Download the following packages (or the latest release of each package):
kernel-2.6.9-78.0.5.EL.x86_64.rpm (This is the most important one)
mkinitrd-4.2.1.13-1.x86_64.rpm (for nash)
lvm2-2.02.37-3.el4.x86_64.rpm (for lvm)
udev-039-10.22.el4.x86_64.rpm (for udev)
module-init-tools-3.1-0.pre5.3.10.x86_64.rpm (for insmod)
iptables-1.2.11-3.1.RHEL4.x86_64.rpm (for iptables)
glibc-2.3.4-2.41.x86_64.rpm (to run iptables)

Step 2: Install the kernel
rpm -i kernel-2.6.9-78.0.5.EL.x86_64.rpm –ignorearch
Ignore the errors from the strip command. We will fix that soon.

Step 3: Extract tools for the initrd
cd /tmp
mkdir initrd
for i in mkinitrd-4.2.1.13-1.x86_64.rpm udev-039-10.22.el4.x86_64.rpm lvm2-2.02.37-3.el4.x86_64.rpm module-init-tools-3.1-0.pre5.3.10.x86_64.rpm; do
(cd initrd ; rpm2cpio $i | cpio -idv )
done

Note: I am using /tmp because this is just a test. If you want these files to stay you should put them somewhere else.

Step 4: Edit /tmp/initrd/sbin/mkinitrd:
Change “inst /sbin/nash” to “inst /tmp/initrd/sbin/nash”
Change “inst /sbin/insmod.static” to “inst /tmp/initrd/sbin/insmod.static”
Change “inst /sbin/udev.static” to “inst /tmp/initrd/sbin/udev.static”
Change “inst /sbin/lvm.static” to “inst /tmp/initrd/sbin/lvm.static”
Change “-x /usr/bin/strip” to “! -x /usr/bin/strip”

Step 5: Make the new initrd
/tmp/initrd/sbin/mkinitrd -v -f /boot/initrd-2.6.9-78.0.5.EL.img 2.6.9-78.0.5.EL

Step 6: Reboot
You should now be able to boot into the 64 bit kernel and load up to a more or less working system. Once you get this far there are a few more things you might want to tweak.

Step 7: haldaemon
haldaemon was printing some errors. I don’t want to use it so I turned it off
chkconfig haldaemon off

Step 8: glibc
In order to run some 64bit applications we need the 64bit libs.
cd /tmp/initrd
rpm2cpio glibc-2.3.4-2.41.x86_64.rpm | cpio -idv
cd /
ln -s /tmp/initrd/lib64 .

Step 9: Magic script
What instructions would be complete without some magic? I created a wrapper script which will invoke the correct binary based on the kernel arch.
make a command called /sbin/archrun
#!/bin/bash
exec $0.$(/bin/uname -m) “$@”

chmod a+x /sbin/archrun

Step 10: Fix modprobe
mv /sbin/modprobe /sbin/modprobe.i686
ln -s /sbin/archrun /sbin/modprobe
cp /tmp/initrd/sbin/modprobe /sbin/modprobe.x86_64

Step 11: Fix iptables
mv /sbin/iptables /sbin/iptables.i686
ln -s /sbin/archrun /sbin/iptables
cp /tmp/initrd/sbin/iptables /sbin/iptables.x86_64
mv /sbin/iptables-restore /sbin/iptables-restore.i686
ln -s /sbin/archrun /sbin/iptables-restore
cp /tmp/initrd/sbin/iptables-restore /sbin/iptables-restore.x86_64

Check with:
service iptables start

Step 12: Quota
If you use quota, you need the 64 bit tools. I don’t use quota so I didn’t do this.

There may be other steps depending on what you run on your system. The steps for fixing them will probably be the same as iptables/modprobe.

This procedure is not automatically repeatable when updates are released. It changes things without the knowledge of RPM which is not very good. It does not use rpm to track the 64 bit applications. They are saved on /tmp etc. These issues would probably need to be addressed if you were going to think about running the 64bit kernel long term.

So how did it work out?
Well I would say that it was a resounding success. I can now run vmware and LVM2 snapshots without the lowmem starvation.

I have not done any performance tests bit it does not seem to be any different.

Syndicated 2008-10-29 03:53:00 from John's test blog

mediawiki on Tyler

I recently looked up upgrading medaiwiki on tyler. Unfortunatly the newer versions require php5 which is not easily available on tyler which runs CentOS-3. Perhaps this is a good reason to upgarade tyler.

Not being able to test on tyler, I decided to test the new on mercury. Andrew has already upgraded swinbrain without any problems but the wiki.ict.swin.edu.au site has a few special requirements. So far, I have not been able to get the new version working correctly with rewritten URLs. The new version of mediawiki keep adding in ‘index.php’ where is is not wanted and even when most links work, some important ones (like save) do not.

I am sure I will find a solution to that but back to tyler. One option is to re-install the tyler PC with a new OS like CentOS-4 or CentOS-5. CentOS-4 is what I use on everything else so I know it well and will have a high level of compatability. It also means i can test on tyler and then use what i learn on the main servers. CentOS-5 is newer so might be more exciting and a good place to test the new features. A final possibility is to move the wiki to mercury and host it along side the faculty wikis. This reduces the maintainence requirements and should prevent compatabiliy problems.

I have alrady moved the database to the faculty database server which is now running mysql 5.

Fell free to email me or leave a comment if you have any suggestions.

Syndicated 2007-11-17 05:03:36 from John's test blog

What is a muxy?

I though it was about time that I posted a blog entry, but what was I going to blog about? Instead of blogging I should be working on muxy so I though that would be a good thing to describe.

Basicly muxy is a transport stream demultiplexor. That means you can use it for recording digital TV programs. You can actuall read more about it at MUXY the Muxer

Muxy forms part of the Digital ORB which will be a TV watching program like MythTV but without feature creep and targeted at Australian DTV.

Syndicated 2007-10-05 12:34:34 from John's test blog

Tyler moves

Tyler has been moved into the ICT server room EN314. It will now enjoy proper aircon, UPS power and high tech networking (10Gig ethernet backbone)

Syndicated 2007-05-15 01:32:00 from John's test blog

Making linux ‘user friendly’

For some years, there have been a few incompatability which turn users off linux. They are more configuration issues than anything else and probably windows is to blame, and specificly PuTTY. Let me explain the problems and then the solution.

Issue 1. UFT-8. UFT-8 is a Unicode encoding which uses variable length strings of 8 bit bytes to encode a character. The first 127 characters are the same as the ANSI characters which makes compatability easy. The problems start when the odd character above 127 gets thrown in but is not respected as the start of a unicode character. This is easily seen in man pages when ‘ turn into â. For the most part you can work around it but it is easy to fix.

The environment variable LANG indicates the current language. On modern linux distros which support unicode, it may well be set to something like ‘en_AU.UTF-8′. This indicates my language, country and unicode support. The simple fix is to set the language to ‘en_AU’ if my terminal does not support unicode.

Now this is all fine until you start using PuTTY to connect from windows, which I find most windows users do. Now your login script has to be able to turn of UFT-8, only if you are using PuTTY. Read on…

Issue 2. Colors. There are a number of factors which impact on the default color selection for programs like ls and vim. The biggest seems to be the terminal type as specified by the TERM environment variable. Common values are linux, vt100, vt220 and xterm. (There are hundreds of others). Now unfortunatly, assumptions are make like ‘the linux console background color is always black’ and ‘the xterm background color is always white’. This do not hold true, mostly because TERM indicated the terminal ‘emulation’, not the terminal it’s self. For example, by default, PuTTY reports that it does xterm emulation but it is clearly a windows program and is only emulating this. This issue is a bit harder to solve because there are many places were color information can come from (see /etc/profile.d/vim.sh). The TERM variable should match your terminal or your output will not be correct and expecting all users to correctly reconfigure there terminal is not realisting. A server side solution is required.

Solution Part 1. Anyone who has accidently viewed the contents of a binary file may well have ended up with PuTTY written all over the screen. This is a giveaway that you can query the terminal for information. (This is also a security issue, but for now it works in our favour.) (If you did view a binary file and now your text is broken, try the ‘reset’ command). Many terminals respond the the character 0×05 by sending the terminal type as a string, just as if the user had typed it on the keyboard. This is obviously quite an old feature (no one would design anything like that these days would they?) but luck for us it works by default with PuTTY.

A program which runs a login time can send an 0×05 and read back the string. This is not as easy as it should be because PuTTY does not ‘press enter’ when it is finished so you have to switch into noncanonical mode to read it. Still, it is possible to write a program which will report the terminal you are using.

Solution Part 2. If you find that your terminal is PuTTY and it is reporting xterm emulation then you can ‘correct’ some assumptions by altering the color ls configuration and the LANG environment variable.

Solution Part 2b. By correcting the COLORS environment variable (used by color ls) then you can make vim decide what color scheme to use. This is easly added to your .exrc

Implementation. Here are 3 code snippits. The first can be added to your ~/.bash_profile (or a global like /etc/profile.d/z_putty.sh). The next is for ~/.exrc or perhaps a gloabl vim rc file like /etc/vimrc. The third is the source to the answerback prorgram. See the code for compile instructions.

I will eventually package all this into a .rpm because we will be using it on our new servers. It will probably be available from http://www.chrysocome.net/

bash profile
eval `/usr/local/bin/answerback`
if [ "${ANSWERBACK}" ] ; then
if [ "${ANSWERBACK}" = PuTTY -a "${TERM}" = xterm ] ; then
echo "Terminal is ${ANSWERBACK}. Assuming dark background."
# Assume a dark background. This is not the assumes default for an xterm
# so we will reverse the default colorls setting
if [ "${COLORS}" = /etc/DIR_COLORS.xterm ] ; then
COLORS=/etc/DIR_COLORS
eval `dircolors --sh "$COLORS"`
fi
# PuTTY does not support UFT8 so we will remove that from our LANG
export LANG=${LANG/.*/}
else
echo "Terminal is ${ANSWERBACK}"
fi
fi

exrc
if $COLORS == "/etc/DIR_COLORS.xterm"
set background=light
else
set background=dark
endif

answerback.c

/* answerback.c version 0.1
Written by John Newbigin <jn@it.swin.edu.au>
GPL Version 2
Compile with 'gcc -Wall -o answerback answerback.c'
*/
#include <stdio.h>
#include <unistd.h>
#include <termios.h>
#include <signal.h>
#include <string.h>
int main(int argc, char **argv)
{
char *name = "ANSWERBACK";
char code = 5;
char buffer[16];
int r;
sigset_t sig, sigsave;
struct termios term, termsave;
FILE *fp;
if(argc == 2)
{
name = argv[1];
}
if((fp = fopen(ctermid(NULL), "r+")) == NULL)
{
perror("open");
return 0;
}
setbuf(fp, NULL);
sigemptyset(&sig);
sigaddset(&sig, SIGINT);
sigaddset(&sig, SIGTSTP);
sigprocmask(SIG_BLOCK, &sig, &sigsave);
tcgetattr(fileno(fp), &termsave);
term = termsave;
term.c_lflag &= ~(ECHO | ECHOE | ECHOK | ECHONL | ICANON);
term.c_cc[VMIN] = 0;
term.c_cc[VTIME] = 10;
tcsetattr(fileno(fp), TCSAFLUSH, &term);
write(fileno(fp), &code, 1);
r = read(fileno(fp), buffer, sizeof(buffer) - 1);
tcsetattr(fileno(fp), TCSAFLUSH, &termsave);
sigprocmask(SIG_SETMASK, &sigsave, NULL);
fclose(fp);
if(r > 0)
{
printf("export %s='%s'\n", name, buffer);
}
return 0;
}

Syndicated 2007-01-25 10:12:59 from John's test blog

16 older entries...

 

jnewbigin certified others as follows:

  • jnewbigin certified Fefe as Master
  • jnewbigin certified jnewbigin as Journeyer
  • jnewbigin certified lance as Journeyer
  • jnewbigin certified gregk433 as Journeyer
  • jnewbigin certified skvidal as Master
  • jnewbigin certified omega as Master
  • jnewbigin certified ladypine as Journeyer
  • jnewbigin certified pp as Journeyer
  • jnewbigin certified Ryan as Journeyer
  • jnewbigin certified cjwatson as Master
  • jnewbigin certified Senra as Master
  • jnewbigin certified jonas as Master
  • jnewbigin certified trow as Master
  • jnewbigin certified gby as Journeyer
  • jnewbigin certified pat as Journeyer
  • jnewbigin certified jsheets as Journeyer
  • jnewbigin certified striker as Journeyer
  • jnewbigin certified rakholh as Journeyer
  • jnewbigin certified vdv as Master
  • jnewbigin certified moray as Apprentice
  • jnewbigin certified sulaiman as Master
  • jnewbigin certified SIrabbi as Master
  • jnewbigin certified harold as Master
  • jnewbigin certified kristian as Journeyer
  • jnewbigin certified fabrice as Apprentice
  • jnewbigin certified willy as Master
  • jnewbigin certified msw as Master
  • jnewbigin certified ndw as Master
  • jnewbigin certified Daryll as Master
  • jnewbigin certified anderson as Journeyer
  • jnewbigin certified demoncrat as Master
  • jnewbigin certified chip as Master
  • jnewbigin certified RyanPavlik as Journeyer
  • jnewbigin certified drc as Journeyer
  • jnewbigin certified adam as Journeyer
  • jnewbigin certified saugart as Journeyer
  • jnewbigin certified mpawlo as Master
  • jnewbigin certified fab as Journeyer
  • jnewbigin certified pavlov as Master
  • jnewbigin certified LaForge as Master
  • jnewbigin certified merlyn as Master
  • jnewbigin certified zeev as Master
  • jnewbigin certified carmstro as Journeyer
  • jnewbigin certified ger as Journeyer
  • jnewbigin certified GrahamAsher as Journeyer
  • jnewbigin certified 4am as Master
  • jnewbigin certified simonstl as Master
  • jnewbigin certified MJ as Journeyer
  • jnewbigin certified aunty as Journeyer
  • jnewbigin certified itamar as Master
  • jnewbigin certified quintela as Journeyer
  • jnewbigin certified edward as Journeyer
  • jnewbigin certified dwiner as Journeyer
  • jnewbigin certified proclus as Master
  • jnewbigin certified weasel as Journeyer
  • jnewbigin certified louie as Master
  • jnewbigin certified thaytan as Journeyer
  • jnewbigin certified mkp as Master
  • jnewbigin certified ShredWheat as Master
  • jnewbigin certified dtype as Journeyer
  • jnewbigin certified stefan as Master
  • jnewbigin certified quinlan as Master
  • jnewbigin certified ping as Journeyer
  • jnewbigin certified rmathew as Master
  • jnewbigin certified notzed as Master
  • jnewbigin certified gtaylor as Journeyer
  • jnewbigin certified besfred as Journeyer
  • jnewbigin certified skx as Master
  • jnewbigin certified timriker as Master
  • jnewbigin certified nether as Journeyer
  • jnewbigin certified tromey as Master
  • jnewbigin certified itp as Master
  • jnewbigin certified malcolm as Journeyer
  • jnewbigin certified madscientist as Master
  • jnewbigin certified matt as Journeyer
  • jnewbigin certified thomasvs as Master
  • jnewbigin certified Ausmosis as Apprentice
  • jnewbigin certified tnt as Journeyer
  • jnewbigin certified ds as Master
  • jnewbigin certified omnic as Journeyer
  • jnewbigin certified bribass as Journeyer
  • jnewbigin certified spiv as Journeyer
  • jnewbigin certified dick as Master
  • jnewbigin certified robilad as Master
  • jnewbigin certified dsandras as Master
  • jnewbigin certified timur as Journeyer
  • jnewbigin certified ajt as Master
  • jnewbigin certified tetron as Journeyer
  • jnewbigin certified jes as Master
  • jnewbigin certified bogart as Journeyer
  • jnewbigin certified abg as Journeyer
  • jnewbigin certified jameson as Journeyer
  • jnewbigin certified jef as Master
  • jnewbigin certified dugsong as Master
  • jnewbigin certified tseaver as Master
  • jnewbigin certified gbowland as Journeyer
  • jnewbigin certified softweyr as Master
  • jnewbigin certified xiphmont as Master
  • jnewbigin certified caio1982 as Apprentice
  • jnewbigin certified jimmac as Master

Others have certified jnewbigin as follows:

  • mattr certified jnewbigin as Apprentice
  • remle certified jnewbigin as Apprentice
  • ncm certified jnewbigin as Apprentice
  • jnewbigin certified jnewbigin as Journeyer
  • trs80 certified jnewbigin as Apprentice
  • mterry certified jnewbigin as Apprentice
  • arasole86 certified jnewbigin as Apprentice
  • phplev certified jnewbigin as Master

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page