Annoying keyboard shortcuts in Xfce

Quickie for mint16/17 using Xfce. You may have found the ctrl-Fn keys have been mapped to change the workspace you are on? This behaviour is highly annoying as applications often define ctrl-Fn to do something.

Do not try to fix it from the graphic configuration window. It does not work.

Instead, as root, open this file in a text editor:

/etc/xdg/xdg-default/xfce4/xfconf/xfce-perchannel-xml/xfce4-keyboard-shortcuts.xml

(Because or links there are about three or four files pointing to this file; it doesn’t matter which one you edit.)

Look for lines like:

<property name="&lt;Control&gt;F5" type="string" value="..."/>

Then delete all those lines, there are 12 in total, from F1 to F12. (In my case they were not in a block, so you had to hunt around to catch all twelve.)

Then you need to logout of your desktop, and log back in again for the change to take effect.

Written with StackEdit.

boost on centos (vs. on ubuntu)

You'd think porting a C++ program from one 64-bit linux to another would be trivial. But, no. A program developed with no issues on Ubuntu 10.04 was a lot of trouble to get to compile on Centos 5.6.
But get it to compile I did... read on for the secret words you have to utter...

First thing I did was:
   yum erase boost boost-devel

This got rid of the very old 1.33 library. I then ran this:

  yum install boost141 boost141-devel boost141-program-options boost141-regex boost141-thread boost141-system

I then had to hack the makefile to add this to my CFLAGS:

  -I/usr/include/boost141/

That got me compiling. But not linking. I changed my LDFLAGs to look like this but still it would not link:

  -L/usr/lib64/boost141 -lboost_regex -lboost_program_options -lboost_system -lboost_thread

I played around with various -L settings, until I had a breakthrough. I noticed the above line complained with:

  /usr/bin/ld: cannot find -lboost_thread

But some different -L settings instead gave me:

  /usr/bin/ld: cannot find -lboost_regex

Blink and you miss it, and indeed I had. My above line was linking with most of Boost, just not boost_thread. In other words, I was really close and hadn't realized it. A bit more poking around discovered that Ubuntu calls it "boost_thread", while Centos calls it "boost_thread-mt" (careful, the first one is an underline, the second one is a hyphen). So, the final magic line for Centos was:

  -L/usr/lib64/boost141 -lboost_regex -lboost_program_options -lboost_system -lboost_thread-mt

NOTE: I did not do any post-installation steps after the yum install steps. I saw some people suggesting copying files from one place to another but I did not need to. (Also, -L/usr/lib64 should be sufficient, as every thing is symlinked; maybe even the -L flag is not needed at all... but I'm in the "If it ain't broke." mindset now, so no more experimentation for me!)

Add a temporary static IP address

At home, with wired ethernet, my (Ubuntu) notebook has a few static IP addresses that I use for developing websites. Out of the house, I use wicd, so I have a dynamic IP address, and those static IPs don't exist. wicd configuration is too complex for me to understand, so I just accept this, but it caught me short the other day when I needed to have both an internet connection and to be able to work on a website running on my notebook.

I failed then, but I'm ready for next time. To temporarily add a static IP address you simply do (as root):

ifconfig eth0:3 10.10.10.10 netmask 255.255.0.0

I'm choosing "eth0:3" for the interface; it can be any unused number after the colon, and you never need to care what this is. netmask can really be anything for our purposes. The 10.10.10.10 is the IP address I've given it. Test with this:

ping 10.10.10.10

To set up a quick virtual host create a file under /etc/apache2/conf.d called 10.10.10.10.conf (any filename is fine) with these contents:

<virtualhost 10.10.10.10:80="">
    DocumentRoot "/var/www/somewhere"
    ServerName 10.10.10.10
</virtualhost> 

Tidyup

To remove just the interface that you added above, use this command:

ip addr del 10.10.10.10/32 dev eth0:3

Or, to restore the network to boot defaults (useful if you have done lots of changes) you can do:

ifdown -a
ifup -a

Either way to then remove the apache config: delete the 10.10.10.10.conf file you created and restart apache.

svn, ssh, /bin/false, git, /etc, etc.

I just spent almost two hours troubleshooting an svn server. It was set up to allow ssh access from a few users over ssh as described in this helpful blog post.

Today I realized none of the svn clients could run svn update: "svn: Network connection closed unexpectedly"

Part of the challenge was that I'd not run "svn update" in 1-2 months, so I had a big time frame for breakage. But all I could remember changing recently was commenting out some lines in my iptables firewall, so I spent quite a while staring at that. And I'd updated some packages and rebooted.

I spent a lot of time looking at ssh verbose output. To do this use

export SVN_SSH="ssh -vv"

(-v for quite verbose, -vv for more, -vvv for even more, etc.) No error messages: it connects, runs the svnserve command, passes up some environmental variables, sends some data, and then simply loses the connection.

Then I wondered if the svn user had somehow lost permission for something important. It has /bin/false as its shell, so I gave it /bin/bash, used su to become root, then "su - svn". Ran the svnserve command. It is fine. Looked at the repository. It is fine. Ran svnadmin to check for locks. None.

Giving up, I started crafting an email to post somewhere asking for help. I went back to get the exact error message svn update was giving me... and it worked. Yep, the other client works now too. My first guess was running svnadmin had quietly fixed something. But then, on a hunch, I changed svn's shell back to /bin/false. Yep, that broke it. It seems the svn user has to have a valid login shell.

Okay, problem solved. But I'm sure that svn has always had /bin/false, and now I wanted to know if that is true, and if and when it changed. It is too late now, but ready for next time, I decided to put all of /etc into a git repository (no need for a central repository, so git is far better choice than svn for this). The git commands (all run as root) are trivial:

cd /etc
git init
git add .
git commit -m "Initial files" -a

I found this page of someone who has done something similar, and he suggested sending the git status report from a daily cron, so I did that too.

It is so easy, and disk space so plentiful, that I think I will do it on all my linux machines.

UPDATE: /etc has some files that get edited a lot, so to reduce noise this is my .gitignore file so far:

/cups/printers.conf*
/cups/subscriptions.conf*
/emacs/site-start*
/mtab
/adjtime
/ld.so.cache
/*-
*~

(Use "git rm --cached cups/printers.conf*" (for instance) to stop git tracking the files if they've already been added.)

Customize less: less annoying

Open ~/.bashrc (or ~/.bash_profile) and add this line at the end:

LESS=-FRX;export LESS

Don't ask why, just trust me. ('cos actually I don't know what it means either...)

This was the magic incantation to tell "git log" to wrap commit messages (instead of just chopping off everything extra).

But it has had the delightful side-effect of man now works properly. About 6 or 7 years ago I used to be able to type "man whatever", press q, and the advice would stay on screen. Then linux (all distros as far as I could tell) changed to hiding the man page as soon as you press q. Frequently I'd have to have two terminal windows open, simply so I could keep the man page open while I type my command. Yes, you're ahead of me; the above LESS command fixes this. If I'd known it would be so easy I'd have done this years ago!!

Even better, the help system in the R language was working just like man pages (and it was even more annoying there), and that has been fixed too.

It is a miracle, I tell you, a miracle! July 13th should go down as "Saint LESS=-FRX" day; in 100 years time it will be a public holiday and children will be taught about it in schools. They might even be taught what it means...

oauth for php (and Ubuntu)

There is a "standard" PHP OAuth library, documented in the manual, and which is installed via pecl. There is no package shortcut under Ubuntu; you still have to use pecl
sudo pecl install oauth

If you get a complaint about no "phpize", install the "php5-dev" ubuntu package. And then if you get an error in "php_pcre.h" when compiling, then you need to install the "libpcre3-dev" ubuntu package.

Finally, you need to enable it. Still as root:
cd /etc/php5/conf.d
echo "extension=oauth.so" >oauth.ini

(This creates a config file just for oauth; you could also simply put the extension line in php.ini.)

Finally, "php -m" should list "OAuth", and you can create a OAuth object in your php scripts.

Ubuntu package manager lists "liboauth-php"; the minimal information, and the lack of mention of pecl, should have given me the clue that it is something different.

Also different is this: http://code.google.com/p/oauth-php/

scp with multiple targets: ssh-add

I sometimes have the need to upload the same file to two or more locations on the target server. In fact I even have a script to help me, which goes something like this:

scp file1 file2 remote01:/path/to/somewhere/
 scp file1 file2 remote01:/path/to/another/

The problem is I need to type the password for remote01 twice. I had never managed to find some clever scp syntax to allow specifying two destinations, and a post on the TLUG mailing list confirmed that. But what I did learn from TLUG is that there is something called ssh-agent that can store passphrases for key pairs; this plugged a gap in my knowledge. There are three ways to login in via ssh/scp:

1. Give your password
2. Make a keypair with no passphrase
3. Make a keypair with a passphrase

The second way is used to allow scp from cron jobs, to automate copying files between two machines. I'd never really got the point of the third way: if you have to type a passphrase why not just use the first method, and save messing around making a key pair. (Well, yes, there is better security: a log-in then requires both something I have and something I know; you have to turn off PasswordAuthentication on your ssh server for this to have any meaning though. Thanks to Kalin for this comment.)

But it turns out there is this program called ssh-agent that remembers passphrases for you. And I found it is already running in the background on Ubuntu.

Enough chat, let's look at the solution. First, I created a one-off keypair for this script, on my machine, using something like this:

cd ~/.ssh
  ssh-keygen -t dsa -C me@example.com -f key_for_remote01
  chmod 600 key_for_remote01*
  scp -p key_for_remote01.pub remote01:~

When generating the keypair give a reasonably secure passphrase (you will have to type it in each time, and it is only of use to people in possession of key_for_remote1, so no need for a 20-random-character monster; I believe it is perfectly fine for it to be the same as your normal ssh login password for remote01).

Then log in to remote01 and append key_for_remote01.pub to ~/.ssh/authorized_keys. If that file does not exist then you can just rename key_for_remote01.pub to authorized_keys and move it into ~/.ssh/

(By the way, there is no need to put your private half of the keypair in ~/.ssh/ but that seems as good as place as anywhere else.)

Now, I modified my script as follows:

ssh-add -t 120 ~/.ssh/key_for_remote01
 scp -i ~/.ssh/key_for_remote01 file1 file2 remote01:/path/to/somewhere/
 scp -i ~/.ssh/key_for_remote01 file1 file2 remote01:/path/to/another/
 ssh-add -d ~/.ssh/key_for_remote01

What happens is the ssh-add line will ask you for your passphrase. The two scp lines then work automatically. Finally the ssh-add -d stops it caching your passphrase (forcing you to type your passphrase each time you run this script).

The -t 120 parameter says the passphrase will expire after two minutes. This is just in case the batch file doesn't complete and so does not get chance to run ssh-add -d.


Note: you can use that same key pair for other machines. Basically anywhere you put the *.pub half of the key pair will let you login. And you can login from anywhere you have the private half of the key pair.

Note: the timeout/deletion code above is deliberate for this application, but you don't have to do it this way. By allowing it to cache it permanently you would only be prompted for your passphrase once, and then all future ssh and scp logins would be automatic. They will be cleared when you log-out of gnome (on ubuntu, at least) or shutdown your machine.

Note: If you don't want to specify -i each time you use ssh/scp then you can add an entry to your ~/.ssh/config file, like this:

Host remote01
        Hostname 10.1.2.3
        Port 22
        IdentityFile ~/.ssh/key_for_remote01

Note: I used -C with my real email address. This is put in the public key, and I wanted the administrator of remote01 to know who put the key there. Without -C it defaulted to "myusername@myhost". The administrator of remote01 knows nothing about my machine names so this seemed unreasonable and I decided to use -C. But the advantage of that default is it seems ssh-agent knows about that name and will prompt automatically the first time you try to use ssh/scp, which means there is no need to run ssh-add first. I have not worked out yet if ssh-agent can be told to know about my email address too.

When bash history no longer works

I've been pulling my hair out: I have all the HISTORY variables set correctly, but history never survives to the next session. I thought it must be an Ubuntu 10.04 bug, as only that machine, but couldn't track down anyone reporting the same problem.

The breakthrough came when I stopped typing "history" to view the history, and decided to "cat ~/.bash_history". I got permission denied... and discovered the file was owned by root. Suddenly it all made sense.

(By the way, I've a bunch of blog articles about setting up a new Ubuntu dual boot, RAIDed, encrypted machine; coming soon to an RSS feed near you...)

Wine broke, or PHP, or xinetd??

Sometimes it can be really educational to sit on the shoulder of brilliant developers and watch how they troubleshoot and debug. So, here is your chance to sit on my shoulder,... and snigger as failure reduces me to tears.
[2010-11-18 UPDATE: a mere 5 months after originally posting, I think I have the answer; see the bottom of this article.]

A program worked fine Saturday. This is on my Ubuntu 8.04 (hardy heron) machine.

Sunday I updated firefox and ghostscript only. I then rebooted, which means kernel 2.6.24-28 finally became active (I updated it July 11th, but hadn't had chance to reboot until yesterday).

Today that program doesn't run. It is one particular wine program: a go program. Another wine program (also a go program) runs fine. And here is the real killer: if I start that problem wine program from gogui (gogui.sf.net) it works. Exactly the same commandline, but it works from gogui, and doesn't work when started from my php script. That PHP script was last changed last Wednesday.

I rebooted into the previous kernel (2.6.24-27), and the problem is exactly the same.

Have you ever done those logic problems where you get a list of clues and have to work out who did what? Applied to the above we discover the only logical explanation is that... reality is warped. It can't be: wine, the go program, php, my php script or the kernel.

Here is some more evidence. The first time I run it after a reboot the program (called valhallgtp.exe) fails to start with a stack dump and backtrace. Here are the first few and last few lines of that:
=>1 0x7bc3b23c __regs_RtlRaiseException+0x4c() in ntdll (0x014ded68)
2 0x7bc76de3 in ntdll (+0x66de3) (0x014df0cc)
3 0x7bc3a936 RtlRaiseException+0x6() in ntdll (0x014df144)
4 0x00415833 in valhallgtp (+0x15833) (0x014df2ac)
5 0x004159dd in valhallgtp (+0x159dd) (0x014dfa24)
... (6..20 are all in valhallgtp)
21 0x0049b02c in valhallgtp (+0x9b02c) (0x014dff08)
22 0x7b8773a7 in kernel32 (+0x573a7) (0x014dffe8)

I also see: "err:seh:raise_exception Exception frame is not in stack limits => unable to dispatch exception."

The 2nd and subsequent times I try to start it I get:
Failed to start wine ValhallGTP.exe ...
(this is an error from my php script; i.e. proc_open() is failing.)

The CWD (current working directory) is correct, the parameters are correct; both are exactly what gogui is using. And running it from a bash shell is fine too.

The fingers are pointing at PHP. But no PHP upgrades either yesterday or in the July 11th batch. And that same PHP is still successfully starting 4 other programs, including another one that uses wine. (BTW, go to synaptic package manager, file menu, history; this is where you can see exactly what Ubuntu updated and when.)

Disk space is fine ("df"), memory is fine ("cat /proc/meminfo"). Machine load is low ("w"). Running it as root has same problem.

When I ran from the -28 kernel I also got errors about no X DISPLAY. And then when I started the other wine program it opened but had no display. And running wine config was the same. I've not seen that again. Let's pretend it never happened and concentrate on what we can reproduce.
(UPDATE: it happened again (this time with the -27 kernel). I had the gtpmfgo.exe running fine under wine; then starting mfgo.exe it came up but with no display. Close and retry showed it was repeatable. After closing gtpmfgo.exe it worked, and then I could open gtpmfgo.exe fine. Note: not repeatable; I haven't managed to get it to happen again.)

Okay, where to next? What I don't understand is how come I cannot reproduce the problem I get when I start it the first time. That seems like a small enough challenge to be achievable: how to get it to crash consistently.

...time passes...

Aha! It seems it is crashing in the same way; it was just crashing and dying before I got chance to read the crash message. So the first time after a reboot it must take longer than a second to start up (perhaps wine initialization).

...aha, aha! Narrowed down further. To explain this I need to give more background: I have a script called frontend.php that calls backend.php over sockets. backend.php uses stdin/stdout and uses xinetd to turn it into a socket server. When I run backend.php directly it works!

Could the updates/reboot have affected xinetd somehow? ...probably not. The xinetd files are all dated Dec 2007 except my own config file which is June 30th. And, the other 4 programs are started in exactly the same way and work fine.

Starting backend.php in exactly the same way as xinetd does works fine. (I've tried it both from bash and from sh.)

...time passes...

I seem to have a solution. (I had started working on a xinetd replacement php script, but that doesn't seem needed now.) My solution was to set xinetd to listen on another port, and use that port for just the problem program. This is working. I cannot explain it however: I had already tried connecting to only that program, so xinetd wouldn't have been doing anything different to now.

UPDATE: 7 weeks later and I do another reboot and it broke in exactly the same way!!
And again the only solution I could find was to give xinetd a new port for just this particular problem.
Weird, very, very weird, ...

2010-11-18 UPDATE: It was bothering me again, but on a different machine, and the trick with setting up xinetd on a different port didn't work. But adding this code to the top of backend.php seems to have solved it:
if(trim(getenv('DISPLAY'))=='')putenv('DISPLAY=:0.0');

The problem program has a GUI you see. I switch the GUI off with a commandline option ("hidewindows") but I think it actually creates windows and just hides them. And yes, none of the other programs uses a GUI... sigh, the clues were there.

Curiously the "hidewindows" option no longer works! That is probably going to annoy me, but not as annoying as not working at all!

Linux keyboard shortcuts

Have you ever gone to hit ctrl-tab (to switch tabs) or ctrl-w (to close current tab) in Firefox and suddenly all your Firefox windows (even the one playing radio on another desktop) disappear? (I'm using Ubuntu and Gnome, but I get the impression this problem affects all Linux distros and all window managers.)

It must be old age making my fingers clumsier but I didn't even know ctrl-q did that until a few weeks ago. Yet I've done it a couple of times by accident recently, and when I did it today I decided Something Has To Be Done.

The solution is not just joyously simple, but I also learnt another cool function while I was there. First the solution: System|Preferences|Keyboard Shortcuts. Assign ctrl-q to do something; then Firefox never gets to see it. I assigned it to the calculator app which is nicely harmless.

I learnt this trick here, which also mentions that the same idea works in XCFE (go to keyboard panel). I'm betting KDE has something similar.

And the cool function? Looking through the other keyboard shortcuts I saw Alt+Print takes a screenshot of just the current window! You have no idea how many times I've clicked Print, then opened up Gimp to crop the screenshot to show just the window of interest. I'm now alternating between feeling very foolish and feeling very empowered.

samba and strange permissions

On my linux server I run a samba share, which is used by both Linux and Windows clients on the LAN. I moved it from an old FedoraCore machine to Ubuntu a few months ago, and ever since have been getting strange permissions: text files kept becoming executable (but only for user, not group or other).

It took me this long to realize what was going on, and when I tried to track it down about a week ago I concluded it was SciTE being strange on just samba partitions. I.e. I'd edit a file with rw-r--r-- permissions, save it, and it would end up with "rwxr--r--". Every time. But not on normal partitions, and gedit didn't do it on the samba partition. I noticed today that files created by a PHP script also got those weird permissions, and the penny dropped: gedit must be explicitly setting permissions when it saves a file. Scite wasn't the cause of the problem at all.

So, I went hunting again. I referred to Mount samba shares with utf8 encoding using cifs a lot, but in fact it didn't give me the answer I wanted: the instructions there gave me the same problem. (It did show me how to set my samba partition to mount from /etc/fstab, replacing my crude entry in rc.local though.)

Hunting through the troubleshooting section I found the "nounix" flag and tried that. Initially it made things worse, giving all files rwxrwxrwx permissions. Then I changed from "file_mode=0777,dir_mode=0777" to "file_mode=0644,dir_mode=0755" (which was what I had originally), and that combined with nounix works! All text files are rw-r--r-- before and after saving. Oh, the other change I added relative to the above page was including "uid=darren,gid=darren". Otherwise files were owned by "nobody:root" and I didn't have permission to edit anything (even with the suggested 0777 settings).

My guess is that my old FedoraCore samba server didn't have the unix extensions, Ubuntu 8 does, and somehow those unix extensions are misconfigured in Ubuntu. My samba server configuration is all defaults however... Anyway, it now works the way it has for the previous few years, so I'm happy.

UPDATE: I just realized this has also fixed another irritation - delete (that moves to the Trash folder) hasn't been working on that partition, but now it does. Trash folder vs. direct delete was only a minor factor; what was really annoying was every time I pressed delete it then popped up a dialog box requiring me to confirm it.

Email on ubuntu (exim)

When I installed real player a few months back I noticed it had installed exim. I thought that highly suspicious - what was real player planning to do behind my back? Well, I thought I'd try removing it today, and discovered realplay depends on lsb-core (linux standards), and lsb-core depends on exim. realplay was innocent.

The magic command to remove exim is:
dpkg --purge --ignore-depends=lsb-core --ignore-depends=mailx exim4 exim4-base exim4-config exim4-daemon-light

But after doing that I changed my mind. As I discovered I couldn't send any email from commandline. Did somebody just say "Duh!". Well I was hoping I could then configure something to tell it to use SMTP directly to my ISP. (Why do I want to send email from commandline? Well, yes to test programs, but much more importantly that that I realized my system has been sending me warning emails and they've been undeliverable, so for instance the daily mail telling me about my RAID problem has not been reaching me...)

Anyway, lsb-core and mailx were screaming at me that their dependencies were missing, so re-installing exim was just a matter of telling them to re-install. Then the key step I had been missing was this:
sudo dpkg-reconfigure exim4-config

Still not easy, but it becomes much clearer if you know that what exim calls a "smarthost" is what the rest of the world calls "my ISP"! I.e. in my case I told it to use a smarthost for outgoing email, no incoming email, and for the smarthost I just looked in my thunderbird configuration for the SMTP server and gave it that address. It also kept asking me for a default domain and I chose "dcook.org" for everything.

Seems to have done the job!

Oh, the raid problem? It is just complaining I didn't set up swap as RAID. This was deliberate, see my original post on raid setup. At the time I wrote that I didn't understand the issues involved regarding RAID and swap. But if the system is complaining I will set it up as raid to make the message go away.

Linux partition advice

When I first partioned my disk I put /boot on the first partition and gave it 96M, following some advice found online I assume. This is not enough. I suggest giving it 256M. It is still a fraction of your hard disk. And because /boot is special it is impossible, or at least complex, to move some files to other partitions and link to them.

About a week ago ubuntu upgrade started complaining about a kernel upgrade problem. I ignored it for a few days assuming it would sort itself out. But it kept happening. Then when I viewed details of the upgrade I noticed (just briefly before it vanished off-screen) it was saying not enough diskspace on /boot.

Not again! When I tried to upgrade from Ubuntu 7 to Ubuntu 8 lack of space on /boot caused problems then too.

Poking around I also found I still had linux-generic packages installed, even though I'd switched to the linux-server kernel (see 6Gb on 32-bit linux). I deleted all packages that had the word "generic" in their name. After a reboot I still had one "generic" file left in /boot which I then just deleted. There was also a *.bak file for my current kernel. Datestamp was for a week ago, so I deleted that too.

It doesn't look like I've broken anything, and I'm now down to using 40M on /boot with 48M free (and I still have Ubuntu 7's linux-server kernel in there, which I think is now pointless, so I could reduce it even more).

Therefore, you can get away with a mere 96M /boot partition, it just requires more time and effort.

Conversely, I think a /boot partition above a certain size (1G?) causes problem at boot time, which is the whole reason for have a separate /boot partition. But I'm no expert, and that may be old-fashioned advice, and every BIOS on every motherboard made in the last 10 years may in fact be fine.

I dunno, and am too busy with more interesting stuff to study up on it, which is why I'll go for a 256M partition on my next computer.

Japanese NLP mailing list

A new mailing list for discussing Japanese NLP (natural language processing) in English has been set up, by Jim Breen (of JMDict fame):
http://groups.google.com/group/nlp-japanese?hl=en

There is a lot of software for processing Japanese text which is only documented in Japanese, and even then only minimally documented. So the new list is an ideal place (for those of us more comfortable in English than Japanese) for asking about how to use chasen, cabocha, namazu, etc. Or to describe what you are trying to do and get program and data suggestions. Hopefully people will also post about new software and data releases, related conferences, new academic papers, and so on.

Also, if the above interests you then you will also want to know about this Ubuntu repository for all kinds of NLP software:
http://cl.aist-nara.ac.jp/~eric-n/ubuntu-nlp/dists/hardy/japanese/

Much of the Japanese stuff is UTF-8 ready (as opposed to the EUC-JP that academic Japanese software still likes to default to).

Adobe PDF reader and Japanese fonts

Another casualty of my recent enforced ubuntu upgrade was Japanese fonts in pdf files. Adobe acroread has also moved from version 8 to version 9. When you meet Japanese in a PDF file it tells you the URL to go to to get asian font pack. Unfortunately that page only has asian fonts for acroread 8 and earlier!

The link should be:
http://www.adobe.com/support/downloads/product.jsp?product=10&platform=unix

(I'm mentioning this as it took a bit of work to discover it.)

Scroll down to the add-ons section, and it seems each language is now its own file, and the files are much bigger. (I don't know the difference between a "Font Pack" and a "Font Packs"; the files I are identical so I chose the latter.)

Unzip the bz2 file with "tar xjf FontPack910_jpn_i486-linux.tar.bz2"
Then move into the JPNKIT directory and type "./INSTALL"

The install process asks:
"Enter the location where you installed the Adobe Reader? /opt"

I didn't install it, Ubuntu did. However it seems Ubuntu is putting it in /opt!
Strange for a package-based distro to put anything there, but I accepted the /opt default and it worked. (This was different in Ubuntu 7, as I remember having to try lots of paths until I guessed the one it was after.)

Incidentally I have already got medibuntu.org as an extra repository, but there is no acroread-fonts or similar package. Perhaps there is some legal issue (though I thought medibuntu.org's raison d'etre was packages with legal issues).

Moving encrypted partition to software RAID

I moved most of my partitions to software RAID a few weeks ago. But I left /home/darren/ because it was encrypted. However a few days ago I moved it too. Here is how.

The quick overview: it is exactly like moving any other type of existing partition to software RAID, except where you would format /dev/md7, prior to copying the existing data over to it, you would set up crypt on /dev/md7 instead.

Detailed Steps

These instructions assume that you have moved other partitions to software RAID, or are at least familiar with the process (see previous article). All commands are run as root.

If you have not got an existing crypt partition you need to prepare for it:
* Install cryptsetup
* modprobe dm-crypt
* Add dm-crypt to /etc/modules

I will be setting up /dev/md7 for software RAID 1, but with just /dev/sdb7 in the raid array initially, i.e.
mdadm --create /dev/md7 --level=1 --raid-disks=2 missing /dev/sdb7

Next I setup /dev/md7 for crypt with these commands:
cryptsetup luksFormat /dev/md7 -c aes -s 256 -h sha256
cryptsetup luksOpen /dev/md7 somecryptraid
mke2fs -j /dev/mapper/somecryptraid -L somecryptraid

Each command only takes a few seconds to run. You will be prompted for a password. Forget that password and the data on your partition is lost forever; there is no recovery ability. So choose carefully.

The next command sets chkdsk to run every 100 days, however many times you boot. This is just personal preference, and completely optional (the default is to run chkdsk more frequently):
tune2fs -c 0 -i 100 /dev/mapper/somecryptraid

Run "mkdir /mnt/md7" then edit /etc/fstab and add this line:
/dev/mapper/somecryptraid /mnt/md7 ext3 defaults,noatime 0 0

What we are doing here is saying we want our new encrypted software raid partition to be mounted somewhere temporary, so we can copy our existing data over to it.
(the "noatime" flag is optional, and nothing to do with software RAID or crypt)

And edit /etc/crypttab to add this line:
somecryptraid /dev/md7 none luks

This is the command that will cause it to prompt for password on boot time. If you already have an encrypted partition then you are adding the above in addition to your existing entry: you will have two encrypted partitions for the next boot.

Now reboot into single-user mode (the recovery kernel). You should be prompted for the password for your new crypt partition (in addition to any existing crypt partition). Now I move /home/darren to the new crypt partition with:
cd /home/darren/
cd -dpRx . /mnt/md7

This took over an hour to run for me.

Once it finishes edit /etc/fstab to change the /dev/mapper/somecryptraid entry from /mnt/md7 to be /home/darren. And comment out the previous entry for /home/darren. I.e. the new entry looks like:
/dev/mapper/somecryptraid /home/darren ext3 defaults,noatime 0 0

Reboot. When running df you should see a line like:
/dev/mapper/somecryptraid ... ... ... ... /home/darren

You can now remove (or comment out) the old "somecrypt" entry from crypttab. Also use fdisk to change the /dev/sda7 entry from "83 Linux" to "fd Linux RAID autodetect" (use the fdisk "t" command to do this). Reboot again.

Now you should be able to run:
mdadm --add /dev/md7 /dev/sda7

This will take a while to run, as it mirrors the data from /dev/sdb7 to /dev/sda7. "watch cat /proc/mdstat" will show its progress.

"rmdir /mnt/md7" as a tidyup step at the end. You might also want to delete commented out lines in /etc/cryptab and /etc/fstab if you like to keep those files lean and clean.


Troubleshooting

When I ran "mdadm --add /dev/md7 /dev/sda7" I got this error:
"mdadm: add new device failed for /dev/sda7 as 2: No space left on device"

I tracked this down to slightly different sizes as reported by fdisk:
/dev/sda7 16988 23361 51199092 fd Linux RAID autodetect
/dev/sdb7 16988 23361 51199123+ fd Linux RAID autodetect

Even though start and end cylinders are the same, the number of sectors is different! When I change the view ("u" command in fdisk), to show start and end sector, the problem is clearer:
/dev/sda7 272896281 375294464 51199092 fd Linux RAID autodetect
/dev/sdb7 272896218 375294464 51199123+ fd Linux RAID autodetect

In other words /dev/sda7 starts a few sectors later. My fix was to delete /dev/sda7 from the partition table, and then recreate it. Then /dev/sda7 showed the same start/end sector as /dev/sdb7. Weird and spooky, but now the "mdadm --add" command worked (after another reboot of course).

Tip: use fdisk to check your partition sizes are exactly the same before starting! The other way to fix this would have been to make /dev/sdb7 smaller but, by the time I realized, it was too late to do that.

Ubuntu Hardy: Hard(y) Work

After last week's upgrade on my notebook, I did the forced upgrade from Ubuntu 7 to Ubuntu 8.10 on my main machine yesterday. Numerous breakages.

Easy Fixes
* Skype disappeared. I still had the deb package and installing it again cured that. All my settings were still around.

* SciTE: It overwrote my old settings. Luckily I keep a backup copy and merged in my differences.

* Evolution was back. So I deleted again all packages it would let me delete (some parts are entwined around the heart of either Ubuntu or Gnome).

* The Permit Cookies firefox plugin didn't like firefox 3. You have to go to developer's own site to get a version that will install.

* SCIM (Japanese input) hot key had stopped working. This fixed itself when I fixed something else (my guess is the xorg.conf change for the Alt-F1/F7 problem - see below).

Hard Fixes
* pcmanfm. This is better than nautilus. Full stop. But after the hardy upgrade trying to launch anything fails - it tries to launch, but after 20s or so gives up. This even includes opening the terminal. I tried reinstalling, and tried getting the latest version, but same problems.
Solution: I gave up. My nautilus problems seem to have gone, and I installed nautilus-terminal package which gives me the "open a terminal in this directory" feature that I'd become addicted to. (Cannot get F4 to open terminal though: the closest I can manage is "Alt-F, t")

* Ctrl-Alt-F1/F7 no longer worked. See my screen blanking blog entry for the xorg.conf fix for this.

* Real Player Plugin. Firefox 3 decided to move the goal posts and keep plugins in a new directory; so you have to move the firefox plugins to the correct place (/usr/lib/firefox-addons/plugins/) yourself.
That still wasn't enough. I've been disabling various other plugins, reinstalling real player, and lots of swearing. I finally have it working, but I cannot tell you exactly which steps were needed.
In the end I used the Real Player's deb, which is highly suspicious as it has dependencies such as "exim" (an email client)?!! Many years ago Real got a bad reputation for malware, and I'm wondering if they are still up to their bad tricks?
But even just that deb didn't work. So, I think that the thing that finally worked was deleting the totem-mozilla package. (I'd already disabled all plugins with the word totem in them however, which was no help, so I'm not entirely sure... the real Real Player plugin calls itself "Helix DNA Plugin", and I think I'd also disabled that... which with hindsight was probably the cause of all my problems?)


Other Fixes

A few months back the red icon to shutdown gnome stopped working. Hardy didn't fix that. But I did find the cause today: Preference | Sessions | Enable Power Manager. A tad confusing - you'd think it was something only needed on notebooks. But, anyway, that was the key.


Things That Did Work Okay:
* Samba (aka windows) shares
* Software RAID
* Encrypted Partitions
* Email/Browser/sound/video card/etc.


Summary

Thankfully Ubuntu 8 is a LTS (Long Term Support) release, so I shouldn't need to go through this for another couple of years.
And while I now hate Ubuntu I have been reminded about its most important feature: a huge user base. Which means typing "Ubuntu 8 some problem" into Google will always find someone having the same problem, usually with some kind of solution.

Thanks to all the hundreds of Ubuntu users who answer questions, and blog about their fixes!

Ubuntu Expired On Me!

So, I boot up my notebook into linux (Ubuntu 7) for first time in a while, maybe 2-3 months, to prepare it for a meeting. I'm a good boy and first go to install the updates I know will be there. It says it cannot find some. Strange.

On a hunch I wonder if Ubuntu 7 has reached end of life. But, no, this page tells me Gutsy Gibbon (7.10) is still current:
https://help.ubuntu.com/community/UpgradeNotes

It is wrong. Ubuntu obviously haven't read their own press releases. It EOL-ed a week ago:
http://www.ubuntu.com/news/ubuntu-7.10-eol

What is annoying is how EOL breaks everything. Packages won't validate. Worse, I cannot install some new software (such as mysql; but I could install lame). This is not simply "we're not going to do updates any more". This is "We've pulled the plug. Should've upgraded earlier. Loser."

Two forehead slaps for Ubuntu in the space of a week. I'll be back to Fedora at this rate!

Anyway, I started the upgrade to 8.04, but after running for 20 minutes it finally told me it was going to take 19hrs to run. Not good timing for my meeting!

By the way, 19hrs was a bad estimate. It takes 10-20 minutes to get it started, then 3-4 hours downloading, then 1+hrs installing. It took 6 hrs in total for me. You can leave it alone for the downloading, but need to be around to answer questions during the install stage.

Screen keeps going blank

I mentioned this in previous entry (on software RAID). My screen keeps going blank. Ctrl-Alt-F1, breathe in, breathe out, Ctrl-Alt-F7 fixes it. I.e. that must reset the video card or X.

It used to happen once a day or less. Since switching to RAID it is happening once an hour roughly. But 9 times in past hour, which was past my threshold, so I went hunting...

...and came up empty-handed. Xorg.0.log has entries but they match what happens if I do the Ctrl-Alt-F1/F7 sequence. /var/log/acpid also has an entry that I think goes along with Ctrl-Alt-F7. But nothing else. No errors in either. Nothing in messages, syslog et al that could be associated.

/var/log/gdm/:0.log also gets an entry when I do ctrl-alt-F7, but nothing beyond that.

Nothing else in /var/log at all.

I'm stumped. I doubt over-heating, and I doubt CPU load (as all I've been doing the past hour is typing in a text file). w tells me "0.00 0.09 0.07".

Not a very informative blog, sorry. Anyone got any suggestions? I don't want to mess around trying different video cards, monitors, etc.; it is still just a minor irritation.

UPDATE: After updating to Ubuntu 8 I still had this problem, but worse my Ctrl-Alt-F1/F7 fix no longer worked! A bit of googling tracked down a solution:
In /etc/X11/xorg.conf, in "InputDevice" section, I commented out these three lines:

  #Option  "XkbLayout" "jp,jp"
  #Option  "XkbVariant" "latin,"
  #Option  "XkbOptions" "grp:alt_shift_toggle,grp_led:scroll"

and replaced with just:

   Option  "XkbLayout" "jp"

Even better, Ctrl-Alt-F1..F6 now give me a terminal! They never have in Ubuntu, instead they just gave me a blank screen. The above xorg.conf has been there since the start - Ubuntu 8 upgrade didn't alter it.
(Oh, and the above change has not fixed the screen going blank problem, so I'm still no closer to fixing that...)

Update: I've noticed if I move the mouse, especially the mouse wheel, just after the screen goes blank, it comes back. It doesn't always work - not sure if there are two problems here, or if I'm too slow reaching for the mouse sometimes.
Anyway, if that suggests a potential cause for someone let me know!
(By the way, I have screensaver set to blank, after 30 minutes of inactivity. Just to see if that is related I've changed screensaver to some animation, and I'll let you know!)

Software RAID on an existing linux system

So, as mentioned in a previous article I bought a terabyte hard disk, mainly to impress the ladies. It doesn't seem to have been working too well in that respect. I considered sending it back, claiming it needs more testerone. But I decided to instead set up software RAID 1.

My main guide for this task was this article: http://www.howtoforge.com/software-raid1-grub-boot-debian-etch
Rather than repeat everything I will just comment on what I did differently. Note that despite the name of that article, it applied fine to Ubuntu 7, and I suspect it will apply equally well to any linux distro of the past two or three years.

First, my disks were different size: 320G vs. 1T. So I opened fdisk -l /dev/sda in one terminal, then ran fdisk /dev/sdb in a second terminal and set up the first six partitions (plus one extended partition) to be the same as /dev/sda. I then set up /dev/sdb8 to be one big partition with the rest of the disk, which I mounted as /backup.

In all software RAID guides they number the partitions /dev/md0, /dev/md1, etc. That is confusing so I decided to keep the same numbering schemes as my hard disks; it turns out this is fine. So I used commands like this:
mdadm --create /dev/md3 --level=1 --raid-disks=2 missing /dev/sdb3

compared to this from the HowToForge article:
mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb3


The above article did all partitions in one go. I wanted to take baby steps. So I started by mirroring one partition, choosing a non-critical one, and ignoring all that stuff about grub configuration. It worked well. I then did /var and /, leaving /boot unmirrored and therefore again ignoring all that grub configuration stuff. /var went smoothly but / did not. It kept saying /dev/sda3 was in use: "mdadm: Cannot open /dev/sda3: Device or resource busy" But df didn't list it. Trawling /var/log/* didn't list it. "Eh?" thought I.

Eventually I realized that grub was set to boot from /dev/sda3, not from /. I.e. grub works at the device level not the partition level.

Key Advice: mirroring one partition at a time is fine, but then do "/", "/boot" and all that grub stuff in one go.

Now, something the HowToForge article didn't mention at all was doing file copies in single-user mode. I know just enough unix administration to scare myself and the idea of taking a copy of /var and / while all the daemons were still running made me nervous. So I tried to boot to single-user mode.

Discovery #1: Ubuntu is weird. They don't use run levels like the rest of the linux world. Sure, their numbering scheme may make more sense, but it also causes much surprise.

Discovery #2: "telinit 1" doesn't work. The GUI shuts down and then nothing. I can see the system is still running but nothing on screen. Nothing on Ctrl-Alt-F1 through to Ctrl-Alt-F12. I think ctrl-alt-del caused a reboot.

So, I booted, and choose the "recovery kernel" from the grub menu. Recovery being another name for single-user mode. It asks for the root password. Ah. Ubuntu doesn't use one. I have to continue into a normal GUI boot.

Key Advice: Set a root password. This has nothing to do with software RAID: you never know when you are going to want to boot into a recovery kernel. I think Ubuntu deserves a forehead slap for this design decision? To set a root password, do "sudo passwd": it will first ask for your user password (that is how sudo works), then it will say "Enter new UNIX password:". I chose the same password as my normal user; easy to remember. (If that makes you nervous, remember that this is equal security to being able to run "sudo passwd"; more importantly, it saves me having to write down my root password somewhere.)

Other comments: the "cp -dpRx" command is the most time-consuming step for those partitions when you have a lot of data. The system will be sluggish while doing this (and if you are in single-user mode you cannot be browsing or doing anything else anyway). But also when you do the "mdadm --add /dev/md3 /dev/sda3" to actually create a genuine RAID partition it will be copying a lot of data, and your system will be sluggish for the time this takes (about 10 minutes per 30G). Bear this in mind.

Swap. I've created /dev/md2 as a swap RAID, but haven't used it. So /dev/md2 just contains /dev/sdb2. And "cat /proc/swaps" tells me only /dev/sda2. Apparently if my sda disk dies my swap will disappear and my system might crash. On the other hand, mirroring swap has a performance downside (apparently). I don't understand the trade-offs well, but this is a workstation, not a server and a crash should /dev/sda ever die is acceptable. I've also got more memory than I need so I suspect I could switch swap off completely. Anyway, so far Inertia born from Igorance means I'm doing nothing about this. Expert opinions are welcome.

Everything works. Just two things I've noticed. Sometimes the system doesn't boot. It comes up in an "ash" shell, which is one of those recursive acronyms standing for "Ash is Satanic Hell". So far the only command I've mastered is "reboot", which works well - I've not had two boot failures in a row. After 10-20 reboots I think this boot failure is happening about 20-25% of the time. I've no idea how to troubleshoot it.
UPDATE: this boot problem doesn't seem to be happening since upgrading to Ubuntu 8. Perhaps the upgrade fixed some mistake in my grub.conf? Maybe related is that I no longer have an (hd1,0) section in grub.conf; they are all (hd0,0). I think I'll leave it like that: if "hd0" dies I can edit the grub boot string that once, and then edit grub.conf.

The other thing is my screen sometimes goes blank. The system is fine, it is just like the video card has died. My solution has always been: Ctrl-Alt-F1, pause for half a second, then Ctrl-Alt-F7. I guess this process resets the video card. It used to happen maybe once a day or so. Now it seems to have happening once an hour on average, and is becoming almost irritating enough that I'll have to look into it. The fact that running RAID could affect the display feels like an important clue.

Finally, my home partition is encrypted, using crypt. I have not moved it to software RAID yet, for that reason. I will really soon (as all my data that could really benefit from the security of software RAID is currently the only data not protected by it)! I will run crypt on top of software RAID, rather than the other way around; I'm just not sure the best way to do the file copy.
UPDATE: now done, it was straightforward, see Moving encrypted partition to software RAID.