After last week's upgrade on my notebook, I did the forced upgrade from Ubuntu 7 to Ubuntu 8.10 on my main machine yesterday. Numerous breakages.
Easy Fixes
* Skype disappeared. I still had the deb package and installing it again cured that. All my settings were still around.
* SciTE: It overwrote my old settings. Luckily I keep a backup copy and merged in my differences.
* Evolution was back. So I deleted again all packages it would let me delete (some parts are entwined around the heart of either Ubuntu or Gnome).
* The Permit Cookies firefox plugin didn't like firefox 3. You have to go to developer's own site to get a version that will install.
* SCIM (Japanese input) hot key had stopped working. This fixed itself when I fixed something else (my guess is the xorg.conf change for the Alt-F1/F7 problem - see below).
Hard Fixes
* pcmanfm. This is better than nautilus. Full stop. But after the hardy upgrade trying to launch anything fails - it tries to launch, but after 20s or so gives up. This even includes opening the terminal. I tried reinstalling, and tried getting the latest version, but same problems.
Solution: I gave up. My nautilus problems seem to have gone, and I installed nautilus-terminal package which gives me the "open a terminal in this directory" feature that I'd become addicted to. (Cannot get F4 to open terminal though: the closest I can manage is "Alt-F, t")
* Ctrl-Alt-F1/F7 no longer worked. See my screen blanking blog entry for the xorg.conf fix for this.
* Real Player Plugin. Firefox 3 decided to move the goal posts and keep plugins in a new directory; so you have to move the firefox plugins to the correct place (/usr/lib/firefox-addons/plugins/) yourself.
That still wasn't enough. I've been disabling various other plugins, reinstalling real player, and lots of swearing. I finally have it working, but I cannot tell you exactly which steps were needed.
In the end I used the Real Player's deb, which is highly suspicious as it has dependencies such as "exim" (an email client)?!! Many years ago Real got a bad reputation for malware, and I'm wondering if they are still up to their bad tricks?
But even just that deb didn't work. So, I think that the thing that finally worked was deleting the totem-mozilla package. (I'd already disabled all plugins with the word totem in them however, which was no help, so I'm not entirely sure... the real Real Player plugin calls itself "Helix DNA Plugin", and I think I'd also disabled that... which with hindsight was probably the cause of all my problems?)
Other Fixes
A few months back the red icon to shutdown gnome stopped working. Hardy didn't fix that. But I did find the cause today: Preference | Sessions | Enable Power Manager. A tad confusing - you'd think it was something only needed on notebooks. But, anyway, that was the key.
Things That Did Work Okay:
* Samba (aka windows) shares
* Software RAID
* Encrypted Partitions
* Email/Browser/sound/video card/etc.
Summary
Thankfully Ubuntu 8 is a LTS (Long Term Support) release, so I shouldn't need to go through this for another couple of years.
And while I now hate Ubuntu I have been reminded about its most important feature: a huge user base. Which means typing "Ubuntu 8 some problem" into Google will always find someone having the same problem, usually with some kind of solution.
Thanks to all the hundreds of Ubuntu users who answer questions, and blog about their fixes!
Wednesday, April 29, 2009
Friday, April 24, 2009
Ubuntu Expired On Me!
So, I boot up my notebook into linux (Ubuntu 7) for first time in a while, maybe 2-3 months, to prepare it for a meeting. I'm a good boy and first go to install the updates I know will be there. It says it cannot find some. Strange.
On a hunch I wonder if Ubuntu 7 has reached end of life. But, no, this page tells me Gutsy Gibbon (7.10) is still current:
https://help.ubuntu.com/community/UpgradeNotes
It is wrong. Ubuntu obviously haven't read their own press releases. It EOL-ed a week ago:
http://www.ubuntu.com/news/ubuntu-7.10-eol
What is annoying is how EOL breaks everything. Packages won't validate. Worse, I cannot install some new software (such as mysql; but I could install lame). This is not simply "we're not going to do updates any more". This is "We've pulled the plug. Should've upgraded earlier. Loser."
Two forehead slaps for Ubuntu in the space of a week. I'll be back to Fedora at this rate!
Anyway, I started the upgrade to 8.04, but after running for 20 minutes it finally told me it was going to take 19hrs to run. Not good timing for my meeting!
By the way, 19hrs was a bad estimate. It takes 10-20 minutes to get it started, then 3-4 hours downloading, then 1+hrs installing. It took 6 hrs in total for me. You can leave it alone for the downloading, but need to be around to answer questions during the install stage.
On a hunch I wonder if Ubuntu 7 has reached end of life. But, no, this page tells me Gutsy Gibbon (7.10) is still current:
https://help.ubuntu.com/community/UpgradeNotes
It is wrong. Ubuntu obviously haven't read their own press releases. It EOL-ed a week ago:
http://www.ubuntu.com/news/ubuntu-7.10-eol
What is annoying is how EOL breaks everything. Packages won't validate. Worse, I cannot install some new software (such as mysql; but I could install lame). This is not simply "we're not going to do updates any more". This is "We've pulled the plug. Should've upgraded earlier. Loser."
Two forehead slaps for Ubuntu in the space of a week. I'll be back to Fedora at this rate!
Anyway, I started the upgrade to 8.04, but after running for 20 minutes it finally told me it was going to take 19hrs to run. Not good timing for my meeting!
By the way, 19hrs was a bad estimate. It takes 10-20 minutes to get it started, then 3-4 hours downloading, then 1+hrs installing. It took 6 hrs in total for me. You can leave it alone for the downloading, but need to be around to answer questions during the install stage.
Wednesday, April 22, 2009
Screen keeps going blank
I mentioned this in previous entry (on software RAID). My screen keeps going blank. Ctrl-Alt-F1, breathe in, breathe out, Ctrl-Alt-F7 fixes it. I.e. that must reset the video card or X.
It used to happen once a day or less. Since switching to RAID it is happening once an hour roughly. But 9 times in past hour, which was past my threshold, so I went hunting...
...and came up empty-handed. Xorg.0.log has entries but they match what happens if I do the Ctrl-Alt-F1/F7 sequence. /var/log/acpid also has an entry that I think goes along with Ctrl-Alt-F7. But nothing else. No errors in either. Nothing in messages, syslog et al that could be associated.
/var/log/gdm/:0.log also gets an entry when I do ctrl-alt-F7, but nothing beyond that.
Nothing else in /var/log at all.
I'm stumped. I doubt over-heating, and I doubt CPU load (as all I've been doing the past hour is typing in a text file). w tells me "0.00 0.09 0.07".
Not a very informative blog, sorry. Anyone got any suggestions? I don't want to mess around trying different video cards, monitors, etc.; it is still just a minor irritation.
UPDATE: After updating to Ubuntu 8 I still had this problem, but worse my Ctrl-Alt-F1/F7 fix no longer worked! A bit of googling tracked down a solution:
In /etc/X11/xorg.conf, in "InputDevice" section, I commented out these three lines:
and replaced with just:
Even better, Ctrl-Alt-F1..F6 now give me a terminal! They never have in Ubuntu, instead they just gave me a blank screen. The above xorg.conf has been there since the start - Ubuntu 8 upgrade didn't alter it.
(Oh, and the above change has not fixed the screen going blank problem, so I'm still no closer to fixing that...)
Update: I've noticed if I move the mouse, especially the mouse wheel, just after the screen goes blank, it comes back. It doesn't always work - not sure if there are two problems here, or if I'm too slow reaching for the mouse sometimes.
Anyway, if that suggests a potential cause for someone let me know!
(By the way, I have screensaver set to blank, after 30 minutes of inactivity. Just to see if that is related I've changed screensaver to some animation, and I'll let you know!)
It used to happen once a day or less. Since switching to RAID it is happening once an hour roughly. But 9 times in past hour, which was past my threshold, so I went hunting...
...and came up empty-handed. Xorg.0.log has entries but they match what happens if I do the Ctrl-Alt-F1/F7 sequence. /var/log/acpid also has an entry that I think goes along with Ctrl-Alt-F7. But nothing else. No errors in either. Nothing in messages, syslog et al that could be associated.
/var/log/gdm/:0.log also gets an entry when I do ctrl-alt-F7, but nothing beyond that.
Nothing else in /var/log at all.
I'm stumped. I doubt over-heating, and I doubt CPU load (as all I've been doing the past hour is typing in a text file). w tells me "0.00 0.09 0.07".
Not a very informative blog, sorry. Anyone got any suggestions? I don't want to mess around trying different video cards, monitors, etc.; it is still just a minor irritation.
UPDATE: After updating to Ubuntu 8 I still had this problem, but worse my Ctrl-Alt-F1/F7 fix no longer worked! A bit of googling tracked down a solution:
In /etc/X11/xorg.conf, in "InputDevice" section, I commented out these three lines:
#Option "XkbLayout" "jp,jp"
#Option "XkbVariant" "latin,"
#Option "XkbOptions" "grp:alt_shift_toggle,grp_led:scroll"
and replaced with just:
Option "XkbLayout" "jp"
Even better, Ctrl-Alt-F1..F6 now give me a terminal! They never have in Ubuntu, instead they just gave me a blank screen. The above xorg.conf has been there since the start - Ubuntu 8 upgrade didn't alter it.
(Oh, and the above change has not fixed the screen going blank problem, so I'm still no closer to fixing that...)
Update: I've noticed if I move the mouse, especially the mouse wheel, just after the screen goes blank, it comes back. It doesn't always work - not sure if there are two problems here, or if I'm too slow reaching for the mouse sometimes.
Anyway, if that suggests a potential cause for someone let me know!
(By the way, I have screensaver set to blank, after 30 minutes of inactivity. Just to see if that is related I've changed screensaver to some animation, and I'll let you know!)
Monday, April 20, 2009
Software RAID on an existing linux system
So, as mentioned in a previous article I bought a terabyte hard disk, mainly to impress the ladies. It doesn't seem to have been working too well in that respect. I considered sending it back, claiming it needs more testerone. But I decided to instead set up software RAID 1.
My main guide for this task was this article: http://www.howtoforge.com/software-raid1-grub-boot-debian-etch
Rather than repeat everything I will just comment on what I did differently. Note that despite the name of that article, it applied fine to Ubuntu 7, and I suspect it will apply equally well to any linux distro of the past two or three years.
First, my disks were different size: 320G vs. 1T. So I opened fdisk -l /dev/sda in one terminal, then ran fdisk /dev/sdb in a second terminal and set up the first six partitions (plus one extended partition) to be the same as /dev/sda. I then set up /dev/sdb8 to be one big partition with the rest of the disk, which I mounted as /backup.
In all software RAID guides they number the partitions /dev/md0, /dev/md1, etc. That is confusing so I decided to keep the same numbering schemes as my hard disks; it turns out this is fine. So I used commands like this:
mdadm --create /dev/md3 --level=1 --raid-disks=2 missing /dev/sdb3
compared to this from the HowToForge article:
mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb3
The above article did all partitions in one go. I wanted to take baby steps. So I started by mirroring one partition, choosing a non-critical one, and ignoring all that stuff about grub configuration. It worked well. I then did /var and /, leaving /boot unmirrored and therefore again ignoring all that grub configuration stuff. /var went smoothly but / did not. It kept saying /dev/sda3 was in use: "mdadm: Cannot open /dev/sda3: Device or resource busy" But df didn't list it. Trawling /var/log/* didn't list it. "Eh?" thought I.
Eventually I realized that grub was set to boot from /dev/sda3, not from /. I.e. grub works at the device level not the partition level.
Key Advice: mirroring one partition at a time is fine, but then do "/", "/boot" and all that grub stuff in one go.
Now, something the HowToForge article didn't mention at all was doing file copies in single-user mode. I know just enough unix administration to scare myself and the idea of taking a copy of /var and / while all the daemons were still running made me nervous. So I tried to boot to single-user mode.
Discovery #1: Ubuntu is weird. They don't use run levels like the rest of the linux world. Sure, their numbering scheme may make more sense, but it also causes much surprise.
Discovery #2: "telinit 1" doesn't work. The GUI shuts down and then nothing. I can see the system is still running but nothing on screen. Nothing on Ctrl-Alt-F1 through to Ctrl-Alt-F12. I think ctrl-alt-del caused a reboot.
So, I booted, and choose the "recovery kernel" from the grub menu. Recovery being another name for single-user mode. It asks for the root password. Ah. Ubuntu doesn't use one. I have to continue into a normal GUI boot.
Key Advice: Set a root password. This has nothing to do with software RAID: you never know when you are going to want to boot into a recovery kernel. I think Ubuntu deserves a forehead slap for this design decision? To set a root password, do "sudo passwd": it will first ask for your user password (that is how sudo works), then it will say "Enter new UNIX password:". I chose the same password as my normal user; easy to remember. (If that makes you nervous, remember that this is equal security to being able to run "sudo passwd"; more importantly, it saves me having to write down my root password somewhere.)
Other comments: the "cp -dpRx" command is the most time-consuming step for those partitions when you have a lot of data. The system will be sluggish while doing this (and if you are in single-user mode you cannot be browsing or doing anything else anyway). But also when you do the "mdadm --add /dev/md3 /dev/sda3" to actually create a genuine RAID partition it will be copying a lot of data, and your system will be sluggish for the time this takes (about 10 minutes per 30G). Bear this in mind.
Swap. I've created /dev/md2 as a swap RAID, but haven't used it. So /dev/md2 just contains /dev/sdb2. And "cat /proc/swaps" tells me only /dev/sda2. Apparently if my sda disk dies my swap will disappear and my system might crash. On the other hand, mirroring swap has a performance downside (apparently). I don't understand the trade-offs well, but this is a workstation, not a server and a crash should /dev/sda ever die is acceptable. I've also got more memory than I need so I suspect I could switch swap off completely. Anyway, so far Inertia born from Igorance means I'm doing nothing about this. Expert opinions are welcome.
Everything works. Just two things I've noticed. Sometimes the system doesn't boot. It comes up in an "ash" shell, which is one of those recursive acronyms standing for "Ash is Satanic Hell". So far the only command I've mastered is "reboot", which works well - I've not had two boot failures in a row. After 10-20 reboots I think this boot failure is happening about 20-25% of the time. I've no idea how to troubleshoot it.
UPDATE: this boot problem doesn't seem to be happening since upgrading to Ubuntu 8. Perhaps the upgrade fixed some mistake in my grub.conf? Maybe related is that I no longer have an (hd1,0) section in grub.conf; they are all (hd0,0). I think I'll leave it like that: if "hd0" dies I can edit the grub boot string that once, and then edit grub.conf.
The other thing is my screen sometimes goes blank. The system is fine, it is just like the video card has died. My solution has always been: Ctrl-Alt-F1, pause for half a second, then Ctrl-Alt-F7. I guess this process resets the video card. It used to happen maybe once a day or so. Now it seems to have happening once an hour on average, and is becoming almost irritating enough that I'll have to look into it. The fact that running RAID could affect the display feels like an important clue.
Finally, my home partition is encrypted, using crypt. I have not moved it to software RAID yet, for that reason. I will really soon (as all my data that could really benefit from the security of software RAID is currently the only data not protected by it)! I will run crypt on top of software RAID, rather than the other way around; I'm just not sure the best way to do the file copy.
UPDATE: now done, it was straightforward, see Moving encrypted partition to software RAID.
My main guide for this task was this article: http://www.howtoforge.com/software-raid1-grub-boot-debian-etch
Rather than repeat everything I will just comment on what I did differently. Note that despite the name of that article, it applied fine to Ubuntu 7, and I suspect it will apply equally well to any linux distro of the past two or three years.
First, my disks were different size: 320G vs. 1T. So I opened fdisk -l /dev/sda in one terminal, then ran fdisk /dev/sdb in a second terminal and set up the first six partitions (plus one extended partition) to be the same as /dev/sda. I then set up /dev/sdb8 to be one big partition with the rest of the disk, which I mounted as /backup.
In all software RAID guides they number the partitions /dev/md0, /dev/md1, etc. That is confusing so I decided to keep the same numbering schemes as my hard disks; it turns out this is fine. So I used commands like this:
mdadm --create /dev/md3 --level=1 --raid-disks=2 missing /dev/sdb3
compared to this from the HowToForge article:
mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb3
The above article did all partitions in one go. I wanted to take baby steps. So I started by mirroring one partition, choosing a non-critical one, and ignoring all that stuff about grub configuration. It worked well. I then did /var and /, leaving /boot unmirrored and therefore again ignoring all that grub configuration stuff. /var went smoothly but / did not. It kept saying /dev/sda3 was in use: "mdadm: Cannot open /dev/sda3: Device or resource busy" But df didn't list it. Trawling /var/log/* didn't list it. "Eh?" thought I.
Eventually I realized that grub was set to boot from /dev/sda3, not from /. I.e. grub works at the device level not the partition level.
Key Advice: mirroring one partition at a time is fine, but then do "/", "/boot" and all that grub stuff in one go.
Now, something the HowToForge article didn't mention at all was doing file copies in single-user mode. I know just enough unix administration to scare myself and the idea of taking a copy of /var and / while all the daemons were still running made me nervous. So I tried to boot to single-user mode.
Discovery #1: Ubuntu is weird. They don't use run levels like the rest of the linux world. Sure, their numbering scheme may make more sense, but it also causes much surprise.
Discovery #2: "telinit 1" doesn't work. The GUI shuts down and then nothing. I can see the system is still running but nothing on screen. Nothing on Ctrl-Alt-F1 through to Ctrl-Alt-F12. I think ctrl-alt-del caused a reboot.
So, I booted, and choose the "recovery kernel" from the grub menu. Recovery being another name for single-user mode. It asks for the root password. Ah. Ubuntu doesn't use one. I have to continue into a normal GUI boot.
Key Advice: Set a root password. This has nothing to do with software RAID: you never know when you are going to want to boot into a recovery kernel. I think Ubuntu deserves a forehead slap for this design decision? To set a root password, do "sudo passwd": it will first ask for your user password (that is how sudo works), then it will say "Enter new UNIX password:". I chose the same password as my normal user; easy to remember. (If that makes you nervous, remember that this is equal security to being able to run "sudo passwd"; more importantly, it saves me having to write down my root password somewhere.)
Other comments: the "cp -dpRx" command is the most time-consuming step for those partitions when you have a lot of data. The system will be sluggish while doing this (and if you are in single-user mode you cannot be browsing or doing anything else anyway). But also when you do the "mdadm --add /dev/md3 /dev/sda3" to actually create a genuine RAID partition it will be copying a lot of data, and your system will be sluggish for the time this takes (about 10 minutes per 30G). Bear this in mind.
Swap. I've created /dev/md2 as a swap RAID, but haven't used it. So /dev/md2 just contains /dev/sdb2. And "cat /proc/swaps" tells me only /dev/sda2. Apparently if my sda disk dies my swap will disappear and my system might crash. On the other hand, mirroring swap has a performance downside (apparently). I don't understand the trade-offs well, but this is a workstation, not a server and a crash should /dev/sda ever die is acceptable. I've also got more memory than I need so I suspect I could switch swap off completely. Anyway, so far Inertia born from Igorance means I'm doing nothing about this. Expert opinions are welcome.
Everything works. Just two things I've noticed. Sometimes the system doesn't boot. It comes up in an "ash" shell, which is one of those recursive acronyms standing for "Ash is Satanic Hell". So far the only command I've mastered is "reboot", which works well - I've not had two boot failures in a row. After 10-20 reboots I think this boot failure is happening about 20-25% of the time. I've no idea how to troubleshoot it.
UPDATE: this boot problem doesn't seem to be happening since upgrading to Ubuntu 8. Perhaps the upgrade fixed some mistake in my grub.conf? Maybe related is that I no longer have an (hd1,0) section in grub.conf; they are all (hd0,0). I think I'll leave it like that: if "hd0" dies I can edit the grub boot string that once, and then edit grub.conf.
The other thing is my screen sometimes goes blank. The system is fine, it is just like the video card has died. My solution has always been: Ctrl-Alt-F1, pause for half a second, then Ctrl-Alt-F7. I guess this process resets the video card. It used to happen maybe once a day or so. Now it seems to have happening once an hour on average, and is becoming almost irritating enough that I'll have to look into it. The fact that running RAID could affect the display feels like an important clue.
Finally, my home partition is encrypted, using crypt. I have not moved it to software RAID yet, for that reason. I will really soon (as all my data that could really benefit from the security of software RAID is currently the only data not protected by it)! I will run crypt on top of software RAID, rather than the other way around; I'm just not sure the best way to do the file copy.
UPDATE: now done, it was straightforward, see Moving encrypted partition to software RAID.
Tuesday, April 14, 2009
Excel and PDF from PHP
Been meaning to post this for ages, but PHP Architect magazine for November 2008 had a useful article on libraries and code for alternatives to HTML.
For PDF html2pdf was introduced. Unlike other PHP PDF libraries (e.g. PDFLib, FPDF) this one has you make your content in HTML then it gets converted. For those of us comfortable with HTML, that means no learning curve.
For Excel, PHPExcel is introduced. This has classes to read and write excel files in a variety of formats. I get this request a lot but so far we have always decided outputting good old CSV is best (simple to make, compact, easy to import into Excel). But if complex layout, or built-in functions, are required this looks worth a look. It can also output into PDF.
Also very interesting is in-memory spreadsheets, with lots of mathematical functions supported. I've not tried it, and neither the article or the website mention explicitly if this will work on Linux (or a Windows machine without Excel installed), but it is described as standalone so it should. It might be an interesting way to port someones spreadsheet to a PHP app!
The article also briefly introduces Google charts, and how to include the produced image in your excel file.
For PDF html2pdf was introduced. Unlike other PHP PDF libraries (e.g. PDFLib, FPDF) this one has you make your content in HTML then it gets converted. For those of us comfortable with HTML, that means no learning curve.
For Excel, PHPExcel is introduced. This has classes to read and write excel files in a variety of formats. I get this request a lot but so far we have always decided outputting good old CSV is best (simple to make, compact, easy to import into Excel). But if complex layout, or built-in functions, are required this looks worth a look. It can also output into PDF.
Also very interesting is in-memory spreadsheets, with lots of mathematical functions supported. I've not tried it, and neither the article or the website mention explicitly if this will work on Linux (or a Windows machine without Excel installed), but it is described as standalone so it should. It might be an interesting way to port someones spreadsheet to a PHP app!
The article also briefly introduces Google charts, and how to include the produced image in your excel file.
Subscribe to:
Posts (Atom)