Markdown Editors (for Linux or cross-platform)

I’ve been using markdown more and more, and use pandoc to make a PDF from it. But it often comes out differently to how I expect, so I have been looking for an editor with live preview.

(This article has been updated, end-March 2016, to add Atom; and I've checked for any improvements in StackEdit, RStudio, NetBeans 8.1 and Remarkable. I've also added spell-checking as a requirement.)

Quick summary: all of them are (fatally) flawed.

Here is a quick review of each; well, more a summary of the flaws with each. (When I say “wrong”, in the below, I am treating pandoc, with default settings, as correct.)

stackedit.io: Online. That is a fatal flaw when looking for an offline editor! It also gets the 1-2-3 case wrong (see below). Not open source. I do like how it exports to Blogger, though. (I use it for writing blog posts.) Has same spell-checking as your browser (which is good). Additional Fatal Flaw: your data is stored in the browser, so almost impossible to backup separately; I've just discovered my recent clearing of all cookies has destroyed all my articles.

RStudio: It supports its own .rmd format (which allows embedding live R code inside markdown), but can also be a normal Markdown editor. But no live preview, so you need to keep clicking “Preview HTML” to see what it looks like (though it does have syntax hilighting as you type). It handles underlines wrongly (see below). No spell-checking.

Remarkable: Gets the 1-2-3 case wrong. Open source and looks nice. But it feels black-box-ish. E.g. I don’t know how the code syntax hilighting works, and I don’t know if writing {php} or {r} is being listened to. (It highlighted a short PHP code snippet, with or without a hint, but not R code.) Another fatal flaw: it resets the preview window to the top every time you add a new line, making it useless for a document longer than one screen. It is also very slow - a distinct sluggishness as you type.

Mark My Words. Gets both the 1-2-3 and underlines case correct in the preview window, but the underline case is wrong in the syntax highlighting in the editor window! The icons at the top are a bit confusing.

NetBeans, with the “Markdown Support” plugin. 8.0 had an unusable preview window, but as of NetBeans 8.1 that is better; however the editor and preview windows scroll completely independently, rather than staying in sync. No word-wrap in the editor window. On the plus side it gets the 1-2-3 and underline cases correct. (NB. NetBeans also has most of the problems I point out with Atom, below: in fact they are very similar.)

Haroopad: This makes me nervous, as it does not appear to be open source, and is a 40MB download. It gets the 1-2-3 case wrong. It also doesn’t do syntax highlighting (but that is not an essential feature for me). No spell-checking. No new releases the past 6 months, so this may be a dead/dying project.

Atom (built-in plugin): Currently (March 2016) this is the number one choice at a comparison of Linux Markdown Editors, so I just installed it. I think it is one to watch, because Atom is actively developed and with some more development the markdown support could become the best of the bunch. It handles 1-2-3 and underline cases correctly. There is live preview, but sadly the two panes are unconnected - when you scroll in one, the other just sits there, with no way to sync them. Also they do not agree what is correct markdown: the left window goes all weird with "*.txt", whereas the preview window handles it just fine. It underlines wrong spellings, but right-click does not suggest the correct spellings. It is more a programmer's editor than a writing tool, e.g. ctrl-b with a word highlighted does not make it bold. No print/export options.


My choice? Initially I went with Remarkable, as the best of the bunch, but I hadn't discovered the one-screenful limit at that point. (it beats Haroopad by being open source, and much smaller). Go with stackedit.io if working in a browser is okay for you. March 2016 Update: I've been using Haroopad the past 6 months, but the lack of spell-checking has become an irritation. Atom and NetBeans have very similar pros and cons; of the two I prefer Atom. Not sure if it is quite good enough yet to make me switch from Haroopad, though...


(Your suggestion? Let me know in the comments.)

---

The 1-2-3 problem. When I type:

1
2
3

(i.e. 1, 2 and 3 each on their own line, with no blank lines between them)

I should see “1 2 3”. A blank line is needed to start a new paragraph. It is nice if it shows the line break, but no good if I send that code to pandoc and all my neat formatting is lost! [BUT, maybe there is a nice flag I can give to pandoc to preserve that formatting, as I’d rather it worked that way!]
The underline problem. When I type:

Then you should open my_special_file.txt

It should not treat those underlines as italics or bold formatting. That formatting only applies with preceding whitespace:

This word is in _italics_ this one is in __bold__

That appears like this:

This word is in italics this one is in bold

Gnucash Timezone Problem Workaround

GnuCash is accounting software. It is usable, which is the most important compliment software can get, but it has one annoying bug, first reported in 2002. Let’s assume it will never be fixed, and just work around it.

The bug is that dates are stored in the xml file as timestamps. E.g. if you say a transaction happened on 2014-06-21, then it gets stored as “2014-06-21 00:00:00”. That is the bug. The problem is that it also stores the users current timezone. So, if I type that in when in the Asia/Tokyo timezone it will actually store it as “2014-06-21 00:00:00 +0900”. If I then open the gnucash file on a server in the Europe/London timezone, and re-save, the file now stores this: “2014-06-20 16:00:00 +0100”. It has moved it to BST timezone, but in the UI all you now see is that the transaction is on the wrong day! (That was a real problem, but if you think that is exotic the original bug report was done by someone who merely moved from one state in the U.S. to another state!)

Anyway, my proposed workaround is to always run gnucash in the UTC timezone. On Linux you do this from commandline by starting it like this:

TZ=UTC gnucash

In windows environments, you might have associated your *.gnucash files, so double-clicking them opens them in gnucash. The following instructions work for xfce (thunar file manager), but I suspect gnome is exactly the same.

1. Right click any gnucash file, and choose properties
2. Choose open with, and at the bottom it says “Other application…”
3. Give bash -c "TZ=UTC gnucash %f"
4. Close it, and test it by double-clicking any gnucash file.

(I confirmed it had worked by looking at the raw XML.)

What to do if you have someone else working on your files who does not want to do this? Or is on Windows (where I don’t know a similar workaround)? Well, the only solution I know is that they temporarily move their whole machine to UTC, open, edit and save your gnucash files, then restore their machine back to their real timezone. (Remember that UTC is different from Europe/London.)

Annoying keyboard shortcuts in Xfce

Quickie for mint16/17 using Xfce. You may have found the ctrl-Fn keys have been mapped to change the workspace you are on? This behaviour is highly annoying as applications often define ctrl-Fn to do something.

Do not try to fix it from the graphic configuration window. It does not work.

Instead, as root, open this file in a text editor:

/etc/xdg/xdg-default/xfce4/xfconf/xfce-perchannel-xml/xfce4-keyboard-shortcuts.xml

(Because or links there are about three or four files pointing to this file; it doesn’t matter which one you edit.)

Look for lines like:

<property name="&lt;Control&gt;F5" type="string" value="..."/>

Then delete all those lines, there are 12 in total, from F1 to F12. (In my case they were not in a block, so you had to hunt around to catch all twelve.)

Then you need to logout of your desktop, and log back in again for the change to take effect.

Written with StackEdit.

Connecting Android 4.x devices to Linux (including Ubuntu 10.04)

The Problem

Android 2.x is easy: plug 'n' play with linux.
But in Android 4.x that component was deprecated/removed, and you have to use "MTP" to connect. And most (all?) Linux distros have no support for "MTP".

Aside
Since writing this, what I actually do is let the device be detected as a camera. Then I open the "Pictures" folder, and just copy everything in there. Movies, PDFs, etc. it does not seem to matter. So I've given up being able to organize stuff intelligently for the convenience of that having to mess around with the below steps.

The Solution

1. Download go-mtpfs.x86_64 from http://hanwen.home.xs4all.nl/public/software/go-mtpfs/
2. Make it executable ( chmod 777 go-mtpfs.x86_64 )
3. Make a mount point directory. E.g. mkdir nexus7  (may have to be root depending on permissions of the parent directory)
4. Plugin Android 4.x device. Make sure it is using MTP.
5. Mount:  sudo ./go-mtpfs.x86_64 nexus7
6. View:  sudo nautilus nexus7    (you could also use the commandline for your file actions: sudo bash )
7. To stop I did three things: close nautilus, then press ctrl-c to stop the go-mtpfs program, then sudo umount nexus7

Warnings 

That is using a pre-compiled binary, that you are going to run as root. Be afraid, be very afraid.
However it is an open source project and you can compile it yourself:  https://github.com/hanwen/go-mtpfs/
You need to install the "go" language, and probably some other stuff.

I used the pre-compiled binary, the 28 Jun 2013 version; here is its md5sum:
  $ md5sum go-mtpfs.x86_64
  bdc71b5e92dbabfa6d33c03a8a7f0e09  go-mtpfs.x86_64 

(This is meaningless without some trusted source telling us it is correct, as it may already have been hacked; but if you get a different value and it still says 28 Jun 2013, I'd be suspicious...)

Notes

This worked on Ubuntu 10.04, doing nothing extra than described above.
It should also work on Ubuntu 12.x


It is slow. 10MB file in 3.7 seconds seems typical. A 435MB and a 480MB file in 4m55s and 5m43s respectively (copied at the same time). A huge file is averaging 2.2MB/sec.

To make sure you are using MTP, go to settings, then Storage, then press the icon with three vertical dots, and choose "USB computer connection". The choices are MTP and PTP. PTP works out of the box with Linux, but only shows you the photographs directory.  That is perfect for plugging in to a printer at your local 7-11 or Fujifilm shop (or local equivalent if you are not in Japan), as it does not expose any other files but your photos. But not useful for anything else.

I suspect you always have to run go-mtpfs as root, but there is probably a way to have the mount point be visible to normal users, not just root. For me it was easier to just run nautilus as root, than try to work out how. Comments welcome!

If you want an alternative, or just more background information, this page looks good: http://bernaerts.dyndns.org/linux/74-ubuntu/247-ubuntu-automount-nexus7-mtp

Changing Rackspace Cloud Instance Sizes

Following on from my post about the actual costs of using Rackspace, this article is about the "Actual downtime of changing the size of a Rackspace cloud server".

I wanted to go from a 10Gb disk to 20Gb disk. I got confused by the interface and accidentally went from 10Gb to 40Gb. So I then also got to try downsizing from 40Gb to 20Gb!

Q1. Does my IP address change? Do I need a new SSH key?
A. No and No.

Q2. How long does it take?
A. The 40Gb to 20Gb downsize took 13 minutes in total. The 10Gb to 40Gb upsize was about the same or slightly quicker. If you like to be careful and take a disk image beforehand, that takes about another 10-15 minutes (for about 9Gb, so allow more if your disk is bigger).

Q3. How much downtime?
A. At least 30 seconds... Prior to the move I saw activity in one program at 06:28:27; the new server was fully running at 06:29:05-ish, and the first new activity in that program was at 06:30:03.

The reason for the almost 60 second delay in that particular program was it gets started from a monitoring script (that I wrote) that runs on a 1-minute cronjob. The cron job didn't run at 06:29:00, so had to wait until 06:30:00. If I had started my program from the init.d scripts, or even from rc.local, the downtime would only have been 30-40 seconds. That is the same downtime you can expect for a webserver.

Q4. Does it depend on the size of the server?
A. I believe so. Bigger servers would have more files to copy.

Q5. Do in-memory caches survive? Do background processes keep running?
A. No and no. The resize implies a reboot. Any background process you'd started manually before needs restarting manually again when the new server comes up. Use /etc/rc.local or cronjobs to auto-start things.

Q6. Do I need to be around when I resize?
A. Yes to get it started. You could then go away (you are supposed to verify it when it comes up, but I think if you don't verify it then after a few hours it assumes you are happy that everything is running smoothly.)  (The point of the verify step is they keep the old image around and can quickly revert - I did not test this to see how quick the revert would be.)

Q7. How do I change server size without any downtime at all?
A. Big question. The glib answer is: If you have to ask, then it is too difficult. The slightly more helpful answer is: if a web server then run two servers, with a load balancer in front of them; if a database server, look into database clusters.

Umlauts, pound signs and more!

Until I learnt this tip, whenever I needed some fancy character (£,°,á,ß, etc.) I went hunting for somewhere to copy and paste it from. Not any more!

I assigned my previously useless windows key to be my Compose key. To type the above pound sign I pressed the windows key, then pressed the L key, then pressed the minus key. Scharfes-S (ß) is simply windows key then press the S key twice. To type á (a with an accent) you press ' (shift-7) and a after the window key.

(Note: until I wrote this post I thought you had to hold the Compose key down while pressing the other keys. That works too, but it is easier to treat it as a modal switch. In other words, you press the Compose key to enter Compose Mode, then the next two characters you type are interpreted together. And if the two characters you type have no special meaning then nothing happens. Either way you exit Compose Mode after your two keypresses.)

See here for the full list of all the compose options

Naturally the above is for linux. This page answers how to do it on Windows and here is the same concept on a Mac



Bonus Tip. Ever wanted to write a Japanese post office sign ( 〒 ) ? Enter your Japanese IME and type yuubin (ゆうびん) (then press space). The ～ symbol can be done by typing kara (から).


˙unɟ ɹoɟ ʇsnɾ ˙uʍop ǝpısdn ʇxǝʇ ɹnoʎ suɹnʇ ʇɐɥʇ ǝʇıs ɐ puıɟ :dıʇ snuoq ɹǝɥʇouɐ
(Yes, this is part of Unicode too.)

boost on centos (vs. on ubuntu)

You'd think porting a C++ program from one 64-bit linux to another would be trivial. But, no. A program developed with no issues on Ubuntu 10.04 was a lot of trouble to get to compile on Centos 5.6.
But get it to compile I did... read on for the secret words you have to utter...

First thing I did was:
   yum erase boost boost-devel

This got rid of the very old 1.33 library. I then ran this:

  yum install boost141 boost141-devel boost141-program-options boost141-regex boost141-thread boost141-system

I then had to hack the makefile to add this to my CFLAGS:

  -I/usr/include/boost141/

That got me compiling. But not linking. I changed my LDFLAGs to look like this but still it would not link:

  -L/usr/lib64/boost141 -lboost_regex -lboost_program_options -lboost_system -lboost_thread

I played around with various -L settings, until I had a breakthrough. I noticed the above line complained with:

  /usr/bin/ld: cannot find -lboost_thread

But some different -L settings instead gave me:

  /usr/bin/ld: cannot find -lboost_regex

Blink and you miss it, and indeed I had. My above line was linking with most of Boost, just not boost_thread. In other words, I was really close and hadn't realized it. A bit more poking around discovered that Ubuntu calls it "boost_thread", while Centos calls it "boost_thread-mt" (careful, the first one is an underline, the second one is a hyphen). So, the final magic line for Centos was:

  -L/usr/lib64/boost141 -lboost_regex -lboost_program_options -lboost_system -lboost_thread-mt

NOTE: I did not do any post-installation steps after the yum install steps. I saw some people suggesting copying files from one place to another but I did not need to. (Also, -L/usr/lib64 should be sufficient, as every thing is symlinked; maybe even the -L flag is not needed at all... but I'm in the "If it ain't broke." mindset now, so no more experimentation for me!)

Actual costs: rackspace cloud

A few months back I decided to put a 24/7 script on a Rackspace Cloud instance, instead of the more obvious Amazon EC2 choice. The reason at the time was my needs were low CPU but relatively high bandwidth and diskspace usage and it worked out cheaper.

Now I've had a few invoices in I am relieved to say there was no catch. My past three invoices have been $11.99, $11.99 and $12.20 (USD). This is for a minimal CPU spec (256MB, 1.6% of a quad core CPU, 10GB disk), 1.1 to 1.3 GB/month of outgoing bandwidth each month (there is no charge for incoming bandwidth), and cloud storage rising from 4 to 8GB. 90% of the monthly cost is for the machine, and the cloud storage has risen from $0.63 to $1.15. The bandwidth is not costing much at all.

In contrast on Amazon EC2, the micro instance would cost $15.65 (including $1 for 10GB of EBS storage), while a small instance would cost $62.25/month, of which $0.03 is the bandwidth usage. (The first year of that micro instance would be free if you are a new customer, but I am not.)

So, at the CPU bottom-end, Rackspace is winning on cost. The other feature of Rackspace Cloud that I love is there is an automatic daily backup of the full disk image, and that backup is stored in the cloud storage. (Storing that backup is basically all my $1/month cloud storage costs.)

What do I not like? I keep using up my 10GB disk space. But there seems no way to move to 20GB without doubling the CPU spec and doubling the monthly cost; with Amazon micro I'd just increase the EBS storage space. With an Amazon small instance I'd get 160GB and would not care.

What do I not like about Rackspace and Amazon? It is that you just get a basic linux distro. You have to spend time installing, configuring and maintaining. And the configuration is not trivial; I've kept a log of all I've had to do, and it includes things like moving ssh off of port 22, setting up an iptables firewall, installing a mail server (not a POP server, just enough so I can *send* email alerts), and writing my own low-diskspace email alert script. The latter was done just the other day after my application broke, yet again, because the machine had run out of disk space.

P.S. As I want 24/7, and have not mentioned the need to scale, what about cheaper shared hosting? Well, I couldn't find a VPS, that gives me root access and no restrictions, for under $10/month. It seems Rackspace is winning that fight too?

(2012-08-15 UPDATE: Rackspace are stopping their minimal server config: no more 256MB option in their "nextgen" V2 API. Also scheduled images are not available in the V2 API (yet). In other words the two things I pointed out that were good, in the above article, are going! I guess the Rackspace marketing department will have to work out a positive spin on "cutting out our competitive advantage so we look just like the competition now" ;-)

Renaming files is easy!

What do you mean? Of course renaming is easy! From the GUI just click it and type the new name. From the linux commandline just type mv oldname newname.

But what about when you have 200 *.JPEG files and you need to rename them to be *.jpg? In the past I've written throwaway PHP scripts to do just this. Well, it turns out linux has a command called rename, that can use regexes. To rename those jpegs:

  rename 's/JPEG/jpg/' *.JPEG

What about a few thousand log files, and I want to remove the datestamp portion from the filename (because I'm going to keep each day's logs in their own subdirectory)? How about this:

  mkdir 20110203
  mv *.20110203.log 20110203/
  rename 's/.20110203//' 20110203/*

(See here for how to then use a bash loop to loop through all the datestamps you have to deal with.)

Add a temporary static IP address

At home, with wired ethernet, my (Ubuntu) notebook has a few static IP addresses that I use for developing websites. Out of the house, I use wicd, so I have a dynamic IP address, and those static IPs don't exist. wicd configuration is too complex for me to understand, so I just accept this, but it caught me short the other day when I needed to have both an internet connection and to be able to work on a website running on my notebook.

I failed then, but I'm ready for next time. To temporarily add a static IP address you simply do (as root):

ifconfig eth0:3 10.10.10.10 netmask 255.255.0.0

I'm choosing "eth0:3" for the interface; it can be any unused number after the colon, and you never need to care what this is. netmask can really be anything for our purposes. The 10.10.10.10 is the IP address I've given it. Test with this:

ping 10.10.10.10

To set up a quick virtual host create a file under /etc/apache2/conf.d called 10.10.10.10.conf (any filename is fine) with these contents:

<virtualhost 10.10.10.10:80="">
    DocumentRoot "/var/www/somewhere"
    ServerName 10.10.10.10
</virtualhost> 

Tidyup

To remove just the interface that you added above, use this command:

ip addr del 10.10.10.10/32 dev eth0:3

Or, to restore the network to boot defaults (useful if you have done lots of changes) you can do:

ifdown -a
ifup -a

Either way to then remove the apache config: delete the 10.10.10.10.conf file you created and restart apache.

svn, ssh, /bin/false, git, /etc, etc.

I just spent almost two hours troubleshooting an svn server. It was set up to allow ssh access from a few users over ssh as described in this helpful blog post.

Today I realized none of the svn clients could run svn update: "svn: Network connection closed unexpectedly"

Part of the challenge was that I'd not run "svn update" in 1-2 months, so I had a big time frame for breakage. But all I could remember changing recently was commenting out some lines in my iptables firewall, so I spent quite a while staring at that. And I'd updated some packages and rebooted.

I spent a lot of time looking at ssh verbose output. To do this use

export SVN_SSH="ssh -vv"

(-v for quite verbose, -vv for more, -vvv for even more, etc.) No error messages: it connects, runs the svnserve command, passes up some environmental variables, sends some data, and then simply loses the connection.

Then I wondered if the svn user had somehow lost permission for something important. It has /bin/false as its shell, so I gave it /bin/bash, used su to become root, then "su - svn". Ran the svnserve command. It is fine. Looked at the repository. It is fine. Ran svnadmin to check for locks. None.

Giving up, I started crafting an email to post somewhere asking for help. I went back to get the exact error message svn update was giving me... and it worked. Yep, the other client works now too. My first guess was running svnadmin had quietly fixed something. But then, on a hunch, I changed svn's shell back to /bin/false. Yep, that broke it. It seems the svn user has to have a valid login shell.

Okay, problem solved. But I'm sure that svn has always had /bin/false, and now I wanted to know if that is true, and if and when it changed. It is too late now, but ready for next time, I decided to put all of /etc into a git repository (no need for a central repository, so git is far better choice than svn for this). The git commands (all run as root) are trivial:

cd /etc
git init
git add .
git commit -m "Initial files" -a

I found this page of someone who has done something similar, and he suggested sending the git status report from a daily cron, so I did that too.

It is so easy, and disk space so plentiful, that I think I will do it on all my linux machines.

UPDATE: /etc has some files that get edited a lot, so to reduce noise this is my .gitignore file so far:

/cups/printers.conf*
/cups/subscriptions.conf*
/emacs/site-start*
/mtab
/adjtime
/ld.so.cache
/*-
*~

(Use "git rm --cached cups/printers.conf*" (for instance) to stop git tracking the files if they've already been added.)

Customize less: less annoying

Open ~/.bashrc (or ~/.bash_profile) and add this line at the end:

LESS=-FRX;export LESS

Don't ask why, just trust me. ('cos actually I don't know what it means either...)

This was the magic incantation to tell "git log" to wrap commit messages (instead of just chopping off everything extra).

But it has had the delightful side-effect of man now works properly. About 6 or 7 years ago I used to be able to type "man whatever", press q, and the advice would stay on screen. Then linux (all distros as far as I could tell) changed to hiding the man page as soon as you press q. Frequently I'd have to have two terminal windows open, simply so I could keep the man page open while I type my command. Yes, you're ahead of me; the above LESS command fixes this. If I'd known it would be so easy I'd have done this years ago!!

Even better, the help system in the R language was working just like man pages (and it was even more annoying there), and that has been fixed too.

It is a miracle, I tell you, a miracle! July 13th should go down as "Saint LESS=-FRX" day; in 100 years time it will be a public holiday and children will be taught about it in schools. They might even be taught what it means...

svn to git: central repository setup

The best thing about git is you can just start working; no messing around setting up the central repository, working out secure access, etc. But the learning curve for git, coming from svn, is much bigger than say from cvs to svn. The thing I'd not got my head around was what do you do when you need that central repository? For instance, when you have a 2nd developer! I think I've got it now, so here is my guide on it.

Let's assume it is a website ("mysite"), and we have three machines involved:
devel: where most development is done
store: the central repository server
www: the web server

The website is on devel currently, under git, and we want to check it out to www.
If not a website, but just two developers, then in this example "www" is the machine of that second developer.

For simplicity we'll assume all developers have ssh access to the "store" machine. And that all our git repositories will be kept under /var/git/

1. On store:
cd /var/git/
mkdir mysite.git
cd mysite.git
git init --bare

(see update#1 below, if your git does not support --bare)

2. On devel (from the root of the website)
git remote add origin darren@store:/var/git/mysite.git
git push origin master
(the "origin master" part is just needed the first time)

3. On www,
cd /var/www/
git clone darren@store:/var/git/mysite.git my.example.com
(this creates /var/www/my.example.com and checks out the current version of the site)
(you need to configure your web server to not serve the .git directory, as well as any other files end users should not see)

4. If we make a change on devel and want to update www.
First on devel, do the commits, then:
git push
Then on www:
git pull

5. If we make a change on www, and want to update devel.
First on www, do the commits, then:
git push
Then on devel do:
git pull origin master

Not quite symmetrical is it? To fix this, so in future you just need to do "git pull" from the devel machine too, edit .git/config. You'll see these lines:

[remote "origin"]
url = darren@store:/var/git/mysite.git
fetch = +refs/heads/*:refs/remotes/origin/*

Don't touch them but, just above them, add this block:

[branch "master"]
remote = origin
merge = refs/heads/master

(unless you are doing something clever, you can use that verbatim). Now "git pull" works without having to specify "origin master" any more.

So, "git commit ...;git push" is like "svn commit ...". And "git pull" is like "svn checkout".


UPDATE #1: Older versions of git don't have the --bare function. Do these steps (on "store" machine) instead:

cd /var/git/
mkdir mysite.git
cd mysite.git
git init
mv .git/* .
rmdir .git

UPDATE #2: What to do if you already have some files on www, with some differences from what is in git? E.g. I had a test and live version of a website, with no source control; the test version had a number of extra files. I scp-ed the live version to devel, created a git repository. Then I set it up and push-ed it to "store". Then on my test site, I had to do these steps:

#Temporarily move current test files out of the way
mv my.example.com{,.old}
#Get the git version
git clone darren@store:/var/git/mysite.git my.example.com
#Merge in the test site
cd my.example.com
cp -a ../my.example.com.old/* .
#See what is different
git status
#For all changed files, revert to the git (live server) version
git reset --hard

By the end of all this, git status will just list (as untracked) the files that were on test but were not on live.

oauth for php (and Ubuntu)

There is a "standard" PHP OAuth library, documented in the manual, and which is installed via pecl. There is no package shortcut under Ubuntu; you still have to use pecl
sudo pecl install oauth

If you get a complaint about no "phpize", install the "php5-dev" ubuntu package. And then if you get an error in "php_pcre.h" when compiling, then you need to install the "libpcre3-dev" ubuntu package.

Finally, you need to enable it. Still as root:
cd /etc/php5/conf.d
echo "extension=oauth.so" >oauth.ini

(This creates a config file just for oauth; you could also simply put the extension line in php.ini.)

Finally, "php -m" should list "OAuth", and you can create a OAuth object in your php scripts.

Ubuntu package manager lists "liboauth-php"; the minimal information, and the lack of mention of pecl, should have given me the clue that it is something different.

Also different is this: http://code.google.com/p/oauth-php/

One man's feature request...

...is another man's bug.
Since Thunderbird 3, if you selected part of a mail then hit reply, only your selected part gets quoted. Incredibly annoying for those of us who think out loud with our mouse.

But it can be switched off. Edit|Preferences, Advanced tab, Config Editor. Then in the search box type:
mailnews.reply_quoting_selection
Change it from true to false (double-click that line is enough).

It turns out this was actually a Thunderbird feature request, dating from 2000! https://bugzilla.mozilla.org/show_bug.cgi?id=23394

scp with multiple targets: ssh-add

I sometimes have the need to upload the same file to two or more locations on the target server. In fact I even have a script to help me, which goes something like this:

scp file1 file2 remote01:/path/to/somewhere/
 scp file1 file2 remote01:/path/to/another/

The problem is I need to type the password for remote01 twice. I had never managed to find some clever scp syntax to allow specifying two destinations, and a post on the TLUG mailing list confirmed that. But what I did learn from TLUG is that there is something called ssh-agent that can store passphrases for key pairs; this plugged a gap in my knowledge. There are three ways to login in via ssh/scp:

1. Give your password
2. Make a keypair with no passphrase
3. Make a keypair with a passphrase

The second way is used to allow scp from cron jobs, to automate copying files between two machines. I'd never really got the point of the third way: if you have to type a passphrase why not just use the first method, and save messing around making a key pair. (Well, yes, there is better security: a log-in then requires both something I have and something I know; you have to turn off PasswordAuthentication on your ssh server for this to have any meaning though. Thanks to Kalin for this comment.)

But it turns out there is this program called ssh-agent that remembers passphrases for you. And I found it is already running in the background on Ubuntu.

Enough chat, let's look at the solution. First, I created a one-off keypair for this script, on my machine, using something like this:

cd ~/.ssh
  ssh-keygen -t dsa -C me@example.com -f key_for_remote01
  chmod 600 key_for_remote01*
  scp -p key_for_remote01.pub remote01:~

When generating the keypair give a reasonably secure passphrase (you will have to type it in each time, and it is only of use to people in possession of key_for_remote1, so no need for a 20-random-character monster; I believe it is perfectly fine for it to be the same as your normal ssh login password for remote01).

Then log in to remote01 and append key_for_remote01.pub to ~/.ssh/authorized_keys. If that file does not exist then you can just rename key_for_remote01.pub to authorized_keys and move it into ~/.ssh/

(By the way, there is no need to put your private half of the keypair in ~/.ssh/ but that seems as good as place as anywhere else.)

Now, I modified my script as follows:

ssh-add -t 120 ~/.ssh/key_for_remote01
 scp -i ~/.ssh/key_for_remote01 file1 file2 remote01:/path/to/somewhere/
 scp -i ~/.ssh/key_for_remote01 file1 file2 remote01:/path/to/another/
 ssh-add -d ~/.ssh/key_for_remote01

What happens is the ssh-add line will ask you for your passphrase. The two scp lines then work automatically. Finally the ssh-add -d stops it caching your passphrase (forcing you to type your passphrase each time you run this script).

The -t 120 parameter says the passphrase will expire after two minutes. This is just in case the batch file doesn't complete and so does not get chance to run ssh-add -d.


Note: you can use that same key pair for other machines. Basically anywhere you put the *.pub half of the key pair will let you login. And you can login from anywhere you have the private half of the key pair.

Note: the timeout/deletion code above is deliberate for this application, but you don't have to do it this way. By allowing it to cache it permanently you would only be prompted for your passphrase once, and then all future ssh and scp logins would be automatic. They will be cleared when you log-out of gnome (on ubuntu, at least) or shutdown your machine.

Note: If you don't want to specify -i each time you use ssh/scp then you can add an entry to your ~/.ssh/config file, like this:

Host remote01
        Hostname 10.1.2.3
        Port 22
        IdentityFile ~/.ssh/key_for_remote01

Note: I used -C with my real email address. This is put in the public key, and I wanted the administrator of remote01 to know who put the key there. Without -C it defaulted to "myusername@myhost". The administrator of remote01 knows nothing about my machine names so this seemed unreasonable and I decided to use -C. But the advantage of that default is it seems ssh-agent knows about that name and will prompt automatically the first time you try to use ssh/scp, which means there is no need to run ssh-add first. I have not worked out yet if ssh-agent can be told to know about my email address too.

Troubleshooting php and xinetd with strace

In a previous entry I mentioned I was trying to track down a difference in a php script between two machines. I've now got closer, close enough in fact to fix the problem but not to understand it. As a troubleshooter, the big discovery for me was how useful strace can be.

First, a PHP code snippet:

while(1){
stream_set_blocking(STDIN,true);
$s=trim(fgets(STDIN));
if($GLOBALS['log_fp'])fwrite($GLOBALS['log_fp'],$s."\n");
$params=explode(" ",$s);
$command=array_shift($params);
...
}

The first two lines say listen on stdin for a command, and wait forever for it to arrive. When it does we log it and then split it up into a command and parameters.

This script is used over xinetd. Xinetd listens on a port for us, and when it gets something it feeds it to stdin; php script output is then fed back over the socket to the client.

Here is the setup:

• machine A: ubuntu 8.04, 32-bit, php 5.2
• machine B: ubuntu 10.04, 64-bit, php 5.3

Both machines run an indentical version of xinetd: "xinetd Version 2.3.14 libwrap loadavg".

Machine A runs fine, machine B sometimes sits there. Analyzing this some more, by connecting over telnet to machine B:

1. I send a command
2. It replies straightaway
3. After 60 seconds or so it gives an error message
4. If I do nothing this error message appears again every 60 seconds.

On machine A, or if I run the php script directly on machine B, it behaves like this:

1. I send a command
2. It replies straightaway
3. It sits there forever waiting for input

Out of desperation I ran strace (using -p you can attach it to a running process; use ps -a | grep php to find the PID), and I was pleasantly surprised to see it was not too verbose.

Over xinetd on machine B (the problem configuration) the interesting snippet is:

...
write(1, "mydata\n", 7)                 = 7
fcntl(0, F_GETFL)                       = 0x2 (flags O_RDWR)
fcntl(0, F_SETFL, O_RDWR)               = 0
poll([{fd=0, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 0 (Timeout)
write(3, "\n", 1)                       = 1
write(5, "\n", 1) 
...

On machine A over xinetd it instead looks like this:

...
write(1, "mydata\n", 7)                 = 7
fcntl64(0, F_GETFL)                     = 0x8002 (flags O_RDWR|O_LARGEFILE)
fcntl64(0, F_SETFL, O_RDWR|O_LARGEFILE) = 0
read(0,

(Yes nothing comes after the "0," until I send some input.)

And direct access to the php script on machine B it practically the same:

...
write(1, "mydata\n", 7)                 = 7
fcntl(0, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
fcntl(0, F_SETFL, O_RDWR|O_LARGEFILE)   = 0
read(0,

So, the cause is clear: fgets() blocks for only 60 seconds instead of blocking forever, but only on this machine and only over xinetd. And the bug in my script then becomes clear: I'm not prepared to handle blank input!

Here is my fix:

while(1){
stream_set_blocking(STDIN,true);
$s=trim(fgets(STDIN));
if($s=='')continue;
if($GLOBALS['log_fp'])fwrite($GLOBALS['log_fp'],$s."\n");
$params=explode(" ",$s);
$command=array_shift($params);
...
}

As an ironic postscript, the real problem was elsewhere (a mismatch in another program version and the configuration being used for it!), and I also used strace to track that mistake down. But, what was so important about fixing the above was that it stopped distracting me with an error message every 60 seconds on machine B, when the configuration error was actually on machine A!

Bash loops

Faced with having to write 160 commands by hand (to split 5000 files into subdirectories, based on the first 3 digits of their 6 digit filenames) I quickly learnt bash loops instead:

for i in `seq 102 189`;
  do
    mkdir $i
    mv $i???.sgf $i/
  done  

I.e. it does commands like this:

mkdir 102
  mv 102???.sgf 102/
  mkdir 103
  mv 103???.sgf 103/
  ...

UPDATE: In a comment, traxplayer kindly explained how I would make, for instance a filename like 120abc.txt; obviously writing $iabc.txt won't work. The trick is to write $i as ${i}. E.g.

mv ${i}abc.sgf $i/

I just needed to use it, and it worked! This bash script tries to find the last reference to each of XXX06..XXX20 in two logfiles.

rm lastorder.log
touch lastorder.log

for i in `seq 6 9`;
    do
        grep -h XXX0${i} order_verbose.log.old order_verbose.log | tail -1 >> lastorder.log
    done

for i in `seq 10 20`;
    do
        grep -h XXX${i} order_verbose.log.old order_verbose.log | tail -1 >> lastorder.log
    done

By the way, traxplayer also said he prefers to use $(...) instead of `...` for readability:

  for i in $(seq 102 189);

When bash history no longer works

I've been pulling my hair out: I have all the HISTORY variables set correctly, but history never survives to the next session. I thought it must be an Ubuntu 10.04 bug, as only that machine, but couldn't track down anyone reporting the same problem.

The breakthrough came when I stopped typing "history" to view the history, and decided to "cat ~/.bash_history". I got permission denied... and discovered the file was owned by root. Suddenly it all made sense.

(By the way, I've a bunch of blog articles about setting up a new Ubuntu dual boot, RAIDed, encrypted machine; coming soon to an RSS feed near you...)

Wine broke, or PHP, or xinetd??

Sometimes it can be really educational to sit on the shoulder of brilliant developers and watch how they troubleshoot and debug. So, here is your chance to sit on my shoulder,... and snigger as failure reduces me to tears.
[2010-11-18 UPDATE: a mere 5 months after originally posting, I think I have the answer; see the bottom of this article.]

A program worked fine Saturday. This is on my Ubuntu 8.04 (hardy heron) machine.

Sunday I updated firefox and ghostscript only. I then rebooted, which means kernel 2.6.24-28 finally became active (I updated it July 11th, but hadn't had chance to reboot until yesterday).

Today that program doesn't run. It is one particular wine program: a go program. Another wine program (also a go program) runs fine. And here is the real killer: if I start that problem wine program from gogui (gogui.sf.net) it works. Exactly the same commandline, but it works from gogui, and doesn't work when started from my php script. That PHP script was last changed last Wednesday.

I rebooted into the previous kernel (2.6.24-27), and the problem is exactly the same.

Have you ever done those logic problems where you get a list of clues and have to work out who did what? Applied to the above we discover the only logical explanation is that... reality is warped. It can't be: wine, the go program, php, my php script or the kernel.

Here is some more evidence. The first time I run it after a reboot the program (called valhallgtp.exe) fails to start with a stack dump and backtrace. Here are the first few and last few lines of that:
=>1 0x7bc3b23c __regs_RtlRaiseException+0x4c() in ntdll (0x014ded68)
2 0x7bc76de3 in ntdll (+0x66de3) (0x014df0cc)
3 0x7bc3a936 RtlRaiseException+0x6() in ntdll (0x014df144)
4 0x00415833 in valhallgtp (+0x15833) (0x014df2ac)
5 0x004159dd in valhallgtp (+0x159dd) (0x014dfa24)
... (6..20 are all in valhallgtp)
21 0x0049b02c in valhallgtp (+0x9b02c) (0x014dff08)
22 0x7b8773a7 in kernel32 (+0x573a7) (0x014dffe8)

I also see: "err:seh:raise_exception Exception frame is not in stack limits => unable to dispatch exception."

The 2nd and subsequent times I try to start it I get:
Failed to start wine ValhallGTP.exe ...
(this is an error from my php script; i.e. proc_open() is failing.)

The CWD (current working directory) is correct, the parameters are correct; both are exactly what gogui is using. And running it from a bash shell is fine too.

The fingers are pointing at PHP. But no PHP upgrades either yesterday or in the July 11th batch. And that same PHP is still successfully starting 4 other programs, including another one that uses wine. (BTW, go to synaptic package manager, file menu, history; this is where you can see exactly what Ubuntu updated and when.)

Disk space is fine ("df"), memory is fine ("cat /proc/meminfo"). Machine load is low ("w"). Running it as root has same problem.

When I ran from the -28 kernel I also got errors about no X DISPLAY. And then when I started the other wine program it opened but had no display. And running wine config was the same. I've not seen that again. Let's pretend it never happened and concentrate on what we can reproduce.
(UPDATE: it happened again (this time with the -27 kernel). I had the gtpmfgo.exe running fine under wine; then starting mfgo.exe it came up but with no display. Close and retry showed it was repeatable. After closing gtpmfgo.exe it worked, and then I could open gtpmfgo.exe fine. Note: not repeatable; I haven't managed to get it to happen again.)

Okay, where to next? What I don't understand is how come I cannot reproduce the problem I get when I start it the first time. That seems like a small enough challenge to be achievable: how to get it to crash consistently.

...time passes...

Aha! It seems it is crashing in the same way; it was just crashing and dying before I got chance to read the crash message. So the first time after a reboot it must take longer than a second to start up (perhaps wine initialization).

...aha, aha! Narrowed down further. To explain this I need to give more background: I have a script called frontend.php that calls backend.php over sockets. backend.php uses stdin/stdout and uses xinetd to turn it into a socket server. When I run backend.php directly it works!

Could the updates/reboot have affected xinetd somehow? ...probably not. The xinetd files are all dated Dec 2007 except my own config file which is June 30th. And, the other 4 programs are started in exactly the same way and work fine.

Starting backend.php in exactly the same way as xinetd does works fine. (I've tried it both from bash and from sh.)

...time passes...

I seem to have a solution. (I had started working on a xinetd replacement php script, but that doesn't seem needed now.) My solution was to set xinetd to listen on another port, and use that port for just the problem program. This is working. I cannot explain it however: I had already tried connecting to only that program, so xinetd wouldn't have been doing anything different to now.

UPDATE: 7 weeks later and I do another reboot and it broke in exactly the same way!!
And again the only solution I could find was to give xinetd a new port for just this particular problem.
Weird, very, very weird, ...

2010-11-18 UPDATE: It was bothering me again, but on a different machine, and the trick with setting up xinetd on a different port didn't work. But adding this code to the top of backend.php seems to have solved it:
if(trim(getenv('DISPLAY'))=='')putenv('DISPLAY=:0.0');

The problem program has a GUI you see. I switch the GUI off with a commandline option ("hidewindows") but I think it actually creates windows and just hides them. And yes, none of the other programs uses a GUI... sigh, the clues were there.

Curiously the "hidewindows" option no longer works! That is probably going to annoy me, but not as annoying as not working at all!