GMail adds holiday (and sports) calendars
Posted by LAV in Calendar sync on September 4th, 2009
GMail recently added a feature that makes it easy to subscribe to official holiday calendars for a variety of countries. Also included are sports schedules, star dates, and (maybe more interestingly) an automatic birthday calender based on your contacts.
I can’t say I really care so much about the Swedish holiday calendar though, it doesn’t even include Midsummer Eve or Christmas Eve, arguably the two most important days in Sweden. Sure they are only de facto holidays, but still. I’ll stick to the nice iCal holiday generator over at oops.se for the time being.
Staying in sync with SpanningSync
Posted by LAV in Calendar sync, Contact sync on May 5th, 2009
Two weeks ago I decided that I really needed a way to enable over-the-air sync of calendar and contact data to my iPhone (after having missed a couple of meetings because I had forgotten to sync manually :P). I basically had three options:
- Subscribe to Apple’s MobileMe service (99 USD / 795 SEK per year).
- Put all my calendar and contact data on Google/GMail and sync with their Exchange interface.
- Start syncing using my company’s Exchange server.
Well, I’m not about to put all my private data on my employer’s Exchange server, so number 3 is out. I was seriously considering MobileMe, but the price tag really puts me off.. They do offer more services than just calendar and contact sync, but I already have working solutions in place for most of those things. Also Apple seems to be doing evil things to customers who cancel their MobileMe accounts.
So, I’m left with option number 2. So how do I get all my data to Google, and how do I keep it synced with iCal and Address Book? Well, Address Book actually has built in Google sync (at least if you have an iPhone), and the Google Calendars can be accessed via Webdav. But there also seemed to be many potential issues… How to trigger the Address Book if you never actually sync your iPhone? Will the webdav-calendars remain editable? Honestly I’d had enough with all “semi working” calender sync solutions, and was prepared to pay to get something that “just works”. Plaxo had been doing a good job of keeping my different Macs in sync, but could not help me with my iPhone, so it was time to find some other solution.
So I found SpanningSync. SpanningSync does exactly what I wanted, i.e. sync OSX calendars and contacts with Google/Gmail. After some initial problems, where repeating events seemed to get infinitely duplicated, most things seem to be running smoothly now. Basically I only have two issues: 1) reminders don’t get synced, but instead get the default value for the given calendar (woke me up at 2am once!), 2) you can’t choose the color of the calendars on the iPhone (issue with iPhone, not Google or SpanningSync). But besides this, my data stays in sync across 3 different Macs and one iPhone (but don’t enable PUSH on the iPhone, it will drain your battery in less than a day!).
Today my 15 day free trial was (almost) over, so I went ahead and coughed up $20 (after the $5 discount) to get a one year subscription. I am, however, quietly hoping that Exchange will be naively supported in the next OSX version, so I won’t have to continue paying for this… 
PS: If anyone reading this is planning to buy SpanningSync, feel free to use my referral code VHKESR and we’ll both earn $5.
Be mindful what you tweet…
So I recently decided to buy, or at least try out, some new sync software (Spanning Sync). It seems they were giving a $5 discount if you could get a referral from someone who has already bought it, so I went ahead and posted on Twitter asking if anyone else was using it. Only one minute later the reply arrives…
Creepy! Oh well. Happy with my choice I once again went ahead and posted:
A little bit later I visited their site again to download the app to another computer, guess what I found on their frontpage?
I suppose I shouldn’t be surprised companies like this monitor what is said about them on Twitter. But I’m still amazed how fast they react, I mean I assume there had to be some kind of review process before my comment ended up on their frontpage?
Anyway, I’ll get back to what I think about Spanning Sync in later posts.
I can haz success! Unison hack to enable Unicode normalization of filenames
After much agony I’ve finally managed to build a hacked version of Unison to make my file sync setup work. The problem, as explained earlier, is that Unison doesn’t support Unicode, and that I have to synchronize files between Mac OSX-machines (using UTF8 NFD-normalized filenames) and Windows machines (using latin1 or UTF8 NFKC-normalized filenames). To make filenames containing non ASCII characters transfer correctly, some kind of conversion has to be made, and as of now Unison does not support this.
In my file sync setup, I have three OSX machines synchronizing files using a Windows server as the central node (all OSX machines sync with the Windows machine). Synchronization is always initiated from one of the OSX-machines. What I have done is to install Cygwin on the Windows machine, and also install a hack for Cygwin which enables UTF8 support.
When I first did this I thought it would be enough, but since Windows/Cygwin and OSX uses different Unicode normalization (NFKC and NFD) the bit-by-bit representation of the filenames are different. This is what I set out to fix. I have inserted a few lines of code in the function the preprocesses filenames before comparison is done in Unison. Those lines uses the Camomile Unicode library to normalize the filename to NFKC, so when the OSX and Windows filenames are compared a little bit later they will be bit-wise identical.
This is DEFINITELY not the best way to do this, and does not by far fix all of Unison’s encoding problems. What one should do is to rewrite all of the filename handling to support Unicode and also other encodings. But I don’t know OCaml very well, in fact I find it quite confusing and frustrating, so for the moment this will have to do for me.
And it seems this is enough to fix my problems. The hack only needs to be applied to the OSX-side of Unison to work, even though it would probably be better if it was applied to both sides (but I’m WAY too lazy to try to compile Unison in Cygwin if it seems I don’t have to :P).
So, if anyone needs to sync an OSX machine with a Windows machine, or perhaps with a Linux machine with a UTF8 filesystem, this could perhaps be of some help to you. (Note that while OSX and Windows/Cygwin enforces NFD and NFKC respectivly, Linux does NOT. So in Linux it would be possible to have to two different files with seemingly identical names, but with different normalization. This would obviously not work well with this hack, but that would probably be a less than ideal situation anyway.)
Quick install:
This is the quick install for people who don’t want to compile stuff.
- Download my precompiled (OSX Leopard) Unison binary here: unison-unicode.zip (600KB, based on Unison 2.27). You only need the modified binary on the OSX side (as long as synchronization is initiated from that side), but all other machines must use the same version of Unison (2.27).
- Download the Camomile data files (5MB). These files must be extracted into /usr/local/share/camomile on your OSX machine (hardcoded, sorry!).
Build yourself:
These are instructions for how to build the modified Unison version yourself (for OSX, but might work on other architectures as well):
- Download and install OCaml.
- Download and install/build Camomile (follow instructions and use the default installation directory).
- Checkout a version of Unison with Subversion (I’m using /branches/2.27, but I think it will work with the latest beta version as well).
- Replace the files src/case.ml and src/src/Makefile.OCaml with these files.
- Compile using “make UISTYLE=text”.
- The new Unison binary will be at src/unison. I would recommend you rename it to unison-unicode or something to tell it apart from your regular Unison version.
Your modified binary (from either the quick or full install) will enable you to synchronize files with Unicode filenames between an OSX machine and another machine with a UTF8 filesystem (for example Linux). If you want to sync with Windows you need to install Cygwin (make sure to select the unison package during installation) and the Cygwin UTF8 hack as well (make sure it’s the cygwin unison binary that is being used during synchronization, use the parameter “-servercmd /usr/bin/unison”).
Note that this version of Unison requires that the two file systems being synchronized are UTF8, if it encounters a filename that is not valid UTF8 it will probably crash!
If anyone actually tries this, please post your comments below! Thanks
Unison Unicode problems
Unison is a pretty awesome file synchronizing utility. It’s free, open source, highly customizable and scriptable. It does, however, have one big flaw: it doesn’t support Unicode. As long as you synchronize between file systems of identical encoding, it doesn’t matter. Unfortunately however, Windows, Linux and MacOSX all use different encodings per default.
My setup synchronizes files between 3 different OSX-machines using a Windows server as the central node. File names containing non-ascii characters like ÅÄÖ gets messed up when transferred, eg. the OSX file räksmörgås.txt will appear as räksmörgaÌŠs.txt on the Windows machine.
This is very annoying. I really like my synchronization setup, and this is the only problem I have with it. What to do? Windows uses latin1 encoding for file names, and OSX uses utf8. What if you could trick windows into using utf8 also? Linux supports utf8 file names, so maybe cygwin can help. Nope, turns out Cygwin does not support Unicode… Googled “cygwin unicode” and found a hack to cygwin which enables Unicode and utf8 support for file names. My hope was rising as räksmörgås.txt seemed to correctly appear on the Windows side. Yes I had done it! Ran unison again to to double check, and the file was now for some reason flagged as new on the windows side, and the whole operation failed when unison tried to copy the file back to the OSX side and failing when discovering that the file was already there.
So, it turns out that there is such a thing as Unicode Normalization. Short story: The same character can be represented in different ways in Unicode, namely composed or decomposed. And, to make matters worse, OSX uses the decomposed form (NFD), and Windows/hacked Cygwin uses the composed form (NFKC). So even though the file is called räksmörgås.txt on both machines, the exact bit representation of the name is different. If I had used a Unicode aware program, this wouldn’t have been a problem and the file names would have been recognized as identical. But as I said, Unison is NOT such a program…
I’ve done some research (ie, googled) there doesn’t seem to be any plans to incorporate Unicode support in Unison. It turns out Unison is written in OCaml, which doesn’t nativly support Unicode, so adding support for this would according to Unisons developers be pretty hard.
But how hard can it really be? I just need to make sure that both filenames are normalized before they are compared. And there are third party libraries to enable Unicode support in OCaml. So I went off and downloaded the Unison source code, the OCaml binaries, and the Unicode library (Camomile). It was pretty easy to locate the piece of code where the normalization should, or at least could, be done. Only one problem remains: Camomile is very poorly documented, and comes with absolutely no example code! Right, two problems: OCaml is a functional languange (like Haskell), and it turns out I hate functional languages!
To be continued (hopefully)…
UPDATE: Problem kind of solved!
