Web 0.1 (or, How to Stop Worrying and Keep All Your Data)

Source

So I was just reading Jono's Unity commercial^H^H^H^H^Hpost about his desktop :), and it reminded me of a post I wanted to write. I run a lot of the same stuff on my desktop as Jono does - particularly Evolution (like Jono, I think Evo has got a very unfair bad rap, but I think it rather depends on the exact configuration you use it in), Rhythmbox and gedit. The big differences are that I don't tweet (it just feels like a waste of time to me) and I don't use Empathy, which leads me into what I really wanted to write about... I've talked about various bits and pieces of my setup before, but looking at it overall the other day I was struck by the extent to which my systems are now really just fat clients. They run all their apps locally, but almost none of my data is localized to any computer any more. I can switch from my laptop to my desktop with virtually no loss of my usual workflow. So, my typical work environment includes Evolution, xchat-gnome, a terminal, Firefox, and Revelation for password management. Evolution on all my systems connects to my mail server, which aggregates mail from six or seven accounts and does all the filtering server-side with procmail. It also acts as an SMTP relay, mainly so I can send mail from my cellphone. So whatever system I'm using, I have the same mail in the same folders with the same read/unread status...but Google doesn't own it all. (I was reading the other day that Hotmail didn't even do end-to-end SSL encryption until recently, and still doesn't do it for non-web connections. I mean, yeesh.) Evo is also connected to my contacts and calendar...which I do keep on Google, unfortunately, because I can't get eGroupWare to sync with my cellphone yet. This means I have access to my contacts and calendar from any system and also from my phone, and I can add a contact or event on any system (or phone) and it's there on all of them. xchat-gnome connects to my IRC bounce server, which runs bip as an IRC bouncer and bitlbee as an IM gateway; all my IRC and IM traffic goes through this setup, so I can connect from any machine and I'll get any traffic since the last time I connected from any machine, I can connect with multiple machines at once, and the logs are stored on my big-lumps-o-data disk (actually a 1TB RAID-5 array which lives in my HTPC because it's the only machine with the space for all the disks, and is mounted on all my machines via NFS). On Firefox I have Firefox Sync set up on all the machines. I love Sync. It synchronizes your history and bookmarks (I don't use bookmarks, but hey) and passwords (again I don't use 'em, but it's there if you do) and most other personal data to a master server; again, it pretty much means that whichever system I'm running on (even if I'm running in Windows to play a game or something) my Firefox is pretty much the same, with access to the same browsing history and stored form data and so on. The first tab in Firefox is always open at the tt-rss installation running on my web server machine (which also hosts this blog); tt-rss is just an RSS feed reader implemented as a web app. So again, whatever machine I'm logged in from, I have all my feeds available, with the correct read/unread status - it's such a silly little thing, but I can get up from my desktop, grab my laptop, go sit out on the deck (or, er, in the little boys' room) and carry right on reading the news wherever I left off on the desktop. The only awkward thing with tt-rss is that its mobile interface doesn't render very well on my creaky Windows Mobile phone's Opera browser, so I can't use it too well from the phone. Then there's the awkward little bits: I use revelation for password management, as I mentioned, and for that I simply copy the database between machines every so often. It'd be nice to have a client/server password manager (anyone know of such a thing?), or perhaps someone could write a password manager independent of Firefox which can store its data via Firefox Sync. That'd be nice. For music, there's lots of newfangled technologies like DAAP and UPnP, but I have a stuck-in-the-1980s approach: all my music files are on the RAID-5 drive on the HTPC and it's NFS mounted on the other systems where I play music. It may not be 2010 buzzword-compliant, but it works fine. For documents and pictures, I use the rather neat unison; it's just a directory synchronization tool, but a good one. I have pairs set up so I can sync my Documents and Pictures directories across desktop, HTPC and both laptops, and every few days I just run unison and run those sync operations (I could schedule this but haven't really felt it necessary yet). I could just store them on the RAID and NFS mount from the other systems, but then they'd be unavailable on my laptops when I'm outside the network. I suppose the next step is to implement them as git repos stored on the RAID, or something. That'd be interesting. Yes, I know I'd basically be doing dropbox the manual way. :) So, why Web 0.1? Well, this setup has most of the flexibility of The Cloud (oooh, cloud!) but has the (to me, at least) rather distinct advantage that it doesn't involve handing off control my data and the server end of things to some random giant corporation for its own benefit. The only exceptions here are calendar/contacts (and I'm trying to make eGroupWare work for that) and Firefox Sync, which I trust just fine because they have an excellent privacy policy and implementation (all the data is encrypted, and Mozilla cannot actually decrypt it; there's literally no way for them to access your data, only you can). It makes for a really enjoyable experience, for me: everything is very streamlined and I don't have to worry too much about any of my systems failing (all the important data is backed up between the server machines via cron jobs). If a Rawhide update screws up my desktop I just grab the laptop instead. If I get a new system or do a re-install I pretty much just install a few apps, set up Evo, Firefox and xchat to connect to the right servers, grab my Revelation database, and I've got my setup right there. I just wish setting things up this way wasn't so arcane; I gradually threw all these bits together over several years and I'm sure many of you reading this have similar stuff with your own personal twists set up for your systems, but it is something you need a bit of enthusiasm and technical knowledge to do. It may be inevitable that for most people the only way to get this effect is to rely on some company to do the server end in exchange for access to all your data, but it's a bit of a shame, really. It'd be nice if we'd somehow managed to make these kinds of setup more accessible. I think it might be interesting if Firefox Sync really takes off, and the Weave protocols prove to be robust enough to handle other types of data from non-Mozilla applications; that could be a really interesting space for app developers to play around in, with a really big potential user base. And it's a system that's done right, with privacy built in right from the start and a good commitment to interoperability and the ability to run entirely independent from Mozilla's own implementation.

Comments

reinouts wrote on 2010-11-11 11:37:
Revelation actually supports opening password files from remote servers! I have it set up to auto-open the database on my server at startup.
Links 11/11/2010: Qt 4.7.1 Released, Firefox 4 Much Faster | Techrights wrote on 2010-11-11 23:44:
[...] Web 0.1 (or, How to Stop Worrying and Keep All Your Data) I just wish setting things up this way wasn’t so arcane; I gradually threw all these bits together over several years and I’m sure many of you reading this have similar stuff with your own personal twists set up for your systems, but it is something you need a bit of enthusiasm and technical knowledge to do. It may be inevitable that for most people the only way to get this effect is to rely on some company to do the server end in exchange for access to all your data, but it’s a bit of a shame, really. It’d be nice if we’d somehow managed to make these kinds of setup more accessible. I think it might be interesting if Firefox Sync really takes off, and the Weave protocols prove to be robust enough to handle other types of data from non-Mozilla applications; that could be a really interesting space for app developers to play around in, with a really big potential user base. And it’s a system that’s done right, with privacy built in right from the start and a good commitment to interoperability and the ability to run entirely independent from Mozilla’s own implementation. [...]
philtrick wrote on 2010-11-12 16:52:
Interesting post from a number of perspectives. Firstly on eGroupware, do you mainly use it as a management tool? I was using it a while ago, but found inconsistencies within the application which made working with it quite difficult. If you have IMAP or Maildir email, you should have a look at z-push: http://z-push.sourceforge.net/soswp/ It works with activesync devices, and has a number of backend plugins. I discovered it when looking at Zarafa, which initially looked promising, but I wanted my email stored in a standard format like Maildir. I suppose all you need to do now is throw asterisk into the mix and you'll have everything covered. Phil
adamw wrote on 2010-11-12 17:07:
I don't actually use eGroupWare for anything at present; I intended to use it purely as somewhere to sync contacts and calendar stuff to. Yes, I've heard of Z-Push, someone pointed me to it at a FUDCon once. I did take a look but found problems of some kind, I kind of forget what, now. It may just have been lack of time to get it working. But remember, I don't want to sync email; my phone can check email via IMAP just fine. I want to sync calendar and contacts. So Z-Push would need to be plugged into eGroupWare or something like that, not my mail server. and yeah, I was thinking about asterisk last night. Seems like kind of a big job though. What I'd *really* like is for someone to found something, call it DP Telecoms (10 geek points for anyone who gets the joke acronym): a cell'phone' provider who simply mails you a stack of SIM cards tied to your account, and all the service you get is a specified level of data transfer per month for a specified price. You can use that data however you like (that's what the stack of SIM cards is for), there's no telephony or other services involved; just data. That would help the Asterisk setup case, obviously, as you'd set up your mobile device as an endpoint for it using that data stream. It'd also get us much closer to the whizzy future where all your devices are connected to Teh Intarwebz; we really have the technology for this already, what's holding it back is the fucking stupid design of the cellphone industry, which is predicated on tying us to a single device with a bunch of legacy features we have to pay through the nose for.
philtrick wrote on 2010-11-12 18:05:
Hmm, Interesting you should mention that: http://www.jonmasters.org/blog/2010/11/08/some-more-fun-with-asterisk/ From what I remember looking at z-push recently, it will sync contacts and calendar, there is a caldav backend: http://z-push.sourceforge.net/phpbb/viewtopic.php?f=4&t=1174 And also a carddav backend, as well as vcard, and most interestingly, LDAP: http://z-push.sourceforge.net/phpbb/viewtopic.php?f=4&t=350 Hmm, asterisk seems more interesting, some of the asterisk distros seem to be heading in the direction of all in one comms server, but there still seems to be some separations that are difficult to overcome. Anyway, HTH
adamw wrote on 2010-11-12 18:18:
yeah, I saw Jon's post. He has a lot more experience in this stuff than me. :) I saw that z-push has a caldav backend, yeah, like I said, I can't recall exactly what the problem was.
adamw wrote on 2010-11-12 18:21:
oh, yeah, the AA thing is more or less exactly what I meant (though slightly expensive at UKP25 per GB...still, not terrible). Pity it's only in the UK :(. Also, they charge per SIM - only a couple of quid, but still. But yeah, that's pretty close. I hope someone starts doing that kind of thing in the US / Canada soon. I'd like to see what kind of interesting stuff you can do with it.
philtrick wrote on 2010-11-13 10:14:
I agree, as soon as either a mobile operator starts offering a phone with a data only SIM for a reasonable monthly price, then we will see an evolution in the way phones are used. Smartphones at the moment are basically a mini computer with a phone bolted on, and an evolution to VOIP on handsets will allow seamless switching between activities, such as IM and voice and video, a little like what can be done with empathy at the moment. Paired with something like this: http://nerdvittles.com/index.php?p=703 Remembering that it runs linux and is fairly hackable: http://pbxinaflash.com/forum/showthread.php?t=8273&page=33 This opens up a whole new development ecosystem for the VOIP world, as the current range of handsets are fairly limited in a third party development scope. I watched Jon's video, he has some good ideas in it, and it is fairly entertaining too. I'm going to be testing a z-push install over the next couple of weeks, I'll let you know how I get on.