Experiments In Owning Data: Part 2

January 23, 2019
Posted in , ,

Checkout part 1

I wanted the setup to have as much less friction as possible, more time maintaining systems and uploading data is less time for things that I actually enjoy. Luckily a lot of open source tooling exists to help in this space, some mature, some not so much.

I stumbled upon these two apps during this experiment and they have become my daily drivers of sort -

Nextcloud

Nextcloud is your one-stop cloud storage app. The core of nextcloud is a file storage server with a web interface, easy enough right? True to the open source nature it is heavily extendable with apps (more like plugins) - and there is an app for everything! hosting calendars, editing office docs, taking markdown notes and even video calling.

Nextcloud comes with companion mobile apps for Android and iOS. The apps let you access and upload data to the nextcloud server. They can also automatically backup new photos to nextcloud! Loosing a device never means loosing photos.

The server also exposes a webdav interface, so one can just mount it as a folder with davfs over fuse. Webdav is not known for being very performant, but I like the simplicity of it and works well for my day today needs where I just need to work on a couple of small files.

Nextcloud has replaced Google Drive/Photos for me. The whole process is not without pains. Documentation exists and is helpful, but sometimes falls short. I am yet to figure out how to setup Collabora (online office suite).

Tiddlywiki

Tiddlywiki claims itself to be a non-linear personal notebook. That’s quite vague, because it depends entierly on the user. I use it to store tiny bits of notes, lists, code etc. I have managed to move everything from evernote to tiddlywiki with the exporter plugin. Formatting of exported notes is not great, but the content is fine - and that’s all I care.

Current and Future

I have been running this for almost an year now, other than nextcloud and tiddlywiki, the server also runs an email server (not my primary one yet), code repo and a CI setup.

Where to now? I am obviously missing out on features, and I had to spent more time setting it up than signing up for a service. But the setup is tailored to my use and so far I am happy with it.

That is not to say I think its perfect - I have a wishlist!

I miss the fast and accurate photos search on google photos, and the current nextcloud photos interface is slow and only has file system hierarchy as a grouping mechanism. Maybe I can inception-v3 or any other similiar image classification models and build an auto tagging system?

2. Disaster proof backups

Current backup system is quite rudimentary, it copys data every day to another server in the same datacenter (also on raid1). Its not great if the DC itself goes under - I would like something offsite, and also to backup server configs.

Well, that’s about it for now. :-) I do have a question to the reader: do you know any applications in this space that I should know about? please let me know over twitter or email.