January 11, 2012

What is ClearSky going to look like as an application?

ClearSky is going to have to be as simple as possible to use. Facebook-level simplicity. The only unavoidable extra requirement is that the node software is going to have to run as a binary on your own computer. The installation process needs to consist of just downloading a single binary. Once you have it, no more extra installation steps.

The first time the binary is run, it opens a web browser pointing to a page that asks you to choose a nickname and password (the binary will have a web server to present the local interface). In the background, it's going to generate a private/public key pair and set up the filesystem directory where you store data for that account. After that, it will let you look up friends online. I'm undecided about how this step will work. There are a lot of alternatives for federated, centrally located directories: just build a web one, XMPP (integration with chat seems like a good idea), IRC, etc. I don't know a lot about how the distributed options work, but they are out there and seem to work ok (Freenet, GNUnet, etc.). It seems like a good idea to make this part modular, but present it behind a consistent and easy to use interface.

The binary will provide an HTTPS port to the outside world to let your friends log in remotely (see previous discussion). If you're behind a NAT gateway, it's going to take some extra work - both the NAT-PMP and IGD protocols will need to be supported.

In general, UDP hole punching needs to be present to let two ClearSky nodes talk to each other, although it alone is not enough to provide remote logins from the outside. UDP hole punching needs an accessible third party to be present - using either a friend's computer that has an unrestricted connection, or a central server if no such friends are available. Using unknown third parties is problematic because it relies on either volunteers (every ClearSky client can be a volunteer, but this opens up possibilities for surveillance) or your mirror provider.

All traffic between you and your friends goes over SSH. Messages and files are cryptographically signed to ensure authenticity, but encryption during transport is provided by SSH. The ClearSky client should encrypt data before writing it to disk to ensure privacy even if the hosting computer gets lost or stolen.

Apps are going to be started in a separate process (via fork(), or by running the same executable with different options) for extra security, stability, and to let the OS handle resource control. Games, content libraries, chat, voice, calendars etc. are all possible apps. One example app that could be built but that's not possible with Facebook is a Tahoe-LAFS that would let you trade harddrive space with your friends to securely and redundantly back up your private data.

Apps will communicate with the ClearSky process via sockets and the file system. Security is going to be provided by language-level virtualization. The app runtime will prompt the user when the app wants to access filesystem folders or other resources. Things like video and audio will ideally be handled by HTML5, if not then by Flash. It's undesireable to have the app display native UI or use the OS multimedia or other capabilities directly - this shouldn't be included in the platform unless there's a good concrete demonstration that it's needed. OTOH, having an app be able to manage its own sockets may be a good idea.

Common Lisp is a good language/runtime to base the ClearSky platform on. A Common Lisp binary can run all the basic ClearSky functions (web server, crypto, discovery, networking, SSH) in the same process, provides excellent support for language-level virtualization (and of course already comes with a compiler), and has very good support for hosting and virtualizing other programming languages - of the popular scripting ones, CLPython and CL-JavaScript are currently available, and new ones are easy to implement: CL-JavaScript was written by 3 people (Alan Pavičić, Marijn Haverbeke, and Iva Jurišić), CL-Python by just one person (Willem Broekema).


Manuel Simoni said...

I agree that CL would be a good PL for that, but actually, I see no need for a downloadable software: all the features you talk about will (shortly) be supported by web browsers.


Vladimir Sedach said...

Thanks for the link to that! I wasn't aware of the project. If there was a way to increase the web storage limit (say to 10 GiB) that would solve most of the distributed problems. One interesting thing they're going to use in the reference implementation is libjingle (http://code.google.com/p/libjingle/) which has all the NAT hole punching techniques rolled together. I think a Common Lisp port is in order.

Vsevolod Dyomkin said...

Somewhat related idea in terms of authentication technology: http://joyeur.com/2012/01/03/a-bit-more-about-the-new-joyent-cloud-api/

Vsevolod Dyomkin said...

One of the questions is how are you going to implement login from 3rd-party computer (like, in internet cafe)?

Vladimir Sedach said...


There's going to have to be a web page you can visit that has your friends' node status and can give you a link to them so you can log in.