8.5 KiB
| title | date |
|---|---|
| MirageOS retreat & Banawá chat | 2023-05-07 |
In beginning of May 2023 I had the joy to participate in the 12th MirageOS hack retreat in Marrakech, Morocco. There I met faces I know from previous retreats as well as many new faces. Some I had interacted with online. In either case it was good to see them.
At the hack retreat we discuss ideas, work on projects related to MirageOS and OCaml in general while also enjoying very nice food and warm weather. As a result knowledge is exchanged, new ideas emerge and seeds for new exciting projects are made. One project was to implement the server part of git by Jules and Paul-Elliot. As part of this project (I believe) they wrote a small unikernel that acts as a ssh server. Since I have some experience with awa-ssh from the first MirageOS hack retreat I attended I tried to help them out. The library is in an experimental state, and while the client part sees some use in the Mirage ecosystem the server part has largely been neglected for some time.
While discussing awa-ssh with them we talked about how authentication works in the library. On every connection you spawn a server that has a list of users and their credentials (password, if applicable, and all their publick keys). This interface is not ideal since on every connection you need to figure out all valid users and their credentials before the client even has suggested who they pretend to be. This is something I (still) would like to change. While on the subject I was reminded of ssh servers I had read about and tested where any public key is accepted and stored for future logins if the user doesn't already exist - a Trust On First Use (TOFU) scheme. I then got motivated to do some quick modifications to awa-ssh to allow this scheme.
Try it yourself
Before I go too deep into the story I will first share the prototype online. It is online at chat.reyn.ir. It can be accessed using an ssh client:
ssh -i path/to/key_ed25519 myusername@chat.reyn.ir
Note that only ed25519 and RSA keys work at the moment.
The connection will silently hang if e.g. an ecdsa key is used (a fix is in the works).
Once connected you will be greeted by your username, and the last many messages are displayed.
At any time you can disconnect by pressing enter and then ~. - this is the default escape sequence in OpenSSH to hard disconnect a session.
You can then reconnect using the same key, but trying to login as your username using a different key will not work.
That is, until the server is restarted.
There are many bugs, and the experience is not quite polished.
Below is the public key of the server and its fingerprint:
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBEQRUNxL+cGcOOIGF0SXFqv0jbhXZJYcGhl6BctMtxN
SHA256:6nWSkqXCM/mBS/CoDzPkQLnHDLDYv/NqNXCOKfATKEE
Back to the story
As mentioned the authentication flow is very opinionated and requires you to list all users before the ssh protocol begins. This does not lend very well to the TOFU scheme I wanted. As a result I replaced the list of users with a mutable hash table mapping usernames to public keys. Then the authentication flow checks first if the user exists (a public key is associated with the username in the hashtable); if so, we verify the signature against that key. Otherwise, we verify the signature against the public key provided by the client (just for good measure) and then add to the hashtable the username and the public key. Now that key is trusted for that particular user for subsequent authentications - the Trust On First Use scheme.
It is possible to reserve usernames by associating them with some (bogus) public key, but otherwise you don't have any control over what usernames can be registered. As this is not a nice property, and the stateful, mutable user database is quite a divergence from the original code I decided to rename my fork of awa-ssh. Awa-ssh is named after the critically endangered indigenous people of Brazil, Awá. Wikipedia has a category of indigenous peoples in Brazil, and there I learned of the Banawá. The fork was then renamed banawa-ssh.
This is all well and cool, but it just authenticating for no purpose is not so exciting. I then started working on a chat service. At the retreat we often have unreliable internet connection, and every now and then we use up all the prepaid data on our shared LTE connection. For this reason we run an opam repository mirroring the official one (from the git repository) using our MirageOS implementation opam-mirror, and the bob file transfer tool. A service I was missing was a messaging service for the local network. My idea was this could be used to announce things such as:
hey, I am going to the fruit market tomorrow morning - who wants to join?
or,
is the internet down for everyone or is it just me? Should I go out and buy more internet credit?
Another point is we don't want to impose too much administration by requiring manual registration prior. On the other hand it is preferrable to have some sort of nick names to distinguish senders, but we also don't people impersonating each other. For this the TOFU scheme is a pretty good balance for a service on a local network.
The chat service is named Banawá chat, and it's source is available online on github.com/reynir/banawa-chat.
First iteration of the chat service
The first iteration multiplexed messages from clients with their usernames to all other connected peers.
This worked okay as a proof of concept, but it had many pain points.
First, the server did not understand terminals.
This meant that each character entered was sent as separate messages(!)
Therefore the preferred way to use it was either by disabling pseudo-terminal allocation in the client (ssh -T in OpenSSH),
or by running a command (ssh hostname lets-chat).
Second, there was no back log upon connection, and users would need to stay connected at all times to ensure they don't miss out on any messages.
All in all a pretty miserable experience even if a working proof of concept.
Second iteration
While talking with @dinosaure about these issues he reminded me of his work-in-progress IRC client catty (or Ca-tty). For that project he implemented a terminal interface layer for awa-ssh and MirageOS. I then started copying much of his terminal user interface code with his help. Quickly I realized that keeping separate message buffers for each connection or using and broadcasting new messages was way more complicated than the more obvious implementation of keeping a shared message backlog and rendering them in each client on updates using Lwd for tracking updates. This both made the code simpler, and gave us the much desired message backlog. Now the service was becoming useful.
During the development several bugs were discovered and many of those fixed and upstreamed to awa-ssh. See for example:
- https://github.com/mirage/awa-ssh/pull/55
- https://github.com/mirage/awa-ssh/pull/56
- https://github.com/mirage/awa-ssh/issues/57
- https://github.com/mirage/awa-ssh/pull/58
The experience of working on banawa-chat has motivated me to work more on awa-ssh. Ssh is a nice protocol with built-in authentication and encryption. A ssh server could be embedded in many unikernels to provide access to inspect, control or modify another service securely.
I would like to give credit to Andrey Petrov who wrote a nice blog post on things you can do with the ssh protocol.
He wrote as well a ssh chat server called ssh-chat that is available on ssh ssh.chat.
While it's been many years since I read his blog post and I had to dig around to find it it's been in the back of my head at the retreat.