diff --git a/content/django-server-sent-events.md b/content/django-server-sent-events.md
index 3bb6706..9db9bb1 100644
--- a/content/django-server-sent-events.md
+++ b/content/django-server-sent-events.md
@@ -8,13 +8,11 @@ Summary: A write-up of how I implemented server-sent events using Django 4.2 and
---
-
-
-With the release of Django 4.2 we got the following [0]:
+With the release of Django 4.2 we got the following [^0]:
> [`StreamingHttpResponse`](https://docs.djangoproject.com/en/4.2/ref/request-response/#django.http.StreamingHttpResponse "django.http.StreamingHttpResponse") now supports async iterators when Django is served via ASGI.
-And the documentation has been expanded with the following [1]:
+And the documentation has been expanded with the following [^1]:
> When serving under ASGI, however, a [`StreamingHttpResponse`](https://docs.djangoproject.com/en/4.2/ref/request-response/#django.http.StreamingHttpResponse "django.http.StreamingHttpResponse") need not stop other requests from being served whilst waiting for I/O. This opens up the possibility of long-lived requests for streaming content and implementing patterns such as long-polling, and server-sent events.
@@ -26,10 +24,14 @@ So I set out to write a small, drumroll please, chat application!
The code for the chat application can be found at
[github.com/valberg/django-sse](https://github.com/valberg/django-sse).
+**Table of contents**
+
+[TOC]
+
### What are server-sent events and why do we want to use them?
Server-sent events is "old tech", as in that is has been supported in major
-browser since around 2010-2011 [2]. The idea is that the client "subscribes" to
+browser since around 2010-2011 [^2]. The idea is that the client "subscribes" to
an HTTP endpoint, and the server can then issue data to the client as long as
the connection is open. This is a great performance boost compared to other
techniques as for instance polling the server.
@@ -98,34 +100,6 @@ probably something we could have done with a synchronous view.
Let's see if we can do better. But first we'll have to talk about how to run
this code.
-#### Aside: Use an ASGI server for development
-
-One thing that took me some time to realise is that the Django runserver is not
-capable of running async views returning `StreamingHttpResponse`.
-
-Running the above view with the runserver results in the following error:
-
- :::text
- .../django/http/response.py:514: Warning: StreamingHttpResponse must
- consume asynchronous iterators in order to serve them synchronously.
- Use a synchronous iterator instead.
-
-Fortunately Daphne, the ASGI server which was developed to power Django Channels, has an async runserver which we can use:
-
-To set this up we'll have to install the `daphne` package, add `daphne` to the top of our installed apps, and set
-the `ASGI_APPLICATION` setting to point to our ASGI application.
-
- :::python
- INSTALLED_APPS = [
- "daphne",
- ...
- "chat", # Our app
- ]
-
- ASGI_APPLICATION = "project.asgi.application"
-
-Now we can just run `./manage.py runserver` as before and we are async ready!
-
### More old tech to the rescue: PostgreSQL LISTEN/NOTIFY
This is where we could reach for more infrastructure which could help us giving
@@ -141,13 +115,13 @@ LISTEN to a channel and then anyone can NOTIFY on that same channel.
This seems like something we can use - but psycopg2 isn't async, so I'm not
even sure if `sync_to_async` would help us here.
-### Enter psycopg 3
+#### Enter psycopg 3
I had put the whole thing on ice until I realized that another big thing (maybe
a bit bigger than StreamingHttpResponse) in Django 4.2 is the support for
psycopg 3 - and psycopg 3 is very much async!
-So I went for a stroll in the psycopg 3 documentation and found this gold[3]:
+So I went for a stroll in the psycopg 3 documentation and found this gold[^3]:
::python
import psycopg
@@ -172,8 +146,17 @@ So by combining the snippet from the psycopg 3 documentation and my previous
from django.db import connection
async def stream_foos() -> AsyncGenerator[str, None]:
+
+ # Get the connection params from Django
connection_params = connection.get_connection_params()
+
+ # Somehow Django 4.2.1 sets the cursor_factory to
+ # django.db.backends.postgresql.base.Cursor
+ # which causes problems. Read more about it in the
+ # "Differences between 4.2 and 4.2.1" section in the Appendix.
+ # Removing it from the connection parameters works around this.
connection_params.pop('cursor_factory')
+
aconnection = await psycopg.AsyncConnection.connect(
**connection_params,
autocommit=True,
@@ -186,13 +169,201 @@ So by combining the snippet from the psycopg 3 documentation and my previous
async for notify in gen:
yield f"data: {notify.payload}\n\n"
-I was almost about to give up again, since this approach didn't work initially.
-All because I for some reason had removed the `autocommit=True` in my attempts
-to async-ify the snippet from the psycopg 3 documentation.
+Appart from problems with the `cursor_factory` (which I'll get back to in the
+[appendix](#difference-between-42-and-421)), this code is pretty straight forward and, most importantly, works!
-#### Aside: Difference between 4.2 and 4.2.1
-the code worked initially in 4.2, but 4.2.1 fixed a regression regarding
+### Test the endpoint with curl
+
+So now we've got the `LISTEN` part in place.
+
+If we connect to the endpoint using curl (`-N` disables buffering and is a way to consume streming content with curl):
+
+ :::console
+ $ curl -N http://localhost:8000/messages/
+
+And connect to our database and run:
+
+ :::sql
+ NOTIFY new_message, 'Hello, world!';
+
+
+We, excitingly, get the following result :
+
+ :::text
+ data: Hello, world!
+
+Amazing!
+
+### Issuing the NOTIFY command from Django
+
+But we want the `NOTIFY` command to be issued when a new chat message is submitted.
+
+For this we'll have a small utility function which does the heavy lifting. Note
+that this is just a very simple synchronous function since everything is just
+happening within a single request-response cycle.
+
+ :::python
+ from django.db import connection
+
+
+ def notify(*, channel: str, event: str, payload: str) -> None:
+ payload = json.dumps({
+ "event": event,
+ "content": payload,
+ })
+ with connection.cursor() as cursor:
+ cursor.execute(
+ f"NOTIFY {channel}, '{payload}'",
+ )
+
+And then we can use this in our view (I'm using `@csrf_exempt` here since this is just a quick proof of concept):
+
+ :::python
+ @csrf_exempt
+ @require_POST
+ def post_message_view(request: HttpRequest) -> HttpResponse:
+ message = request.POST.get("message")
+ user = request.POST.get("user")
+ message = ChatMessage.objects.create(user=user, text=message)
+ notify(
+ channel="lobby",
+ event="message_created",
+ content=json.dumps({
+ "text": message.text,
+ "user": message.user,
+ })
+ )
+ return HttpResponse("OK")
+
+The keen observer will notice that we are storing the payload content as a JSON string within a JSON string.
+
+This is because we have two recipients of the payload. The first is the `stream_messages` function which is going to
+send the payload to the client with a `event`, and the second is the browser which is going to parse the payload and use
+the `event` to determine what to do with the payload.
+
+For this we'll have to update our `stream_messages` function as follows:
+
+ :::python
+ async def stream_messages() -> AsyncGenerator[str, None]:
+ connection_params = connection.get_connection_params()
+
+ # Remove the cursor_factory parameter since I can't get
+ # the default from Django 4.2.1 to work.
+ # Django 4.2 didn't have the parameter and that worked.
+ connection_params.pop('cursor_factory')
+
+ aconnection = await psycopg.AsyncConnection.connect(
+ **connection_params,
+ autocommit=True,
+ )
+ channel_name = "lobby"
+ async with aconnection.cursor() as acursor:
+ await acursor.execute(f"LISTEN {channel_name}")
+ gen = aconnection.notifies()
+ async for notify in gen:
+ payload = json.loads(notify.payload)
+ event = payload.pop("event")
+ data = payload.pop("data")
+ yield f"event: {event}\ndata: {data}\n\n"
+
+Everything is the same except that we now parse the payload from the `NOTIFY` command and construct the SSE payload with
+an `event` and a `data` field. This will come in handy when dealing with the frontend.
+
+Another way to do this would be to use Django's
+[signals](https://docs.djangoproject.com/en/4.2/topics/signals/) or event
+writing a PostgreSQL
+[trigger](https://www.postgresql.org/docs/15/plpgsql-trigger.html) which issues
+the `NOTIFY` command.
+
+### Hooking up the frontend
+
+Now that we've got the backend in place, we can get something up and running on
+the frontend.
+
+We could use HTMX's [SSE
+extension](https://htmx.org/extensions/server-sent-events/) but for this
+example we'll just use the
+[EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) API
+directly.
+
+ :::html
+
+