DR McCullochHOMESHOPABOUT

Archived Β· 18 February 2024

prometheusrun

Two people typing into the same sentence at the same moment end up with the same sentence, and nobody arbitrates. I dropped a Liveblocks example into an empty repo on 18 February 2024, made nine commits across that one Sunday and stopped, and the machinery inside it that made that true was Yjs. That machinery is running on this page, so the two panes below are two real replicas of one document, each with its own Yjs client id, and the wire between them is yours to hold shut.

To the wire

One Β· the wire

Hold it shut, type into both, then open it in whatever order you like.

Each pane owns a separate Y.Doc with a Y.Textbound to its textarea by diffing the common prefix and common suffix on every keystroke. Nothing passes between them except the update bytes on the wire, and while the wire is held those bytes pile up where you can see them. The strip underneath compares the two strings every frame and says diverged when they differ, because while you are holding the wire they really do differ.

replica Adoc.clientID …

yours

replica Bdoc.clientID …

also yours, until somebody else turns up

A → B
B → A

Every chip is one real doc.on('update') payload, and clicking one decodes it.

bootingloading yjsa.getText('doc').toString() === b.getText('doc').toString()
operation inspectorY.decodeUpdate(bytes)

Hold the wire, type into a pane, then click one of the chips on the rail to decode the bytes it is carrying.

Both panes are yours while you are here on your own, and joining the room hands pane B over to whoever else is reading this page, across a Supabase Realtime broadcast channel, with the same hold and release controls working on real network traffic.

The collide button seeds both replicas with “how do you actually feel?”, the real title from the archive, holds the wire, puts both carets at index 11 where “actually” begins, then types “really ” into A and “honestly ” into B. Release them in either order or shuffle them, and the two panes land on the same sentence every time. Then press reseed the client ids to tear both docs down, build fresh ones with new random ids and replay the identical keystrokes. The keystrokes are the same and only the two ids have changed, so the panes go on agreeing with each other while the word that lands first may well swap over.

The docs, the updates, the merge and the tiebreak here are the real Yjs ones, running the yjs package in your browser, but the transport is not the original. The archive synced through Liveblocks’ hosted provider (@liveblocks/yjs 1.9.8), and here the same bytes travel over a queue inside one tab, or over a Supabase channel if you join the room.

Two Β· why it lands the same

A random 32-bit number settles it, and nobody picked that number.

When two inserts share an origin, meaning both were typed into the same gap with the same character on their left, Yjs has to put one of them first and make the same choice on every replica without asking anyone. It does that by comparing the two client ids and sending the smaller one left, which is the whole of the rule; line 467 alongside is where that comparison happens.

OT attempts to transform index positions to ensure convergence (all clients end up with the same content), while CRDTs use mathematical models that usually do not involve index transformations, like linked lists.
yjs README.md, shipped inside the archive at node_modules/yjs/README.md

The same README notes that operational transformation without a central server requires “too much bookkeeping to be viable in practice”, which is the reason a CRDT is in the box at all. There is no server behind the two panes above, and the one in the original was there only to carry bytes, never to decide anything.

Where the number comes from

export const generateNewClientId = random.uint32

yjs src/utils/Doc.js line 22

export const uint32 = () => getRandomValues(new Uint32Array(1))[0]

lib0 0.2.78 random.js line 16

export const compareIDs = (a, b) => a === b || (a !== null && b !== null && a.client === b.client && a.clock === b.clock)

yjs src/utils/ID.js line 34. An ID is nothing but { client, clock }.

Flip the coin a few hundred times

Each pair here is two fresh Y.Docs with two fresh random client ids, seeded with the same sentence, given the identical keystrokes at index 11 and then delivered in a random order. The counts are worked out in your browser the moment you press the button rather than stored in the page, so no two visitors see the same numbers.

Nothing run yet.

node_modules/yjs/src/structs/Item.jsyjs 13.6.7 Β· lines 414–487 Β· Item.integrate()
414  /**415   * @param {Transaction} transaction416   * @param {number} offset417   */418  integrate (transaction, offset) {419    if (offset > 0) {420      this.id.clock += offset421      this.left = getItemCleanEnd(transaction, transaction.doc.store, createID(this.id.client, this.id.clock - 1))422      this.origin = this.left.lastId423      this.content = this.content.splice(offset)424      this.length -= offset425    }426 427    if (this.parent) {428      if ((!this.left && (!this.right || this.right.left !== null)) || (this.left && this.left.right !== this.right)) {429        /**430         * @type {Item|null}431         */432        let left = this.left433 434        /**435         * @type {Item|null}436         */437        let o438        // set o to the first conflicting item439        if (left !== null) {440          o = left.right441        } else if (this.parentSub !== null) {442          o = /** @type {AbstractType<any>} */ (this.parent)._map.get(this.parentSub) || null443          while (o !== null && o.left !== null) {444            o = o.left445          }446        } else {447          o = /** @type {AbstractType<any>} */ (this.parent)._start448        }449        // TODO: use something like DeleteSet here (a tree implementation would be best)450        // @todo use global set definitions451        /**452         * @type {Set<Item>}453         */454        const conflictingItems = new Set()455        /**456         * @type {Set<Item>}457         */458        const itemsBeforeOrigin = new Set()459        // Let c in conflictingItems, b in itemsBeforeOrigin460        // ***{origin}bbbb{this}{c,b}{c,b}{o}***461        // Note that conflictingItems is a subset of itemsBeforeOrigin462        while (o !== null && o !== this.right) {463          itemsBeforeOrigin.add(o)464          conflictingItems.add(o)465          if (compareIDs(this.origin, o.origin)) {466            // case 1467            if (o.id.client < this.id.client) {468              left = o469              conflictingItems.clear()470            } else if (compareIDs(this.rightOrigin, o.rightOrigin)) {471              // this and o are conflicting and point to the same integration points. The id decides which item comes first.472              // Since this is to the left of o, we can break here473              break474            } // else, o might be integrated before an item that this conflicts with. If so, we will find it in the next iterations475          } else if (o.origin !== null && itemsBeforeOrigin.has(getItem(transaction.doc.store, o.origin))) { // use getItem instead of getItemCleanEnd because we don't want / need to split items.476            // case 2477            if (!conflictingItems.has(getItem(transaction.doc.store, o.origin))) {478              left = o479              conflictingItems.clear()480            }481          } else {482            break483          }484          o = o.right485        }486        this.left = left487      }

Copied byte for byte out of the archive’s own node_modules, highlighted by line number rather than by rewriting it. Yjs is MIT, Copyright (c) 2014 Kevin Jahns, Chair of Computer Science 5 (Databases & Information Systems), RWTH Aachen University, Germany.

The archive pinned yjs 13.6.7 and the two panes above run yjs 13.6.31, the copy this site already had installed for its own collaborative editor, so I compared the two files: lines 414 to487 of Item.js are character identical across both trees, sitting one line lower in the newer copy, so the tiebreak you are watching is the same text the archive shipped.

Every character remembers who was on its left

0h1o2w3␣4d5o6␣7y8o9u10␣11a12c13t14u15a16l17l18y19␣20f21e22e23l24?
right = new Item(createID(ownClientId, getState(doc.store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, content)

yjs src/types/YText.js line 269. Each inserted run keeps a reference to the item on its left, which is what makes an index-free merge possible.

Runs typed in sequence collapse inside the document afterwards, and the yjs README gives the case exactly: text.insert(0, 'a'), text.insert(1, 'b'); is “first represented as two structs”,[{id: {client, clock: 0}, content: 'a'}, {id: {client, clock: 1}, content: 'b'}], “and then merged into a single struct”,[{id: {client, clock: 0}, content: 'ab'}]. The wire does not collapse the same way, because every keystroke is its own transaction and so its own chip: typing “really” and the space after it one character at a time puts 7 separate updates on the rail, where collide inserts the same 7 characters in one transaction and puts up one chip six bytes longer than any single keystroke would have been. That is why the room mode coalesces with Y.mergeUpdates on a hundred-millisecond tick before anything goes out.

Three Β· the whole life of the repo

Nine commits, all on one Sunday, 10 hours and 1 minute end to end.

The first commit is the example landing in one go, 82 files and 6,957 insertions. Everything after it is mine, and it comes to 11 files, 186 insertions and 35 deletions. Hovering or tapping any of the nine below finds it on the axis. The subjects are verbatim, typos included, because I wasn’t writing them for anyone.

That axis runs in real time rather than in commit order, so the crowding is real: 7 of the nine land inside the first hour, and then there are 8.8 hours of nothing between max widht:we at 07:37 and colors & styling at 16:28. Whatever I did that afternoon, git has no record of it.

6,957

lines in the first commit, 82 files, the downloaded example

186

lines git can attribute to me after that, across 11 files

3,632

lines under src/ today, 76 files, 364K

429M

the folder on disk, 14,492 files, of which node_modules is 318M and a dead .next build is 110M

The line between the example and me isn’t clean, and git can’t draw it. That first commit already contains TextInlineAdvanced.tsx,marks/Prompt.tsx and api/chat/route.ts with the whole OpenAI layer and its system message inside them, so the prompt layer further down this page was already standing before git started watching and whatever led up to it is gone. What git can show me on top of it is 44 lines in TextInlineAdvanced.tsx and 26 in marks/Prompt.tsx, which is where the wording and the colour list came from. The eleven files below are only what git can attribute to me after the import, which is not the same thing as everything I did.

Four Β· what git can attribute to me

Eleven files carry my changes, and they come to 186 insertions against 35 deletions.

  • src/app/api/liveblocks-auth/route.ts+8 −8
  • src/app/layout.tsx+1 −1
  • src/components/TextEditor.module.css+11 −1
  • src/components/TextInlineAdvanced.tsx+44 −17
  • src/components/Toolbar.module.css+65 −2
  • src/components/Toolbar.tsx+2 −2
  • src/components/marks/Prompt.tsx+26 −2
  • src/icons/Robot.tsx+19 −0
  • src/icons/index.ts+1 −0
  • src/styles/globals.css+2 −2
  • src/styles/text-editor.css+7 −0
  • git -C ~/prometheusrun diff --numstat 2a166b7 HEAD

src/app/layout.tsx

export const metadata = {
  title: "Liveblocks",
};
export const metadata = {
  title: "How do you actually feel?",
};

The two favicon <link> tags immediately below it still point at liveblocks.io.

Five Β· four minutes after the import

I renamed the demo users and left their colours alone.

The second commit lands at 06:48, 4 minutes after the example, and it is one file with eight lines changed. The example ships eight fictional collaborators for the avatar stack. I gave them the names of eight theorists and touched nothing else, so every colour and every avatar URL below is still the example’s.

  • Charlie LayneJudith Butler#D583F0
  • Mislav AbhaDonna Haraway#F08385
  • Tatum PaoloSara Ahmed#F0D885
  • Anjali WandaGloria AnzaldΓΊa#85EED6
  • Jody HeklaLuciana Parisi#85BBF0
  • Emil JoyceAudre Lorde#8594F0
  • Jory QuispeJudith Halberstam#85DBF0
  • Quinn EltonKimberly Crenshaw#87EE85

“Kimberly Crenshaw” is spelled that way in the file. The theorist is KimberlΓ© Crenshaw, and I got it wrong at 06:48 on a Sunday morning and never went back, in src/app/api/liveblocks-auth/route.ts.

Six Β· the layer that was already there

Commune with the machine.

In the archived editor you select some text and press the button with the gradient on it, then a popover opens asking what you want done. The model behind it is model: 'gpt-4-0125-preview', streamed, on export const runtime = 'edge', through the openai package at 4.28.0 and ai at 2.2.35. Text that comes back is wrapped in a Tiptap mark called prompt and painted with a colour off a fixed list. All of it is in src/app/api/chat/route.ts and src/components/TextInlineAdvanced.tsx.

The system message sent with every request, verbatim including its typo
You are not a bot and you do not chat to do user. You just respond with the text updates or the content they have requested based on their request format. This is not a discussion. You respond with the new text/copy in british English with British spellings. The text will be sent as TipTap JSON, responses should ensure they fit within the TipTap formatting and the response is formatted as TipTap JSON if it has been sent

src/components/TextInlineAdvanced.tsx

How the mark gets on the page

export const inputRegex = /(?:^|\s)((?:==)((?:[^~=]+))(?:==))$/ in src/components/marks/Prompt.tsx, so typing ==like this== marks a span. There is a keyboard shortcut too, Mod-Shift-h.

The button that opens the model is .shinyButton, an eight-second gradient animation with a two-second pulse over the top of it. Its hover rule sets color: pink !important; border-color: red !Important; with the capital I still in.

The popover label reads “Commune with the machine” over an italic “(ask it to modify, expand, improve, lengthen etc the selected text)”, and the submit button reads “Activate your communion”.

That is the submit button, .toolbarShinyButton, rebuilt out of Toolbar.module.css with its own gradient, its own GradientAnimation and PulseAnimation keyframes and the black pill its span carries inline. The button described just above, the one that opens the popover, is a separate rule in the same file running those two keyframes slower: eight seconds for the gradient against four here and two seconds for the pulse against half a second.

The colour a model’s sentence gets, drawn at random from the 15 strings in getRandomHexValue(), with the words laid on top set Black or White out of the same numbers

  • Black#dfddff225.5
  • Black#f4f3ff244.7
  • Black#e3e2ff229.6
  • Black#d8dbff222.2
  • White#11236b37.8
  • White#163c8356.7
  • White#1f398e58.9
  • White#17267742.7
  • White#1e4c8a69.3
  • White#171d6735.6
  • White#217fb5105.0
  • White#1e82b6106.0
  • White#11225034.2
  • White#11225034.2
  • White#1a448062.3

The label on each swatch is the word the code returns for it, printed in the colour it names. 4 come back Black and 11 come back White, decided by const brightness = (r * 299 + g * 587 + b * 114) / 1000; then return brightness > 128 ? 'Black' : 'White';. There are 15 entries and only 14 distinct colours, because 112250 is in the list twice, marked above.

This list is the one thing here that is definitely mine, and git can prove it. At 16:28 that afternoon, in the commit called colors & styling, I wrote getRandomHexValue() with these 15 strings in it and pointed both call sites at it. Until then the example had been colouring the model’s text with generateRandomColor(), which put all three channels between 180 and 255 and gave you a pale nothing.

generateRandomColor() is still sitting there, and the same commit that stopped calling it is the one that rewrote its body to hold red and green under 20 so the blue would come through, and wrote the note it has now: “Cyanotype printing typically results in a range of blue tones, so we’ll adjust our function to generate colors within a range of blues.” I explained to a function how it ought to behave in the same breath as I stopped asking it anything. The luminance test I wrote to go with the new list is in the project twice over, taking hex in TextInlineAdvanced.tsx and rgb() in marks/Prompt.tsx, and only the second copy is ever called.

Coda

The repo still thinks it is somebody else’s.

  • package.json"name": "@liveblocks-examples/nextjs-yjs-tiptap-advanced", licence Apache-2.0, described as “This example shows how to build a collaborative text editor with Liveblocks, Yjs, Tiptap, and Next.js.”
  • README.mduntouched since the first commit, still headed “Advanced Collaborative Rich Text Editor (Tiptap)” and still telling you to run npx create-liveblocks-app@latest --example nextjs-yjs-tiptap-advanced --api-key
  • Room.tsxthe room id was never changed: const roomId = useOverrideRoomId("nextjs-yjs-tiptap-advanced");
  • layout.tsxthe title became “How do you actually feel?” at 16:45, and the two favicon links directly below it still point at liveblocks.io/favicon-32x32.png and liveblocks.io/favicon-16x16.png
  • mtimesnothing under src/ has moved since 2024-02-18 16:45, leaving aside the .DS_Store files Finder rewrote in May, and 504 of the 506 files in the abandoned .next build are stamped 18 February 2024, the last at 17:36

Locked in package-lock.json, lockfileVersion 3, 262 packages

  • yjs13.6.7
  • @liveblocks/client1.9.8
  • @liveblocks/node1.9.8
  • @liveblocks/react1.9.8
  • @liveblocks/yjs1.9.8
  • lib00.2.78
  • y-prosemirror1.0.20
  • y-protocols1.0.5
  • @tiptap/core2.0.4
  • @tiptap/pm2.0.4
  • @tiptap/react2.0.4
  • @tiptap/starter-kit2.0.4
  • @tiptap/extension-collaboration2.0.4
  • @tiptap/extension-collaboration-cursor2.0.4
  • next13.5.4
  • react18.2.0
  • openai4.28.0
  • ai2.2.35
  • typescript5.2.2

The darker rows are the ones package.json asks for by name, which I never edited; the rest arrived underneath them.

The title, and then the name

“How do you actually feel?” is a real work of mine, described in my paper Algorithmic Entanglement as a durational collaborative performance exploring NHS care through an 8-metre collaborative canvas, one-to-one performances, fictional pamphlets, unstructured interviews and an interactive website, in which I performed as “co-coordinator, practitioner, interviewer, programmer, performer, fictitious commissioned artist and algorithm.” At 16:45 that Sunday I changed one line of this repo to carry that title, and then I stopped.

The same paper lists this repo as a portfolio output, “A functioning tool for collaborative human-AI document creation built with Next.js, Liveblocks, Tiptap and GPT-4”, deployed at prometheusrun.vercel.app. That address comes from the paper rather than from me checking it, and I have no idea whether it still answers.

The name went on without the code, because prometheusrun is now the Google identity my Kew calendar tooling authenticates as, the account that actually holds that work calendar while my primary address holds none of it. A repo whose whole git history is one Sunday in 2024 became the login I run my automation under, which is a stranger kind of afterlife than a live URL.

Where all of this came from

Every number, name, quote and line of code above is read out of the archive at ~/prometheusrun or out of yjs’s own source shipped inside it. The commit stats come from git log --numstat, the eleven files from git diff --numstat 2a166b7 HEAD, the palette numbers are computed with the project’s own luminance formula rather than mine and the Item.integrate block is a byte-for-byte copy. Nothing is invented. The five things I can’t vouch for are named where they appear: the prompt layer’s history is inside the first commit and unrecoverable, the Vercel deploy is unverified, the page runs yjs 13.6.31 where the archive pinned 13.6.7, the wire here is not Liveblocks and the Google account just above is my own word rather than anything the archive can show you.

The generated data:commits.jsonmy-diff.jsonnames.jsonpalette.jsonstack.jsoncensus.jsonitem-integrate.txt

prometheusrun, nine commits and one changed title.← back home