The world's most unhinged video wall (made out of Chromebooks)
This is the story of our three year long journey to turn a fleet of laptops into what can only be described as the worldâs most unhinged video wall.
This project was a collaboration with my friend Aksel Salmi. I was responsible for the software, and he designed the incredible hardware, see his blog to learn about the unexpectedly complex hardware needed to mount these dismantled computers[1].
About three years ago, my Design teacher (The amazing Mr. Bush) came to us with an idea - our school was about to dispose of its fleet of Chromebooks, and he was wondering if we could build anything with them.
Meet the Lenovo ThinkPad 11e #
The Lenovo ThinkPad 11e could very well be the worldâs worst laptop. It is also the standard-issue school laptop that reinforced eight-year-old meâs interest in computers.
Despite my emotional connection to them, today these devices are, for all intents and purposes, junk. And for that reason, my school began the process of replacing them (with marginally less junky laptops)
These things donât receive software updates from Google anymore, they struggle loading most webpages and to top it off, theyâre tied to some long forgotten Enterprise Enrolment system, so they canât even be used without a school Google account.
What is a video wall? #
A video wall is a large display made up of multiple screens arranged together to create a single, seamless display across all the screens. In the case of our project, we decided to try reusing the laptop screens to build a video wall.
Can we drive the screens using separate hardware? #
Our first idea was to harvest just the laptop display panels and somehow drive them using a powerful computer that could power the 10 screens simultaneously. We did not go this route (due to the fact that we had no idea what we were doing, and a quick estimate of the time and costs involved scared us away).
Okay, before we try anything else, letâs just try synchronising a video across two devices #
Since the screens were attached to perfectly functional laptops, it was quickly apparent that weâd probably be better off letting each screen be driven independently by their own laptop motherboards.
At this point, there were still many questions (eg. how were we going to do that on Chromebooks), so we put aside that challenge to focus on the new issue this brings up: Can we synchronise a single video across multiple computers?
Our experiments brought us to the schoolâs computer lab, where we experimented with VLCâs streaming abilities to get a stream synchronised across devices on a single network, but this posed two challenges: This system is not designed for videos being perfectly in sync, nor was it designed for two clients to receive different video inputs (because the whole point of the video wall is to display one loooooong video across the screens, not 10 repeat copies of the same video).
We were stuck here until my ââbreakthroughââ.
For context, the story is currently in 2022. Two years earlier, I had been locked up in my room due to the COVID lockdown, and in this time, I had loads of fun building random realtime web apps, like a chat app and multiplayer drawing game. These apps worked thanks to socket.io
, a (primarily) WebSocket based library that allows for low-latency, bi-directional communication.
I realised that my best bet to get videos synchronised would be by using a web page that used socket.io
to sync the video playback across clients. Yes, there are better approaches, but simply doing something like this worked unreasonably well, all things considered.
<video src="..." id="video">
Your browser does not support the video tag.
</video>
const socket = io();
// ...
socket.on("play", () => {
const videoElement = document.getElementById("video");
videoElement.play();
});
I named this ExpressJS server/client system c-sync
[3].
Thanks to c-sync (and tons of tinkering), after some time we had decently synchronised videos across computer screens through a webpage (or at least it seemed like it, testing on these desktop computers)
As it turns out, in reality, the Chromebooks are too slow for this to be a reliable approach to synchronising playback, and tiny discrepancies in loading times + latency + system clocks etc. lead to videos not being synchronised.
Now, Iâm not entirely sure why this works so well, but I came up with a ridiculous solution by accident. When videos reach the end of playback, each client emits the start event.
video.onended = () => {
socket.emit("start");
// yes seriously, this is all I needed
};
This means that the slowest computers hold back the fastest computers, and get the chance to load the videos. This also means looping can be a very slightly jittery process (with each screen receiving 10 âstartâ events), but as long as the first couple frames of the video are identical, nobody would even notice.
Sidenote: why not schedule with timestamps
Modern computers have clocks you can rely on to be extremely precise. This plus regular NTP synchronisations means a reasonable person might just try to ensure the full video is cached, then just send a 'start' event to each client that schedules the client to start playback at a given timestamp. Unfortunately, these Chromebooks could not reliably keep track of time within milliseconds of each other, so this method didn't work for us.Using this method, we have nearly perfectly synchronised video playback, and can play any video on any screen (meaning we can split a wide video into 10 segments, and each computer displays its respective part, all in sync with eachother)
Putting it together #
We reached this stage within a month or two. Believe it or not, this project still had three years of work ahead of us. The biggest issue was Chromebook software. At this point, we had a website that we could manually open on each laptop to display a fullscreen synchronised video.
Ideally, we would want this to be entirely automated, so that as soon as a Chromebook receives power, it boots up automagically to the c-sync client page. Unfortunately, right now, booting the Chromebook would just take you to a Google login page (and one that was locked to our school domain to boot).
Also, just to add insult to injury, when batteries are removed, the laptops donât turn themselves on when they receive power (you have to hold down the power button)
This meant that our next step would have to be to replace ChromeOS with something else.
The âChromeOS Firmware Recovery Scriptâ is a magical piece of technology that somehow supports many different Chromebook motherboards. Ours was called âGLIMMERâ. We just had to enter the built-in âRecovery Modeâ, enable âDeveloper Modeâ and use the ChromeOS Shell to run the script.
Now weâre basically on the home stretch. All we needed to do was pick up some stable Linux distro, write a hacky startup script that loads up Chromium and simulates the keystrokes to fullscreen the video and weâre done!
We ran in to two main issues: Some Chromebooks (roughly half of our working laptops) would refuse to enter developer mode due to the enterprise enrolment, and while we were able to get the other half onto a Linux distro, video playback would consistently freeze after some time (actually they would lock up entirely).
It took us several months of on-and-off experimentation to figure out what to do. Essentially, the solution was to overwrite the entire default firmware with coreboot
(which is also possible using MrChromeboxâs script). We just needed to remove the âWrite Protectionâ screw from each laptop motherboard, and this seemed to bypass the enrolment too.
Doing this for 20+ computers was slow and tedious. We only really needed the WiFi, motherboard and screen in working condition, but we decided to be (mostly) gentle and keep the laptops looking like laptops so that we had a keyboard and mouse for the rest of the installation steps.
After âcorebootingâ the Chromebooks, we were also pleasantly surprised to find out that âWake on ACâ was a feature of the firmware, and that video playback no longer randomly breaks. By this point we had enough non-bricked Chromebooks left over for a line of 10 screens and a handful of spares.
The Final Stretch #
Now weâre really on the final stretch. Aksel worked on the mounting hardware, which you can read about on his blog, while I worked on figuring out a less flaky way to âboot to a webpageâ than the keystroke simulation and startup script I bodged together.
I previously used the aptly named âFullPageOSâ for a different project (which I briefly mention in my TED talk, which you should watch), but it doesnât run on x86 hardware.
I landed on using âPorteus Kioskâ, which is just a minimal Linux distro that opens a fullscreen Chromium browser with all the correct flags for hands-off usage (eg. allowing video playback without user interaction)
This honestly worked totally fine, but left me unsatisfied for two reasons. Firstly, I didnât like how we couldnât customise the splash screen, so our project would be forever stamped with the Porteus logo on every startup (which would be every morning). And secondly, in search of a better issue to justify the extra work, I realised we canât remotely do anything to the installations (eg. changing the page URL) without re-doing them, which would be definitely a problem once these get mounted on the wall.
For those good reasons, I embarked on the journey of building âmy own distroâ that we could install on the laptops. The system should start with something minimal (no desktop environment), and have an elegant script to autostart a kiosk mode Chromium instance.
I first tried NixOS before quickly realising there was no way it would work with the tiny amount of storage on these Chromebooks (and it failed to install with every single attempt).
Then I gave up, started with a Debian minimal install and just wrote a script that would provision a client (generate a âKIOSK_IDâ, set its hostname to csync-client-$KIOSK_ID
, connect to the schoolâs WiFi, create users/permissions and set up openbox
to autostart a fullscreen kiosk mode Chromium).
Then after attempting to repeat this on a second machine, I realised I would be wasting so much time (installing Debian is very âhands-onâ - you need to press lots of buttons), and I discovered âFAI - Fully Automatic Installationâ and the web FAI.me tool. To cut a long story short[4], after redoing everything for the millionth time, I had a single USB that I can plug in to any âcorebootedâ Chromebook which provisions it as a c-sync client. Woohoo!
I also built out a âcontrollerâ for c-sync which lets us manage connected clients and assign them videos.
After a successful three day stress test where the playback remained butter-smooth (and I sacrificed my ability to sleep for the greater good of testing with the backlight on), we were ready to mount these laptops on the wall.
Mounting #
The mounting is mostly Akselâs thing, so I implore you to read his blog, but here are some cool photos from the process. (also arenât our cable splices so pretty and not terrifying?? đâ€ïž)
One last thing⊠#
After we painstakingly mounted everything, I realised something sort-of important. Computers generate heat. Somewhere along the way of wiping away the firmwares, the laptop fans stopped spinning, which meant things get quite hot quite quickly. I had to figure out a way to get those working again before we could comfortably leave this up 24/7 (well, actually 12/7).
Embedded Controllers #
You can apparently interface with the âChromeOS Embedded Controllerâ using a tool called ectool
, which should allow you to manually set fan speeds (among other things). The online documentation for this is lacking, and thereâs apparently a slightly different ectool
from coreboot
and from Google directly. None of this made much sense at all to me, and no built ectool
binary I could find would work. At some point, I found a dead link, but thanks to the magic of the Wayback Machine, I was able to get my hands on something that wouldnât immediately crash.
By some miracle, this version of the tool actually works perfectly fine at setting fan speeds, and after some testing, I found some goldilocks values that balance noise and temperature.
Aside: Making Videos for the Thing #
As it turns out, making such a wide video is actually not easy. Each display has a resolution of 1366Ăâ768, and very few pieces of software will let you edit a 13660âĂâ768 video. Final Cut Pro and Blender are the only programs we were able to do anything this wide in.
Then itâs just a matter of rendering the wide video and splitting it into 10 segments.
#!/bin/bash
# incase anyone ever has this insane use case again
input_video="input.mp4"
prefix="v8"
width=1366
height=768
segments=10
for ((i=1; i<=segments; i++)); do
x_offset=$(((i - 1) * width))
output_file="${prefix}-${i}.mp4"
ffmpeg -i "$input_video" -vf "crop=$width:$height:$x_offset:0" -c:a copy "$output_file"
done
echo "Splitting complete!"
In all its glory #
Boot Sequence and âSelf Calibrationâ #
Synced videos! #
Now thereâs an enclosure and cable routing! #
Yes, itâs imperfect #
Our video wall is imperfect. TN panel viewing angles suck, and the screens vary in colours and stuff. Yes, the synchronisation isnât perfect, and yes, Iâm sure there were better alternatives for nearly every decision we made along the way.
Yet I love our video wall, despite how absurdly weird it is. Itâs a perfect representation of the iterative design process and a true testament to teamwork and collaboration. We turned E-Waste into something interesting. And maybe, just maybe, the real video wall was the friends we made along the way.
This project was made possible by the incredible work of so many people. Aside from my collaborator Aksel Salmi, our Design teacher Daniel Bush played a huge role in guiding us through the project.
Additionally, I wanted to thank the coreboot project and Matt âMrChromeboxâ DeVillier for putting together the firmware and tools that allowed any of this to work. I would also like to thank Thomas Lange of the FAI project for his help in building the FAI.me based automated installer that saved us so many many many hours, as well as his support over email.
As silly as it sounds, this project was a backbone in my high-school experience. We hacked away at it every Monday for the past few years, and we grew up along the way too.