The Beatles get by, with a little help from AI. Now And Then.

The Beatles are probably the most famous artists of all time. Their work was seen and heard billions of times in an era where millions was considered a lot. They were fascinated by technology, and their extreme fame allowed them to create experimental electronic collages that were played around the world. The band and their fans were game to try new things.

It makes sense that Now and Then makes use of custom machine learning and AI. In both the music and the video. That’s what The Beatles do.

The story of the technology begins in 2018, when director Peter Jackson began work on what would become the award-winning series Get Back for Apple (both Apple Records and Apple Computers, that long ago agreed not to sue each other)

You probably know Jackson from The Lord of the Rings movies and the digital effects company he founded, Weta. Weta Digital is often mentioned as the world’s most innovative visual effects house. (more on LOTR creator JRR Tolkien)

The very popular 2021 documentary was made using footage shot fifty years ago, and required updating and clean-up. Jackson used his own company, Wingnut Films, to develop special restoration technology that was based on machine learning. It was first used for the powerful World War I documentary They Shall Not Grow Old, then Get Back. If you haven’t seen Get Back yet, I highly recommend that you at least scan through it. It looks and sounds amazing (available on Disney+).

For the music film they developed software called Machine Audio Learning, or MAL, to reliably identify and extract voices and other sounds from a mix. There is other software that can demix music, but this appears to be the best quality.

The story of the song goes back to the 90s, when Paul, George, and Ringo learned there were unfinished John Lennon songs they could complete. Yoko Ono found some cassette tapes from 1977 that had good ideas, but sounded very rough. Using the technology of the time they created the first Beatles tunes in twenty-five years: Free As A Bird and Real Love. Now and Then was part of that session, but they couldn’t make anything useful because the tape was too noisy. Every time they would try to boost John’s voice, the piano would climb up along with him and ruin it, since it was all based on frequencies.

Flash forward to 2022, and Paul McCartney asks Peter Jackson if they could use MAL to extract John’s voice from the cassette. He said yes, and the result is a remarkably natural, clean sounding solo voice.

This is the same category of software that you may know from video calls, were the voice is emphasized and noise filtered out. It is not generative artificial intelligence. The voice we’re hearing is the sound of the original person’s voice, as they sang it. There is nothing synthesized about John’s voice. It has been simply extracted from the background noise and cleaned up.

This still capture from the documentary video of the song shows how many parameters can be carefully adjusted so only the frequencies of the voice come through. The process can be very labor intensive. For example, there may be overlap between John’s voice frequency and the piano sound. So every note has to be double-checked by hand.

A similar machine learning algorithm is used in the delightful music video, and directed by Peter Jackson himself. The software extracts a moving image of John or George and places it into the footage of Paul and Ringo. It’s both wonderful and a bit startling, like there must be time travel and magic of some kind at work here. Peter Jackson is a great artist in his own right and the video really enhances the song experience.

Just as the song utilizes bits of previous Beatles songs (a common Beatles technique), the video also combines new footage with classic film clips, and also includes never-before-seen archival footage. Jackson doesn’t really care much about awards, but this work is outstanding.

Now and Then went to #1 on iTunes, and the video already has millions of views in less than a day. Lots of people like it. But those who have don’t seem upset that it sounds modern. That seems a bit humorous, since the people who didn’t like The Beatles back in the day said the same thing! Tech can be used for good and this project, starting with Get Back, is a great example. Now and Then is, at the end of everything that the Beatles have done and been, a poignant and beautiful final work from a band that has passed the test of time while we watched, and listened, and danced.

Thank you, lads. Well done.

-30-