Turn a story into a comic with Lore Machine

12 years ago, author Will Douglas Heaven, in his spare time, wrote a humorous short story about a man who eats luxury cat food. And now, there is a Generative AI platform called Lore Machine, which helps turn this story into interesting comics.

The author puts his story into the text box and gets this message: 'We're identifying scenes and locations and characters and emotions. This process may take up to 2 minutes'. Lore Machine analyzed the text, extracted descriptions of the characters and places mentioned, then passed that information to an image-generating model. An illustrated storyboard appears on the screen. As the author clicks through the vivid comic book illustrations of his nearly forgotten characters, things are surprising.

Turn a story into a comic with Lore Machine Picture 1

After more than a year of development, Lore Machine has launched to the public for the first time. For $10/month, you can upload up to 100,000 words of text (up to 30,000 words at a time) and create 80 images for short stories, scripts, podcast recordings, etc. There are also pricing tiers. for power users, including a business plan priced at $160/month with 2.24 million words and 1,792 images. Illustrations come in a variety of built-in styles, from manga to watercolor to 80s TV shows.

Zac Ryder, founder of creative agency Modern Arts, has been using an early access version of the tool since Thobey Campion, founder of Lore Machine, first showed Ryder what it could do. Ryder submitted the script for a short film, and Campion used Lore Machine to turn it into a 16-page graphic novel overnight.

'I remember Thobey sharing his screen. We were all completely stunned,' Ryder said. 'It's not the image-making aspect. That's the level of storytelling. From the story line to the characters' emotions, everything was perfect from the beginning.'

Modern Arts is currently using Lore Machine to develop a fictional universe for a text-based comic series written by the creator of Netflix's Love, Death & Robots.

Turn a story into a comic with Lore Machine Picture 2

The Lore Machine is built from familiar parts. A large language model scans your text, identifying descriptions of people and places as well as its overall sentiment. A Stable Diffusion instance will create the image. What sets Lore Machine apart is that it is easy to use.

That makes it one of the new wave of user-friendly tools that hide the incredible power of generative models behind a one-click web interface. Ben Palmer, CEO of New Computer Corporation, a content creation company, said: 'There is a lot of work to be done to keep up to date with new AI tools, and the interfaces and workflows of each. tools are all different'. 'It's very appealing to use a large tool with a consistent user interface. I feel like this is where the industry will find its footing.'

Campion founded the company behind Lore Machine two years ago to develop a blockchain version of Wikipedia. But when he saw how people were adopting generative models, Campion switched gears, using the free text-to-image model Midjourney to create a comic version of Samuel Taylor Coleridge's The Rime of the Ancient Mariner. He said it went viral but there was nothing interesting about doing it.

Turn a story into a comic with Lore Machine Picture 3

'My wife hated that project,' Campion said. 'Every night, I stay up until 4am trying to get these photos perfect.' The problem is that text-to-image models like Midjourney create one image at a time. That makes it difficult to maintain consistency between different images of the same character. Even locking in on a specific style across multiple images can be difficult. 'Finally the author moved towards an abstract expression,' he said.

This experience made him see that the technology needed to be much easier to use. Campion wouldn't say exactly how Lore Machine handles keeping its imagery and style consistent across a series of illustrations. It's pretty good, but not perfect: In one scene in the author's story, a character with short hair and bangs appears twice, in a different place.

Illustrations can also start to feel generic. The author was disappointed that many details describing the characters were not shown in the illustrations. But compared to manually doing each reminder one by one, Lore Machine is a big step forward.

Ryder said: 'The consistency has been great. It gave Modern Arts the confidence to use Lore Machine on a project with its client." 'If we constantly had to go back and fix consistency issues there was no way for on-time delivery'.

Turn a story into a comic with Lore Machine Picture 4

Like all generative models, the technology behind Lore Machine generates malicious content on demand. Campion said it tried to prevent the creation of images depicting violence or hatred, but at the same time did not limit artists' creative expression.

Much of the initial interest in Lore Machine came from marketing agencies. But Campion hopes the wider release will encourage more users to try it. Campion said, 6 months ago, he received a call from the principal of a school for disabled children in Manhattan. The principal wants to run his textbooks through the tool so the kids can see the images. 'I hadn't even thought about that. I was so stuck in the Hollywood mindset,' Campion said.

Jessica Tanner

Update 30 March 2024