Subscribe to Matt Cynamon
<100 subscribers
Subscribe to Matt Cynamon
<100 subscribers
Share Dialog
Share Dialog
Yesterday Albert asked me to generate an image for his newest blog post Scale is All You Need? I obliged but thought that was a little boring so I made him a trailer as well. Because well, trailers are awesome.
I've been making trailers for a little while now. Mostly around fictional concepts that I find funny. Like my rom-com Hearts of Chamonix or my sci-fi thriller AGI. It's been a while since I've made one but on average they would take me about half a day or a couple evenings. I didn't have that kind of time for Albert's so I gave myself the challenge - Can you develop a trailer for this blog post in under one hour. To accomplish this, I used 4 tools:
The only way to get a project like this to work on that time frame is to be comfortable giving up creative control. Understandably, creative and editing choices by far take up the most time in AI video creation. So, I needed to try making something that required minimal human input and I made it a point to provide little to no editing.
The key to making the whole process work was starting with a comprehensive prompt. Having done this a few times, I knew all the inputs I needed to get to work quickly. My prompt was:
You are an expert film maker and advertising executive who has been tasked with developing a fully AI generated trailer of the below blog post. Please write a 30 second trailer. It should include:
1) A time stamped script.
2) A list of b roll shots structured as prompts that can be entered into an LLM. The B roll should all be strong images that contain no logos or specific companies. No shots should be longer than 4 seconds.
3) A system prompt for an overarching style across all B roll shots (no more than 300 characters).
4) a prompt for lyricless music that will play on this trailer. It should be thematically aligned (no more than 300 characters).
You'll likely notice I didn't provide much in the way of stylistic guidelines to the prompt. This is simply because Albert's post already had such a strong sci-fi slant. If I was trying to design something from scratch for a work of fiction, I would have wanted to be a bit more explicit in my prompt. Maybe describe the genre, provide color palettes, or outline the emotion I wanted in the music.
Absent of more specific instructions, with only Albert's blog post and my prompt, here's what was produced:
Title: "Scale is All You Need? Exploring the Future of AI"
Voiceover: "In the world of AI, a handful of tricks have proven unreasonably effective, all driven by the power of scale. But is even more scale all we need?" (8 seconds)
"Or might we need some more tricks - simulated environments for learning causality, multi-agent systems for modeling interaction, reliable tool use and structured knowledge?" (8 seconds)
"These known tricks show promise, raising the possibility of getting further with open-source models. But we may also hit fundamental limitations, requiring expensive breakthroughs." (7 seconds)
"The implications are profound. Will we see an open, thriving AI ecosystem or a concentrated, closed-source market? Much remains to be figured out." (7 seconds)
B-roll shots as prompts:
B-roll 1 (4 seconds): /imagine prompt: A vast, abstract data visualization representing the exponential growth and potential of AI, with swirling tendrils of light and color forming intricate patterns and structures.
B-roll 2 (3 seconds): /imagine prompt: A monolithic, enigmatic structure towering over a desolate landscape, symbolizing the transformative power and mystery of AI.
B-roll 3 (3 seconds): /imagine prompt: A lone figure standing at the edge of a precipice, silhouetted against a backdrop of swirling, ethereal light, representing the profound implications and challenges of AI development.
B-roll 4 (3 seconds): /imagine prompt: A surreal, dreamlike environment where AI agents interact and learn, with fluid, morphing landscapes and structures adapting to their actions.
B-roll 5 (3 seconds): /imagine prompt: A choreographed dance of robotic entities, moving in perfect unison and harmony, symbolizing the potential of multi-agent collaboration and coordination.
B-roll 6 (3 seconds): /imagine prompt: A robotic hand deftly manipulating a complex, luminescent tool, showcasing the precision and skill of AI in leveraging advanced technologies.
From there I just took all the prompts and pasted them into different programs. The voice over was pasted into Eleven Labs to create the narration. the style prompt was pasted into RunwayML to create a preset and then the b roll prompts were were independently added to create each shot. The musical prompt was entered into Suno to create the backing track. This whole process took around half an hour, mostly due to the rendering times required for each asset.
Next, I assembled all of the individual elements together inside RunwayML. If you can use legos, you can do this as well. In fact, I made it a point to do no real editing and just follow the time stamps recommended from the initial prompt.
All in all, this took about 20 minutes. But once complete I realized the timing didn't perfectly sync up and needed a slightly punchier conclusion. So, I took it upon myself to add a Title slide at the end. That was my only creative contribution.
And that was it....video trailer end to end in about an hour. To say that I made this would be a bit of an overstatement, I simply assembled the pieces. By next week, I'm sure the tools will exist to do this even quicker and better. That's just where we are in the tech cycle.
Of course, none of that would have been possible with Albert providing the initial source material. So give the post a read, it's worth it.
Yesterday Albert asked me to generate an image for his newest blog post Scale is All You Need? I obliged but thought that was a little boring so I made him a trailer as well. Because well, trailers are awesome.
I've been making trailers for a little while now. Mostly around fictional concepts that I find funny. Like my rom-com Hearts of Chamonix or my sci-fi thriller AGI. It's been a while since I've made one but on average they would take me about half a day or a couple evenings. I didn't have that kind of time for Albert's so I gave myself the challenge - Can you develop a trailer for this blog post in under one hour. To accomplish this, I used 4 tools:
The only way to get a project like this to work on that time frame is to be comfortable giving up creative control. Understandably, creative and editing choices by far take up the most time in AI video creation. So, I needed to try making something that required minimal human input and I made it a point to provide little to no editing.
The key to making the whole process work was starting with a comprehensive prompt. Having done this a few times, I knew all the inputs I needed to get to work quickly. My prompt was:
You are an expert film maker and advertising executive who has been tasked with developing a fully AI generated trailer of the below blog post. Please write a 30 second trailer. It should include:
1) A time stamped script.
2) A list of b roll shots structured as prompts that can be entered into an LLM. The B roll should all be strong images that contain no logos or specific companies. No shots should be longer than 4 seconds.
3) A system prompt for an overarching style across all B roll shots (no more than 300 characters).
4) a prompt for lyricless music that will play on this trailer. It should be thematically aligned (no more than 300 characters).
You'll likely notice I didn't provide much in the way of stylistic guidelines to the prompt. This is simply because Albert's post already had such a strong sci-fi slant. If I was trying to design something from scratch for a work of fiction, I would have wanted to be a bit more explicit in my prompt. Maybe describe the genre, provide color palettes, or outline the emotion I wanted in the music.
Absent of more specific instructions, with only Albert's blog post and my prompt, here's what was produced:
Title: "Scale is All You Need? Exploring the Future of AI"
Voiceover: "In the world of AI, a handful of tricks have proven unreasonably effective, all driven by the power of scale. But is even more scale all we need?" (8 seconds)
"Or might we need some more tricks - simulated environments for learning causality, multi-agent systems for modeling interaction, reliable tool use and structured knowledge?" (8 seconds)
"These known tricks show promise, raising the possibility of getting further with open-source models. But we may also hit fundamental limitations, requiring expensive breakthroughs." (7 seconds)
"The implications are profound. Will we see an open, thriving AI ecosystem or a concentrated, closed-source market? Much remains to be figured out." (7 seconds)
B-roll shots as prompts:
B-roll 1 (4 seconds): /imagine prompt: A vast, abstract data visualization representing the exponential growth and potential of AI, with swirling tendrils of light and color forming intricate patterns and structures.
B-roll 2 (3 seconds): /imagine prompt: A monolithic, enigmatic structure towering over a desolate landscape, symbolizing the transformative power and mystery of AI.
B-roll 3 (3 seconds): /imagine prompt: A lone figure standing at the edge of a precipice, silhouetted against a backdrop of swirling, ethereal light, representing the profound implications and challenges of AI development.
B-roll 4 (3 seconds): /imagine prompt: A surreal, dreamlike environment where AI agents interact and learn, with fluid, morphing landscapes and structures adapting to their actions.
B-roll 5 (3 seconds): /imagine prompt: A choreographed dance of robotic entities, moving in perfect unison and harmony, symbolizing the potential of multi-agent collaboration and coordination.
B-roll 6 (3 seconds): /imagine prompt: A robotic hand deftly manipulating a complex, luminescent tool, showcasing the precision and skill of AI in leveraging advanced technologies.
From there I just took all the prompts and pasted them into different programs. The voice over was pasted into Eleven Labs to create the narration. the style prompt was pasted into RunwayML to create a preset and then the b roll prompts were were independently added to create each shot. The musical prompt was entered into Suno to create the backing track. This whole process took around half an hour, mostly due to the rendering times required for each asset.
Next, I assembled all of the individual elements together inside RunwayML. If you can use legos, you can do this as well. In fact, I made it a point to do no real editing and just follow the time stamps recommended from the initial prompt.
All in all, this took about 20 minutes. But once complete I realized the timing didn't perfectly sync up and needed a slightly punchier conclusion. So, I took it upon myself to add a Title slide at the end. That was my only creative contribution.
And that was it....video trailer end to end in about an hour. To say that I made this would be a bit of an overstatement, I simply assembled the pieces. By next week, I'm sure the tools will exist to do this even quicker and better. That's just where we are in the tech cycle.
Of course, none of that would have been possible with Albert providing the initial source material. So give the post a read, it's worth it.
B-roll 7 (3 seconds): /imagine prompt: An intricate, holographic network of nodes and connections, pulsing with energy and data, representing the vast potential of structured knowledge in AI systems.
B-roll 8 (4 seconds): /imagine prompt: A diverse group of individuals, rendered as silhouettes, collaborating against a backdrop of flowing, abstract data visualizations, symbolizing the power of open-source AI development.
B-roll 9 (4 seconds): /imagine prompt: A lone researcher standing before a colossal, inscrutable wall of complex equations and data, representing the daunting challenges and potential limitations in AI development.
B-roll 10 (4 seconds): /imagine prompt: A vibrant, interconnected web of AI-powered systems and applications, pulsing with energy and potential, symbolizing a thriving, open AI ecosystem.
B-roll 11 (3 seconds): /imagine prompt: A delicate balance of glowing, abstract shapes, teetering on the edge of stability, representing the precarious equilibrium between AI progress and safety.
B-roll 12 (3 seconds): /imagine prompt: A diverse, unified crowd of individuals, standing together as a powerful force, their faces illuminated by the glow of holographic AI interfaces, symbolizing the collaborative future of AI development.
Overarching style prompt:
An ethereal, abstract digital realm with fluid geometries, pulsing light networks, and holographic data. Cool, futuristic colors with neon accents. Diffused and focused lighting create depth.
Musical prompt:
A captivating, cinematic soundtrack that perfectly complements the visionary exploration of AI's future. Predominantly electronic, with rich, layered synth and pulsing rhythms
B-roll 7 (3 seconds): /imagine prompt: An intricate, holographic network of nodes and connections, pulsing with energy and data, representing the vast potential of structured knowledge in AI systems.
B-roll 8 (4 seconds): /imagine prompt: A diverse group of individuals, rendered as silhouettes, collaborating against a backdrop of flowing, abstract data visualizations, symbolizing the power of open-source AI development.
B-roll 9 (4 seconds): /imagine prompt: A lone researcher standing before a colossal, inscrutable wall of complex equations and data, representing the daunting challenges and potential limitations in AI development.
B-roll 10 (4 seconds): /imagine prompt: A vibrant, interconnected web of AI-powered systems and applications, pulsing with energy and potential, symbolizing a thriving, open AI ecosystem.
B-roll 11 (3 seconds): /imagine prompt: A delicate balance of glowing, abstract shapes, teetering on the edge of stability, representing the precarious equilibrium between AI progress and safety.
B-roll 12 (3 seconds): /imagine prompt: A diverse, unified crowd of individuals, standing together as a powerful force, their faces illuminated by the glow of holographic AI interfaces, symbolizing the collaborative future of AI development.
Overarching style prompt:
An ethereal, abstract digital realm with fluid geometries, pulsing light networks, and holographic data. Cool, futuristic colors with neon accents. Diffused and focused lighting create depth.
Musical prompt:
A captivating, cinematic soundtrack that perfectly complements the visionary exploration of AI's future. Predominantly electronic, with rich, layered synth and pulsing rhythms
No comments yet