Explore the exciting world of AI-powered video creation, focusing on how to create compelling videos from static images using tools like Runway ML and Kling AI.
Whether you’re a content creator, marketer, or just curious about the latest advances in AI technology, this guide will help you harness the power of image-to-video creation to bring your creative visions to life.
Here’s a sample video of mine.
I attempted to close the visor on my helmet with the right hand.
Since the visor is hidden under the white surface, the request to close the visor is not a trivial matter. Several concepts have to be linked together:
- The visor rotates on a hinge
- The closed visor should be the same size as the opening of the helmet
- Astronaut visors are mirrored
Runway ML and Kling AI
Runway ML and Kling AI are both amazing generative AI video generators. Depending on your video needs you will want to play around with both.
To make matters worse, the two are facing increasing competition from Luma Dream Machine, and OpenAI’s Sora is not even released yet.
While Kling AI offers an interesting feature that lets you specify detailed movements, it didn’t quite work out for closing the visor.
Finally, Runway ML did the trick using the first and last frame functionality.
Video prompting best practices
- Use clear and specific descriptions for desired scenes and actions
- But then again, often times very simple prompts do the job
- Incorporate relevant details like lighting, camera angles, camera movements and mood
- Based on the tool you are using, experiment with different prompt lengths to find the optimal balance
- Include references to specific visual styles or genres for more targeted results
- Iterate and refine prompts based on generated outputs for better results
Video generators like these now appear on a daily basis. Below are a few more attempts.
Same prompt in Hailuo AI Minimax
Hailuo Minimax is in early Beta and not quite on par with the others just yet.
Same prompt in Luma Labs Dream Machine
Same prompt in Haiper AI
Step-by-step guide to video prompting
Step 1: Choose Your AI Video Generation Tool
Select a platform like Runway ML or Kling AI that offers image-to-video capabilities.
Step 2: Prepare Your Input Image
Choose or create a high-quality static image that will serve as the basis for your video.
Step 3: Craft Your Prompt
Write a clear, detailed prompt describing the desired video outcome.
Include:- Specific actions or movements
- Desired mood or atmosphere
- Lighting and color preferences
- Camera angles or movements
- Any style or genre references
Step 4: Experiment with Prompt Length
Try different prompt lengths to find the optimal balance between brevity and detail.
Step 5: Generate the video
Input your image and prompt into the AI tool and initiate the video generation process.
Step 6: Review and iterate
Analyze the generated video and refine your prompt based on the results. Repeat the process until satisfied.
Mind you, this can come at a cost. At one point, I had to pay an additional $10 on top of my monthly plan because I had tried so many different prompt variations.
Step 7: Fine-tune and download
Make any final adjustments and download your video in the desired format.
By the way, the format depends on your image that served as the base of the image-to-video generation. If you want a vertical video, you need to upload an according image in the 9:16 format.
Image creation for image-to-video prompts
In my example I used a generative AI photo of mine that I created using Flux and a low-rank adaption (LoRA) containing my portrait photos. See the post below.
If it doesn’t matter who or what is on the image, you may as well use Midjourney to create your basis for image-to-video.
In this other post you will find a few more video generation tools for purposes like lip-synch.
