Different methods
Go to the initial settings, and paste the path in init image;
To enable, check the use init box
when you uncheck the box to disable, you don’t need to delete the path
VIDEO INPUT
First let’s switch the animation mode to Video input.
All the settings for Motion Parameters and Coherence, do not work in animation mode as one cancels the other.
You should always match the Resolution settings of Deforum with the aspect ratio from the initial video you’ll use.
Also a squere resolution (512x512) will work mostly fine.
The strength parameter is going to tell how much of your initial image Deforum is going to keep during the diffusion of the very first frame.
the higher the strength, the more of your image is going to be preserved.
A small trick for initial image is to use it as a source for colors that you want to have in your animation and set a low strenght value (ex: 0.25)
In this way Dorum is basically just going to keep the colors of the image.
This can save you tons of time (especially when you have a commissioned job) and you have the right prompt but you can't get the right colors.
A clear way to utilize it is to begin an animation with an image that you personally created or prefer and set the strength to medium or high.
Another good way to use it is to resume an animation with it.
Use last frame of a previous animation as an initial image with a 0.95 strenght value
Upload your video in colab or locate it from your google drive (same as you did for the image)
There are more ways you can create your animations.
One of these, is using an initial image as starting point of your render.
To upload your own images into Deforum, simply drag and drop it into this tab of Google collab’s window
or locate it from your Google Drive.
Right click and then copy path
You can include a mask image to prevent certain parts of your animation from being animated.
Locate your mask file, copy the path, then paste it into mask_file: right below and check the use-mask box
MASK
IMAGE
ANIMATION
This technique alone is not perfect, as some parts of the image still gets diffused.
You can recover that in video editing phase
Now paste it into the video init path and then choose how many frames you want to extract from your video.
For example if the video, you insert and 240 frames, and if you set this parameter to, then, is going to use 120 frames from that video.
The extracted frames are going to be used as the initial image to create your animation
use the strength to decide the ammount of diffusion exacly like with an image
Seed behavior
Seed behavior is going to set the way your seed “travels”.
By default it's set to “iter” and it gives you new seed every new frame during the animation render:
Example: If set seed to 1, iter travels like this 1, 2, 3, 4, 5, 6, 7, 8, 9…
When using video input mode you might wanna set it on “fixed” because it's going to keep the same seed and give you more coherence
Be careful not to use fixed seed in 3d mode or is gonna just render a bunch of nonsense
”Random” is… Well… Random.
"ladder": Increments + 2 then decrements -1 for a pattern like 0, 2, 1, 3, 2, 4, 3, 5…
"alternate": Increments + 1 then decrements -1 for a pattern like 0, 1, 0, 1, 0, 1, 0, 1…
THIS LAST 3 OPTIONS WORK VERY WELL IN 3D MODE, but they can get a bit “exotic”
The magic of Deforum is in these modes.
I never use 2D mode and interpolate mode because they just don’t look so interesting and have a LOT of limits.
I personally think that Video input mode is working the best when you “use the initial video as a brush to paint”.
I rarely have the strength set to a high value when i use it for my work.
Only in very specific cases i use it as “the video that I want my animation to look like”
ANIMATION
INITIAL VIDEO
You can also use a video mask by doing the same process
Copy the path of your mask video into the right line and check the box
You can also use a video mask by doing the same process you did with the image.
Copy the path of your mask video into the right line and check the box
Make sure that the number of frames (length) of the MASK matches the number of frames of the video or is longer,
If the mask runs out of frames before the video is done, it will stop rendering.
Keep this in mind when you set the number of frames you extract.
This is one of my favorite tricks to use a steady image or something like that to mask a video because it lower the chance to get errors in your render. And it looks cool!
Arrived at this point there is not wrong and right anymore, (or if there ever was into Deforum). All these are just tools and it's up to you to find the best way to use them and get creative with them.