ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

Changing pre-rendered video footage using csv data

COW Forums : Adobe Media Encoder

VIEW ALL   •   PRINT
Perry Sheppard
Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 2:21:28 am

I'm trying to plan a project to create about 4,000 videos using a set of data in csv format. Each video/comp would have several 'segments' customized based on the data row in the csv file. Each segment would be exactly the same length for any row of data. The general breakdown would be something like:

Comp Structure
Segment 1: Textual content customized from the data set (4,000 possible options)
Segment 2: Pre-rendered video footage (1 option)
Segment 3: Pre-rendered video footage (120 options)
Segment 4: Pre-rendered video footage (15 options)
Segment 5: Pre-rendered video footage (1 option)

The textual content in Segment 1 is easy enough. My original thought for the pre-rendered footage was to use the data set to reference the specific pieces of footage in 'placeholders' using and expressions and swap them using the csv data. I would then duplicate the comp 4,000 times (I'm sure there's a better way, but I'm still learning), and render them out.

However, I haven't been able to find anything that clearly states if this is possible or where to find resources aside from this about aligning video, which is not really what I'm looking for.

I know I could create a single comp that has all 120 options for Segment 3 and the 15 options for Segment 4 and use if/else statements to turn the layer opacity on or off based on the data in the csv, but that seems really inefficient and I imagine that would have a substantial impact on render times.

I'm looking for suggestions or insight on this and any resources available that you can point me to. I'd love to learn, but right now I'm not even sure I'm asking the right questions. Thanks!


Return to posts index

Tomas Bumbulevičius
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 10:08:22 am

Hey Perry, to begin this - this could be solved relatively easy with scripting, but I have one question:

Considering segment 1 has 4000 options, and segment 3 and 4 contains even more customisations... Wouldn't it turn into 4000x120x15 variations (blows my mind thinking about results, haha), if videos suppose to be unique? Elaborating further on what you have in mind for those segment-variations would be a good idea to help you out better!

Find out more:
Motion Graphics Design & After Effects Tutorials
On YT
On VH


Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 12:36:31 pm

Hi Tomas,

Thanks for your reply. Permutations and combinations abound lol, but, no, it's just 4,000.

There are 4,000 rows of data and each row contains the information for one individual. Their name, location, and area of interest. There are 4,000 'people', 120 'locations' possible for each individual, and 15 possible 'areas of interest' for each individual.

It's essentially a video mail merge:
Segment 1: The person's name. This would appear as dynamic, animated text (provided by the csv) in the comp.
Segment 2: Common to all
Segment 3: The person's location. This would appear as a pre-rendered video specific to the location (video footage to appear would be driven by the csv data).
Segment 4: The person's area of interest. This would appear as a pre-rendered video specific to the person's area of interest (video footage to appear would be driven by the csv data).
Segment 5: Common to all

To map out the comp, it would be something like:
Segment 1: Text animation - 8 seconds (text dynamically changes via csv)
Segment 2: Pre-render into - 10 seconds (static)
Segment 3: Pre-render location - 8 seconds (footage dynamically changes via csv)
Segment 4: Pre-render area of interest - 15 seconds (footage dynamically changes via csv)
Segment 5: Pre-render outro - 4 seconds (static)

Thanks again!


Return to posts index


Tomas Bumbulevičius
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 4:08:34 pm

Perry, thanks for reassuring me down to 4,000 videos only, haha! This all makes a lot more sense and clearer in general.

Now what is important here, is how you are going to do that 'linkage' between pre-rendered assets and the .csv.
1. Will it be based in some way on file names, or otherwise?
2. Assuming files replacement is sorted - what is your rendering preference?
3. Is it a single-outputs generation only, or might be further updated in the future with other data sets?

Cheers!

Find out more:
Motion Graphics Design & After Effects Tutorials
On YT
On VH


Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 5:03:51 pm

Thanks Tomas,

1. Re: linkage to csv - I need to refine the data set regardless of what approach I take (it'll only take a few minutes to do), so I have options. I was thinking my best options would be adding a column in the csv for, example, 'Location' with content in the format of either 'United Kingdom.mov' or simply 'United Kingdom'. Whichever is most efficient. The comps would ideally be named to match with the originating csv document. Ex. row 2 = comp 2.

2. Re: rendering preference - are you referring to render engine, settings, both? If engine, then either AE Render Engine or Media Encoder. I know that aerender would probably be faster, but I'm not familiar with Terminal and wouldn't be comfortable using a solution I couldn't troubleshoot. For output settings, it would be mp4, 1080x1080... I've done a fair bit of research on maximizing render efficiency (source file colour space, codecs, etc.), but always open to new info.

3. Re: one-off or not - Ideally, the setup would work with a new data set (the new data set would have exactly the same structure though) and new comp materials. So, the content of the csv rows could change, but the columns would remain identical and the master comp might change, but the approach would remain consistent.

Let me know if I understood your questions. Thanks again :)


Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 7:08:25 pm

One additional nice-to-have would be to also be able to render out the first second of each comp as an animated gif. I know that media encoder can export these, but would either need an in point at 0 time and an out point at 1 second, or the length of the comp would need to be one second.


Return to posts index


Filip Vandueren
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 9:18:44 pm

Have you taken a look at templater from Dataclay?



Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 9:29:34 pm

Hi Filip, hope you're doing well. Thanks for this.

I just took a quick look at Templater and I'm already wishing I'd known about this two months ago. They need to bump up their SEO/marketing because it looks great, but this is the first I've heard of it in all of my searches (including searches on batch, render farms, etc.). I'm going to do a deep dive into the demo this weekend and see how I might be able to incorporate this into the workflow.


Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 9:39:07 pm

Templator looks great, but I'm going to need to get this first project out as a proof-of-concept before I can try to make the case for it. At 3-6k/year, I'll need buy-in on consistent use.


Return to posts index


Filip Vandueren
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 10:00:18 pm

I thought you could get a 1 month subscription?
Have not used it myself, but if I had a client ask for this kind of project with 4000 movies, I wouldn’t hesitate



Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 10:05:51 pm

The trial is fully open for 30 days--which is great--but videos include a watermark. My proof-of-concept is going to require the production of the videos, but also the delivery to the recipients via email; otherwise, it won't be considered.


Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Jul 31, 2020 at 10:41:03 pm

I want to mention--because this community is so fantastic and I'd hate to leave the impression that I'm trying to get 'free sub-contractor help'--that I work for a public institution and this is one of two projects I've been developing on nights and weekends (this isn't for a paying client).

The institution isn't asking for these solutions because they don't know what's possible, so I'm trying to provide the proof-of-concepts to help improve our processes and provide better experiences for our community. So, I've been, admittedly, outside my coding comfort zone because I'm a solution designer more than an actual developer.

Filip, you Tomas (and others) have been really gracious with your time and support as I've been trying to learn expressions and the more complex ins-and-outs of AE, so I wanted to let you know how much it's appreciated.


Return to posts index

Filip Vandueren
Re: Changing pre-rendered video footage using csv data
on Aug 1, 2020 at 9:01:38 am

Since only the first segment is truly dynamic.
I would just render those 4000 (don’t know if AE like 4000 comps inside of 1 project though, but easy to distribute over a few files if not)
This is pretty trivial to do.
You could even just have a simple expression that looks at a number in the name of the comp to fetch that name from a csv.
Comp 1 -> name in row 1
Comp 2 -> name in row 2
Etc.

And then either automate premiere or even ffmpeg to stitch together the rendered titles with the other segments.
Doing it with ffmpeg would be lightning fast but involves some CLI scripting. https://trac.ffmpeg.org/wiki/Concatenate

Rendering the whole movie over and over again in After Effects would be by far the worst option. Because that’s just not what AE is built for, it’s not an editor.



Return to posts index

Filip Vandueren
Re: Changing pre-rendered video footage using csv data
on Aug 1, 2020 at 3:28:48 pm

Did a quick test with 1000 comps.
a simple script to duplicate a project-item 999 times and add each copy to the renderqueue
took just a few minutes to complete.
After Effects might not allow you to scroll down through past the first few 100 render queue items (a problem with drawing the interface), but it'll render out all of them.

I juist loaded in a csv with mock username.
set the expresison for text-source to this:

i=parseInt(thisComp.name.split(" ")[1]);
t=footage("MOCK_DATA.csv").dataValue([1,i]) + " " + footage("MOCK_DATA.csv").dataValue([2,i])

(takes a number from the second word of comp name like "comp 123" = 123 and looks that up in the)

then ran this simple script:

app.beginUndoGroup("999")

for (i=0; i < 999; i++) {
cmp = app.project.items[1].duplicate();
app.project.renderQueue.items.add(cmp);
}

app.endUndoGroup();


Obvioulsy just a quick &dirty proof of concept, you would want to change the name of the queued item to the name or some other identifier etc.



Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Aug 1, 2020 at 5:15:31 pm

Thanks Filip, that gives me a better sense of it.

I've been researching FFmpeg (haven't used it in years), Premiere Extendscript and AE scripting this morning. I know AE isn't really for editing and it might not be sustainable on an ongoing basis, but it looks like it might be the best option for me for the proof-of-concept.

I'll take a closer look at that script you provided and see if I can wrap my head around the syntax. I found a couple of reference sites, but still working through it.


Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Aug 2, 2020 at 3:57:22 am

Filip, can you private message me or email me. For some reason, the email response option fails on my computer. Thanks!


Return to posts index

Filip Vandueren
Re: Changing pre-rendered video footage using csv data
on Aug 2, 2020 at 10:21:26 am

I don't have your e-mail ?



Return to posts index

Perry Sheppard
Re: Changing pre-rendered video footage using csv data
on Aug 3, 2020 at 3:08:37 am

perry.sheppard at gmail.com


Return to posts index

Tomas Bumbulevičius
Re: Changing pre-rendered video footage using csv data
on Aug 4, 2020 at 11:09:07 am

Hey Perry, sorry for my late response, yet I see Filip might gave you a path to chase already. While I am an advocat for Templater all the time whenever I have a chance to do that (use it myself and for clients), before committing now or later you should consider whether it will be beneficiary in the long run.

Reason why I do say so - it has some learning curve as well, thus going in an easier path from a first sight (to get things finished now) might seem a great choice, but it also provides cons, when in the future you will still need scripting, sooner or later. Thus its better to go that harder path sooner, IMO! (: Unless you don't have intentions for automation - then its out of question. Yet, considering where everything is going to in the tech, knowing automation well enough is beneficial!

On a different note, as Filip mentioned - approaches are trivial and depends on everyones experiences/taste. Personally I wouldn't recommend parsing .csv, but instead - run a batch script which creates N projects and do replacement in each of them. Having XXXX comps in a project is not a sustainable choice, nor rendering all of them in a single project through Render Queue.

Find out more:
Motion Graphics Design & After Effects Tutorials
On YT
On VH


Return to posts index

VIEW ALL   •   PRINT
© 2020 CreativeCOW.net All Rights Reserved
[TOP]