FORUMS: list search recent posts

Spoken Word Search in Video Editing Applications

COW Forums : Apple Final Cut Pro Legacy

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Kip Watters
Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 8:49:02 pm

My name is Kip Watters – I am the Director or Product Management for Media at Nexidia. Nexidia is the company whose technology powers ScriptSync in Avid’s Media Composer. ScriptSync is one way to utilize Nexidia’s “phonetic search” capability – jumping to specific lines in scenes based on text selected from the shooting script.

I am very interested in gauging the interest in and ultimately bringing these types of features to the Apple community as well.

First, a couple notes on how Nexidia's technology is different from Speech-to-text:

1) Rather than producing a text transcript, Nexidia’s indexing process (which produces a small index file for each media file) is several hundred times faster than speech-to-text. As an example, an hour long video can be indexed in several seconds with Nexidia. The index file is not human readable, but can be searched for any number of terms/phrases once it is created.

2) Phonetic search, unlike speech to text, does not require every word to be in the “dictionary” of possible terms for it to be found. This is especially important when searching regional or topical content with unique proper name and colloquialisms. Nexidia is also more tolerant of background noise and does not require a perfect recording to be useful.

I am also interested in hearing your thoughts on other ways that search of spoken word content in audio and video would benefit the professional video editing community.

For example, would having access to a search box through Avid client applications that located all occurrences of a word or phrase (anything you typed in) within a local and or network-based content archive – similar to a Google search for untranscribed audio and video – be valuable? Search results would jump you directly to each place in the audio where the search term was spoken.

This search would not rely on manual tagging, descriptions, captions, etc. of content, and is independent of any existing metadata (title, tags, descriptions, character names, etc.) that may already be searchable. In fact, dialogue/spoken-word and metadata-based search could be combined to ensure that all information, including what the actors say, is searched together.

Would an audio search be more useful in a particular client application that you use, and if so, which applications? Does a server-based search solution, able to access content on networked drives and from multiple integrated Avid tools on the network be more useful?

Do you have examples of jobs you’re working on in which fast, on-demand access to target content via keyword/phrase search would make it easier or possible to do things that you cannot do today?


Return to posts index

Shane Ross
Re: Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 9:02:14 pm

That would be...pardon my LA vernacular... FREAKING AWESOME! SO many times I work on documentaries that have transcripts, but without timecode. So while I know what they say, finding that is, difficult, to say the least. If I could have a searchable way to find a word or phrase a person says, that would save TONS of time watching clips.

And if you can convince Apple to have a Script Sync feature too... I won't look a gift horse in the mouth.



Shane



GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it.
Read my blog, Little Frog in High Def


Return to posts index

David Roth Weiss
Re: Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 9:55:03 pm

Kip,

There is no doubt that, as Shane mentioned, having spoken text search capabilities in Final Cut Pro would be a real bonus for editors working on documentaries, and I wholeheartedly support any effort you and your company can do to bring that about.

However, while the advantages that your app has over speech-to-text apps seem very nice, your solution ultimately offers only about 50% of total equation that documentary filmmakers really need. Until there is a way to fully integrate searchable transcripts and spoken text search capabilities into a cohesive and comprehensive solution, documentary post-production will unfortunately continue to be both inefficient and antiquated.

Meanwhile, if there is anything I can do to help and/or beta test, please feel free to call on me anytime.

THNX,
David

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


A forum host of Creative COW's Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.


Return to posts index


Atticus Culver-Rease
Re: Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 10:09:44 pm

Yeah, while I can't imagine any sane editor that wouldn't love to have something like this, its usefulness is somewhat crippled unless it also includes the ability to generate transcripts.

That said, even without speech-to-text capability I'd love to have something that was integrated with Final Cut Pro to allow me to speech search all clips in a project, or to restrict the search based on other metadata criteria like reel name, or to only search within a specific bin, etc. It would also be good to have the option to point it at a folder that contains media that isn't necessarily imported into your project and search that way too. But the holy grail would be something that generates timcoded transcripts and then lets you search the transcript to locate bits of speech within the source media.


Return to posts index

Shane Ross
Re: Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 10:17:06 pm

[Atticus Culver-Rease] " its usefulness is somewhat crippled unless it also includes the ability to generate transcripts. "

I disagree. Now you are getting into something that Premiere does...which is FAR different here. Avid editors who load scripts into Script Sync will disagree with this feature being crippled. Heck, even doc editors who have transcripts (done the standard way, sent to a transcription company) find this VERY useful when on an Avid and looking for sound bytes.

And many time the "transcript" made by Premiere is nearly useless, due to the way the speaker talks. PLUS, this technology that he is pitching works one way, not the reverse. You want both ways. I can see wanting that...wanting to save a few bucks on transcripts. Still..."crippled?" I don't see that.




Shane



GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it.
Read my blog, Little Frog in High Def


Return to posts index

David Roth Weiss
Re: Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 10:22:31 pm

[Shane Ross] "And many time the "transcript" made by Premiere is nearly useless, due to the way the speaker talks. PLUS, this technology that he is pitching works one way, not the reverse. You want both ways. I can see wanting that...wanting to save a few bucks on transcripts. Still..."crippled?" I don't see that. "

Shane,

I agree with you both, but I still maintain that until the solution is does the transcript from the spoken word as well as any search capabilities that we are still dealing with an antiquated workflow.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


A forum host of Creative COW's Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.


Return to posts index


Atticus Culver-Rease
Re: Spoken Word Search in Video Editing Applications
on Jan 7, 2010 at 12:59:03 am

Maybe crippled is too strong... what the original post is proposing would definitely be cool regardless. I guess I'm just saying that in a perfect world you would have this functionality coupled with the ability to generate a text transcript. I know that Premiere does this, and from what I've heard their implementation of it isn't that great. I am hopeful that someone else could do it better, because if it worked well it would be a huge timesaver.


Return to posts index

David Roth Weiss
Re: Spoken Word Search in Video Editing Applications
on Jan 6, 2010 at 10:17:29 pm

[Atticus Culver-Rease] "the holy grail would be something that generates timcoded transcripts and then lets you search the transcript to locate bits of speech within the source media."

Yes, but it also has to do so with at least a degree of accuracy that makes it a time saver rather a waste of time.

If you check out my Cow review of Adobe Speech Search at http://library.creativecow.net/articles/weiss_roth_david/speech_search.php you'll see that Speech Search is a great idea, but one whose time has not yet arrived.


David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


A forum host of Creative COW's Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.


Return to posts index

Kip Watters
Re: Spoken Word Search in Video Editing Applications
on Jan 7, 2010 at 3:56:56 pm

Thanks for the input. I agree that a very good transcript, time-aligned accurately and complete, is an ideal scenario. Today's automated transcription technology is very mature, and given the tools I've tested, as well as actual experiences documented on this and other forums, the quality of automated transcripts isnt likely to improve anytime soon.

I'm looking at it from a different angle -- although I have limited experience using video editing and production tools, with a large number of examples I've seen, *some form of text-based artifacts exist if not transcripts -- shooting scripts, rundowns, teleprompter text, CC -- that approximate some portion of what is spoken.

In your experience, what % of the content being edited has or had some form of dialogue in text form?

Our Sync technology is robust in that text can be chopped up, spelled incorrectly, a superset or subset of the audio - and relevant words will still be lined up with the dialogue.

Another question is around what you'd actually like to do with the the time-aligned transctipt once it's produced? I'd ultimately like to see the text included as a native part of any encoded video, enabling jump-to navigation and browsing within a video from any search box (i.e., iTunes, web portals, other content distribution points, allowing the actual audio to be included as another searchable metadata element.

Assuming no text existed that was of high enough quality to be included with the video, I envision including the audio index that Nexidia creates - alternatively included as a native part of an encoded video. The result would be identical to the scenario above -fully searchable audio with higher accuracy and potentially more coverage than a poor transcript.

Another idea thrown out on this forum is to manually transcribe only select sentences/lines in a video and have Sync find those relevant portions to be used as chapter markers in the video. Same scenario exists - any text that can sync'ed should be natively integrated into the video.

What are your thoughts on other uses of the time-aligned text during and after the editing process?


Return to posts index


David Roth Weiss
Re: Spoken Word Search in Video Editing Applications
on Jan 8, 2010 at 12:32:10 am

[Kip Watters] "Another idea thrown out on this forum is to manually transcribe only select sentences/lines in a video and have Sync find those relevant portions to be used as chapter markers in the video. Same scenario exists - any text that can sync'ed should be natively integrated into the video. "

Kip,

This is in fact very similar to the way I do things now when I am not given transcripts. It's not a bad direction to go in, but still not the ideal.

As you said in your opening, "a very good transcript, time-aligned accurately and complete, is an ideal scenario." There simply is still nothing better, and any of the alternatives are a compromise that ultimately probably save neither time nor money.

For me, the two most important reasons a complete transcript are imperative are the following:

1) When dealing with voluminous interviews, anything less than the ideal of a complete transcript is often too difficult to follow.

2) Vetting edited material for legal reasons can only be accomplished by comparing to the context of the complete transcript.

Nonetheless Kip, I still support what your company is proposing. It still sounds like it could be a great addition to FCP.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


A forum host of Creative COW's Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.


Return to posts index

Mark Raudonis
Re: Spoken Word Search in Video Editing Applications
on Jan 7, 2010 at 4:01:53 am

Kip, Where have you been? For years we've been waiting to see Apple adopt a "script sync" like
feature for FCP. You ask are we interested? Of course we are. This is the best news I've read so far this year!

Our company spends thousands of dollars a year generating transcripts of interviews. If we could leverage that investment and somehow make searches more efficient within FCP I would be the first person buy it. I'd be happy to have a more in depth, detailed conversation with you regarding your technology and how it would benefit our workflow.

Feel free to give me a call.

Mark Raudonis
VP Post Production
Bunim-Murray Productions
818 756 5100



Return to posts index

Mitch Jacobson
Re: Spoken Word Search in Video Editing Applications
on Jan 7, 2010 at 2:25:59 pm

YES, Kip Yes!!! Do it! As everyone has pointed out so far, we could use it in the FCP world. If you are looking for votes, you have another one.

Mitch Jacobson
Producer/Editor: Avid & FCP
New York City


Return to posts index


Al Falaschi
Re: Spoken Word Search in Video Editing Applications
on Jan 7, 2010 at 3:37:25 pm

Hi Kip,

I am the product manager for a SaaS based digital asset management system. I'm very interested in this topic. Our customers are large brand name companies with large amounts of digital assets,....images, brochures, audio and video files...

You ask: "Do you have examples of jobs you’re working on in which fast, on-demand access to target content via keyword/phrase search would make it easier or possible to do things that you cannot do today?"

Answer: YES!

I give you a call....!




Return to posts index

Andrew Kimery
Re: Spoken Word Search in Video Editing Applications
on Jan 8, 2010 at 1:06:01 am

Just wanted to chime in that I would be very interested in this software as well. Avid's Script Sync is a big reason I'm probably going to pick up a copy of Avid MC.


-Andrew

3.2GHz 8-core, FCP 6.0.4, 10.5.5
Blackmagic Multibridge Eclipse (6.8.1)



Return to posts index

Joshua Pearson
Re: Spoken Word Search in Video Editing Applications
on Jan 8, 2010 at 4:28:46 pm

YES PLEASE!.... i work on a lot of documentary TV shows... I have the luxury of having transcribers who use software called Pilotware to make nice transcriptions with TC ins at the start of paragraphs, and for the past year or so i have been literally copying and pasting the paragraphs from the transcript into FCP's marker comment box... its actually very nice to use markers as a way of jumping from paragraph to paragraph in the sequence and see in a nutshell & quickly what is being said... and of course it is roughly searchable, you can search for a word or phrase and it will take you to that marker... but it won't highlight the word or anything, and it won't take you to that exact spot in the sequence, you still have to do a little searching...

unfortunately i just yesterday noticed that for some reason FCP 7 now has a limit on the number of characters that will display in the canvas... after pasting a paragraph into the co0mment box, only about half of it displays in the canvas, unlike FCP 6 where it would all display.

There is something nice about being able to see a whole paragraph in a box, as opposed to a "subtitle" type thing, where you'd have to play it out or scroll to see the whole phrase.




Return to posts index


Grady Knight
Re: Spoken Word Search in Video Editing Applications
on Oct 4, 2010 at 6:41:06 am

I work on oral histories and use the new, beta, free download, Adobe Story to transcribe my interviews. I then use Adobe Premiere Pro to attach the script to the video as meta data. Then I export the video to Adobe Encore and output it as an Adobe Flash Video that is fully searchable, just like a text doc.

8mm & 16mm film to DVD


Return to posts index

Ken Isaacson
Re: Spoken Word Search in Video Editing Applications
on May 31, 2011 at 9:13:37 pm

Hello Kip.
Just ran across your post and am wondering if you can help direct me. I am president of a public access community TV station in Wayland, MA. We upload most of our recorded programs to a web server, in addition to broadcasting on the local cable stations.

We've been looking for a way to search these uploaded videos for particular words or phrases. They typically are local governmental meetings. I ran across your article on "spoken-word search" from about 18 months ago.

Is there any direction you can give me towards finding an application that we can utilize?

Many thanks,
Ken Isaacson
President, WayCAM
http://www.waycam.tv


Return to posts index

David Roth Weiss
Re: Spoken Word Search in Video Editing Applications
on May 31, 2011 at 9:46:14 pm

Ken,

GET from AV3 Software ($249) became the working product that Kip alluded to, using technology from Nexidia. If you use Final Cut Pro GET will do the job for you. Avid also has a similar add-on called Phrasefind ($500) to it's latest version of Media Composer, which also uses Nexidia technology. Hope this answers your question.

BTYW, here's a link to my Creative Cow review of GET: http://library.creativecow.net/weiss_roth_david/get-review/1

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
http://www.drwfilms.com

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.


Return to posts index


Ken Isaacson
Re: Spoken Word Search in Video Editing Applications
on Jun 1, 2011 at 2:32:13 am

David:
Thanks so much for taking the time to help me out. GET looks like a great product. I'll look into it further.

Ken


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]