Assembly notes are a vital a part of collaboration, but they usually fall by means of the cracks. Between main discussions, listening carefully, and typing notes, it’s straightforward for key info to slide away unrecorded. Even when notes are captured, they are often disorganized or illegible, rendering them ineffective.
On this submit, we discover the right way to use Amazon Transcribe and Amazon Bedrock to routinely generate clear, concise summaries of video or audio recordings. Whether or not it’s an inside staff assembly, convention session, or earnings name, this method may help you distill hours of content material all the way down to salient factors.
We stroll by means of an answer to transcribe a mission staff assembly and summarize the important thing takeaways with Amazon Bedrock. We additionally talk about how one can customise this resolution for different widespread eventualities like course lectures, interviews, and gross sales calls. Learn on to simplify and automate your note-taking course of.
Resolution overview
By combining Amazon Transcribe and Amazon Bedrock, it can save you time, seize insights, and improve collaboration. Amazon Transcribe is an computerized speech recognition (ASR) service that makes it easy so as to add speech-to-text functionality to purposes. It makes use of superior deep studying applied sciences to precisely transcribe audio into textual content. Amazon Bedrock is a totally managed service that gives a selection of high-performing basis fashions (FMs) from main AI firms like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single API, together with a broad set of capabilities it’s essential to construct generative AI purposes. With Amazon Bedrock, you possibly can simply experiment with quite a lot of prime FMs, and privately customise them together with your information utilizing methods equivalent to fine-tuning and Retrieval Augmented Era (RAG).
The answer offered on this submit is orchestrated utilizing an AWS Step Capabilities state machine that’s triggered while you add a recording to the designated Amazon Easy Storage Service (Amazon S3) bucket. Step Capabilities allows you to create serverless workflows to orchestrate and join elements throughout AWS companies. It handles the underlying complexity so you possibly can concentrate on software logic. It’s helpful for coordinating duties, distributed processing, ETL (extract, rework, and cargo), and enterprise course of automation.
The next diagram illustrates the high-level resolution structure.
The answer workflow consists of the next steps:
A person shops a recording within the S3 asset bucket.
This motion triggers the Step Capabilities transcription and summarization state machine.
As a part of the state machine, an AWS Lambda operate is triggered, which transcribes the recording utilizing Amazon Transcribe and shops the transcription within the asset bucket.
A second Lambda operate retrieves the transcription and generates a abstract utilizing the Anthropic Claude mannequin in Amazon Bedrock.
Lastly, a closing Lambda operate makes use of Amazon Easy Notification Service (Amazon SNS) to ship a abstract of the recording to the recipient.
This resolution is supported in Areas the place Anthropic Claude on Amazon Bedrock is obtainable.
The state machine orchestrates the steps to carry out the precise duties. The next diagram illustrates the detailed course of.
Conditions
Amazon Bedrock customers must request entry to fashions earlier than they’re accessible to be used. It is a one-time motion. For this resolution, you’ll must allow entry to the Anthropic Claude (not Anthropic Claude Instantaneous) mannequin in Amazon Bedrock. For extra info, seek advice from Mannequin entry.
Deploy resolution sources
The answer is deployed utilizing an AWS CloudFormation template, discovered on the GitHub repo, to routinely provision the required sources in your AWS account. The template requires the next parameters:
E mail deal with used to ship abstract – The abstract might be despatched to this deal with. You could acknowledge the preliminary Amazon SNS affirmation e-mail earlier than receiving extra notifications.
Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract.
Run the answer
After you deploy the answer utilizing AWS CloudFormation, full the next steps:
Acknowledge the Amazon SNS e-mail affirmation that it’s best to obtain just a few moments after creating the CloudFormation stack.
On the AWS CloudFormation console, navigate to stack you simply created.
On the stack’s Outputs tab, and search for the worth related to AssetBucketName; it should look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
On the Amazon S3 console, navigate to your asset bucket.
That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.
Add your recording to the recordings folder.
Importing recordings will routinely set off the Step Capabilities state machine. For this instance, we use a pattern staff assembly recording within the sample-recording listing of the GitHub repository.
On the Step Capabilities console, navigate to the summary-generator state machine.
Select the title of the state machine run with the standing Operating.
Right here, you possibly can watch the progress of the state machine because it processes the recording.
After it reaches its Success state, it’s best to obtain an emailed abstract of the recording.
Alternatively, you possibly can navigate to the S3 belongings bucket and consider the transcript there within the transcripts folder.
Overview the abstract
You’ll get the recording abstract emailed to the deal with you supplied while you created the CloudFormation stack. In the event you don’t obtain the e-mail in just a few moments, just be sure you acknowledged the Amazon SNS affirmation e-mail that it’s best to have acquired after you created the stack after which add the recording once more, which is able to set off the abstract course of.
This resolution features a mock staff assembly recording that you need to use to check the answer. The abstract will look just like the next instance. Due to the character of generative AI, nevertheless, your output will look a bit totally different, however the content material must be shut.
Listed below are the important thing factors from the standup:
Joe completed reviewing the present state for job EDU1 and created a brand new job to develop the longer term state. That new job is within the backlog to be prioritized. He’s now beginning EDU2 however is blocked on useful resource choice.
Rob created a tagging technique for SLG1 primarily based on finest practices, however could must coordinate with different groups who’ve created their very own methods, to align on a uniform method. A brand new job was created to coordinate tagging methods.
Rob has made progress debugging for SLG2 however might have extra assist. This job might be moved to Dash 2 to permit time to get additional sources.Subsequent Steps:
Joe to proceed engaged on EDU2 as ready till useful resource choice is set
New job to be prioritized to coordinate tagging methods throughout groups
SLG2 moved to Dash 2
Standups transferring to Mondays beginning subsequent week
Develop the answer
Now that you’ve got a working resolution, listed below are some potential concepts to customise the answer on your particular use instances:
Attempt altering the method to suit your accessible supply content material and desired outputs:
For conditions the place transcripts can be found, create an alternate Step Capabilities workflow to ingest current text-based or PDF-based transcriptions.
As an alternative of utilizing Amazon SNS to inform recipients by way of e-mail, you need to use it to ship the output to a unique endpoint, equivalent to a staff collaboration website, or to the staff’s chat channel.
Attempt altering the abstract directions CloudFormation stack parameter supplied to Amazon Bedrock to supply outputs particular to your use case (that is the generative AI immediate):
When summarizing an organization’s earnings name, you possibly can have the mannequin concentrate on potential promising alternatives, areas of concern, and issues that it’s best to proceed to watch.
If you’re utilizing this to summarize a course lecture, the mannequin may establish upcoming assignments, summarize key ideas, record info, and filter out any small discuss from the recording.
For a similar recording, create totally different summaries for various audiences:
Engineers’ summaries concentrate on design selections, technical challenges, and upcoming deliverables.
Undertaking managers’ summaries concentrate on timelines, prices, deliverables, and motion gadgets.
Undertaking sponsors get a short replace on mission standing and escalations.
For longer recordings, strive producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, it’s possible you’ll wish to alter the max_tokens_to_sample parameter to accommodate totally different content material lengths.
Clear up
To scrub up the answer, delete the CloudFormation stack that you just created earlier. Be aware that deleting the stack is not going to delete the asset bucket. In the event you not want the recordings or transcripts, you possibly can delete this bucket individually. Amazon Transcribe will routinely delete transcription jobs after 90 days, however you possibly can delete these manually earlier than then.
Conclusion
On this submit, we explored the right way to use Amazon Transcribe and Amazon Bedrock to routinely generate clear, concise summaries of video or audio recordings. We encourage you to proceed evaluating Amazon Bedrock, Amazon Transcribe, and different AWS AI companies, like Amazon Textract, Amazon Translate, and Amazon Rekognition, to see how they may help meet what you are promoting targets.
In regards to the Authors
Rob Barnes is a principal advisor for AWS Skilled Companies. He works with our prospects to handle safety and compliance necessities at scale in complicated, multi-account AWS environments by means of automation.
Jason Stehle is a Senior Options Architect at AWS, primarily based within the New England space. He works with prospects to align AWS capabilities with their biggest enterprise challenges. Outdoors of labor, he spends his time constructing issues and watching comedian e book films together with his household.