logo

AutomatedVideoInsightExtractor

InfoGenerateCreated ByPackages

The provided code snippet defines a class `VideoTranscriptionAndAnalysisAgent` within a Ruby module `Sublayer::Agents`. This class is designed to handle the processes of transcribing, segmenting, extracting metadata, and analyzing a given video source. The primary purpose of this class is to facilitate the automated processing and analysis of video content in order to extract useful insights.

### Key Functionalities:

1. **Initialization:**
- The class initializes with parameters `video_source` and an optional `mode`, defaulting to `:youtube`. It also sets up internal state flags for tracking the completion of various processing stages.

2. **Trigger and Goal Conditions:**
- `.trigger_on_source_ready`: Initiated when the video source is ready.
- `.goal_condition`: Specifies when the full processing workflow is considered complete, i.e., when transcription, segmentation, and metadata extraction are all completed successfully.

3. **Processing Stages:**
- **Transcription:** Uses `Sublayer::Actions::TranscribeVideoAction` to transcribe the video. Success updates a flag (`@transcription_done`). The method includes a visual spinner to indicate ongoing processing.
- **Segmentation:** Utilizes `Sublayer::Actions::SegmentVideoAction` to segment the video based on the transcription output. Completes with a success flag (`@segmentation_done`) and a spinner for user feedback.
- **Metadata Extraction:** Extracts metadata via `Sublayer::Actions::ExtractMetadataAction`, updating a success flag (`@metadata_extracted`) and providing a visual spinner.

4. **Analysis and Insights Generation:**
- Once the video is transcribed, segmented, and metadata is extracted, the video is further processed to generate scene thumbnails. A `SoftwareIdentificationAnalyzer` examines these thumbnails, and the results are vectorized and stored in `@indexed_data` for insights.

5. **User Feedback:**
- An additional utility method `with_spinner` is introduced to visually indicate processing states to users by displaying a rotating character spinner during time-consuming operations.

This class encapsulates a robust workflow combining multiple actions to automate the analysis of videos, positioning it as an effective tool for video content processing and insights derivation.

module Sublayer
  module Agents
    class VideoTranscriptionAndAnalysisAgent < Base
      def initialize(video_source:, mode: :youtube)
        @video_source = video_source
        @mode = mode
        @indexed_data = {}
        @transcription_done = false
        @segmentation_done = false
        @metadata_extracted = false
      end

      trigger_on_source_ready do
        [@video_source]
      end

      check_transcription_status do
        stdout, stderr, status = Sublayer::Actions::TranscribeVideoAction.new(video_source: @video_source, mode: @mode).call
        puts stdout

        @transcription_output = stdout
        @transcription_done = (status.exitstatus == 0)
        @transcription_done ? puts("Transcription completed") : puts("Transcription failed")
      end

      check_segmentation_status do
        stdout, stderr, status = Sublayer::Actions::SegmentVideoAction.new(transcription_output: @transcription_output).call
        puts stdout

        @segmentation_output = stdout
        @segmentation_done = (status.exitstatus == 0)
        @segmentation_done ? puts("Segmentation completed") : puts("Segmentation failed")
      end

      check_metadata_extraction_status do
        stdout, stderr, status = Sublayer::Actions::ExtractMetadataAction.new(segmentation_output: @segmentation_output).call
        puts stdout

        @metadata_output = stdout
        @metadata_extracted = (status.exitstatus == 0)
        @metadata_extracted ? puts("Metadata extraction completed") : puts("Metadata extraction failed")
      end

      goal_condition do
        @transcription_done && @segmentation_done && @metadata_extracted
      end

      step do
        puts "Processing video"
        scene_thumbnails = Sublayer::Generators::SceneThumbnailGenerator.new(
          video_source: @video_source,
          segmentation_data: @segmentation_output
        ).generate

        analysis = Sublayer::Analyzers::SoftwareIdentificationAnalyzer.new(
          scene_thumbnails: scene_thumbnails
        ).analyze

        vectorized_insights = Sublayer::Actions::VectorizeInsightsAction.new(
          analysis_data: analysis
        ).call

        @indexed_data.merge!(vectorized_insights)
      end

      # New method to add a spinner
      def with_spinner(message)
        chars = ['|', '/', '-', '\\']
        delay = 0.1
        thread = Thread.new do
          while true do
            chars.each do |char|
              print "\r#{message} #{char}"
              sleep(delay)
            end
          end
        end
        yield
      ensure
        thread.kill
        print "\r"
      end

      # Modify check_transcription_status to include the spinner
      def check_transcription_status_with_spinner
        with_spinner('Transcribing video') do
          check_transcription_status
        end
      end

      # Modify check_segmentation_status to include the spinner
      def check_segmentation_status_with_spinner
        with_spinner('Segmenting video') do
          check_segmentation_status
        end
      end

      # Modify check_metadata_extraction_status to include the spinner
      def check_metadata_extraction_status_with_spinner
        with_spinner('Extracting metadata') do
          check_metadata_extraction_status
        end
      end

      # Modify step to include the spinner
      def step_with_spinner
        with_spinner('Processing video and analyzing software identification') do
          step
        end
      end
    end
  end
end