Recherche avancée

Sur d’autres sites (6745)

• How to Stream Audio from Google Cloud Storage in Chunks and Convert Each Chunk to WAV for Whisper Transcription

  14 novembre 2024, par Douglas Landvik

  I'm working on a project where I need to transcribe audio stored in a Google Cloud Storage bucket using OpenAI's Whisper model. The audio is stored in WebM format with Opus encoding, and due to the file size, I'm streaming the audio in 30-second chunks.

  


  To convert each chunk to WAV (16 kHz, mono, 16-bit PCM) compatible with Whisper, I'm using FFmpeg. The first chunk converts successfully, but subsequent chunks fail to convert. I suspect this is because each chunk lacks the WebM container's header, which FFmpeg needs to interpret the Opus codec correctly.

  


  Here’s a simplified version of my approach :

  


  Download Chunk : I download each chunk from GCS as bytes.
Convert with FFmpeg : I pass the bytes to FFmpeg to convert each chunk from WebM/Opus to WAV.

  


  async def handle_transcription_and_notify(
    consultation_service: ConsultationService,
    consultation_processor: ConsultationProcessor,
    consultation: Consultation,
    language: str,
    notes: str,
    clinic_id: str,
    vet_email: str,
    trace_id: str,
    blob_path: str,
    max_retries: int = 3,
    retry_delay: int = 5,
    max_concurrent_tasks: int = 3
):
    """
    Handles the transcription process by streaming the file from GCS, converting to a compatible format, 
    and notifying the client via WebSocket.
    """
    chunk_duration_sec = 30  # 30 seconds per chunk
    logger.info(f"Starting transcription process for consultation {consultation.consultation_id}",
                extra={'trace_id': trace_id})

    # Initialize GCS client
    service_account_key = os.environ.get('SERVICE_ACCOUNT_KEY_BACKEND')
    if not service_account_key:
        logger.error("Service account key not found in environment variables", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Service account key not found for consultation {consultation.consultation_id}.\nTrace ID: {trace_id}"
        )
        return

    try:
        service_account_info = json.loads(service_account_key)
        credentials = service_account.Credentials.from_service_account_info(service_account_info)
    except Exception as e:
        logger.error(f"Error loading service account credentials: {str(e)}", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error loading service account credentials for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        return

    # Initialize GCS client
    service_account_key = os.environ.get('SERVICE_ACCOUNT_KEY_BACKEND')
    if not service_account_key:
        logger.error("Service account key not found in environment variables", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Service account key not found for consultation {consultation.consultation_id}.\nTrace ID: {trace_id}"
        )
        return

    try:
        service_account_info = json.loads(service_account_key)
        credentials = service_account.Credentials.from_service_account_info(service_account_info)
    except Exception as e:
        logger.error(f"Error loading service account credentials: {str(e)}", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error loading service account credentials for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        return

    storage_client = storage.Client(credentials=credentials)
    bucket_name = 'vetz_consultations'
    blob = storage_client.bucket(bucket_name).get_blob(blob_path)
    bytes_per_second = 16000 * 2  # 32,000 bytes per second
    chunk_size_bytes = 30 * bytes_per_second
    size = blob.size

    async def stream_blob_in_chunks(blob, chunk_size):
        loop = asyncio.get_running_loop()
        start = 0
        size = blob.size
        while start < size:
            end = min(start + chunk_size - 1, size - 1)
            try:
                logger.info(f"Requesting chunk from {start} to {end}", extra={'trace_id': trace_id})
                chunk = await loop.run_in_executor(
                    None, lambda: blob.download_as_bytes(start=start, end=end)
                )
                if not chunk:
                    break
                logger.info(f"Yielding chunk from {start} to {end}, size: {len(chunk)} bytes",
                            extra={'trace_id': trace_id})
                yield chunk
                start += chunk_size
            except Exception as e:
                logger.error(f"Error downloading chunk from {start} to {end}: {str(e)}", exc_info=True,
                             extra={'trace_id': trace_id})
                raise e

    async def convert_to_wav(chunk_bytes, chunk_idx):
        """
        Convert audio chunk to WAV format compatible with Whisper, ensuring it's 16 kHz, mono, and 16-bit PCM.
        """
        try:
            logger.debug(f"Processing chunk {chunk_idx}: size = {len(chunk_bytes)} bytes")

            detected_format = await detect_audio_format(chunk_bytes)
            logger.info(f"Detected audio format for chunk {chunk_idx}: {detected_format}")
            input_io = io.BytesIO(chunk_bytes)
            output_io = io.BytesIO()

            # ffmpeg command to convert webm/opus to WAV with 16 kHz, mono, and 16-bit PCM

            # ffmpeg command with debug information
            ffmpeg_command = [
                "ffmpeg",
                "-loglevel", "debug",
                "-f", "s16le",            # Treat input as raw PCM data
                "-ar", "48000",           # Set input sample rate
                "-ac", "1",               # Set input to mono
                "-i", "pipe:0",
                "-ar", "16000",           # Set output sample rate to 16 kHz
                "-ac", "1",               # Ensure mono output
                "-sample_fmt", "s16",     # Set output format to 16-bit PCM
                "-f", "wav",              # Output as WAV format
                "pipe:1"
            ]

            process = subprocess.Popen(
                ffmpeg_command,
                stdin=subprocess.PIPE,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE
            )

            stdout, stderr = process.communicate(input=input_io.read())

            if process.returncode == 0:
                logger.info(f"FFmpeg conversion completed successfully for chunk {chunk_idx}")
                output_io.write(stdout)
                output_io.seek(0)

                # Save the WAV file locally for listening
                output_dir = "converted_chunks"
                os.makedirs(output_dir, exist_ok=True)
                file_path = os.path.join(output_dir, f"chunk_{chunk_idx}.wav")

                with open(file_path, "wb") as f:
                    f.write(stdout)
                logger.info(f"Chunk {chunk_idx} saved to {file_path}")

                return output_io
            else:
                logger.error(f"FFmpeg failed for chunk {chunk_idx} with return code {process.returncode}")
                logger.error(f"Chunk {chunk_idx} - FFmpeg stderr: {stderr.decode()}")
                return None

        except Exception as e:
            logger.error(f"Unexpected error in FFmpeg conversion for chunk {chunk_idx}: {str(e)}")
            return None

    async def transcribe_chunk(idx, chunk_bytes):
        for attempt in range(1, max_retries + 1):
            try:
                logger.info(f"Transcribing chunk {idx + 1} (attempt {attempt}).", extra={'trace_id': trace_id})

                # Convert to WAV format
                wav_io = await convert_to_wav(chunk_bytes, idx)
                if not wav_io:
                    logger.error(f"Failed to convert chunk {idx + 1} to WAV format.")
                    return ""

                wav_io.name = "chunk.wav"
                chunk_transcription = await consultation_processor.transcribe_audio_whisper(wav_io)
                logger.info(f"Chunk {idx + 1} transcribed successfully.", extra={'trace_id': trace_id})
                return chunk_transcription
            except Exception as e:
                logger.error(f"Error transcribing chunk {idx + 1} (attempt {attempt}): {str(e)}", exc_info=True,
                             extra={'trace_id': trace_id})
                if attempt < max_retries:
                    await asyncio.sleep(retry_delay)
                else:
                    await send_discord_alert(
                        f"Max retries reached for chunk {idx + 1} in consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
                    )
                    return ""  # Return empty string for failed chunk

    await notification_manager.send_personal_message(
        f"Consultation {consultation.consultation_id} is being transcribed.", vet_email
    )

    try:
        idx = 0
        full_transcription = []
        async for chunk in stream_blob_in_chunks(blob, chunk_size_bytes):
            transcription = await transcribe_chunk(idx, chunk)
            if transcription:
                full_transcription.append(transcription)
            idx += 1

        combined_transcription = " ".join(full_transcription)
        consultation.full_transcript = (consultation.full_transcript or "") + " " + combined_transcription
        consultation_service.save_consultation(clinic_id, vet_email, consultation)
        logger.info(f"Transcription saved for consultation {consultation.consultation_id}.",
                    extra={'trace_id': trace_id})

    except Exception as e:
        logger.error(f"Error during transcription process: {str(e)}", exc_info=True, extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error during transcription process for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        return

    await notification_manager.send_personal_message(
        f"Consultation {consultation.consultation_id} has been transcribed.", vet_email
    )

    try:
        template_service = TemplateService()
        medical_record_template = template_service.get_template_by_name(
            consultation.medical_record_template_id).sections

        sections = await consultation_processor.extract_structured_sections(
            transcription=consultation.full_transcript,
            notes=notes,
            language=language,
            template=medical_record_template,
        )
        consultation.sections = sections
        consultation_service.save_consultation(clinic_id, vet_email, consultation)
        logger.info(f"Sections processed for consultation {consultation.consultation_id}.",
                    extra={'trace_id': trace_id})
    except Exception as e:
        logger.error(f"Error processing sections for consultation {consultation.consultation_id}: {str(e)}",
                     exc_info=True, extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error processing sections for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        raise e

    await notification_manager.send_personal_message(
        f"Consultation {consultation.consultation_id} is fully processed.", vet_email
    )
    logger.info(f"Successfully processed consultation {consultation.consultation_id}.",
                extra={'trace_id': trace_id})



  


• The Ultimate Guide to HeatMap Software

  20 septembre 2021, par Ben Erskine — Analytics Tips, Plugins, Heatmaps

  One of the most effective ways to improve the user experience on your website is to use heatmap software. As well as in-depth insight on how to improve your website and funnels, user behaviour analytics complement traditional web metrics with insights from your customers’ point of view. 

  Heatmap software shows actual user behaviour. That means that you have a visual representation of why a customer might not be converting instead of guessing. 

  By tracking clicks, mouse movement, and page scrolling as well as analysing above the fold content engagement and overall session recordings, heatmap software helps improve user experience and therefore customer retention and conversions.  

  

  What is heatmap software ?

  Heatmap software is a data visualisation tool that uses colour to show what actions a user is taking on a website. 

  If there is a design element on a page that many users engage with, it will show as red/hot. For elements that are less engaging, it will show on the analysis as blue/cold. 

   

  Heatmap software like Matomo helps businesses to improve user experience and increase conversions by tracking elements such as :

  • Flow through sales funnels 
  • Individual session recordings for user experience 
  • A/B page comparisons 
  • Form analytics for sign ups and conversions 
  • Media interaction such as instruction or sales videos
  • Clicks on important links and information 

  Using data visualisation software like a heatmap provides more in-depth data when combined with standard website metrics. 

  What is heatmap software used for ?

  Heatmap software tracks website user behaviour to improve website performance and increase conversions. 

  Heatmaps can show you a detailed analysis of : 

  • Where visitors are clicking (or not clicking) 
  • Where visitors are hovering with their mouse
  • How far users are scrolling or stopping 
  • Where the focus is above the fold 
  • What roadblocks or frictions customers are facing in the sales funnel

  Analysing activity on your website and across channels from your customers point of view is critical in developing a customer-centric business model. 

  This is because heatmaps not only show you what customers are doing but why they are doing it. 

  Heatmap software is ideal for businesses updating and redesigning websites. It also helps to answer important growth questions such as “how can we improve our user experience ?” and “why is our sales funnel not converting better ?”. 

  The benefits of using data visualisation like heatmaps for your website

  Heatmaps are critical for improving websites because they drastically improve customer experience. 

  Customer experience is one of the most important factors in modern business success. A Walker study found that customer experience is one of the biggest differentiators between brands, overtaking other factors such as price. 

  Where straightforward website metrics show customers left a page without action, data visualisation and session recordings show what happens in between them arriving and leaving. This gives web developers and marketers invaluable insights to improve website design and ultimately increase conversions. 

  How heatmap software improves your website and conversions

  There are a few key ways that heatmap software boosts website performance and conversions. All of them focus on both creating a seamless buyer journey and using data to improve results over time. 

  How heatmap software improves conversions ; 

  • By improving UX and usability– 70% of online businesses fail due to bad usability. Heatmaps identify user frustrations and optimise accordingly 
  • By improving content structure – Heatmaps take the guesswork out of design layout and content structure by showing real visitor experiences on your website 
  • By comparing A/B landing pages – Using heatmaps on alternate landing pages can show you why conversions are working or not working based on user activity on the page
  • By optimising across devices – See how your visitors are interacting with your content to learn how well optimised your website is for various devices and remove roadblocks 

  Heatmap analytics you need to improve website user experience

  Click heatmap

  Click heatmaps are useful for two key reasons.

  Firstly, it shows where website users are clicking. 

  Heatmaps that show clicks give you a visual representation of whether copy and CTA links are clear from the customers’ point of view. It can also show whether a customer is clicking on a design feature that doesn’t link anywhere. 

  Secondly, it shows where website users are not clicking. This is just as important when developing funnels and improving user experiences.

  For example, you may have a CTA button for a free trial or purchase. A click heatmap analysis would show if this isn’t clicked on mobile devices and informs developers that it needs to be more mobile-friendly.

  

  Mouse move or hover heatmap

  Like a click heatmap, a mouse hover heatmap shows how you can improve the overall user experience.

  For example, hover heatmaps identify where your visitors engage on a particular webpage. Ideally, of course, you want them to engage with CTAs. Analysing their mouse movements or where they are hovering for more information gives you an indication of any page elements that are distracting them or not working.

  

  Scroll heatmap

  A scroll heatmap uses colours to visualise how far down in a page your visitors scroll. For most web pages, the top will have the most impressions and will naturally get less views (i.e. get “colder” on the heatmap) further down the page. 

  This lets you find out if there is important content positioned too far down the page or if the page is designed to encourage users to keep scrolling.

  No matter how good your product or service is, it won’t convert if potential customers aren’t engaged and scrolling far enough to see it.

  

  Above the fold analysis 

  Above the fold is the content that a visitor sees without scrolling. 

  In a heatmap, the “Average Above the Fold” line will show you how much content your visitors see on average when they open your page. It also shows whether the page design is engaging, whether it encourages visitors to keep scrolling, and whether important information is too far down the page and therefore being missed. 

  Above the fold analysis is arguably the most important as this is the section that the highest number of traffic will see. Using this information ensures that the right content for conversion is seen by the highest number of visitors. 

  

  Session recording

  Session Recording lets you record a real visitor session, so you can see clicks, mouse movements, scrolls, window resizes, page changes, and form interactions all in one. 

  They allow you to understand the experience from the point of view of your visitor and then optimise your website to maximise your success.

  Heatmap software like Matomo takes this one step further and allows you to gather session recordings for individual segments. By analysing sessions based on segments, you can further personalise and optimise based on customer history and patterns.

  

  Final thoughts on heatmap software 

  Heatmap software improves your user experience by easily spotting critical issues that you can then address. 

  As well as that, heatmap analytics like clicks, mouse movement, scroll, above the fold analysis and session recordings increase your marketing ROI by making the most of your existing traffic. 

  It’s a win-win ! 

  Now that you know what heatmap software is, the benefits of using heatmaps on your website and how it can improve your user experience, check out this user guide on heatmap analytics. 

• Upload part of a video files to server

  10 septembre 2014, par Peter Lur

  I want to make a webpage where you can upload a video to the server (let say using a File Input HTML TAG). I would like to know if it’s possible in ANY languages to upload only small parts of the video files without uploading the whole file first and then splitting the file.

  Let me give you an example of what I am trying to achieve. I am trying to create a service where one upload a long & big video file and I give them 10 samples segment of the video file as a result.

  Let say I have a 1 hours long & /1 Gig size video (avi) file.

  I would like to get a sample of 10 segments of 1 minutes spread across the entire file uploaded to the server.

  I know that I could upload the whole file and then split it but uploading a 1G file would be way long for the user.

  I want to know if upfront there is a way to choose what part of the file need to be uploaded so I could upload only specific segments and then re-construct the files to make them readable.

  I know this is more of a subjective question and it may not be well fitted for Stack Overflow but I really need to figure out how to do that.