Recherche avancée

Sur d’autres sites (5994)

• Rails 5 - Concurrent large Video uploads using Carrierwave eats up the server memory/space

  22 mars 2020, par Milind

  I have a working Rails 5 apps using Reactjs for frontend and React dropzone uploader to upload video files using carrierwave.

  So far, what is working great is listed below -

  1. User can upload videos and videos are encoded based on the selection made by user - HLS or MPEG-DASH for online streaming.
  2. Once the video is uploaded on the server, it starts streaming it by :-
     • FIRST,copying video on /tmp folder.
     • Running a bash script that uses ffmpeg to transcode uploaded video using predefined commands to produce new fragments of videos inside /tmp folder.
     • Once the background job is done, all the videos are uploaded on AWS S3, which is how the default carrierwave works
  3. So, when multiple videos are uploaded, they are all copied in /tmp folder and then transcoded and eventually uploaded to S3.

  My questions, where i am looking some help are listed below -

  1- The above process is good for small videos, BUT what if there are many concurrent users uploading 2GB of videos ? I know this will kill my server as my /tmp folder will keep on increasing and consume all the memory, making it to die hard.How can I allow concurrent videos to upload videos without effecting my server’s memory consumption ?

  2- Is there a way where I can directly upload the videos on AWS-S3 first, and then use one more proxy server/child application to encode videos from S3, download it to the child server, convert it and again upload it to the destination ? but this is almost the same but doing it on cloud, where memory consumption can be on-demand but will be not cost-effective.

  3- Is there some easy and cost-effective way by which I can upload large videos, transcode them and upload it to AWS S3, without effecting my server memory. Am i missing some technical architecture here.

  4- How Youtube/Netflix works, I know they do the same thing in a smart way but can someone help me to improve this ?

  Thanks in advance.

• Problems with Python's azure.cognitiveservices.speech when installing together with FFmpeg in a Linux web app

  15 mai 2024, par Kakobo kakobo

  I need some help.
I'm building an web app that takes any audio format, converts into a .wav file and then passes it to 'azure.cognitiveservices.speech' for transcription.I'm building the web app via a container Dockerfile as I need to install ffmpeg to be able to convert non ".wav" audio files to ".wav" (as azure speech services only process wav files). For some odd reason, the 'speechsdk' class of 'azure.cognitiveservices.speech' fails to work when I install ffmpeg in the web app. The class works perfectly fine when I install it without ffpmeg or when i build and run the container in my machine.

  


  I have placed debug print statements in the code. I can see the class initiating, for some reason it does not buffer in the same when when running it locally in my machine. The routine simply stops without any reason.

  


  Has anybody experienced a similar issue with azure.cognitiveservices.speech conflicting with ffmpeg ?

  


  Here's my Dockerfile :

  


  # Use an official Python runtime as a parent imageFROM python:3.11-slim

#Version RunRUN echo "Version Run 1..."

Install ffmpeg

RUN apt-get update && apt-get install -y ffmpeg && # Ensure ffmpeg is executablechmod a+rx /usr/bin/ffmpeg && # Clean up the apt cache by removing /var/lib/apt/lists saves spaceapt-get clean && rm -rf /var/lib/apt/lists/*

//Set the working directory in the container

WORKDIR /app

//Copy the current directory contents into the container at /app

COPY . /app

//Install any needed packages specified in requirements.txt

RUN pip install --no-cache-dir -r requirements.txt

//Make port 80 available to the world outside this container

EXPOSE 8000

//Define environment variable

ENV NAME World

//Run main.py when the container launches

CMD ["streamlit", "run", "main.py", "--server.port", "8000", "--server.address", "0.0.0.0"]`and here's my python code:


  


  def transcribe_audio_continuous_old(temp_dir, audio_file, language):
    speech_key = azure_speech_key
    service_region = azure_speech_region

    time.sleep(5)
    print(f"DEBUG TIME BEFORE speechconfig")

    ran = generate_random_string(length=5)
    temp_file = f"transcript_key_{ran}.txt"
    output_text_file = os.path.join(temp_dir, temp_file)
    speech_recognition_language = set_language_to_speech_code(language)
    
    speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region)
    speech_config.speech_recognition_language = speech_recognition_language
    audio_input = speechsdk.AudioConfig(filename=os.path.join(temp_dir, audio_file))
        
    speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_input, language=speech_recognition_language)
    done = False
    transcript_contents = ""

    time.sleep(5)
    print(f"DEBUG TIME AFTER speechconfig")
    print(f"DEBUG FIle about to be passed {audio_file}")

    try:
        with open(output_text_file, "w", encoding=encoding) as file:
            def recognized_callback(evt):
                print("Start continuous recognition callback.")
                print(f"Recognized: {evt.result.text}")
                file.write(evt.result.text + "\n")
                nonlocal transcript_contents
                transcript_contents += evt.result.text + "\n"

            def stop_cb(evt):
                print("Stopping continuous recognition callback.")
                print(f"Event type: {evt}")
                speech_recognizer.stop_continuous_recognition()
                nonlocal done
                done = True
            
            def canceled_cb(evt):
                print(f"Recognition canceled: {evt.reason}")
                if evt.reason == speechsdk.CancellationReason.Error:
                    print(f"Cancellation error: {evt.error_details}")
                nonlocal done
                done = True

            speech_recognizer.recognized.connect(recognized_callback)
            speech_recognizer.session_stopped.connect(stop_cb)
            speech_recognizer.canceled.connect(canceled_cb)

            speech_recognizer.start_continuous_recognition()
            while not done:
                time.sleep(1)
                print("DEBUG LOOPING TRANSCRIPT")

    except Exception as e:
        print(f"An error occurred: {e}")

    print("DEBUG DONE TRANSCRIPT")

    return temp_file, transcript_contents


  


  The transcript this callback works fine locally, or when installed without ffmpeg in the linux web app. Not sure why it conflicts with ffmpeg when installed via container dockerfile. The code section that fails can me found on note #NOTE DEBUG"

  


• avcodec/nvdec : Round up odd width/height values

  24 novembre 2017, par Philip Langdale

  avcodec/nvdec : Round up odd width/height values
  nvdec will not produce odd width/height output, and while this is
  
  basically never an issue with most codecs, due to internal alignment
  
  requirements, you can get odd sized jpegs.
  If an odd-sized jpeg is encountered, nvdec will actually round down
  
  internally and produce output that is slightly smaller. This isn't
  
  the end of the world, as long as you know the output size doesn't
  
  match the original image resolution.
  However, with an hwaccel, we don't know. The decoder controls
  
  the reported output size and the hwaccel cannot change it. I was
  
  able to trigger an error in mpv where it tries to copy the output
  
  surface as part of rendering and triggers a cuda error because
  
  cuda knows the output frame is smaller than expected.
  To fix this, we can round up the configured width/height passed
  
  to nvdec so that the frames are always at least as large as the
  
  decoder's reported size, and data can be copied out safely.
  In this particular jpeg case, you end up with a blank (green) line
  
  at the bottom due to nvdec refusing to decode the last line, but
  
  the behaviour matches cuviddec, so it's as good as you're going to
  
  get.
  

  • [DH] libavcodec/nvdec.c