Recherche avancée

Sur d’autres sites (10110)

• avformat/iamf : fix setting channel layout for Scalable layers

  17 juin, par James Almer

  avformat/iamf : fix setting channel layout for Scalable layers
  The way streams are coded in an IAMF struct follows a scalable model where the
  
  channel layouts for each layer may not match the channel order our API can
  
  represent in a Native order layout.
  For example, an audio element may have six coded streams in the form of two
  
  stereo streams, followed by two mono streams, and then by another two stereo
  
  streams, for a total of 10 channels, and define for them four scalable layers
  
  with loudspeaker_layout values "Stereo", "5.1ch", "5.1.2ch", and "5.1.4ch".
  
  The first layer references the first stream, and each following layer will
  
  reference all previous streams plus extra ones.
  
  In this case, the "5.1ch" layer will reference four streams (the first two
  
  stereo and the two mono) to encompass six channels, which does not match out
  
  native layout 5.1(side) given that FC and LFE come after FL+FR but before
  
  SL+SR, and here, they are at the end.
  For this reason, we need to build Custom order layouts that properly represent
  
  what we're exporting.
  
  Before :
    Stream group #0:0[0x12c] : IAMF Audio Element :
  
      Layer 0 : stereo
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
      Layer 1 : 5.1(side)
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
        Stream #0:1[0x1] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:2[0x2] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:3[0x3] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
      Layer 2 : 5.1.2
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
        Stream #0:1[0x1] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:2[0x2] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:3[0x3] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:4[0x4] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
      Layer 3 : 5.1.4
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
        Stream #0:1[0x1] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:2[0x2] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:3[0x3] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:4[0x4] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:5[0x5] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
  AFter :
    Stream group #0:0[0x12c] : IAMF Audio Element :
  
      Layer 0 : stereo
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
      Layer 1 : 6 channels (FL+FR+SL+SR+FC+LFE)
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
        Stream #0:1[0x1] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:2[0x2] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:3[0x3] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
      Layer 2 : 8 channels (FL+FR+SL+SR+FC+LFE+TFL+TFR)
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
        Stream #0:1[0x1] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:2[0x2] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:3[0x3] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:4[0x4] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
      Layer 3 : 10 channels (FL+FR+SL+SR+FC+LFE+TFL+TFR+TBL+TBR)
  
        Stream #0:0[0x0] : Audio : opus, 48000 Hz, stereo, fltp (default)
  
        Stream #0:1[0x1] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:2[0x2] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:3[0x3] : Audio : opus, 48000 Hz, mono, fltp (dependent)
  
        Stream #0:4[0x4] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  
        Stream #0:5[0x5] : Audio : opus, 48000 Hz, stereo, fltp (dependent)
  Signed-off-by : James Almer <jamrial@gmail.com>
  

  • [DH] libavformat/iamf_parse.c
  • [DH] libavformat/iamf_writer.c
  • [DH] libavformat/iamfdec.c
  • [DH] tests/ref/fate/iamf-5_1-copy
  • [DH] tests/ref/fate/iamf-5_1-demux
  • [DH] tests/ref/fate/iamf-5_1_4
  • [DH] tests/ref/fate/iamf-7_1_4
  • [DH] tests/ref/fate/iamf-9_1_6
  • [DH] tests/ref/fate/mov-mp4-iamf-5_1_4
  • [DH] tests/ref/fate/mov-mp4-iamf-7_1_4-video-first
  • [DH] tests/ref/fate/mov-mp4-iamf-7_1_4-video-last

• How to fix av_interleaved_write_frame() broken pipe error in php

  31 mars, par Adekunle Adeyeye

  I have an issue using ffmpeg to stream audio and parse to google cloud speech to text in PHP.

  


  It returns this output.
I have tried delaying some part of the script, that did not solve it.
I have also checked for similar questions. however, they are mostly in python and none of the solutions actually work for this.

  


    built with gcc 8 (GCC)
  cpudetect
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mp3, from 'https://npr-ice.streamguys1.com/live.mp3':
  Metadata:
    icy-br          : 96
    icy-description : NPR Program Stream
    icy-genre       : News and Talk
    icy-name        : NPR Program Stream
    icy-pub         : 0
    StreamTitle     :
  Duration: N/A, start: 0.000000, bitrate: 96 kb/s
    Stream #0:0: Audio: mp3, 32000 Hz, stereo, fltp, 96 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (mp3 (mp3float) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, s16le, to 'pipe:':
  Metadata:
    icy-br          : 96
    icy-description : NPR Program Stream
    icy-genre       : News and Talk
    icy-name        : NPR Program Stream
    icy-pub         : 0
    StreamTitle     :
    encoder         : Lavf58.29.100
    Stream #0:0: Audio: pcm_s16le, 16000 Hz, mono, s16, 256 kb/s
    Metadata:
      encoder         : Lavc58.54.100 pcm_s16le
**av_interleaved_write_frame(): Broken pipe** 256.0kbits/s speed=1.02x
**Error writing trailer of pipe:: Broken pipe**
size=      54kB time=00:00:01.76 bitrate= 250.8kbits/s speed=0.465x
video:0kB audio:55kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!


  


  this is my PHP code

  


  require_once 'vendor/autoload.php';
    
    $projectId = "xxx-45512";
    putenv('GOOGLE_APPLICATION_CREDENTIALS=' . __DIR__ . '/xxx-45512-be3eb805f1d7.json');
    
    // Database connection
    $pdo = new PDO('mysql:host=localhost;dbname=', '', '');
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    
    $url = "https://npr-ice.streamguys1.com/live.mp3";
    
    $ffmpegCmd = "ffmpeg -re -i $url -acodec pcm_s16le -ac 1 -ar 16000 -f s16le -";
    
    $fp = popen($ffmpegCmd, "r");
    if (!$fp) {
        die("Failed to open FFmpeg stream.");
    }
    sleep(5);

    try {
        $client = new SpeechClient(['transport' => 'grpc', 'credentials' => json_decode(file_get_contents(getenv('GOOGLE_APPLICATION_CREDENTIALS')), true)]);
    } catch (Exception $e) {
        echo 'Error: ' . $e->getMessage(); 
        exit;
    }
    
    $recognitionConfig = new RecognitionConfig([
        'auto_decoding_config' => new AutoDetectDecodingConfig(),
        'language_codes' => ['en-US'],
        'model' => 'long',
    ]);
    
    $streamingConfig = new StreamingRecognitionConfig([
        'config' => $recognitionConfig,
    ]);
    
    $configRequest = new StreamingRecognizeRequest([
        'recognizer' => "projects/$projectId/locations/global/recognizers/_",
        'streaming_config' => $streamingConfig,
    ]);
    
    
    function streamAudio($fp)
    {
        while (!feof($fp)) {
            yield fread($fp, 4096);
        }
    }
    
    $responses = $client->streamingRecognize([
    'requests' => (function () use ($configRequest, $fp) {
            yield $configRequest; // Send initial config
            foreach (streamAudio($fp) as $audioChunk) {
                yield new StreamingRecognizeRequest(['audio' => $audioChunk]);
            }
        })()]
    );
    
    // $responses = $speechClient->streamingRecognize();
    // $responses->writeAll([$request,]);
    
    foreach ($responses as $response) {
        foreach ($response->getResults() as $result) {
            $transcript = $result->getAlternatives()[0]->getTranscript();
            // echo "Transcript: $transcript\n";
    
            // Insert into the database
            $stmt = $pdo->prepare("INSERT INTO transcriptions (transcript) VALUES (:transcript)");
            $stmt->execute(['transcript' => $transcript]);
        }
    }
    
    
    pclose($fp);
    $client->close();


  


  I'm not sure what the issue is at this time.

  


  UPDATE

  


  I've done some more debugging and i have gotten the error to clear and to stream actually starts.
However, I expect the audio to transcribe and update my database but instead I get this error when i close the stream

  


  

  


  this is my updated code

  


      $handle = popen($ffmpegCommand, "r");

    try {
        $client = new SpeechClient(['transport' => 'grpc', 'credentials' => json_decode(file_get_contents(getenv('GOOGLE_APPLICATION_CREDENTIALS')), true)]);
    } catch (Exception $e) {
        echo 'Error: ' . $e->getMessage(); 
        exit;
    }
    
    try {
    $recognitionConfig = (new RecognitionConfig())
        ->setAutoDecodingConfig(new AutoDetectDecodingConfig())
        ->setLanguageCodes(['en-US'], ['en-UK'])
        ->setModel('long');
    } catch (Exception $e) {
        echo 'Error: ' . $e->getMessage(); 
        exit;
    }
    
    try {
        $streamConfig = (new StreamingRecognitionConfig())
        ->setConfig($recognitionConfig);
    } catch (Exception $e) {
        echo 'Error: ' . $e->getMessage();
        exit;
    }
    try {
        $configRequest = (new StreamingRecognizeRequest())
        ->setRecognizer("projects/$projectId/locations/global/recognizers/_")
        ->setStreamingConfig($streamConfig);
    } catch (Exception $e) {
        echo 'Error: ' . $e->getMessage(); 
        exit;
    }
    
    $stream = $client->streamingRecognize();
    $stream->write($configRequest);
    
    mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('bef')");
    
    while (!feof($handle)) {
        $chunk = fread($handle, 25600);
        // printf('chunk: ' . $chunk);
        if ($chunk !== false) {
            try {
                $request = (new StreamingRecognizeRequest())
                        ->setAudio($chunk);
                    $stream->write($request);
            } catch (Exception $e) {
                printf('Errorc: ' . $e->getMessage());
            }
        }
    }
    
    
    $insr = json_encode($stream);
    mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('$insr')");
    
    foreach ($stream->read() as $response) {
        mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('loop1')");
        foreach ($response->getResults() as $result) {
            mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('loop2')");
            foreach ($result->getAlternatives() as $alternative) {
                $trans = $alternative->getTranscript();
                mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('$trans')");
            }
        }
    }
    
    pclose($handle);
    $stream->close();
    $client->close();```


  


• avformat/iamf_writer : add support for expanded channel layouts

  10 décembre 2024, par James Almer

  avformat/iamf_writer : add support for expanded channel layouts
  Defined in Immersive Audio Model and Formats 1.1.0, sections 3.6.2 and 3.7.3
  Signed-off-by : James Almer <jamrial@gmail.com>
  

  • [DH] libavformat/iamf_writer.c