Recherche avancée

Sur d’autres sites (9539)

• Transcoding & displaying user-uploaded videos for cross-browser/platform compatibility

  5 avril 2012, par Jonathan Amend

  I want to display user-uploaded videos in high quality on different browsers/platforms. I think I have a pretty good start, but there are a few issues. I have at my disposal ffmpeg 0.10 (called from PHP) and jwPlayer 5.8 (licensed).

  Ideally I would like to support :

  • Browsers with Flash 9+
  • Modern Firefox/Chrome/Safari without Flash
  • iOS 4.1+
  • Android 2.2+

  My current transcoding commands are :

  $commands = array(
  
      'flv' => "/usr/bin/ffmpeg -y -i {$sourceFile} -b 500k -ar 22050 -ab 64 {$tmpFileName}.flv 2>&1",
  
      'webm' => "/usr/bin/ffmpeg -i {$sourceFile} -acodec libvorbis -ac 2 -ab 96k -ar 44100 -b 500k {$tmpFileName}.webm 2>&1",
  
      'mp4' => "/usr/bin/ffmpeg -i {$sourceFile} -acodec libfaac -ab 96k -vcodec libx264 -level 21 -refs 2 -vf 'scale=trunc(ih*a/2)*2:trunc(iw/a/2)*2' -b 500k -bt 1000k -threads 0 -f mp4 {$tmpFileName}.pre.mp4 2>&1",
  
      'jpg' => "/usr/bin/ffmpeg -y -i {$sourceFile} -ss 5 -vcodec mjpeg -vframes 1 -an -f rawvideo {$tmpFileName}.jpg 2>&1"
  
  );
  
  
  
  $fastStartCommand = "/usr/bin/qt-faststart {$tmpFileName}.pre.mp4 {$tmpFileName}.mp4");

  My current display code is this :

  <video controls="controls" width="{$width}" height="{$height}" poster="{$fileJPG}" style="width: {$width}px; height: {$height}px;">
  
      <source src="{$fileMP4}" type="video/mp4" data-ext="mp4"></source>
  
      <source src="{$fileWEBM}" type="video/webm" data-ext="webm"></source>
  
      <a href="http://stackoverflow.com/feeds/tag/{$fileMP4}" title="Play Video">
  
          <img border="0" src="http://stackoverflow.com/feeds/tag/{$fileJPG}" width='0' height='0' alt="Play Video" />
  
      </a>
  
  </video>
  
  <code class="echappe-js">&lt;script type=&quot;text/javascript&quot;&gt;<br />
  
  swfobject.embedSWF(<br />
  
      &amp;#39;/flash/jwPlayer.swf&amp;#39;,<br />
  
      &amp;#39;video&amp;#39;,<br />
  
      {$width},<br />
  
      {$height},<br />
  
      &amp;#39;9.0.0&amp;#39;,<br />
  
      &amp;#39;&amp;#39;,<br />
  
      {<br />
  
          file: &amp;#39;{$fileFLV}&amp;#39;,<br />
  
          width: {$width},<br />
  
          height: {$height},<br />
  
          provider: &amp;#39;video&amp;#39;,<br />
  
          stretching: &amp;#39;uniform&amp;#39;,<br />
  
          smoothing: &amp;#39;true&amp;#39;,<br />
  
          dock: &amp;#39;true&amp;#39;<br />
  
      },<br />
  
      {<br />
  
          menu: &amp;#39;false&amp;#39;,<br />
  
          allowfullscreen: &amp;#39;true&amp;#39;,<br />
  
          allowscriptaccess: &amp;#39;always&amp;#39;,<br />
  
          allownetworking: &amp;#39;always&amp;#39;,<br />
  
          wmode: &amp;#39;transparent&amp;#39;<br />
  
      }<br />
  
  );<br />
  
  &lt;/script&gt;
  
  

  The idea is to prefer the Flash player (we have many custom skins that we would like to keep using), then fall back to HTML 5 video, and if that fails, show the JPEG with a link to the mp4 file (that seems to work as a last-ditch effort for iOS 2/3 and old browsers so they can at least play the video by launching QuickTime).

  The main issues that I'm not sure how to solve right now are :

  • How can I show a higher quality video in Flash ? I tried using the mp4 video instead of the flv but it doesn't work and I only get sound, no video. I think Flash is supposed to support h.264 these days, but how ? And do I have to use a different transcoding method for older and newer versions of Flash ?
  • How can I transcode the video with better quality ? Right now I am using the same bitrate for all resolutions. Any chance there is a way for ffmpeg to adjust the bitrate automatically ? Or can I detect and match the resolution of the uploaded video to a list of resolution -> bitrates somehow ?

• RTP packets detected as UDP

  8 juillet 2024, par fritz

  Here is what I am trying to do :

  



  WebRTC endpoint > RTP Endpoint > ffmpeg > RTMP server.


  



  This is what my SDP file looks like.

  



  var cm_offer = "v=0\n" +
              "o=- 3641290734 3641290734 IN IP4 127.0.0.1\n" +
              "s=nginx\n" +
              "c=IN IP4 127.0.0.1\n" +
              "t=0 0\n" +
              "m=audio 60820 RTP/AVP 0\n" +
              "a=rtpmap:0 PCMU/8000\n" +
              "a=recvonly\n" +
              "m=video 59618 RTP/AVP 101\n" +
              "a=rtpmap:101 H264/90000\n" +
              "a=recvonly\n";


  



  What's happening is that wireshark can detect the incoming packets at port 59618, but not as RTP packets but UDP packets. I am trying to capture the packets using ffmpeg with the following command :

  



  ubuntu@ip-132-31-40-100:~$ ffmpeg -i udp://127.0.0.1:59618 -vcodec copy stream.mp4
ffmpeg version git-2017-01-22-f1214ad Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
  libavutil      55. 44.100 / 55. 44.100
  libavcodec     57. 75.100 / 57. 75.100
  libavformat    57. 63.100 / 57. 63.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 69.100 /  6. 69.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100 


  



  All I get is a blinking cursor and The stream.mp4 file is not written to disk after I exit (ctrl+c).

  



  So can you help me figure out :

  



  

  1. why wireshark cannot detect the packets as RTP (I suspect it has something to do with SDP)

  


  2. How to handle SDP answer when the RTP endpoint is pushing to ffmpeg which doesn't send an answer back.

  


  



  Here is the entire code (hello world tutorial modified)

  



  /*
     * (C) Copyright 2014-2015 Kurento (http://kurento.org/)
     *
     * Licensed under the Apache License, Version 2.0 (the "License");
     * you may not use this file except in compliance with the License.
     * You may obtain a copy of the License at
     *
     *   http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */

    function getopts(args, opts)
    {
      var result = opts.default || {};
      args.replace(
          new RegExp("([^?=&]+)(=([^&]*))?", "g"),
          function($0, $1, $2, $3) { result[$1] = decodeURI($3); });

      return result;
    };

    var args = getopts(location.search,
    {
      default:
      {
        ws_uri: 'wss://' + location.hostname + ':8433/kurento',
        ice_servers: undefined
      }
    });

    function setIceCandidateCallbacks(webRtcPeer, webRtcEp, onerror)
    {
      webRtcPeer.on('icecandidate', function(candidate) {
        console.log("Local candidate:",candidate);

        candidate = kurentoClient.getComplexType('IceCandidate')(candidate);

        webRtcEp.addIceCandidate(candidate, onerror)
      });

      webRtcEp.on('OnIceCandidate', function(event) {
        var candidate = event.candidate;

        console.log("Remote candidate:",candidate);

        webRtcPeer.addIceCandidate(candidate, onerror);
      });
    }


    function setIceCandidateCallbacks2(webRtcPeer, rtpEp, onerror)
    {
      webRtcPeer.on('icecandidate', function(candidate) {
        console.log("Localr candidate:",candidate);

        candidate = kurentoClient.getComplexType('IceCandidate')(candidate);

        rtpEp.addIceCandidate(candidate, onerror)
      });
    }


    window.addEventListener('load', function()
    {
      console = new Console();

      var webRtcPeer;
      var pipeline;
      var webRtcEpt;

      var videoInput = document.getElementById('videoInput');
      var videoOutput = document.getElementById('videoOutput');

      var startButton = document.getElementById("start");
      var stopButton = document.getElementById("stop");

      startButton.addEventListener("click", function()
      {
        showSpinner(videoInput, videoOutput);

        var options = {
          localVideo: videoInput,
          remoteVideo: videoOutput
        };


        if (args.ice_servers) {
         console.log("Use ICE servers: " + args.ice_servers);
         options.configuration = {
           iceServers : JSON.parse(args.ice_servers)
         };
        } else {
         console.log("Use freeice")
        }

        webRtcPeer = kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error)
        {
          if(error) return onError(error)

          this.generateOffer(onOffer)
        });

        function onOffer(error, sdpOffer)
        {
          if(error) return onError(error)

          kurentoClient(args.ws_uri, function(error, client)
          {
            if(error) return onError(error);

            client.create("MediaPipeline", function(error, _pipeline)
            {
              if(error) return onError(error);

              pipeline = _pipeline;

              pipeline.create("WebRtcEndpoint", function(error, webRtc){
                if(error) return onError(error);

                webRtcEpt = webRtc;

                setIceCandidateCallbacks(webRtcPeer, webRtc, onError)

                webRtc.processOffer(sdpOffer, function(error, sdpAnswer){
                  if(error) return onError(error);

                  webRtcPeer.processAnswer(sdpAnswer, onError);
                });
                webRtc.gatherCandidates(onError);

                webRtc.connect(webRtc, function(error){
                  if(error) return onError(error);

                  console.log("Loopback established");
                });
              });



            pipeline.create("RtpEndpoint", function(error, rtp){
                if(error) return onError(error);

                //setIceCandidateCallbacks2(webRtcPeer, rtp, onError)


                var cm_offer = "v=0\n" +
                      "o=- 3641290734 3641290734 IN IP4 127.0.0.1\n" +
                      "s=nginx\n" +
                      "c=IN IP4 127.0.0.1\n" +
                      "t=0 0\n" +
                      "m=audio 60820 RTP/AVP 0\n" +
                      "a=rtpmap:0 PCMU/8000\n" +
                      "a=recvonly\n" +
                      "m=video 59618 RTP/AVP 101\n" +
                      "a=rtpmap:101 H264/90000\n" +
                      "a=recvonly\n";



                rtp.processOffer(cm_offer, function(error, cm_sdpAnswer){
                  if(error) return onError(error);

                  //webRtcPeer.processAnswer(cm_sdpAnswer, onError);
                });
                //rtp.gatherCandidates(onError);

                webRtcEpt.connect(rtp, function(error){
                  if(error) return onError(error);

                  console.log("RTP endpoint connected to webRTC");
                });
              });









            });
          });
        }
      });
      stopButton.addEventListener("click", stop);


      function stop() {
        if (webRtcPeer) {
          webRtcPeer.dispose();
          webRtcPeer = null;
        }

        if(pipeline){
          pipeline.release();
          pipeline = null;
        }

        hideSpinner(videoInput, videoOutput);
      }

      function onError(error) {
        if(error)
        {
          console.error(error);
          stop();
        }
      }
    })


    function showSpinner() {
      for (var i = 0; i < arguments.length; i++) {
        arguments[i].poster = 'img/transparent-1px.png';
        arguments[i].style.background = "center transparent url('img/spinner.gif') no-repeat";
      }
    }

    function hideSpinner() {
      for (var i = 0; i < arguments.length; i++) {
        arguments[i].src = '';
        arguments[i].poster = 'img/webrtc.png';
        arguments[i].style.background = '';
      }
    }

    /**
     * Lightbox utility (to display media pipeline image in a modal dialog)
     */
    $(document).delegate('*[data-toggle="lightbox"]', 'click', function(event) {
      event.preventDefault();
      $(this).ekkoLightbox();
    });


  


• RTP packets detected as UDP

  28 février 2017, par user3172852

  Here is what I am trying to do :

  WebRTC endpoint > RTP Endpoint > ffmpeg > RTMP server.

  This is what my SDP file looks like.

  var cm_offer = "v=0\n" +
  
                "o=- 3641290734 3641290734 IN IP4 127.0.0.1\n" +
  
                "s=nginx\n" +
  
                "c=IN IP4 127.0.0.1\n" +
  
                "t=0 0\n" +
  
                "m=audio 60820 RTP/AVP 0\n" +
  
                "a=rtpmap:0 PCMU/8000\n" +
  
                "a=recvonly\n" +
  
                "m=video 59618 RTP/AVP 101\n" +
  
                "a=rtpmap:101 H264/90000\n" +
  
                "a=recvonly\n";

  What’s happening is that wireshark can detect the incoming packets at port 59618, but not as RTP packets but UDP packets. I am trying to capture the packets using ffmpeg with the following command :

  ubuntu@ip-132-31-40-100:~$ ffmpeg -i udp://127.0.0.1:59618 -vcodec copy stream.mp4
  
  ffmpeg version git-2017-01-22-f1214ad Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  
    configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
  
    libavutil      55. 44.100 / 55. 44.100
  
    libavcodec     57. 75.100 / 57. 75.100
  
    libavformat    57. 63.100 / 57. 63.100
  
    libavdevice    57.  2.100 / 57.  2.100
  
    libavfilter     6. 69.100 /  6. 69.100
  
    libavresample   3.  2.  0 /  3.  2.  0
  
    libswscale      4.  3.101 /  4.  3.101
  
    libswresample   2.  4.100 /  2.  4.100
  
    libpostproc    54.  2.100 / 54.  2.100 

  All I get is a blinking cursor and The stream.mp4 file is not written to disk after I exit (ctrl+c).

  So can you help me figure out :

  1. why wireshark cannot detect the packets as RTP (I suspect it has something to do with SDP)
  2. How to handle SDP answer when the RTP endpoint is pushing to ffmpeg which doesn’t send an answer back.

  Here is the entire code (hello world tutorial modified)

  /*
  
       * (C) Copyright 2014-2015 Kurento (http://kurento.org/)
  
       *
  
       * Licensed under the Apache License, Version 2.0 (the "License");
  
       * you may not use this file except in compliance with the License.
  
       * You may obtain a copy of the License at
  
       *
  
       *   http://www.apache.org/licenses/LICENSE-2.0
  
       *
  
       * Unless required by applicable law or agreed to in writing, software
  
       * distributed under the License is distributed on an "AS IS" BASIS,
  
       * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  
       * See the License for the specific language governing permissions and
  
       * limitations under the License.
  
       */
  
  
  
      function getopts(args, opts)
  
      {
  
        var result = opts.default || {};
  
        args.replace(
  
            new RegExp("([^?=&]+)(=([^&]*))?", "g"),
  
            function($0, $1, $2, $3) { result[$1] = decodeURI($3); });
  
  
  
        return result;
  
      };
  
  
  
      var args = getopts(location.search,
  
      {
  
        default:
  
        {
  
          ws_uri: 'wss://' + location.hostname + ':8433/kurento',
  
          ice_servers: undefined
  
        }
  
      });
  
  
  
      function setIceCandidateCallbacks(webRtcPeer, webRtcEp, onerror)
  
      {
  
        webRtcPeer.on('icecandidate', function(candidate) {
  
          console.log("Local candidate:",candidate);
  
  
  
          candidate = kurentoClient.getComplexType('IceCandidate')(candidate);
  
  
  
          webRtcEp.addIceCandidate(candidate, onerror)
  
        });
  
  
  
        webRtcEp.on('OnIceCandidate', function(event) {
  
          var candidate = event.candidate;
  
  
  
          console.log("Remote candidate:",candidate);
  
  
  
          webRtcPeer.addIceCandidate(candidate, onerror);
  
        });
  
      }
  
  
  
  
  
      function setIceCandidateCallbacks2(webRtcPeer, rtpEp, onerror)
  
      {
  
        webRtcPeer.on('icecandidate', function(candidate) {
  
          console.log("Localr candidate:",candidate);
  
  
  
          candidate = kurentoClient.getComplexType('IceCandidate')(candidate);
  
  
  
          rtpEp.addIceCandidate(candidate, onerror)
  
        });
  
      }
  
  
  
  
  
      window.addEventListener('load', function()
  
      {
  
        console = new Console();
  
  
  
        var webRtcPeer;
  
        var pipeline;
  
        var webRtcEpt;
  
  
  
        var videoInput = document.getElementById('videoInput');
  
        var videoOutput = document.getElementById('videoOutput');
  
  
  
        var startButton = document.getElementById("start");
  
        var stopButton = document.getElementById("stop");
  
  
  
        startButton.addEventListener("click", function()
  
        {
  
          showSpinner(videoInput, videoOutput);
  
  
  
          var options = {
  
            localVideo: videoInput,
  
            remoteVideo: videoOutput
  
          };
  
  
  
  
  
          if (args.ice_servers) {
  
           console.log("Use ICE servers: " + args.ice_servers);
  
           options.configuration = {
  
             iceServers : JSON.parse(args.ice_servers)
  
           };
  
          } else {
  
           console.log("Use freeice")
  
          }
  
  
  
          webRtcPeer = kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error)
  
          {
  
            if(error) return onError(error)
  
  
  
            this.generateOffer(onOffer)
  
          });
  
  
  
          function onOffer(error, sdpOffer)
  
          {
  
            if(error) return onError(error)
  
  
  
            kurentoClient(args.ws_uri, function(error, client)
  
            {
  
              if(error) return onError(error);
  
  
  
              client.create("MediaPipeline", function(error, _pipeline)
  
              {
  
                if(error) return onError(error);
  
  
  
                pipeline = _pipeline;
  
  
  
                pipeline.create("WebRtcEndpoint", function(error, webRtc){
  
                  if(error) return onError(error);
  
  
  
                  webRtcEpt = webRtc;
  
  
  
                  setIceCandidateCallbacks(webRtcPeer, webRtc, onError)
  
  
  
                  webRtc.processOffer(sdpOffer, function(error, sdpAnswer){
  
                    if(error) return onError(error);
  
  
  
                    webRtcPeer.processAnswer(sdpAnswer, onError);
  
                  });
  
                  webRtc.gatherCandidates(onError);
  
  
  
                  webRtc.connect(webRtc, function(error){
  
                    if(error) return onError(error);
  
  
  
                    console.log("Loopback established");
  
                  });
  
                });
  
  
  
  
  
  
  
              pipeline.create("RtpEndpoint", function(error, rtp){
  
                  if(error) return onError(error);
  
  
  
                  //setIceCandidateCallbacks2(webRtcPeer, rtp, onError)
  
  
  
  
  
                  var cm_offer = "v=0\n" +
  
                        "o=- 3641290734 3641290734 IN IP4 127.0.0.1\n" +
  
                        "s=nginx\n" +
  
                        "c=IN IP4 127.0.0.1\n" +
  
                        "t=0 0\n" +
  
                        "m=audio 60820 RTP/AVP 0\n" +
  
                        "a=rtpmap:0 PCMU/8000\n" +
  
                        "a=recvonly\n" +
  
                        "m=video 59618 RTP/AVP 101\n" +
  
                        "a=rtpmap:101 H264/90000\n" +
  
                        "a=recvonly\n";
  
  
  
  
  
  
  
                  rtp.processOffer(cm_offer, function(error, cm_sdpAnswer){
  
                    if(error) return onError(error);
  
  
  
                    //webRtcPeer.processAnswer(cm_sdpAnswer, onError);
  
                  });
  
                  //rtp.gatherCandidates(onError);
  
  
  
                  webRtcEpt.connect(rtp, function(error){
  
                    if(error) return onError(error);
  
  
  
                    console.log("RTP endpoint connected to webRTC");
  
                  });
  
                });
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
              });
  
            });
  
          }
  
        });
  
        stopButton.addEventListener("click", stop);
  
  
  
  
  
        function stop() {
  
          if (webRtcPeer) {
  
            webRtcPeer.dispose();
  
            webRtcPeer = null;
  
          }
  
  
  
          if(pipeline){
  
            pipeline.release();
  
            pipeline = null;
  
          }
  
  
  
          hideSpinner(videoInput, videoOutput);
  
        }
  
  
  
        function onError(error) {
  
          if(error)
  
          {
  
            console.error(error);
  
            stop();
  
          }
  
        }
  
      })
  
  
  
  
  
      function showSpinner() {
  
        for (var i = 0; i < arguments.length; i++) {
  
          arguments[i].poster = 'img/transparent-1px.png';
  
          arguments[i].style.background = "center transparent url('img/spinner.gif') no-repeat";
  
        }
  
      }
  
  
  
      function hideSpinner() {
  
        for (var i = 0; i < arguments.length; i++) {
  
          arguments[i].src = '';
  
          arguments[i].poster = 'img/webrtc.png';
  
          arguments[i].style.background = '';
  
        }
  
      }
  
  
  
      /**
  
       * Lightbox utility (to display media pipeline image in a modal dialog)
  
       */
  
      $(document).delegate('*[data-toggle="lightbox"]', 'click', function(event) {
  
        event.preventDefault();
  
        $(this).ekkoLightbox();
  
      });