Recherche avancée

Sur d’autres sites (4399)

• send browser camera to rtmp server

  22 juin 2022, par anamul

  i am using python and in frotnend javascript.

  


  I am using ffmpeg to push video to rtmp server.

  


  ffmpeg -y -f mp4 -stream_loop -1 -re -i myvideo.mp4
-b:v 3M -vcodec libx264 -r 30.0 -s 1280x720
-c:a copy -b:a 96k
-f flv "rtmps://dfdf222.global-contribute.live-video.net:443/app/"


  


  It is working well but i am not getting how to push to rtmp server of my browser camera ? i mean from javascript ?

  


  i am trying like this :

  


  &#xA;    <video autoplay="autoplay"></video>&#xA;&#xA;    <a href="#">open and stream</a>&#xA;&#xA;    <code class="echappe-js">&lt;script&gt;&amp;#xA;        function streamVideo () {&amp;#xA;            console.log(&amp;#x27;ddd&amp;#x27;)&amp;#xA;            navigator.mediaDevices.getUserMedia({&amp;#xA;            video: {&amp;#xA;&amp;#xA;            },&amp;#xA;            audio: {&amp;#xA;                &amp;#xA;            }&amp;#xA;            }).then(stream=&gt;{&amp;#xA;                console.log(stream)&amp;#xA;                document.getElementById(&amp;#x27;vid&amp;#x27;).srcObject = stream; &amp;#xA;            })&amp;#xA;&amp;#xA;        }&amp;#xA;    &lt;/script&gt;&#xA;&#xA;

  


  It is working but not getting any way to push from browser to rtmp server. I know browser doesnt support rtmp protocol, we need to send it from the backend but can i push to server so that it push to rtmp server ?

  


  Anyone know the way ?

  


• Adventures in Unicode

  29 novembre 2012, par Multimedia Mike — Programming, php, Python, sqlite3, unicode

  Tangential to multimedia hacking is proper metadata handling. Recently, I have gathered an interest in processing a large corpus of multimedia files which are likely to contain metadata strings which do not fall into the lower ASCII set. This is significant because the lower ASCII set intersects perfectly with my own programming comfort zone. Indeed, all of my programming life, I have insisted on covering my ears and loudly asserting “LA LA LA LA LA ! ALL TEXT EVERYWHERE IS ASCII !” I suspect I’m not alone in this.

  Thus, I took this as an opportunity to conquer my longstanding fear of Unicode. I developed a self-learning course comprised of a series of exercises which add up to this diagram :

  
  
  

  Part 1 : Understanding Text Encoding
  Python has regular strings by default and then it has Unicode strings. The latter are prefixed by the letter ‘u’. This is what ‘ö’ looks like encoded in each type.

  python

  < view plain text >

  1. >>> ’ö’, u’ö’
  2. (’\xc3\xb6’, u’\xf6’)

  A large part of my frustration with Unicode comes from Python yelling at me about UnicodeDecodeErrors and an inability to handle the number 0xc3 for some reason. This usually comes when I’m trying to wrap my head around an unrelated problem and don’t care to get sidetracked by text encoding issues. However, when I studied the above output, I finally understood where the 0xc3 comes from. I just didn’t understand what the encoding represents exactly.
  

  I can see from assorted tables that ‘ö’ is character 0xF6 in various encodings (in Unicode and Latin-1), so u’\xf6′ makes sense. But what does ‘\xc3\xb6′ mean ? It’s my style to excavate straight down to the lowest levels, and I wanted to understand exactly how characters are represented in memory. The UTF-8 encoding tables inform us that any Unicode code point above 0x7F but less than 0×800 will be encoded with 2 bytes :

   110xxxxx 10xxxxxx
  

  Applying this pattern to the \xc3\xb6 encoding :

              hex : 0xc3      0xb6
             bits : 11000011  10110110
   important bits : ---00011  —110110
        assembled : 00011110110
       code point : 0xf6
  

  I was elated when I drew that out and made the connection. Maybe I’m the last programmer to figure this stuff out. But I’m still happy that I actually understand those Python errors pertaining to the number 0xc3 and that I won’t have to apply canned solutions without understanding the core problem.

  I’m cheating on this part of this exercise just a little bit since the diagram implied that the Unicode text needs to come from a binary file. I’ll return to that in a bit. For now, I’ll just contrive the following Unicode string from the Python REPL :

  python

  < view plain text >

  1. >>> u = u’Üñìçôđé’
  2. >>> u
  3. u’\xdc\xf1\xec\xe7\xf4\u0111\xe9’

  Part 2 : From Python To SQLite3
  The next step is to see what happens when I use Python’s SQLite3 module to dump the string into a new database. Will the Unicode encoding be preserved on disk ? What will UTF-8 look like on disk anyway ?

  python

  < view plain text >

  1. >>> import sqlite3
  2. >>> conn = sqlite3.connect(’unicode.db’)
  3. >>> conn.execute("CREATE TABLE t (t text)")
  4. >>> conn.execute("INSERT INTO t VALUES (?)", (u, ))
  5. >>> conn.commit()
  6. >>> conn.close()

  Next, I manually view the resulting database file (unicode.db) using a hex editor and look for strings. Here we go :

  000007F0   02 29 C3 9C  C3 B1 C3 AC  C3 A7 C3 B4  C4 91 C3 A9
  

  Look at that ! It’s just like the \xc3\xf6 encoding we see in the regular Python strings.

  Part 3 : From SQLite3 To A Web Page Via PHP
  Finally, use PHP (love it or hate it, but it’s what’s most convenient on my hosting provider) to query the string from the database and display it on a web page, completing the outlined processing pipeline.

  php

  < view plain text >

  1. < ?php
  2. $dbh = new PDO("sqlite:unicode.db") ;
  3. foreach ($dbh->query("SELECT t from t") as $row) ;
  4. $unicode_string = $row[’t’] ;
  5.  ?>
  6.  
  7. <html>
  8. <head><meta http-equiv="Content-Type" content="text/html ; charset=utf-8"></meta></head>
  9. <body><h1>< ?=$unicode_string ?></h1></body>
  10. </html>

  I tested the foregoing PHP script on 3 separate browsers that I had handy (Firefox, Internet Explorer, and Chrome) :

  
  
  

  I’d say that counts as success ! It’s important to note that the “meta http-equiv” tag is absolutely necessary. Omit and see something like this :

  
  
  

  Since we know what the UTF-8 stream looks like, it’s pretty obvious how the mapping is operating here : 0xc3 and 0xc4 correspond to ‘Ã’ and ‘Ä’, respectively. This corresponds to an encoding named ISO/IEC 8859-1, a.k.a. Latin-1. Speaking of which…

  Part 4 : Converting Binary Data To Unicode
  At the start of the experiment, I was trying to extract metadata strings from these binary multimedia files and I noticed characters like our friend ‘ö’ from above. In the bytestream, this was represented simply with 0xf6. I mistakenly believed that this was the on-disk representation of UTF-8. Wrong. Turns out it’s Latin-1.

  However, I still need to solve the problem of transforming such strings into Unicode to be shoved through the pipeline diagrammed above. For this experiment, I created a 9-byte file with the Latin-1 string ‘Üñìçôdé’ couched by 0′s, to simulate yanking a string out of a binary file. Here’s unicode.file :

  00000000   00 DC F1 EC  E7 F4 64 E9  00         ......d..
  

  (Aside : this experiment uses plain ‘d’ since the ‘đ’ with a bar through it doesn’t occur in Latin-1 ; shows up all over the place in Vietnamese, at least.)

  I’ve been mashing around Python code via the REPL, trying to get this string into a Unicode-friendly format. This is a successful method but it’s probably not the best :

  python

  < view plain text >

  1. >>> import struct
  2. >>> f = open(’unicode.file’, ’r’).read()
  3. >>> u = u’’
  4. >>> for c in struct.unpack("B"*7, f[1 :8]) :
  5. ... u += unichr(c)
  6. ...
  7. >>> u
  8. u’\xdc\xf1\xec\xe7\xf4d\xe9’
  9. >>> print u
  10. Üñìçôdé

  Conclusion
  Dealing with text encoding matters reminds me of dealing with integer endian-ness concerns. When you’re just dealing with one system, you probably don’t need to think too much about it because the system is usually handling everything consistently underneath the covers.

  However, when the data leaves one system and will be interpreted by another system, that’s when a programmer needs to be cognizant of matters such as integer endianness or text encoding.

• Fragmented MP4 - problem playing in browser

  12 juin 2019, par PookyFan

  I try to create fragmented MP4 from raw H264 video data so I could play it in internet browser’s player. My goal is to create live streaming system, where media server would send fragmented MP4 pieces to browser. The server would buffer input data from RaspberryPi camera, which sends video as H264 frames. It would then mux that video data and make it available for client. The browser would play media data (that were muxed by server and sent i.e. through websocket) by using Media Source Extensions.

  For test purpose I wrote the following pieces of code (using many examples I found in the intenet) :

  C++ application using avcodec which muxes raw H264 video to fragmented MP4 and saves it to a file :

  #define READBUFSIZE 4096
  
  #define IOBUFSIZE 4096
  
  #define ERRMSGSIZE 128
  
  
  
  #include <cstdint>
  
  #include <iostream>
  
  #include <fstream>
  
  #include <string>
  
  #include <vector>
  
  
  
  extern "C"
  
  {
  
      #include <libavformat></libavformat>avformat.h>
  
      #include <libavutil></libavutil>error.h>
  
      #include <libavutil></libavutil>opt.h>
  
  }
  
  
  
  enum NalType : uint8_t
  
  {
  
      //NALs containing stream metadata
  
      SEQ_PARAM_SET = 0x7,
  
      PIC_PARAM_SET = 0x8
  
  };
  
  
  
  std::vector outputData;
  
  
  
  int mediaMuxCallback(void *opaque, uint8_t *buf, int bufSize)
  
  {
  
      outputData.insert(outputData.end(), buf, buf + bufSize);
  
      return bufSize;
  
  }
  
  
  
  std::string getAvErrorString(int errNr)
  
  {
  
      char errMsg[ERRMSGSIZE];
  
      av_strerror(errNr, errMsg, ERRMSGSIZE);
  
      return std::string(errMsg);
  
  }
  
  
  
  int main(int argc, char **argv)
  
  {
  
      if(argc &lt; 2)
  
      {
  
          std::cout &lt;&lt; "Missing file name" &lt;&lt; std::endl;
  
          return 1;
  
      }
  
  
  
      std::fstream file(argv[1], std::ios::in | std::ios::binary);
  
      if(!file.is_open())
  
      {
  
          std::cout &lt;&lt; "Couldn't open file " &lt;&lt; argv[1] &lt;&lt; std::endl;
  
          return 2;
  
      }
  
  
  
      std::vector inputMediaData;
  
      do
  
      {
  
          char buf[READBUFSIZE];
  
          file.read(buf, READBUFSIZE);
  
  
  
          int size = file.gcount();
  
          if(size > 0)
  
              inputMediaData.insert(inputMediaData.end(), buf, buf + size);
  
      } while(!file.eof());
  
      file.close();
  
  
  
      //Initialize avcodec
  
      av_register_all();
  
      uint8_t *ioBuffer;
  
      AVCodec *codec = avcodec_find_decoder(AV_CODEC_ID_H264);
  
      AVCodecContext *codecCtxt = avcodec_alloc_context3(codec);
  
      AVCodecParserContext *parserCtxt = av_parser_init(AV_CODEC_ID_H264);
  
      AVOutputFormat *outputFormat = av_guess_format("mp4", nullptr, nullptr);
  
      AVFormatContext *formatCtxt;
  
      AVIOContext *ioCtxt;
  
      AVStream *videoStream;
  
  
  
      int res = avformat_alloc_output_context2(&amp;formatCtxt, outputFormat, nullptr, nullptr);
  
      if(res &lt; 0)
  
      {
  
          std::cout &lt;&lt; "Couldn't initialize format context; the error was: " &lt;&lt; getAvErrorString(res) &lt;&lt; std::endl;
  
          return 3;
  
      }
  
  
  
      if((videoStream = avformat_new_stream( formatCtxt, avcodec_find_encoder(formatCtxt->oformat->video_codec) )) == nullptr)
  
      {
  
          std::cout &lt;&lt; "Couldn't initialize video stream" &lt;&lt; std::endl;
  
          return 4;
  
      }
  
      else if(!codec)
  
      {
  
          std::cout &lt;&lt; "Couldn't initialize codec" &lt;&lt; std::endl;
  
          return 5;
  
      }
  
      else if(codecCtxt == nullptr)
  
      {
  
          std::cout &lt;&lt; "Couldn't initialize codec context" &lt;&lt; std::endl;
  
          return 6;
  
      }
  
      else if(parserCtxt == nullptr)
  
      {
  
          std::cout &lt;&lt; "Couldn't initialize parser context" &lt;&lt; std::endl;
  
          return 7;
  
      }
  
      else if((ioBuffer = (uint8_t*)av_malloc(IOBUFSIZE)) == nullptr)
  
      {
  
          std::cout &lt;&lt; "Couldn't allocate I/O buffer" &lt;&lt; std::endl;
  
          return 8;
  
      }
  
      else if((ioCtxt = avio_alloc_context(ioBuffer, IOBUFSIZE, 1, nullptr, nullptr, mediaMuxCallback, nullptr)) == nullptr)
  
      {
  
          std::cout &lt;&lt; "Couldn't initialize I/O context" &lt;&lt; std::endl;
  
          return 9;
  
      }
  
  
  
      //Set video stream data
  
      videoStream->id = formatCtxt->nb_streams - 1;
  
      videoStream->codec->width = 1280;
  
      videoStream->codec->height = 720;
  
      videoStream->time_base.den = 60; //FPS
  
      videoStream->time_base.num = 1;
  
      videoStream->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
      formatCtxt->pb = ioCtxt;
  
  
  
      //Retrieve SPS and PPS for codec extdata
  
      const uint32_t synchMarker = 0x01000000;
  
      unsigned int i = 0;
  
      int spsStart = -1, ppsStart = -1;
  
      uint16_t spsSize = 0, ppsSize = 0;
  
      while(spsSize == 0 || ppsSize == 0)
  
      {
  
          uint32_t *curr =  (uint32_t*)(inputMediaData.data() + i);
  
          if(*curr == synchMarker)
  
          {
  
              unsigned int currentNalStart = i;
  
              i += sizeof(uint32_t);
  
              uint8_t nalType = inputMediaData.data()[i] &amp; 0x1F;
  
              if(nalType == SEQ_PARAM_SET)
  
                  spsStart = currentNalStart;
  
              else if(nalType == PIC_PARAM_SET)
  
                  ppsStart = currentNalStart;
  
  
  
              if(spsStart >= 0 &amp;&amp; spsSize == 0 &amp;&amp; spsStart != i)
  
                  spsSize = currentNalStart - spsStart;
  
              else if(ppsStart >= 0 &amp;&amp; ppsSize == 0 &amp;&amp; ppsStart != i)
  
                  ppsSize = currentNalStart - ppsStart;
  
          }
  
          ++i;
  
      }
  
  
  
      videoStream->codec->extradata = inputMediaData.data() + spsStart;
  
      videoStream->codec->extradata_size = ppsStart + ppsSize;
  
  
  
      //Write main header
  
      AVDictionary *options = nullptr;
  
      av_dict_set(&amp;options, "movflags", "frag_custom+empty_moov", 0);
  
      res = avformat_write_header(formatCtxt, &amp;options);
  
      if(res &lt; 0)
  
      {
  
          std::cout &lt;&lt; "Couldn't write container main header; the error was: " &lt;&lt; getAvErrorString(res) &lt;&lt; std::endl;
  
          return 10;
  
      }
  
  
  
      //Retrieve frames from input video and wrap them in container
  
      int currentInputIndex = 0;
  
      int framesInSecond = 0;
  
      while(currentInputIndex &lt; inputMediaData.size())
  
      {
  
          uint8_t *frameBuffer;
  
          int frameSize;
  
          res = av_parser_parse2(parserCtxt, codecCtxt, &amp;frameBuffer, &amp;frameSize, inputMediaData.data() + currentInputIndex,
  
              inputMediaData.size() - currentInputIndex, AV_NOPTS_VALUE, AV_NOPTS_VALUE, 0);
  
          if(frameSize == 0) //No more frames while some data still remains (is that even possible?)
  
          {
  
              std::cout &lt;&lt; "Some data left unparsed: " &lt;&lt; std::to_string(inputMediaData.size() - currentInputIndex) &lt;&lt; std::endl;
  
              break;
  
          }
  
  
  
          //Prepare packet with video frame to be dumped into container
  
          AVPacket packet;
  
          av_init_packet(&amp;packet);
  
          packet.data = frameBuffer;
  
          packet.size = frameSize;
  
          packet.stream_index = videoStream->index;
  
          currentInputIndex += frameSize;
  
  
  
          //Write packet to the video stream
  
          res = av_write_frame(formatCtxt, &amp;packet);
  
          if(res &lt; 0)
  
          {
  
              std::cout &lt;&lt; "Couldn't write packet with video frame; the error was: " &lt;&lt; getAvErrorString(res) &lt;&lt; std::endl;
  
              return 11;
  
          }
  
  
  
          if(++framesInSecond == 60) //We want 1 segment per second
  
          {
  
              framesInSecond = 0;
  
              res = av_write_frame(formatCtxt, nullptr); //Flush segment
  
          }
  
      }
  
      res = av_write_frame(formatCtxt, nullptr); //Flush if something has been left
  
  
  
      //Write media data in container to file
  
      file.open("my_mp4.mp4", std::ios::out | std::ios::binary);
  
      if(!file.is_open())
  
      {
  
          std::cout &lt;&lt; "Couldn't open output file " &lt;&lt; std::endl;
  
          return 12;
  
      }
  
  
  
      file.write((char*)outputData.data(), outputData.size());
  
      if(file.fail())
  
      {
  
          std::cout &lt;&lt; "Couldn't write to file" &lt;&lt; std::endl;
  
          return 13;
  
      }
  
  
  
      std::cout &lt;&lt; "Media file muxed successfully" &lt;&lt; std::endl;
  
      return 0;
  
  }
  
  </vector></string></fstream></iostream></cstdint>

  (I hardcoded a few values, such as video dimensions or framerate, but as I said this is just a test code.)

  ---

  Simple HTML webpage using MSE to play my fragmented MP4

      
  
  
  
  
  
      <video width="1280" height="720" controls="controls">
  
      </video>
  
  
  
  <code class="echappe-js">&lt;script&gt;<br />
  
  var vidElement = document.querySelector('video');<br />
  
  <br />
  
  if (window.MediaSource) {<br />
  
    var mediaSource = new MediaSource();<br />
  
    vidElement.src = URL.createObjectURL(mediaSource);<br />
  
    mediaSource.addEventListener('sourceopen', sourceOpen);<br />
  
  } else {<br />
  
    console.log(&quot;The Media Source Extensions API is not supported.&quot;)<br />
  
  }<br />
  
  <br />
  
  function sourceOpen(e) {<br />
  
    URL.revokeObjectURL(vidElement.src);<br />
  
    var mime = 'video/mp4; codecs=&quot;avc1.640028&quot;';<br />
  
    var mediaSource = e.target;<br />
  
    var sourceBuffer = mediaSource.addSourceBuffer(mime);<br />
  
    var videoUrl = 'my_mp4.mp4';<br />
  
    fetch(videoUrl)<br />
  
      .then(function(response) {<br />
  
        return response.arrayBuffer();<br />
  
      })<br />
  
      .then(function(arrayBuffer) {<br />
  
        sourceBuffer.addEventListener('updateend', function(e) {<br />
  
          if (!sourceBuffer.updating &amp;amp;&amp;amp; mediaSource.readyState === 'open') {<br />
  
            mediaSource.endOfStream();<br />
  
          }<br />
  
        });<br />
  
        sourceBuffer.appendBuffer(arrayBuffer);<br />
  
      });<br />
  
  }<br />
  
  &lt;/script&gt;
  

  ---

  Output MP4 file generated by my C++ application can be played i.e. in MPC, but it doesn’t play in any web browser I tested it with. It also doesn’t have any duration (MPC keeps showing 00:00).

  To compare output MP4 file I got from my C++ application described above, I also used FFMPEG to create fragmented MP4 file from the same source file with raw H264 stream. I used the following command :

  ffmpeg -r 60 -i input.h264 -c:v copy -f mp4 -movflags empty_moov+default_base_moof+frag_keyframe test.mp4

  This file generated by FFMPEG is played correctly by every web browser I used for tests. It also has correct duration (but also it has trailing atom, which wouldn’t be present in my live stream anyway, and as I need a live stream, it won’t have any fixed duration in the first place).

  MP4 atoms for both files look very similiar (they have identical avcc section for sure). What’s interesting (but not sure if it’s of any importance), both files have different NALs format than input file (RPI camera produces video stream in Annex-B format, while output MP4 files contain NALs in AVCC format... or at least it looks like it’s the case when I compare mdat atoms with input H264 data).

  I assume there is some field (or a few fields) I need to set for avcodec to make it produce video stream that would be properly decoded and played by browsers players. But what field(s) do I need to set ? Or maybe problem lies somewhere else ? I ran out of ideas.

  ---

  EDIT 1 :
  As suggested, I investigated binary content of both MP4 files (generated by my app and FFMPEG tool) with hex editor. What I can confirm :

  • both files have identical avcc section (they match perfectly and are in AVCC format, I analyzed it byte after byte and there’s no mistake about it)
  • both files have NALs in AVCC format (I looked closely at mdat atoms and they don’t differ between both MP4 files)

  So I guess there’s nothing wrong with the extradata creation in my code - avcodec takes care of it properly, even if I just feed it with SPS and PPS NALs. It converts them by itself, so no need for me to do it by hand. Still, my original problem remains.

  EDIT 2 : I achieved partial success - MP4 generated by my app now plays in Firefox. I added this line to the code (along with rest of stream initialization) :

  videoStream->codec->time_base = videoStream->time_base;

  So now this section of my code looks like this :

  //Set video stream data
  
  videoStream->id = formatCtxt->nb_streams - 1;
  
  videoStream->codec->width = 1280;
  
  videoStream->codec->height = 720;
  
  videoStream->time_base.den = 60; //FPS
  
  videoStream->time_base.num = 1;
  
  videoStream->codec->time_base = videoStream->time_base;
  
  videoStream->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
  formatCtxt->pb = ioCtxt;