Recherche avancée

Sur d’autres sites (6909)

• RTMP Broadcast packet body structure for Twitch

  22 mai 2018, par Dobby

  I’m currently working on a project similar to OBS, where I’m capturing screen data, encoding it with the x264 library, and then broadcasting it to a twitch server.

  Currently, the servers are accepting the data, but no video is being played - it buffers for a moment, then returns an error code "2000 : network error"

  Like OBS Classic, I’m dividing each NAL provided by x264 by its type, and then making changes to each

  int frame_size = x264_encoder_encode(encoder, &nals, &num_nals, &pic_in, &pic_out);
  
  
  
      //sort the NAL's into their types and make necessary adjustments
  
  
  
      int timeOffset = int(pic_out.i_pts - pic_out.i_dts);
  
  
  
      timeOffset = htonl(timeOffset);//host to network translation, ensure the bytes are in the right format
  
      BYTE *timeOffsetAddr = ((BYTE*)&timeOffset) + 1;
  
  
  
      videoSection sect;
  
      bool foundFrame = false;
  
  
  
      uint8_t * spsPayload = NULL;
  
      int spsSize = 0;
  
  
  
      for (int i = 0; i/std::cout << "VideoEncoder: EncodedImages Size: " << encodedImages->size() << std::endl;
  
          x264_nal_t &nal = nals[i];
  
          //std::cout << "NAL is:" << nal.i_type << std::endl;
  
  
  
          //need to account for pps/sps, seems to always be the first frame sent
  
          if (nal.i_type == NAL_SPS) {
  
              spsSize = nal.i_payload;
  
              spsPayload = (uint8_t*)malloc(spsSize);
  
              memcpy(spsPayload, nal.p_payload, spsSize);
  
          } else if (nal.i_type == NAL_PPS){
  
              //pps always happens after sps
  
              if (spsPayload == NULL) {
  
                  std::cout << "VideoEncoder: critical error, sps not set" << std::endl;
  
              }
  
              uint8_t * payload = (uint8_t*)malloc(nal.i_payload + spsSize);
  
              memcpy(payload, spsPayload, spsSize);
  
              memcpy(payload, nal.p_payload + spsSize, nal.i_payload);
  
              sect = { nal.i_payload + spsSize, payload, nal.i_type };
  
              encodedImages->push(sect);
  
          } else if (nal.i_type == NAL_SEI || nal.i_type == NAL_FILLER) { 
  
              //these need some bytes at the start removed
  
              BYTE *skip = nal.p_payload;
  
              while (*(skip++) != 0x1);
  
              int skipBytes = (int)(skip - nal.p_payload);
  
  
  
              int newPayloadSize = (nal.i_payload - skipBytes);
  
  
  
              uint8_t * payload = (uint8_t*)malloc(newPayloadSize);
  
              memcpy(payload, nal.p_payload + skipBytes, newPayloadSize);
  
              sect = { newPayloadSize, payload, nal.i_type };
  
              encodedImages->push(sect);
  
  
  
          } else if (nal.i_type == NAL_SLICE_IDR || nal.i_type == NAL_SLICE) { 
  
              //these packets need an additional section at the start
  
              BYTE *skip = nal.p_payload;
  
              while (*(skip++) != 0x1);
  
              int skipBytes = (int)(skip - nal.p_payload);
  
  
  
              std::vector<byte> bodyData;
  
              if (!foundFrame) {
  
                  if (nal.i_type == NAL_SLICE_IDR) { bodyData.push_back(0x17); } else { bodyData.push_back(0x27); } //add a 17 or a 27 as appropriate
  
                  bodyData.push_back(1);
  
                  bodyData.push_back(*timeOffsetAddr);
  
  
  
                  foundFrame = true;
  
              }
  
  
  
              //put into the payload the bodyData followed by the nal payload
  
              uint8_t * bodyDataPayload = (uint8_t*)malloc(bodyData.size());
  
              memcpy(bodyDataPayload, bodyData.data(), bodyData.size() * sizeof(BYTE));
  
  
  
              int newPayloadSize = (nal.i_payload - skipBytes);
  
  
  
              uint8_t * payload = (uint8_t*)malloc(newPayloadSize + sizeof(bodyDataPayload));
  
              memcpy(payload, bodyDataPayload, sizeof(bodyDataPayload));
  
              memcpy(payload + sizeof(bodyDataPayload), nal.p_payload + skipBytes, newPayloadSize);
  
              int totalSize = newPayloadSize + sizeof(bodyDataPayload);
  
              sect = { totalSize, payload, nal.i_type };
  
              encodedImages->push(sect);
  
          } else {
  
              std::cout &lt;&lt; "VideoEncoder: Nal type did not match expected" &lt;&lt; std::endl;
  
              continue;
  
          }
  
      }
  
  </byte>

  The NAL payload data is then put into a struct, VideoSection, in a queue buffer

  //used to transfer encoded data
  
  struct videoSection {
  
      int frameSize;
  
      uint8_t* payload;
  
      int type;
  
  };

  After which it is picked up by the broadcaster, a few more changes are made, and then I call rtmp_send()

  videoSection sect = encodedImages->front();
  
  encodedImages->pop();
  
  
  
  //std::cout << "Broadcaster: Frame Size: " << sect.frameSize << std::endl;
  
  
  
  //two methods of sending RTMP data, _sendpacket and _write. Using sendpacket for greater control
  
  
  
  RTMPPacket * packet;
  
  
  
  unsigned char* buf = (unsigned char*)sect.payload;
  
  
  
  int type = buf[0]&0x1f; //I believe &0x1f sets a 32bit limit
  
  int len = sect.frameSize;
  
  long timeOffset = GetTickCount() - rtmp_start_time;
  
  
  
  //assign space packet will need
  
  packet = (RTMPPacket *)malloc(sizeof(RTMPPacket)+RTMP_MAX_HEADER_SIZE + len + 9);
  
  memset(packet, 0, sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE);
  
  
  
  packet->m_body = (char *)packet + sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE;
  
  packet->m_nBodySize = len + 9;
  
  
  
  //std::cout << "Broadcaster: Packet Size: " << sizeof(RTMPPacket) + RTMP_MAX_HEADER_SIZE + len + 9 << std::endl;
  
  //std::cout << "Broadcaster: Packet Body Size: " << len + 9 << std::endl;
  
  
  
  //set body to point to the packetbody
  
  unsigned char *body = (unsigned char *)packet->m_body;
  
  memset(body, 0, len + 9);
  
  
  
  
  
  
  
  //NAL_SLICE_IDR represents keyframe
  
  //first element determines packet type
  
  body[0] = 0x27;//inter-frame h.264
  
  if (sect.type == NAL_SLICE_IDR) {
  
      body[0] = 0x17; //h.264 codec id
  
  }
  
  
  
  
  
  //-------------------------------------------------------------------------------
  
  //this section taken from https://stackoverflow.com/questions/25031759/using-x264-and-librtmp-to-send-live-camera-frame-but-the-flash-cant-show
  
  //in an effort to understand packet format. it does not resolve my previous issues formatting the data for twitch to play it
  
  
  
  //sets body to be NAL unit
  
  body[1] = 0x01;
  
  body[2] = 0x00;
  
  body[3] = 0x00;
  
  body[4] = 0x00;
  
  
  
  //>> is a shift right
  
  //shift len to the right, and AND it
  
  /*body[5] = (len >> 24) & 0xff;
  
  body[6] = (len >> 16) & 0xff;
  
  body[7] = (len >> 8) & 0xff;
  
  body[8] = (len) & 0xff;*/
  
  
  
  //end code sourced from https://stackoverflow.com/questions/25031759/using-x264-and-librtmp-to-send-live-camera-frame-but-the-flash-cant-show
  
  //-------------------------------------------------------------------------------
  
  
  
  //copy from buffer into rest of body
  
  memcpy(&body[9], buf, len);
  
  
  
  //DEBUG
  
  
  
  //save individual packet body to a file with name rtmp[packetnum]
  
  //determine why some packets do not have 0x27 or 0x17 at the start
  
  //still happening, makes no sense given the above code
  
  
  
  /*std::string fileLocation = "rtmp" + std::to_string(packCount++);
  
  std::cout << fileLocation << std::endl;
  
  const char * charConversion = fileLocation.c_str();
  
  
  
  FILE* saveFile = NULL;
  
  saveFile = fopen(charConversion, "w+b");//open as write and binary
  
  if (!fwrite(body, len + 9, 1, saveFile)) {
  
      std::cout << "VideoEncoder: Error while trying to write to file" << std::endl;
  
  }
  
  fclose(saveFile);*/
  
  
  
  //END DEBUG
  
  
  
  //other packet details
  
  packet->m_hasAbsTimestamp = 0;
  
  packet->m_packetType = RTMP_PACKET_TYPE_VIDEO;
  
  if (rtmp != NULL) {
  
      packet->m_nInfoField2 = rtmp->m_stream_id;
  
  }
  
  packet->m_nChannel = 0x04;
  
  packet->m_headerType = RTMP_PACKET_SIZE_LARGE;
  
  packet->m_nTimeStamp = timeOffset;
  
  
  
  //send the packet
  
  if (rtmp != NULL) {
  
      RTMP_SendPacket(rtmp, packet, TRUE);
  
  }

  I can see that Twitch is receiving the data in the inspector, at a steady 3kbps. so I’m sure something is wrong with how I’m adjusting the data before sending it. Can anyone advise me on what I’m doing wrong here ?

• ffmpeg - Determine what parameters

  27 juin 2018, par AdmiralJonB

  I’ve got a video file that I’m trying to determine what parameters I can use to reproduce the encoding with ffmpeg.

  Here’s the ffprobe of the particular stream in question.

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4': Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf57.51.107   
  
  Duration: 00:05:02.84, start: 0.000000, bitrate: 4324 kb/s
  
      Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 4323 kb/s, 25 fps, 25 tbr, 1200k tbn, 2400k tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler

  The key part is that I’m noticing that the bitrate is quite low at 4324 kb/s, but this is actually an incredibly high quality video. To approach this sort of quality, I’ve only been able to make videos at 40000 kb/s (which is a huge increase in filesize). I also notice that it mentions yuvj420p which is an image format, but I have no idea what parameters with ffmpeg could produce that image format (if this at all would make a difference).

  Would appreciate any help I can get.

  Edit :

  Here’s the output based on the comment below :

  [STREAM]
  
  index=0
  
  codec_name=h264
  
  codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
  
  profile=Main
  
  codec_type=video
  
  codec_time_base=123187/3701760
  
  codec_tag_string=avc1
  
  codec_tag=0x31637661
  
  width=1920
  
  height=1080
  
  coded_width=1920
  
  coded_height=1080
  
  has_b_frames=0
  
  sample_aspect_ratio=1:1
  
  display_aspect_ratio=16:9
  
  pix_fmt=yuvj420p
  
  level=41
  
  color_range=pc
  
  color_space=bt709
  
  color_transfer=bt709
  
  color_primaries=bt709
  
  chroma_location=center
  
  field_order=unknown
  
  timecode=N/A
  
  refs=1
  
  is_avc=true
  
  nal_length_size=4
  
  id=N/A
  
  r_frame_rate=15/1
  
  avg_frame_rate=1850880/123187
  
  time_base=1/15360
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=246374
  
  duration=16.039974
  
  bit_rate=13795677
  
  max_bit_rate=N/A
  
  bits_per_raw_sample=8
  
  nb_frames=241
  
  nb_read_frames=N/A
  
  nb_read_packets=N/A
  
  DISPOSITION:default=1
  
  DISPOSITION:dub=0
  
  DISPOSITION:original=0
  
  DISPOSITION:comment=0
  
  DISPOSITION:lyrics=0
  
  DISPOSITION:karaoke=0
  
  DISPOSITION:forced=0
  
  DISPOSITION:hearing_impaired=0
  
  DISPOSITION:visual_impaired=0
  
  DISPOSITION:clean_effects=0
  
  DISPOSITION:attached_pic=0
  
  DISPOSITION:timed_thumbnails=0
  
  TAG:language=und
  
  TAG:handler_name=VideoHandler
  
  [/STREAM]

  Edit 2 :

  Here’s information from mediainfo

  Video
  
  ID                                       : 1
  
  Format                                   : AVC
  
  Format/Info                              : Advanced Video Codec
  
  Format profile                           : Main@L4.1
  
  Format settings, CABAC                   : Yes
  
  Format settings, RefFrames               : 1 frame
  
  Format settings, GOP                     : M=1, N=32
  
  Codec ID                                 : avc1
  
  Codec ID/Info                            : Advanced Video Coding
  
  Duration                                 : 5 min 2 s
  
  Bit rate                                 : 4 323 kb/s
  
  Width                                    : 1 920 pixels
  
  Height                                   : 1 080 pixels
  
  Display aspect ratio                     : 16:9
  
  Frame rate mode                          : Constant
  
  Frame rate                               : 25.000 FPS
  
  Color space                              : YUV
  
  Chroma subsampling                       : 4:2:0
  
  Bit depth                                : 8 bits
  
  Scan type                                : Progressive
  
  Bits/(Pixel*Frame)                       : 0.083
  
  Stream size                              : 156 MiB (100%)
  
  Color range                              : Full
  
  Color primaries                          : BT.709
  
  Transfer characteristics                 : BT.709
  
  Matrix coefficients                      : BT.709

• Solution for VB6 to broadcast Webcam

  14 septembre 2018, par vantrung -cuncon

  Sorry, I know VB6 is decades ago, but I’m in a situation that I have to use VB6 to deliver live webcam stream beetween 2 PC in Server - Client Model program. Vb6-code holds the connection then I have no choice but to transfer all data via that connection.

  I’ve tried weeks for this, uncountable approaches but went to nowhere.
  My efforts focused on 3 major approaches :

  1/ Use ffmpeg to record live webcam as ".avi" file on hard disk, transfer parts of file to other end & play it. But I’ve stucked with a media-player that can play a "being written" avi file.

  Windows Media Player control told me "file already in use..." & VLC Plugin can’t even be added to VB6 (axvlc.dll).

  2/ Use ffmpeg to save live webcam as avi file, transfer each bit of that file to the other end, then in other end, extract 24 images / second from the avi to display continously in a picture box.
  This approach is ok except that my hard disk get fulled of images in a time of wink and my program get very slow before hanging.

  3/ Use ffmpeg to stream the live webcam to a rtp-port like this :

  ffmpeg -f dshow -i video="Lenovo EasyCamera" -vcodec mpeg2video -pix_fmt yuv422p -f rtp -an rtp://224.1.2.3:8191

  I’ve successfully watch the stream in VLC, but VLC(axvlc.dll) refused to be integrated into ancient VB6. And more important, I don’t know how to redirect/reroute the rtp stream to other PC with VB6.

  Any one please light me up ? (Any 3rd party component is welcomed)