Recherche avancée

Sur d’autres sites (11876)

• mmsh:_implement a minimal mmsh_seek() so the stream position can be found out.

  31 octobre 2011, par Michael Niedermayer

  mmsh:_implement a minimal mmsh_seek() so the stream position can be found out.

• Android Exoplayer garbled mp4 video output on imagereader surface while playing fine on SurfaceView (Samsung galaxy tab s7)

  29 mars 2023, par Orcun

  Problem :

  


  I am trying to play an mp4 video file on Samsung Galaxy Tab S7 for video processing purposes. For the purpose, I use Exoplayer(v2.18.4) library to be able to play the video onto an ImageReader surface so that I can grab the frames in ImageReader.OnImageAvailableListener.

  


  I do not have any problems in getting the frames. However, the frames I acquire are garbled/corrupted like :

  


  

  


  When I saw this frame, it made me think that it is some sort of color format or codec issue for the mp4 file or device I use. In logcat, there are no errors reported.
Device uses [OMX.qcom.video.decoder.avc] decoder. Also, same video plays nicely on some other devices that use exactly the same decoder.

  


  What I tried so far :

  


  I tried a lot of things to understand the root cause and also find a solution, so my brain currently is free flowing stream. I will try to summarise, please bear with me.

  


  I use a CustomMediaCodecVideoRenderer in my app which overrides MediaCodecVideoRenderer.getMediaFormat of Exoplayer to set color format like :

  


  setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible)


  


  This step proved itself necessary. Without it I get a crash simply because I cannot access the image planes of an image frame in expected YUV image format.
This unfortunately was not enough for the Samsung Galaxy Tab 7 although it was already enough for another device like Xiaomi to render the image correctly.

  


  Then I decided to use a library like ffmpeg to modify the mp4 file itself to maybe use a different profile, color format, resolution that is widely compatible in android devices. No luck so far.
a quick ffprobe on my mp4 file shows :

  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'outputyuv169.mp4':
Metadata:
  major_brand     : isom
  minor_version   : 512
  compatible_brands: isomiso2avc1mp41
  encoder         : Lavf59.16.100
Duration: 00:03:00.02, start: 0.000000, bitrate: 4823 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 
1280x720 [SAR 1:1 DAR 16:9], 4820 kb/s, 30 fps, 30 tbr, 360k tbn (default)


  


  I tried even recording a simple video on the tablet itself and using that as my input file. Nevertheless, still getting similarly garbled frames.

  


  One thing I keep noticing in logcat is this line :

  


  [OMX.qcom.video.decoder.avc] using color format 0x7fa30c04 in place of 0x7f420888

  


  and

  


  [OMX.qcom.video.decoder.avc] Requested output format 0x7f420888 and got 0x7fa30c04

  


  where 0x7fa30c04 refers to YCbCr_420_SP_VENUS_UBWC. Not really sure If this has anything to do with it. I did not dive too deep into that because same log is present also on other device that renders the mp4 correctly.

  


  I would be grateful for any clue.

  


• moviepy black border around png when compositing into an MP4

  27 août 2022, par OneWorld

  compositing a png into an MP4 video creates a black border around the edge.

  


  This is using moviepy 1.0.0

  


  Code below reproduces the MP4 with the attached red text png.

  


  

  


  import numpy as np
import moviepy.editor as mped
def composite_txtpng_on_colour():
    bg_color = mped.ColorClip(size=[400, 300], color=np.array([0, 255, 0]).astype(np.uint8),
                          duration=2).set_position((0, 0))
    text_png_postition = [5, 5]
    text_png = mped.ImageClip("./txtpng.png", duration=3).set_position((text_png_postition))

    canvas_size = bg_color.size
    stacked_clips = mped.CompositeVideoClip([bg_color, text_png], size=canvas_size).set_duration(2)
    stacked_clips.write_videofile('text_with_black_border_video.mp4', fps=24)

composite_txtpng_on_colour()


  


  The result is an MP4 that can be played in VLC player. A screenshot of the black edge can be seen below :-

  


  

  


  Any suggestions to remove the black borders would be much appreciated.

  


  Update : It looks like moviepy does a blit instead of alpha compositing.

  


  def blit(im1, im2, pos=None, mask=None, ismask=False):
    """ Blit an image over another.  Blits ``im1`` on ``im2`` as position ``pos=(x,y)``, using the
    ``mask`` if provided. If ``im1`` and ``im2`` are mask pictures
    (2D float arrays) then ``ismask`` must be ``True``.
    """
    if pos is None:
        pos = [0, 0]

    # xp1,yp1,xp2,yp2 = blit area on im2
    # x1,y1,x2,y2 = area of im1 to blit on im2
    xp, yp = pos
    x1 = max(0, -xp)
    y1 = max(0, -yp)
    h1, w1 = im1.shape[:2]
    h2, w2 = im2.shape[:2]
    xp2 = min(w2, xp + w1)
    yp2 = min(h2, yp + h1)
    x2 = min(w1, w2 - xp)
    y2 = min(h1, h2 - yp)
    xp1 = max(0, xp)
    yp1 = max(0, yp)

    if (xp1 >= xp2) or (yp1 >= yp2):
        return im2

    blitted = im1[y1:y2, x1:x2]

    new_im2 = +im2

    if mask is None:
        new_im2[yp1:yp2, xp1:xp2] = blitted
    else:
        mask = mask[y1:y2, x1:x2]
        if len(im1.shape) == 3:
            mask = np.dstack(3 * [mask])
        blit_region = new_im2[yp1:yp2, xp1:xp2]
        new_im2[yp1:yp2, xp1:xp2] = (1.0 * mask * blitted + (1.0 - mask) * blit_region)
    
    return new_im2.astype('uint8') if (not ismask) else new_im2


  


  and so, Rotem is right.

  


  new_im2[yp1:yp2, xp1:xp2] = (1.0 * mask * blitted + (1.0 - mask) * blit_region)


  


  is

  


  (alpha * img_rgb + (1.0 - alpha) * bg)


  


  and this is how moviepy composites. And this is why we see black at the edges.