Sur d’autres sites (7755)

• How to get video pixel location from screen pixel location ?

  22 février 2024, par AmLearning

  Wall of Text so I tried breaking it up into sections to make it better sorry in advance

  


  The problem

  


  I have some video files that I am reading with ffmpeg to get the colors at specific pixels, and all seems well, but I just ran into a problem with finding the right pixel to input. I realized (or mistakingly believe) that the pixel location (x,y) on the screen will be different than the local pixel location so to speak of the video (ie. If I want to get pixel 50,0 of the video that will be different than my screen's pixel 50,0 because the resolutions don't match). I was trying to think of a way to convert my screen's pixel location into the "local pixel location", and I have two ideas but I am not sure if any of them is any good. Note I am currently using cmd+shift+4 on macos to get the screen coordinates and the video is playing fullscreen like in the screenshot below.

  


  Ideas

  


  


  1. If I manually measure and account for this vertical offset, would it effectively convert the screen coordinate into the "local" one ?

     


  


  2. If I instead adjust my SwsContext to put the destination height and width as that of my screen, will it effectively replace the need to convert screen coordinates to the video coordinates ?

     


  


  


  Problems with the Ideas

  


  The problems I see with the first solution are that I am assuming there is no hidden horizontal offset (or conversely that all of the width of the video is actually renderable on the screen). Additionally, this solution would only get an approximate result as I would need to manually measure the offsets, screen width, and screen height using the method I currently am using to get the screen coordinates.

  


  With the second solution, aside from the question of if it will even work, the problem becomes that I can no longer measure what the screen coordinates I want are because I can't seem to get rid of those black bars in VLC.

  


  Some Testing I did

  


  Given that if the black bars are part of the video itself, my entire problem would be fixed (maybe ?) I tried seeing if the black bars were part of the video, and when I looked at the frame data's first pixel, it was black. The problem then is that if the black bars are entirely part of the video, then why are the colors I get for some pixels slightly off (I am checking with ColorSync Utility). These colors aren't just slightly off as in wrong but it seems more that they belong to a slightly offset region of the video.

  


  However, this may be somewhat explained if ffmpeg reads right to left. When I put the top left corner of the video into the program and looked again at the pixel data in the frame for that location (location again was calculated by assuming the video location would be the same as the screen location) instead of getting white, I got a bluish color much like the glove in the top right corner.

  


  The Watered Down Code

  


      struct SwsContext *rescaler = NULL;
    rescaler = sws_getContext(codec_context->width, codec_context->height, codec_context->pix_fmt, codec_context->width, codec_context->height, AV_PIX_FMT_RGB0, SWS_FAST_BILINEAR, NULL, NULL, 0);

// Get Packets (containers for frames but not guaranteed to have a full frame) and Frames
    while (av_read_frame(avformatcontext, packet) >= 0)
    {
        
        // determine if packet is video packet
        if (packet->stream_index != video_index)
        {
            continue;
        }
        
        // send packet to decoder
        if (avcodec_send_packet(codec_context, packet) < 0)
        {
            perror("Failed to decode packet");
        }
        
        // get frame from decoder
        int response = avcodec_receive_frame(codec_context, frame);
        if (response == AVERROR(EAGAIN))
        {
            continue;
        }
        else if (response < 0)
        {
            perror("Failed to get frame");
        }
        
        // convert frame to RGB0 colorspace 4 bytes per pixel 1 per channel
        response = sws_scale_frame(rescaler, scaled_frame, frame);
        if(response < 0){
            perror("Failed to change colorspace");
        }
        // get data and write it
        int pixel_number = y*(scaled_frame->linesize[0]/4)+x; // divide by four gets pixel linesize (4 byte per pixel)
        int byte_number = 4*(pixel_number-1); // position of pixel in array
        // start of debugging things
        int temp = scaled_frame->data[0][byte_number]; // R
        int one_after = scaled_frame->data[0][byte_number+1]; // G
        int two_after = scaled_frame->data[0][byte_number+2]; // B
        int als; // where i put the breakpoint
        // end of debugging things
    }


  


  In Summary

  


  I have no idea what is happening.

  


  I take the data for a pixel and compare it to what colorsync utility says should be there, but it is always slightly off as though the pixel I was actually reading was offset from what I thought I was reading. Therefore, I want to find a way to get the pixel location in a video given a screen coordinate when the video is in fullscreen, but I have no idea how to (aside from a few ideas that are probably bad at best).

  


  Also does FFMPEG put the frame data right to left ?

  


  A Video Better Showing My Problem

  


  https://www.youtube.com/watch?v=NSEErs2lC3A

  


• 12 ffmpeg xfade transitions : "option not found"

  17 juillet 2023, par nimda

  I'm running into a strange error where each of these following 12 of the 56 xfade transitions are throwing errors :

  


  

  • hlwind

  


  • hrwind

  


  • vuwind

  


  • vdwind

  


  • coverleft

  


  • coverright

  


  • coverup

  


  • coverdown

  


  • revealleft

  


  • revealright

  


  • revealup

  


  • revealdown

  


  


  I've tested all filter transitions with the following command, which is being generated by a nodeJS app with the fluent-ffmpeg library, and only the twelve listed above fail with the same error "option not found" :

  


  ffmpeg -f lavfi -t 169
  -i color=c=000000:s=960x540:r=30:duration=169
  -i file:///Users/johnbandy/Work/Projects/Active/CNCT/XFade/_assets/crosstrekvx-1.jpg
  -i file:///Users/johnbandy/Work/Projects/Active/CNCT/XFade/_assets/crosstrekvx-2.jpg 
  -filter_complex
    [1:v]format=pix_fmts=yuva420p,scale=w=4608:h=-1,zoompan=z='1.2+(0*(ot/4))':x='(on/(30*4))*(iw-iw/1.2)':y='(ih-ih/zoom)/2':d=120:s=960x540[1_up];
    [1_up]scale=w=960:h=-1[1_down];
    [2:v]format=pix_fmts=yuva420p,scale=w=4608:h=-1,zoompan=z='1.2+(0*(ot/4))':x='(on/(30*4))*(iw-iw/1.2)':y='(ih-ih/zoom)/2':d=120:s=960x540[2_up];
    [2_up]scale=w=960:h=-1[2_down];
    [1_down][2_down]xfade=transition=revealdown:duration=1:offset=3[2_out];
    [0:v][2_out]overlay=x=0:y=0:enable='lte(t,7)'[all_out]
  -vcodec libx264 -r 30 -f mp4 -map [all_out] -preset veryfast -crf 18 -movflags frag_keyframe+empty_moov -pix_fmt yuv420p all.mp4


  


  fflog output :

  


  ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, lavfi, from 'color=c=000000:s=960x540:r=30:duration=169':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, yuv420p, 960x540 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 30 tbn
Input #1, image2, from 'file:///Users/johnbandy/Work/Projects/Active/CNCT/XFade/_assets/crosstrekvx-1.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 36486 kb/s
  Stream #1:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 960x540, 25 fps, 25 tbr, 25 tbn
Input #2, image2, from 'file:///Users/johnbandy/Work/Projects/Active/CNCT/XFade/_assets/crosstrekvx-2.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 35680 kb/s
  Stream #2:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 960x540, 25 fps, 25 tbr, 25 tbn
[Parsed_xfade_8 @ 0x60000137cb00] [Eval @ 0x16f9b5348] Undefined constant or missing '(' in 'revealdown'
[Parsed_xfade_8 @ 0x60000137cb00] Unable to parse option value "revealdown"
Error applying option 'transition' to filter 'xfade': Option not found
Error initializing complex filters.
Option not found


  


  The fflog for a successful video generation follows (for comparison to the erroneous log above) :

  


  ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, lavfi, from 'color=c=000000:s=960x540:r=30:duration=169':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, yuv420p, 960x540 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 30 tbn
Input #1, image2, from 'file:///Users/johnbandy/Work/Projects/Active/CNCT/XFade/_assets/crosstrekvx-1.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 36486 kb/s
  Stream #1:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 960x540, 25 fps, 25 tbr, 25 tbn
Input #2, image2, from 'file:///Users/johnbandy/Work/Projects/Active/CNCT/XFade/_assets/crosstrekvx-2.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 35680 kb/s
  Stream #2:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 960x540, 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 (wrapped_avframe) -> overlay
  Stream #1:0 (mjpeg) -> format:default
  Stream #2:0 (mjpeg) -> format:default
  overlay:default -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[swscaler @ 0x140158000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x120098000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x130078000] deprecated pixel format used, make sure you did set range correctly
    Last message repeated 2 times
[swscaler @ 0x140598000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x1302d8000] deprecated pixel format used, make sure you did set range correctly
    Last message repeated 1 times
[libx264 @ 0x138e09080] using SAR=1/1
[libx264 @ 0x138e09080] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x138e09080] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x138e09080] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'all.mp4':
  Metadata:
    encoder         : Lavf60.3.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(progressive), 960x540 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 15360 tbn
    Metadata:
      encoder         : Lavc60.3.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
frame=   32 fps=0.0 q=24.0 size=       0kB time=00:00:01.00 bitrate=   0.3kbits/s speed=1.94x    
frame=   87 fps= 84 q=24.0 size=       0kB time=00:00:02.83 bitrate=   0.1kbits/s speed=2.75x    
frame=  164 fps=107 q=24.0 size=       0kB time=00:00:05.40 bitrate=   0.1kbits/s speed=3.52x    
frame=  812 fps=399 q=24.0 size=     512kB time=00:00:26.96 bitrate= 155.5kbits/s speed=13.3x    
frame= 1821 fps=718 q=24.0 size=     512kB time=00:01:00.60 bitrate=  69.2kbits/s speed=23.9x    
frame= 2469 fps=813 q=24.0 size=     512kB time=00:01:22.23 bitrate=  51.0kbits/s speed=27.1x    
frame= 3421 fps=967 q=24.0 size=     512kB time=00:01:53.96 bitrate=  36.8kbits/s speed=32.2x    
frame= 4522 fps=1120 q=24.0 size=     768kB time=00:02:30.63 bitrate=  41.8kbits/s speed=37.3x    
frame= 5070 fps=1184 q=-1.0 Lsize=     812kB time=00:02:48.90 bitrate=  39.4kbits/s speed=39.5x    
video:769kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 5.626653%
[libx264 @ 0x138e09080] frame I:21    Avg QP: 2.73  size:  3710
[libx264 @ 0x138e09080] frame P:1305  Avg QP: 6.83  size:   449
[libx264 @ 0x138e09080] frame B:3744  Avg QP: 9.07  size:    33
[libx264 @ 0x138e09080] consecutive B-frames:  1.1%  1.4%  0.0% 97.5%
[libx264 @ 0x138e09080] mb I  I16..4: 95.4%  0.7%  3.8%
[libx264 @ 0x138e09080] mb P  I16..4:  0.1%  0.3%  0.1%  P16..4:  2.3%  0.2%  0.1%  0.0%  0.0%    skip:97.0%
[libx264 @ 0x138e09080] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.1%  0.0%  0.0%  direct: 0.1%  skip:99.8%  L0:70.9% L1:27.3% BI: 1.8%
[libx264 @ 0x138e09080] 8x8 transform intra:13.8% inter:71.9%
[libx264 @ 0x138e09080] coded y,uvDC,uvAC intra: 19.1% 12.6% 5.2% inter: 0.3% 0.5% 0.0%
[libx264 @ 0x138e09080] i16 v,h,dc,p: 94%  2%  4%  1%
[libx264 @ 0x138e09080] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 41% 22%  2%  6%  2%  9%  3%  8%
[libx264 @ 0x138e09080] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 32% 16%  2% 11%  4% 12%  3%  8%
[libx264 @ 0x138e09080] i8c dc,h,v,p: 88%  9%  2%  1%
[libx264 @ 0x138e09080] Weighted P-Frames: Y:1.6% UV:1.6%
[libx264 @ 0x138e09080] kb/s:37.25


  


• Revision 36958 : Petite mise à jour de base de donnée... Pixelformat n’est pas un int mais ...

  5 avril 2010, par kent1@… — Log

  Petite mise à jour de base de donnée... Pixelformat n’est pas un int mais un string
  On ne refile pas de valeurs hallucinantes à l’encodeur non plus